From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 266A0C7618D for ; Thu, 6 Apr 2023 20:07:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A56796B0071; Thu, 6 Apr 2023 16:07:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A06B76B0074; Thu, 6 Apr 2023 16:07:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F7326B0075; Thu, 6 Apr 2023 16:07:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 814966B0071 for ; Thu, 6 Apr 2023 16:07:25 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5B803160E23 for ; Thu, 6 Apr 2023 20:07:25 +0000 (UTC) X-FDA: 80652050850.01.1468BF3 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf08.hostedemail.com (Postfix) with ESMTP id 94F02160019 for ; Thu, 6 Apr 2023 20:07:23 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=T9IQUPkI; dmarc=none; spf=pass (imf08.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680811643; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2IimjLy1ND6Tgpx+QXJ2YfCff+Q0oUlqw0i50k7xbyo=; b=DY+8e2snhScBs7CnR3T1P/wp4LSzRqaj9wEx4awfN1XNzV/VVYcp0ZBMciRIbWBA2LaWOg dzpxzMRQSaFJ7TEq83D8BRWRPNwo0CRCN9TgllakL5WZN8pZBDUVOKK34YTOhhX7HI6DNM l+vMhnG5lmevkVn/+/esA5EysCdF1js= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=T9IQUPkI; dmarc=none; spf=pass (imf08.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680811643; a=rsa-sha256; cv=none; b=pEv+hwFOcBA/OUe7cirewjEuHXebxUW5jNSv6J9pPRmeRtK9mIBTU1a8fPbXvsu46lb+kB I8+d1PoBvZmkqoJPCjTSVxMxIfxF3y+2gBJRlpO+2MilyYfb3DadRUK3cBjWuFLtgi7wT7 bFYb+3zQWGlqBuyc/L/xIr0GuyaCzCI= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A3C3264B4F; Thu, 6 Apr 2023 20:07:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9A6BCC433EF; Thu, 6 Apr 2023 20:07:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1680811642; bh=gQwoDTS3ub+TjZRhwYbtU1P4YaBDlo3MUyuPnnHIKbg=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=T9IQUPkIdSgBlNKH2SPFrNClp0DoyyRNLc/dtDjdkZtdvvigCTNIcqngKO+ukOiyI GigGHuQxXzI6BVSos1cJ7mRN3YLqUb0gbd1bwUW6K15LS+rRGAEG1OyPQxeS0Pilg2 4sE6f5KmFR3N1hx5gyd/41YYQ1cwNeuHCwjSxOxs= Date: Thu, 6 Apr 2023 13:07:20 -0700 From: Andrew Morton To: John Hubbard Cc: Catalin Marinas , Will Deacon , Ard Biesheuvel , Anshuman Khandual , Mark Rutland , Kefeng Wang , Feiyang Chen , Alistair Popple , Ralph Campbell , , LKML , , Subject: Re: [PATCH] arm64/mm: don't WARN when alloc/free-ing device private pages Message-Id: <20230406130720.ba11ed2de992cc4c2485ad5d@linux-foundation.org> In-Reply-To: <20230406040515.383238-1-jhubbard@nvidia.com> References: <20230406040515.383238-1-jhubbard@nvidia.com> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 94F02160019 X-Stat-Signature: dsbbgkmp5kdswqohmmwkh9e3yxdprdwb X-HE-Tag: 1680811643-30054 X-HE-Meta: U2FsdGVkX1/B3qG14beps1tdySU0CIwb3sfzd7C6sSAmXCVyywJI4+fbv6T0aKO+zUqnsTNhulG3XA1qbFfxr2iNhmMuxUkteyzcSEGaY1dAxeJp1ZqpaUMq04lP+YPbMtJQDT0r1NiP1DsyOy36Vdh67UAIMu8ljgURnfQn1Gqh4HwkTG3SEZwaWw7RJldNCkiGLlnLlrQaXjcSJY8AlTBci9SVUZcJwMHfKA1InJDsRbzc/P4/dNVlo+J/Z66zz37hb62S/mstfkd5dCMWytP5Quaum+M5ZCipiT5efivLjlrA8uPBLImp70qzF8aY+hyc5dFzxLFCm2ueyD3GJx9h4tAC7wL/VxuNhGMgooUry81gfELQIptG6loFnDiUMcVlitn9vxT5g4bPFo+yewusmU2FOsC3abl1/OS0D+padLX4RyGDjl6LZCfEcbWoF0OJBiN4vyYJp01rWHoedD12jqMp68RB3pqv8t8BEyjtoUAV1Nz0mAXBKacCriREH5Dz1tFzVOcPc/2p2UvncgW0FSpJl/NvlQJwnCvxj7TB6efz8krtBW19PRUwMjHn+ZhhbwYam8d6oQ6EFpGemCLUGZxBmbuqB4O8NRT0lE19kmePw7Hcg54tgE+VcwPTtY0hOK65pNBFjY4DAmRztMqfb5GhEw7AnHuXwKM8TOD6VQl39zIMirlDbR86T/jNxevSdkek1tCI0JNL1lnBoxhEvxoK7dus6jxxdCMYQncY+yiE01yihxpl3B6bM2KY2TyIAA+lBjdiX8vnKtWnsoce9NKCDmBDuJEyYSr8KzS2md5WM2A34zTeLiD7ILzdq1cSLjUbTQaADyqdnybREUiI3HWguse4xcBpMp4jDbs27DFcxBlZ12UInK2rCr5twQ3bi4iQAfTxAz2AJnJutyMz6vNkT3aXl7vHNuWl5CC6TuzQXDhCgo5C3RyT0XLxycglb0LPIc2RRQ8Q9e1 Gi9DnE9i 4/NRxpqzrMC/1ct4bqiN//YtQf5mBzYcZwDTt/pf4fWXNDqEY0kXu2bdjx5ba0EU3KlLyVRw0D2fJ2bHvSMwNQYOhc4vL0XWlwYvMAppspRxieZ9tGBkrR2orRCNFBFyZcCHfWsyuc606R+A/EpTN2J4COCmF0OX7JDh5qXDncwjxlxUZd1Zyk7aiOxEUuM/1/S2hMh5r0bStv83rC7FyiYAxX24DmeQbhX/xxLYCdNaMg0SoML76GsvzHoZG4YTZuNQunmBRyYHslLmAfbIdtimPV39fMA7xHEhJnM2+v2mhV6HfvQL2Y5gGHGUUBLIKwixWK4XUdaRcscM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 5 Apr 2023 21:05:15 -0700 John Hubbard wrote: > Although CONFIG_DEVICE_PRIVATE and hmm_range_fault() and related > functionality was first developed on x86, it also works on arm64. > However, when trying this out on an arm64 system, it turns out that > there is a massive slowdown during the setup and teardown phases. > > This slowdown is due to lots of calls to WARN_ON()'s that are checking > for pages that are out of the physical range for the CPU. However, > that's a design feature of device private pages: they are specfically > chosen in order to be outside of the range of the CPU's true physical > pages. > > ... > > --- a/arch/arm64/mm/mmu.c > +++ b/arch/arm64/mm/mmu.c > @@ -1157,8 +1157,10 @@ int __meminit vmemmap_check_pmd(pmd_t *pmdp, int node, > int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, > struct vmem_altmap *altmap) > { > +/* Device private pages are outside of the CPU's physical page range. */ > +#ifndef CONFIG_DEVICE_PRIVATE > WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END)); For a simple expression like this to cause a "massive slowdown", I assume the WARN is triggering. But changelog doesn't mention massive dmesg spewage? Given Ard's comments, perhaps a switch to WARN_ON_ONCE() would suit?