From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 354ACC74A5B for ; Wed, 29 Mar 2023 08:13:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 90CF46B0072; Wed, 29 Mar 2023 04:13:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8BC836B0074; Wed, 29 Mar 2023 04:13:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7AB156B0075; Wed, 29 Mar 2023 04:13:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 6BD0B6B0072 for ; Wed, 29 Mar 2023 04:13:27 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 345F1AB30C for ; Wed, 29 Mar 2023 08:13:27 +0000 (UTC) X-FDA: 80621221254.01.149C336 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf23.hostedemail.com (Postfix) with ESMTP id 52D98140007 for ; Wed, 29 Mar 2023 08:13:25 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=KgmXk3WD; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf23.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680077605; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4uKJsI6kcmXX5PcI4vzLQ7gfadOE59ADRMPHNu9uM4g=; b=QR0SCLlz1ZRzVOAkN2pQ9GkxrsoMjZo0e5M5ZQo5SGeGAULnhR25Wt4QCuICRGGbWOolqD ZmM0gWXNnZEqPeRwCkQTQr4iRdGjd3DGvdXXjrGSLWEw4tWhcecYfpY9dux0uuqoY4Pwju 56ve1U4ur594SnzSeH/NCy+/EoSqE/8= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=KgmXk3WD; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf23.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680077605; a=rsa-sha256; cv=none; b=W5YkmJ0jzAn0h8tBMrOzYbKsYoQtlYbGu66IJ7NYINw7WMJTAMtRasKLEqw9YH2oCgWA9c Rs8r9KvFmwAFziMqdm9jY0BsEA7Q6XH0WxfroO4qSsFbNdDTIJ0wg0/Ge5zu0OyD1O8WcU CHjTUA2zk39F+w4qqtBmGwtZZxBSbYA= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id B00141FDF3; Wed, 29 Mar 2023 08:13:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1680077603; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=4uKJsI6kcmXX5PcI4vzLQ7gfadOE59ADRMPHNu9uM4g=; b=KgmXk3WDiPzBXzdj45vKo4Vxz9vi4sI6hNTwjCUsKoQDpB57b7/5//GKc082Or6lCMx/b9 ZKDzzzwTxi4ao+87xcxzxrVyODS5gjn/Lku2in5ezddE1za04vzIij6QMQpnfKlTJeGdIU mG87qg70katMuzooi3uJR+33T3GLTfE= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id A26B0139D3; Wed, 29 Mar 2023 08:13:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id CwdQJyPzI2TmWQAAMHmgww (envelope-from ); Wed, 29 Mar 2023 08:13:23 +0000 Date: Wed, 29 Mar 2023 10:13:23 +0200 From: Michal Hocko To: Mike Rapoport Cc: linux-mm@kvack.org, Andrew Morton , Dave Hansen , Peter Zijlstra , Rick Edgecombe , Song Liu , Thomas Gleixner , Vlastimil Babka , linux-kernel@vger.kernel.org, x86@kernel.org Subject: Re: [RFC PATCH 1/5] mm: intorduce __GFP_UNMAPPED and unmapped_alloc() Message-ID: References: <20230308094106.227365-1-rppt@kernel.org> <20230308094106.227365-2-rppt@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 52D98140007 X-Stat-Signature: oq7nm37dkxkdhr3uu7kdqkwyw564tzxj X-HE-Tag: 1680077605-873003 X-HE-Meta: U2FsdGVkX1/IvapljkZWtHcsGFTQ3Kk29i89F/sVjdcK9A2StIAyUPmFOKI0fznPw9kysOlFSi1t2ObxZhnuVE37P+3i7MyTSo7jOPsSkrEJ36N1oO47wFL/gOy3kSVXDuKVS61+Ws+9EVmeHG+d8r25ht0p+igZ9wQZw3H2vnCVsYSVDZaix3RwfS9lRfQNHF9meVMHrHF7raABSoogc5dg06YeK+W8AEGIBaeSu+6wkGxRU4CAlsRFBvo/jJPkZvH0q4nFIm387qAqeDOOQMQ8uhQ05EhUIJ0RII4VJ+tvKMD/Y3svcbxtlKLLJKlnw4lS3TGJbqvfPGxOMnlQNJ1P7MAzs1RvPAUEyTd65RVlu3coPYyRSNgLSN+x8nkLlSNy1HXf5U6RYJXazxR7yIF7H/7ECdR84K6vyQIO/mGyhOMsfeNsjBkvwvy+srQd8cbPdf23mJnUiyoDF0rB1AhW44HZrM5vCRCFKRv2W8r8zedRXJw13lhOypJDIwHGPgPyIu3CWvi6FbajgYrH3JS7/eOF8tDxxD+y5uH9elU+u9QlB7/CMuede3cYGWtwgCrDwRcifKQS2dpNNHGf1r1lWLB9RT6jx5nHr2RY1AlqFljdqz3m3rikVxfmSPiB27WbsqB3uh2wuDZLM8IJ9NQ+ZgGcHW/8VxMf1hjFkzbGUuaZwdCkiDdMmrxXiu5VPXvoKqGNo7r7MaKcOYGqOuV6hjZ9joMSUkHc/9a+cCTNMwK4ZZARIIweRJe2xXqXaWV4gG42wDULpzkuxbIIi+WfdK9RCrml9XJlmtBho8GBFv0Ir+RIo6k3RCgCeIfkYKKYJ60MCyFzqK61m7csUe1YCwAOiul8SDKPh2G+4UuYwBpDkNjqD+WaprkwMxdHlFxKb10G1Rys3tUYRd/WzmFbVmK8BCb3TK789dDTOdc0qB41BnvZk3e+27FJ1ZmD07mFG7mAILIYEI466WL pWJH+264 TOwqTQLaqo6k5lsCQGAhCOaZ8vXlFU2FE0Lv+VEe0sM3araSOtmgGuNGntcGy3Im0iOMvoPPCnP59+TUKV8PM1givw1aukKbQXlYmMUlZv6oWXHvEot8YfkKxJ1acdwFShdwbigNVjvILmxq9nXa6Z3GLLtEC4BzXa7o7rPsdfXCKbA2Rcs68tA9pHu6W9YpIgWYDEY1TTvn9LUz39aZ1s44vnQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 29-03-23 10:28:02, Mike Rapoport wrote: > On Tue, Mar 28, 2023 at 05:24:49PM +0200, Michal Hocko wrote: > > On Tue 28-03-23 18:11:20, Mike Rapoport wrote: > > > On Tue, Mar 28, 2023 at 09:39:37AM +0200, Michal Hocko wrote: > > [...] > > > > OK, so you want to reduce that direct map fragmentation? > > > > > > Yes. > > > > > > > Is that a real problem? > > > > > > A while ago Intel folks published report [1] that showed better performance > > > with large pages in the direct map for majority of benchmarks. > > > > > > > My impression is that modules are mostly static thing. BPF > > > > might be a different thing though. I have a recollection that BPF guys > > > > were dealing with direct map fragmention as well. > > > > > > Modules are indeed static, but module_alloc() used by anything that > > > allocates code pages, e.g. kprobes, ftrace and BPF. Besides, Thomas > > > mentioned that having code in 2M pages reduces iTLB pressure [2], but > > > that's not only about avoiding the splits in the direct map but also about > > > using large mappings in the modules address space. > > > > > > BPF guys suggested an allocator for executable memory [3] mainly because > > > they've seen performance improvement of 0.6% - 0.9% in their setups [4]. > > > > These are fair arguments and it would have been better to have them in > > the RFC. Also it is not really clear to me what is the actual benefit of > > the unmapping for those usecases. I do get they want to benefit from > > caching on the same permission setup but do they need unmapping as well? > > The pages allocated with module_alloc() get different permissions depending > on whether they belong to text, rodata, data etc. The permissions are > updated in both vmalloc address space and in the direct map. The updates to > the direct map cause splits of the large pages. That much is clear (wouldn't hurt to mention that in the changelog though). > If we cache large pages as > unmapped we take out the entire 2M page from the direct map and then > if/when it becomes free it can be returned to the direct map as a 2M page. > > Generally, the unmapped allocations are intended for use-cases that anyway > map the memory elsewhere than direct map and need to modify direct mappings > of the memory, be it modules_alloc(), secretmem, PKS page tables or maybe > even some of the encrypted VM memory. I believe we are still not on the same page here. I do understand that you want to re-use the caching capability of the unmapped_pages_alloc for modules allocations as well. My question is whether module_alloc really benefits from the fact that the memory is unmapped? I guess you want to say that it does because it wouldn't have to change the permission for the direct map but I do not see that anywhere in the patch... Also consinder the slow path where module_alloc needs to allocate a fresh (huge)page and unmap it. Does the overhead of the unmapping suits needs of all module_alloc users? Module loader is likely not interesting as it tends to be rather static but what about BPF programs? [...] > > > I also think vmalloc with unmmapped pages can provide backing pages for > > > execmem_alloc() Song proposed. > > > > Why would you need to have execmem_alloc have its memory virtually > > mapped into vmalloc space? > > Currently all the memory allocated for code is managed in a subset of > vmalloc() space. The intention of execmem_alloc() was to replace > module_alloc() for the code pages, so it's natural that it will use the > same virtual ranges. Ohh, sorry my bad. I have only now realized I have misread and thought about secretmem_alloc... -- Michal Hocko SUSE Labs