From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C1BEC76196 for ; Tue, 28 Mar 2023 07:39:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 066A36B0072; Tue, 28 Mar 2023 03:39:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F31806B0074; Tue, 28 Mar 2023 03:39:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD1AB900002; Tue, 28 Mar 2023 03:39:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id CAD356B0072 for ; Tue, 28 Mar 2023 03:39:42 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8AE081C39A2 for ; Tue, 28 Mar 2023 07:39:42 +0000 (UTC) X-FDA: 80617507404.03.F0A6ABB Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf21.hostedemail.com (Postfix) with ESMTP id 982C11C001D for ; Tue, 28 Mar 2023 07:39:39 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=jHH+FS9q; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf21.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679989180; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6LsNQFW9qDJbRtDT8d0v52+HSjljoW5023G5IXOpgfw=; b=V2cDZKBmxC71p/h69HAbELuBOTsGKfPqK0FJiHVR6R7t6JNdS389v1SnOoDJsPxnOJlFNI dR4Zqru6Zr/knpqdHGGA0XiiCRkmnkOtZQHNedt3GduCW5Xb8DoDGLUWkPPclG2cdY6RFq nbkfsC+/EXTGTZAIdIMiYGe1dQCxuBI= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=jHH+FS9q; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf21.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679989180; a=rsa-sha256; cv=none; b=Wf+2qQi9NiPHUoDti5jtOtGjPAw6MnUE5W7fSO2B1qNHuUxVQMh1SObQUpEFcN32XrIRJp egsDpaRE2yMzhv4Kphtgotjw88rId8hgQntYHY/IxDbZAbYjxKQPiixDpap0DVeNeGdsAx BDOZMb8Cv9XusoBaBn3Ab70Jk7S2I7g= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id E37D81FD6A; Tue, 28 Mar 2023 07:39:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1679989177; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6LsNQFW9qDJbRtDT8d0v52+HSjljoW5023G5IXOpgfw=; b=jHH+FS9qODj00E1eRKg6/VYtJn7vkmlf4B0mnpK86VQ+9DisQyhb6dJ6bfYSvAlpJYrjUR meLggKv3uhru4lKroWou8mhhpT1a0SMGu0JCgDYpfHVoLiK8krIR5uCKcvCPpOf0+kAh2J E8xoamhubbBxrSisYFF0Yg1EcE3rYo4= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C555B1390B; Tue, 28 Mar 2023 07:39:37 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 2VNgLLmZImR8CAAAMHmgww (envelope-from ); Tue, 28 Mar 2023 07:39:37 +0000 Date: Tue, 28 Mar 2023 09:39:37 +0200 From: Michal Hocko To: Mike Rapoport Cc: linux-mm@kvack.org, Andrew Morton , Dave Hansen , Peter Zijlstra , Rick Edgecombe , Song Liu , Thomas Gleixner , Vlastimil Babka , linux-kernel@vger.kernel.org, x86@kernel.org Subject: Re: [RFC PATCH 1/5] mm: intorduce __GFP_UNMAPPED and unmapped_alloc() Message-ID: References: <20230308094106.227365-1-rppt@kernel.org> <20230308094106.227365-2-rppt@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 982C11C001D X-Stat-Signature: 8bhwk6t37x1cnpuccthnhp1yqumqyghi X-HE-Tag: 1679989179-913988 X-HE-Meta: U2FsdGVkX19mYRiCPVYD6oxrwZPT9A73IQ4xxKbCwr/kX/j5XiL6jimJpHRwRNU0+2kPNCckfserN4lcZLiWw3m1lml1UGcosdh3bFJZE8F7ntOg3QZsVYxqG17O0teTTZpL6ypd5/Fld37GypflTVhECqYAoWz7QmotlnBSRcLjLD90rluW0F5psweW5l3vqXztHUa318d3vE6FLAqq5JFg/qwAjbnmiDEcmJtoc5X+rGDG2sKoMmNPueNKEC9nXEbExIe+9wb6Z31XrkqFxQeiLLUeTWUB/7c6IqDSZKr64MTPdldyJZTtgOmCRylCl7PVy0HGLYZEwFKo5Q3Hesgwxg3lgO2eEVhmfEy8I8nmKmLv+yCuhLZCe4XFwjlNSdeNur190X2jayyklUrLLLH8kUDCKVrAe79KmZHrWoP8zr3qZjyuyLgYCqRi3kXJcXLfEtLB9U0hWW5zWpqTc0NsGsOSzY2PJZSDH7ns/O94gqKhkOVDpY84jUj4E3hDFUBOF4QJYAezmai622/vSisMcEVaqOc5vyDpO8kXOFoxJb0fFftg2yH51sJ6mTSE+sM4IPHwoNiOeGPMhxpIldX+oy6Ta7Wvhrg7yOhsoHy6h+eTm2n8U1WymHLXXpfFMYgbaN2tgyzsRmzR+wMWF9R3vqRU7TCTjJuIGu747UUlL7S4VX5ydj7dZaxBDjeOwcbZTQ5sjSqio1NMBmCzadTac/l8voAs0xtNpMs31PbAPFUPunQBOcXZocp4OPp0aNE+AEO+FvYzEDBfob77AIAuFeiUVO3nH2jrmFbJGo1GMh37rVtvuY5hFwOi9C+1AFpJhv89iDP6iMjmLv0JerfusA7fdf8gsNvv5iLyFERjluGyyJCKoo0oV121tLtEPe38r+nfh5fOTdIm8uFSIxY6FSDSP8mywPPzEzkYwmsKaM3lRI/BnpqR/iigLA18YIrq5suePQud2I52t9S fU0w7G93 7rWHKPTY5DRrgcawX+9c6S7rKHx8gh0WbjSi7OVDZ3VCAnqnDIpW/gqujhY8GAn/3ffu9uTYhdTmzY3MfxjB8sRVNSM875ufC0EVi8ZSJC2qxxnUIcdPw/r0fZEDrHVtkPeZpW/CmlLKZQ+Z1cpCe4gJPcWtq9dexAM0SZYgkcYm590c+0a7aiF41RVqiAZHlRC/EOwNcaK8/Zi9F+J2vWxDUZBZqZS3QwqyfyzTczuO/Z83BPgrk5k8xtG7dPdb86SZonzC4nfj8AST1SISJw806iA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue 28-03-23 09:25:35, Mike Rapoport wrote: > On Mon, Mar 27, 2023 at 03:43:27PM +0200, Michal Hocko wrote: > > On Sat 25-03-23 09:38:12, Mike Rapoport wrote: > > > On Fri, Mar 24, 2023 at 09:37:31AM +0100, Michal Hocko wrote: > > > > On Wed 08-03-23 11:41:02, Mike Rapoport wrote: > > > > > From: "Mike Rapoport (IBM)" > > > > > > > > > > When set_memory or set_direct_map APIs used to change attribute or > > > > > permissions for chunks of several pages, the large PMD that maps these > > > > > pages in the direct map must be split. Fragmenting the direct map in such > > > > > manner causes TLB pressure and, eventually, performance degradation. > > > > > > > > > > To avoid excessive direct map fragmentation, add ability to allocate > > > > > "unmapped" pages with __GFP_UNMAPPED flag that will cause removal of the > > > > > allocated pages from the direct map and use a cache of the unmapped pages. > > > > > > > > > > This cache is replenished with higher order pages with preference for > > > > > PMD_SIZE pages when possible so that there will be fewer splits of large > > > > > pages in the direct map. > > > > > > > > > > The cache is implemented as a buddy allocator, so it can serve high order > > > > > allocations of unmapped pages. > > > > > > > > Why do we need a dedicated gfp flag for all this when a dedicated > > > > allocator is used anyway. What prevents users to call unmapped_pages_{alloc,free}? > > > > > > Using unmapped_pages_{alloc,free} adds complexity to the users which IMO > > > outweighs the cost of a dedicated gfp flag. > > > > Aren't those users rare and very special anyway? > > > > > For modules we'd have to make x86::module_{alloc,free}() take care of > > > mapping and unmapping the allocated pages in the modules virtual address > > > range. This also might become relevant for another architectures in future > > > and than we'll have several complex module_alloc()s. > > > > The module_alloc use is lacking any justification. More context would be > > more than useful. Also vmalloc support for the proposed __GFP_UNMAPPED > > likely needs more explanation as well. > > Right now module_alloc() boils down to vmalloc() with the virtual range > limited to the modules area. The allocated chunk contains both code and > data. When CONFIG_STRICT_MODULE_RWX is set, parts of the memory allocated > with module_alloc() remapped with different permissions both in vmalloc > address space and in the direct map. The change of permissions for small > ranges causes splits of large pages in the direct map. OK, so you want to reduce that direct map fragmentation? Is that a real problem? My impression is that modules are mostly static thing. BPF might be a different thing though. I have a recollection that BPF guys were dealing with direct map fragmention as well. > If we were to use unmapped_pages_alloc() in modules_alloc(), we would have > to implement the part of vmalloc() that reserves the virtual addresses and > maps the allocated memory there in module_alloc(). Another option would be to provide an allocator for the backing pages to vmalloc. But I do agree that a gfp flag is a less laborous way to achieve the same. So the primary question really is whether we really need vmalloc support for unmapped memory. -- Michal Hocko SUSE Labs