From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77E02C0015E for ; Thu, 3 Aug 2023 08:52:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E80B828021E; Thu, 3 Aug 2023 04:52:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E099B2801EB; Thu, 3 Aug 2023 04:52:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA9E928021E; Thu, 3 Aug 2023 04:52:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B44D42801EB for ; Thu, 3 Aug 2023 04:52:16 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 866E4410CE for ; Thu, 3 Aug 2023 08:52:16 +0000 (UTC) X-FDA: 81082176672.18.E4D41E6 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf24.hostedemail.com (Postfix) with ESMTP id 9C52018001B for ; Thu, 3 Aug 2023 08:52:13 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=sF3+SbxN; spf=pass (imf24.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691052734; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=t9D3HOm7zMWuXjtU8sIhIIEaYx21Igl7pf2ICGEcc9E=; b=ABqi6BzcBw1J+KnaJfX9orUnv1YWQLxpNzNs9R72LJhriC2bpYdmAdA068EB1eV1amoHwu jYxr2c0YwkHKLPYWEqtxNmNKTU8dKjxjCgbGBEHcP8HgwbA1Y8zFDmSw1mmxTc7n4V/zDP OidDX0WBDIJ4hmjp9pQiCR9vDez5ge4= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=sF3+SbxN; spf=pass (imf24.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691052734; a=rsa-sha256; cv=none; b=jtVb3j1BI+9H73NbzjqS/VuWW+FMZI+eH/vIobAhDiaZiJigb2yD+evUOTdCzxodM9z+Ea LPE6LXlbo3ulqaInlTQwbnM5M8A87bOcqHsIlqPHjgmuO75MJAVD6dU0pav/IOWKXqeSav iEeVXmjaqZQJrqR6LmXCPeM4pSeaVZc= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id F31B41F381; Thu, 3 Aug 2023 08:52:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1691052732; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=t9D3HOm7zMWuXjtU8sIhIIEaYx21Igl7pf2ICGEcc9E=; b=sF3+SbxNeHQ1K1dRKmTniekm7Mo3Up1JFOTPUv8l0nWxxP8sfEw4wh0mHufE2QPA8f1E7G Kq2eQbQAoXymq4B2oGuAzNKGhB4EbHRa/fDfXwVNBRz1Xvqu7F/pNU6OttugNUoRIX5z+P MN5k0UV+NSXronooFVVvM5WAlEZMuTE= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id CEEC1134B0; Thu, 3 Aug 2023 08:52:11 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id sDuRL7tqy2TFYAAAMHmgww (envelope-from ); Thu, 03 Aug 2023 08:52:11 +0000 Date: Thu, 3 Aug 2023 10:52:11 +0200 From: Michal Hocko To: David Hildenbrand Cc: Aneesh Kumar K V , linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu, Oscar Salvador , Vishal Verma Subject: Re: [PATCH v7 7/7] mm/memory_hotplug: Enable runtime update of memmap_on_memory parameter Message-ID: References: <20230801044116.10674-1-aneesh.kumar@linux.ibm.com> <20230801044116.10674-8-aneesh.kumar@linux.ibm.com> <31305ab7-1e65-80aa-ee91-9190c8f67430@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 9C52018001B X-Rspam-User: X-Stat-Signature: md9wqiicqtbbeyj6gd1e5gdjame9ap5t X-Rspamd-Server: rspam01 X-HE-Tag: 1691052733-818131 X-HE-Meta: U2FsdGVkX1/QHL84jfh3YIzjHJ5qqFhAU0r17RUYqKpI5G4zOWV15EpGd4NKjpEhH3wWYoN0TT0uqt0kXMpg5NUIdug3TYmV0Qt0JX4o8KfxXaBAOjpsNWodRDYabEZDkw/6GmNO8u7NxrkwzKzp3tKANTXL+v7hTX3WU/bbaCbngt9LwtERQuNjRytDTj+9E/MsBY6yzekj9U5BjyFaGuQRy2ICp/8Gl+djclT03KYA7RLbgdzfnLWeRouQx5s9FIpuRzHkRCKgsLBoTZ+h0dwe3foycko0arzAuTeyjB0/NVrg5ejqVUduO6As9qBiYojXshsKH9drPNf79d/X0QK/MfoaKH3xmKUKDu2INYLzLlVsBirO9TQr6AgufBPOUQ2ooLadIbHz4GCKMjyQpTPm+JGv8gup99d5NXwPDLUKmk7nYgL/Vd/TjTLdIW4kZq4VlQgTgJWvuGh7SyflEoPwkSqbZV/6ZMsy6Rates4ttQWUjvbitdAnUBKD/cBdR93lbEkzb1P7DLH2QztxPrhOeSDJbN6fWSNdYq0zihBv0qwUDBGjfdPv432RGHHV309NpRu515SODU2Gm6VSGQYK7qWTf0lWXJ1V6USmJCnpYTF+MXE+VC+JOiIKsmKp7HHr5zZ8mQXfv9mkvvOwc9dF5uipwjQZpCjhoxoBIHDyfzy2RbLPdz9h1h8li8zrUNbK0eMtz0wWWicbB12+4st2ccwSHmC/Y9m5PDYyv5DRcKw7UXqjw+PUB+9b5RlcfDo0zE0fV4hwJC8Ey/W5sJJ1zSl3rAmX4WNhQx85x6nE+cKx3OeyQoD8R99f4d37ubR9zl8OEzojscDfaIn+He3ewanQN9j5iK5DyGp5PxC/9mQvhIJYi/3ntWsvcIwosGbzB0dXnubTrgwVBAl8OqAg2OI8fatfPjSplT/0NPniWsUC+K4xKUF9/Lw0RZ8XFmI9Y07vMeeZJ6r38rB 5i3qJtNc NVIrmFO6nj3xsPhw8EwNacM0w8kF9wxAyUKFUnLQn8t5LNdKvmOt12bYMPAr26NIpbGeeI2Fn0AkUaSX9Lz1iRXtGQHkrQYv8z9mXQXhoucR3r0Tynnpz/WeoBT0G3eB0t65iIWPfveTbosqSVNm7lXuxe6mQmffWGQadcep841Hgi6G9mmKO5vXDKmTtRNk/D91KjCebK6+XMrE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 02-08-23 18:59:04, Michal Hocko wrote: > On Wed 02-08-23 17:54:04, David Hildenbrand wrote: > > On 02.08.23 17:50, Michal Hocko wrote: > > > On Wed 02-08-23 10:15:04, Aneesh Kumar K V wrote: > > > > On 8/1/23 4:20 PM, Michal Hocko wrote: > > > > > On Tue 01-08-23 14:58:29, Aneesh Kumar K V wrote: > > > > > > On 8/1/23 2:28 PM, Michal Hocko wrote: > > > > > > > On Tue 01-08-23 10:11:16, Aneesh Kumar K.V wrote: > > > > > > > > Allow updating memmap_on_memory mode after the kernel boot. Memory > > > > > > > > hotplug done after the mode update will use the new mmemap_on_memory > > > > > > > > value. > > > > > > > > > > > > > > Well, this is a user space kABI extension and as such you should spend > > > > > > > more words about the usecase. Why we could live with this static and now > > > > > > > need dynamic? > > > > > > > > > > > > > > > > > > > This enables easy testing of memmap_on_memory feature without a kernel reboot. > > > > > > > > > > Testing alone is rather weak argument to be honest. > > > > > > > > > > > I also expect people wanting to use that when they find dax kmem memory online > > > > > > failing because of struct page allocation failures[1]. User could reboot back with > > > > > > memmap_on_memory=y kernel parameter. But being able to enable it via sysfs makes > > > > > > the feature much more useful. > > > > > > > > > > Sure it can be useful but that holds for any feature, right. The main > > > > > question is whether this is worth maintaing. The current implementation > > > > > seems rather trivial which is an argument to have it but are there any > > > > > risks long term? Have you evaluated a potential long term maintenance > > > > > cost? There is no easy way to go back and disable it later on without > > > > > breaking some userspace. > > > > > > > > > > All that should be in the changelog! > > > > > > > > I updated it as below. > > > > > > > > mm/memory_hotplug: Enable runtime update of memmap_on_memory parameter > > > > > > > > Allow updating memmap_on_memory mode after the kernel boot. Memory > > > > hotplug done after the mode update will use the new mmemap_on_memory > > > > value. > > > > > > > > It is now possible to test the memmap_on_memory feature easily without > > > > the need for a kernel reboot. Additionally, for those encountering > > > > struct page allocation failures while using dax kmem memory online, this > > > > feature may prove useful. Instead of rebooting with the > > > > memmap_on_memory=y kernel parameter, users can now enable it via sysfs, > > > > which greatly enhances its usefulness. > > > > > > > > > I do not really see a solid argument why rebooting is really a problem > > > TBH. Also is the global policy knob really the right fit for existing > > > hotplug usecases? In other words, if we really want to make > > > memmap_on_memory more flexible would it make more sense to have it per > > > memory block property instead (the global knob being the default if > > > admin doesn't specify it differently). > > > > Per memory block isn't possible, due to the creation order. > > I am not sure I follow. Could you elaborate more? Must have been a tired brain. Now I see what you mean of course. vmemmap is allocated at the time the range is registered and therefore memory block sysfs created. So you are right that it is too late in some sense but I still believe that this would be doable. The memmap_on_memory file would be readable only when the block is offline and it would reallocate vmemmap on the change. Makes sense? Are there any risks? Maybe pfn walkers? -- Michal Hocko SUSE Labs