From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95B17C282EC for ; Mon, 17 Mar 2025 08:23:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D91F5280002; Mon, 17 Mar 2025 04:23:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D4172280001; Mon, 17 Mar 2025 04:23:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C0B79280002; Mon, 17 Mar 2025 04:23:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A3E97280001 for ; Mon, 17 Mar 2025 04:23:51 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BE0391A1A2E for ; Mon, 17 Mar 2025 08:23:51 +0000 (UTC) X-FDA: 83230354662.18.FD10F96 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf29.hostedemail.com (Postfix) with ESMTP id 39C5C120007 for ; Mon, 17 Mar 2025 08:23:48 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of rakie.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=rakie.kim@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742199830; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wItXePYLUKn92EYF8lOrrdlrRJSLV/x/GeLZfPG+e48=; b=PQJvxPR16V7deunXXrQYNaGWTwY+tbx4Wu8HQjZnGoSJEGay1Fcq3zpGytF6ugo1bjDFTb VY8vZ376hIeqiHndx4kH44FMJUJoZ90l9b5y0QC4dRLfVKbX0UU9fndt2jw/z3lmO3qgbU wQiwexj2pm8WD5LOddMyw5AJt9EaNgs= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of rakie.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=rakie.kim@sk.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742199830; a=rsa-sha256; cv=none; b=gs5nFzk9D2xGdHonzUwNeQNjfRpY4FPHcRc88gCZYraPfmoPIMB8kRhD9dyXPQHm2FJZIg 1hjbPP3vbg3UEKfc42Z4w3bmKRqzuLdvhtWYqywR8+MrVLV9Hqg8Y3b1gERdFIgyphTIa4 jg2dGWmj12lFmOLvngiFiCHP1Wy80Xc= X-AuditID: a67dfc5b-669ff7000002311f-bc-67d7dc127398 From: Rakie Kim To: David Hildenbrand Cc: akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-cxl@vger.kernel.org, joshua.hahnjy@gmail.com, dan.j.williams@intel.com, ying.huang@linux.alibaba.com, kernel_team@skhynix.com, honggyu.kim@sk.com, yunjeong.mun@sk.com, Gregory Price , Rakie Kim Subject: Re: [PATCH v2 2/4] mm/mempolicy: Support memory hotplug in weighted interleave Date: Mon, 17 Mar 2025 17:23:36 +0900 Message-ID: <20250317082342.807-1-rakie.kim@sk.com> X-Mailer: git-send-email 2.48.1.windows.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrKLMWRmVeSWpSXmKPExsXC9ZZnka7QnevpBuf261nMWb+GzWL61AuM Fl/X/2K2+Hn3OLvF8a3z2C3OzzrFYnF51xw2i3tr/rNarF6T4cDpsXPWXXaP7rbL7B6L97xk 8tj0aRK7x4kZv1k8dj609Hi/7yqbx+dNcgEcUVw2Kak5mWWpRfp2CVwZy7o3sxbcVqn4feoD SwPjc6kuRk4OCQETie5j65m7GDnA7KXvhEBMNgEliWN7Y0AqRAQ0JDa1bQCq4OJgFrjCJNFy dQNYubBAuMTxt1UgNSwCqhJtby4ygti8AsYS9w6/Z4eYrinRcOkeE4jNKWAncePAQbAaIQEe iVcb9kPVC0qcnPmEBcRmFpCXaN46G2yXhMBtNolpT9pZIQZJShxccYNlAiP/LCQ9s5D0LGBk WsUolJlXlpuYmWOil1GZl1mhl5yfu4kRGNjLav9E72D8dCH4EKMAB6MSD6/B+mvpQqyJZcWV uYcYJTiYlUR4WXZcTxfiTUmsrEotyo8vKs1JLT7EKM3BoiTOa/StPEVIID2xJDU7NbUgtQgm y8TBKdXAmMhWv3nOYxk+1XM+VYqskxeGxU0XE3Ovkn24o/1+X0EEh9f3Deb2PYy9vzJ+RYkY 8lzLP8HLNId9yasHpfcbjVvsfm30WHv8dMFzC0H3h+18s859DVOui+/od5ypUBgw+8nOk8+d ZTemLu99GPL0FV/ntbSqWt3L5q9kXy/0TjbSebr3hIW0EktxRqKhFnNRcSIAbUzvl2gCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrPLMWRmVeSWpSXmKPExsXCNUNNS1fozvV0gyuz1C3mrF/DZjF96gVG i6/rfzFb/Lx7nN3i87PXzBbHt85jtzg89ySrxflZp1gsLu+aw2Zxb81/VotD156zWqxek2Hx e9sKNgdej52z7rJ7dLddZvdYvOclk8emT5PYPU7M+M3isfOhpcf7fVfZPL7d9vBY/OIDk8fn TXIBXFFcNimpOZllqUX6dglcGcu6N7MW3Fap+H3qA0sD43OpLkYODgkBE4ml74RATDYBJYlj e2O6GDk5RAQ0JDa1bWDuYuTiYBa4wiTRchXE4eAQFgiXOP62CqSGRUBVou3NRUYQm1fAWOLe 4ffsILaEgKZEw6V7TCA2p4CdxI0DB8FqhAR4JF5t2A9VLyhxcuYTFhCbWUBeonnrbOYJjDyz kKRmIUktYGRaxSiSmVeWm5iZY6pXnJ1RmZdZoZecn7uJERjMy2r/TNzB+OWy+yFGAQ5GJR5e g/XX0oVYE8uKK3MPMUpwMCuJ8LLsuJ4uxJuSWFmVWpQfX1Sak1p8iFGag0VJnNcrPDVBSCA9 sSQ1OzW1ILUIJsvEwSnVwLgsIl50cm7+5+JsHV2G0v/dbc4zxfp5pIs+XfBOX9Kl/y7lnrtr oGd7S1uvwrmzE5QyjcXDyu9dkDU8aWQsfrTTNDDpeqkbk+v987nyXBw/slkZON/+CX5Q5mmR /uCUsvPKuEeb7c9FH9o+OSjtn8JTG9fyDPt7QVMPfdwcb3xoi9jVH1ttlViKMxINtZiLihMB 5KlrLWICAAA= X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Queue-Id: 39C5C120007 X-Rspamd-Server: rspam03 X-Stat-Signature: g19x59mx1wxgtd6zgb79d17wimoc94uf X-HE-Tag: 1742199828-429699 X-HE-Meta: U2FsdGVkX18qY1iyxRWbt5SZrVLaOdVpIvOe5Qvm7PDkPD61n6CToJiHCC12oSNJiUlU8yuMOma3X+1KPlcGAzWWvmqIqv23UHbTRKYZMBj3snsrYOXWKKVXC0NQS3aNMgK5hA9z2n3xRUb1e0ofdvxZ79FZK+x2AXxwXyUgyDOIoFCF9QSMzzmQNP0tXvnq/hAx1NWexDay3GnaKAS/rV54tHx/psLbL+GINBwC/jFwpYoeMhJ9ljeCQ0rvD3YJFsWoLhTwX0CV1zkPC5QsEcsuMqK4ntD1byMUfOF34rak0+DlqDwVKabytkE7cbH4XkUENf6CV3G3FumfepsN2DpmklEWhkiQHWz5b6DBwv1fKbg1++oYgZzZ2vYUogXDQ/Pj1YaUbfEXPY+XHFZou94q2YfQfpsX6hvErq9THjBxcL8C8238k7UoLVZ6nv4CMS6sKhMbMCFyxTE95y5kHGnvanyrLqxeOulSD93099brrURmobtOeHIpF45jDdzDF9mKWtDbw/1p7k8SbeaImOXdVSksFpnNULMvkrzRRWafhS7UiE9gpMjtpruq35ICLzYHtTq3ZDeVYxG8Fv8/JMd6OfhlXw79+xcrIheXr4lVuYnRajMbRVMXj1tF4Bo4uS+7SC2cU5S1u5nQk8CFG0ZIJCMZM786+bfCr3O2OYct5kjpinGWO2IWYj4Jr46i44MgiXhZH1Cenui4hpVsd7SukPNmACSupKRqKpNvN1vqiAPAYIkSsQqJK15PXYLVLlnxNfGafxg7oRWRtldiSCxgJOul7YqlMqHaPjExu1er/vYuOhTdx+auSPL96nJG6I3OEB9dFNC+K6jcLN6Mm3QPfjSnKtMEZQjbnB8TFghoW+fKw4CN6SZHQnXMldP7ks8BXnQ0Q4bMvJraTfQrxKgdTOHqWRnoZDC3oEBB9RrJw+/YMtkJUAVxgw4O9nVGJYeWRziuDCqDqw67yHl lxuWB673 sYlIcGmoYYQWzqMi21DNvyho/EZmVd8UVUGRvIon45pFCypfXrfZqxIqiPX20TyGaIimN X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, 14 Mar 2025 10:17:26 +0100 David Hildenbrand wrote: > > Hi David, > > Hi :) > > > > > I am currently working on adding memory hotplug-related functionality > > to the weighted interleave feature. While discussing this with Gregory, > > a question came up. If you happen to know the answer to the following, > > I would greatly appreciate your input. > > > > I have added the following logic to call add_weight_node when a node > > transitions to the N_MEMORY state to create a sysfs entry. Conversely, > > when all memory blocks of a node go offline (!N_MEMORY), > > I call sysfs_wi_node_release to remove the corresponding sysfs entry. > > > > As a spoiler: I don't like how we squeezed the status_change_nid / > status_change_nid_normal stuff into the memory notifier that works on a > single memory block -> single zone. But decoupling it is not as easy, > because we have this status_change_nid vs. status_change_nid_normal thing. > > For the basic "node going offline / node going online", a separate > notifier (or separate callbacks) would make at least your use case much > clearer. > > The whole "status_change_nid_normal" is only used by slab. I wonder if > we could get rid of it, and simply let slab check the relevant > zone->present pages when notified about onlining/offlining of a singe > memory block. > > Then, we would only have status_change_nid and could just convert that > to a NODE_MEM_ONLINE / NODE_MEM_OFFLINE notifier or sth like that. > > Hmmm, if I wouldn't be on PTO, I would prototype that real quick :) Hi David :) I completely agree with your perspective on this. Having separate callbacks for "node going offline/node going online" would certainly lead to clearer code. For now, I shall proceed with developing the code based on the current structure. I will also continue monitoring updates related to "node online/ offline" and plan on revising the code once those are integrated. Thank you for your valuable input on this matter. > > > +static int wi_node_notifier(struct notifier_block *nb, > > + unsigned long action, void *data) > > +{ > > + int err; > > + struct memory_notify *arg = data; > > + int nid = arg->status_change_nid; > > + > > + if (nid < 0) > > + goto notifier_end; > > + > > + switch(action) { > > + case MEM_ONLINE: > > + err = add_weight_node(nid, wi_kobj); > > + if (err) { > > + pr_err("failed to add sysfs [node%d]\n", nid); > > + kobject_put(wi_kobj); > > + return NOTIFY_BAD; > > + } > > + break; > > + case MEM_OFFLINE: > > + sysfs_wi_node_release(node_attrs[nid], wi_kobj); > > + break; > > + } > > + > > +notifier_end: > > + return NOTIFY_OK; > > +} > > > > One question I have is whether the MEM_OFFLINE action in the code > > below will be triggered when a node that consists of multiple memory > > blocks has only one of its memory blocks transitioning to the offline state. > > > > node_states_check_changes_offline() should be making sure that that is > the case. > > Only if offlining the current block will make the node (all zones) have > no present pages any more, then we'll set status_change_nid to >= 0. > Thank you for reviewing this matter. > > > + int nid = arg->status_change_nid; > > + > > + if (nid < 0) > > + goto notifier_end; > > > > Based on my analysis, wi_node_notifier should function as expected > > because arg->status_change_nid only holds a non-negative value > > under the following conditions: > > > > 1) !N_MEMORY -> N_MEMORY > > When the first memory block of a node transitions to the online state, > > it holds a non-negative value. > > In all other cases, it remains -1 (NUMA_NO_NODE). > > > > 2) N_MEMORY -> !N_MEMORY > > When all memory blocks of a node transition to the offline state, > > it holds a non-negative value. > > In all other cases, it remains -1 (NUMA_NO_NODE). > > > > I would truly appreciate it if you could confirm whether this analysis is correct. > > Below is a more detailed explanation of my findings. > > > > Yes, that's at least how it is supposed to work (-bugs, but I am not > aware of any) :) > Thank you once again for reviewing this matter. Your insightful feedback has been instrumental in crafting a more robust structure. Rakie > -- > Cheers, > > David / dhildenb >