From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FBF1C369A4 for ; Tue, 8 Apr 2025 13:52:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A05496B000D; Tue, 8 Apr 2025 09:52:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B3D86B0010; Tue, 8 Apr 2025 09:52:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 855196B0011; Tue, 8 Apr 2025 09:52:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 64B256B000D for ; Tue, 8 Apr 2025 09:52:06 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 991E2ADFE7 for ; Tue, 8 Apr 2025 13:52:07 +0000 (UTC) X-FDA: 83311015494.09.C41D8B9 Received: from mail-yb1-f169.google.com (mail-yb1-f169.google.com [209.85.219.169]) by imf20.hostedemail.com (Postfix) with ESMTP id B972E1C0004 for ; Tue, 8 Apr 2025 13:52:05 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=C66Z30+J; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf20.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.219.169 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744120325; a=rsa-sha256; cv=none; b=exELGNbURlSL49o9UnCc8do9m1vtpGvKDihfxWNevHKNS4DxEL6gOj8WjZVrbPwAAevgv5 Roc5nR4ACS9qznpfqHvoFKCMzGCxULp5+d0uDoWdjgvCr2gaO/6Ei4jzDkJIDKfKSSz2o9 yut+J8tH5DO1/ZQZPYraHzyPP/hmYtQ= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=C66Z30+J; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf20.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.219.169 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744120325; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iggtIaOdKwCaj0c8UnJKvVOsn69XQmtF4Ox/sMEh3ho=; b=6QMRO1YPB1xow7pY+cLHXhxlfHlSkf6WaETIn5MMwcu9WKTfDg6mzTu9MWw86zbhQJOUG4 mqaIseZgGYlCgGa34mkja/lzYwOumn8adk0cBemFc4opNCCInyTmWL//TB9PLHbgGKLnMr ifmMObglHr4R9oNImYRRKbABYrt1ct8= Received: by mail-yb1-f169.google.com with SMTP id 3f1490d57ef6-e5372a2fbddso4789893276.3 for ; Tue, 08 Apr 2025 06:52:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1744120325; x=1744725125; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iggtIaOdKwCaj0c8UnJKvVOsn69XQmtF4Ox/sMEh3ho=; b=C66Z30+Jx2UrNWpeFDWP094tYBrBBkHS4b2fpuwMRggJTXiVCRTbxGoMApNRQiyfOP MkN/tSCe9NtjEyffegFjVzBHRyYrRT1Ubd073W71Zrt7G+4Ki3NNp/HHY8plN+Fgz9yf uufZJRXQJci+JfPkbxrDrB5g/39pGMnJyKbzrlmaULteN39iNhqcLS7FzjwAkkvHbawV 0Xub9fOQOXy3wGlq69HeFr/+d2AfCRM/o0cquRw9X5RyPkDT21HhareSZyE6JYfyqS9R LiBpD/e4PS81H81M0GeXob8ytfybEx6M22lQFYAMByyohPWv5XTRMCd67Y9BKeq1/t4B wN+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744120325; x=1744725125; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iggtIaOdKwCaj0c8UnJKvVOsn69XQmtF4Ox/sMEh3ho=; b=dmZG6z00zYZcmRrL54mE9ehEDcfjwuIqcAu6wAXb9KdNVvV9DZr7JA1etTO1W7jA43 9es5O12k1GD1CaYypEGozTzNEledJvwEhc1kx+wHOLBd0Hdm3piBIHMsUte8z2c1dHxc owX0kyltpYfgu+U7LwnK86ttBk9M47MORSlWnqA4vqm0wuwVDVQ0Xs2wzxjfIV5GcsF/ 5YgWAxWxt1gCKVQuHqgL36rajmDzRC9yvoiU/vzJ+tLrx+DNOx9++769aLqA53LqazVr fzm1B/Rpi4MlZTjxp4tnnqJXLBNg70id7XvGdNFup0DfX9J69zqYAIw4SnJ2H5+Nuobi 8mWA== X-Forwarded-Encrypted: i=1; AJvYcCXPpMfVCidrQlOEBRw8F/xHtR+J5Z1EcghtqFxmi/L8BSbf9LgBK/4YP2r8C3VQt3rCx6KXyFcCfg==@kvack.org X-Gm-Message-State: AOJu0YxyxOC5Lw0PBGYGSCK/reabA9QDGiCwBWY4FMZ9jkk0jNG9PBDb 1sWzbZOLrEQJzpmBdQaJzj92K5JkR33TEwYh0B3NC27nICOJssnZ X-Gm-Gg: ASbGnctQEuCitYmx7ycpDp4Epht2+5lDtxCBJq7Jz5oboi8Lmwo3AXebT/zHusL7tCP wfx7BItkcIrYDayOfSLUzOkuRdetXBmc4uF1V45odBSXuXi0kolb0fQv3NfGOgWFPVjD12mocpR uJvTjbvJC6OGhV5rM3svaUaaux0nbAMjzqwP6lcynd6s5+sKsNTtT8gKEpLf53QLSe9LpgsqP80 hu5cqVGj+jS2laqrqAzGUhbWJk9ejzfAw0YwsIq/c5UlPDg3F6o+lrAXaH10aFiJ9T2VwuDtA2Z rx0IQ8IW0IKRqUW/TVv8rZtBLreef3g0pv5PsmoB X-Google-Smtp-Source: AGHT+IEEOIFV5JW5y4gJNyzsPGTfIpKDTk2nVR3Zxvi/DZjydcsNnJirWV6ukyAOhPkuSSu+3YtRSQ== X-Received: by 2002:a05:6902:2e0a:b0:e6d:f7ea:c722 with SMTP id 3f1490d57ef6-e6e1c286d11mr28221779276.17.1744120324656; Tue, 08 Apr 2025 06:52:04 -0700 (PDT) Received: from localhost ([2a03:2880:25ff::]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e6e0ca96a08sm2853800276.45.2025.04.08.06.52.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Apr 2025 06:52:04 -0700 (PDT) From: Joshua Hahn To: Rakie Kim Cc: akpm@linux-foundation.org, gourry@gourry.net, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-cxl@vger.kernel.org, joshua.hahnjy@gmail.com, dan.j.williams@intel.com, ying.huang@linux.alibaba.com, david@redhat.com, Jonathan.Cameron@huawei.com, osalvador@suse.de, kernel_team@skhynix.com, honggyu.kim@sk.com, yunjeong.mun@sk.com Subject: Re: [PATCH v7 3/3] mm/mempolicy: Support memory hotplug in weighted interleave Date: Tue, 8 Apr 2025 06:52:01 -0700 Message-ID: <20250408135202.3001047-1-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250408073243.488-4-rakie.kim@sk.com> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: B972E1C0004 X-Stat-Signature: atf4fjxxawmfpxayc6wgto1hsnimiqu8 X-HE-Tag: 1744120325-275310 X-HE-Meta: U2FsdGVkX1+etxR0WKPJLNMOvvsbgC9eSGz/4gKq4O+b0+CYZWfN0bA/sQ71ywNhvr/8JknRLebOcvrc+Kz7+ooOHWcNMItEjrYNDi9Y4kXK2S9WANPmwBpmMUIeycR+/6vyQo9xNO6LCGbQ36/A0UY+TcM9QWOJ01BoWmypV1NICE/1CSfpU066Ux4HEnn/6dEa97V2mLOiQoAZ0SUyKYpG7n7AbNTzWRbwjPiK1KVFdWP/Mk3/BybmgzXzRFZyc1AIMS8wt+S0JpEJ1ZMFWDgQK7nb5NAgUSStj2LuZIh3e5JIDxpBQwYf57a+aKbakdE1dkHsXYj712s6ymvNVSC9fGaeLXOOf6zR4kkfyTygz8zKKqJUM1ek0861yvg7cT4rxvh0+FaVUlJuGjRM0JIcYY5DpSmimxa+k5OXQ84SjkMyXJD9juCxiSZfNcxTTY9zcxe/EppdHnSDAwjNbM3+KL3RtzwlzO+OLRGI+ocNjtU8RPWP9yrPPHWMFU9UyxwNnb09Ig3QXCU8dIcoUronuu11BmyNTV+4brAC12n3/uDQ8lMgzDNo+GJY7FNVGTToqVb/9fu1LgVzrb8fUVvuUINhHh6ZTK+s8FincVHnT1e/pXNVJNvlghMLStJwUFTrZ2PpKxqtQDIDrvsRspz+U+a78cZcMX3LFB/8dirDVmqtzMebPWryqchRMSuAx+LPs+weliM1vrHiAWk9VeLEYsAfA/JtNTXRFc9XraHDB1fNK7YJPvwTJkXoioUbAVxHzrh2MWAmb+zNP53GJYrbt80vRiHsHutNXg+udaTneTO3ElqdZwobeNfd/LMSPjnk/++uxxj0xai2GOX+Uhz7qj8NnCJ/I7M2OZQc/K7Fgs3HuidbKye63wvy+InhnfV5vf2XZF9VHJ5AepBip8A1gJcZZP2m41jXpNE7hJLCyP4n/tgD1qDXk0wJuKwxMlnXYFAvFuU/uTqCp0o LlgWwYom qU0KYSy8xfQjX4zqtKHv7SonuZKePzZUVt/iqsxuV79s4R/eqhwUk/oVKZEmvV/tTRQt4TjDq6AoFgMX8WJDfWuVUKcaUKm1OKkA2NONwBxH9KqOH7SCJ3gxs4Jai27wgcEOpjD9sf/hiUJ1Z+bYnRJwHasrIb9eaGDirLn97XYNmN+Ou50E32jiFrH227ZFm08ul1/OKfZ2jY9wnU/K9AZHRmWMed0+9kSVGuqXjdRnrbNSDfP/KiHAv+KBvdAl0gcgZt7dwcCB0Thk46zAQh9h+gr29wmVO5v74K1sxCNqvxvfb55QEuNrRmRACLzmDa3DoOUu1KxQ0OBxtFgEi9Ki2R05qurrTvTHM2A9EULtl7cSaBnkwiq9dWx6NOyZDJPhlyt0KCZImfuWvZTfhm+L6dZzP558JrN09Gnfigo17Mscf1yepjwHyZIe0SR21q/5eGmzcK23adT/3OE/QdwlMcN2RFDIXG+3Pf+S3YqN1TfuYhDhCd9zVk7OKNMOYqNrXW7CxZCCr5HmMP+TRqnwpIQxUha6UX4sx7wgl0fLqotFWA+pfhyW7RuUMOc4szeudOszkzLYPfCgJ84hcCRpMMc0rZM7Ctj9ucxXSVPCN7As= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, 8 Apr 2025 16:32:42 +0900 Rakie Kim wrote: Hi Rakie, Looks good to me as well : -) Thank you for working on this! Reviewed-by: Joshua Hahn > The weighted interleave policy distributes page allocations across multiple > NUMA nodes based on their performance weight, thereby improving memory > bandwidth utilization. The weight values for each node are configured > through sysfs. > > Previously, sysfs entries for configuring weighted interleave were created > for all possible nodes (N_POSSIBLE) at initialization, including nodes that > might not have memory. However, not all nodes in N_POSSIBLE are usable at > runtime, as some may remain memoryless or offline. > This led to sysfs entries being created for unusable nodes, causing > potential misconfiguration issues. > > To address this issue, this patch modifies the sysfs creation logic to: > 1) Limit sysfs entries to nodes that are online and have memory, avoiding > the creation of sysfs entries for nodes that cannot be used. > 2) Support memory hotplug by dynamically adding and removing sysfs entries > based on whether a node transitions into or out of the N_MEMORY state. > > Additionally, the patch ensures that sysfs attributes are properly managed > when nodes go offline, preventing stale or redundant entries from persisting > in the system. > > By making these changes, the weighted interleave policy now manages its > sysfs entries more efficiently, ensuring that only relevant nodes are > considered for interleaving, and dynamically adapting to memory hotplug > events. > > Signed-off-by: Rakie Kim > Signed-off-by: Honggyu Kim > Signed-off-by: Yunjeong Mun > Reviewed-by: Oscar Salvador > --- > mm/mempolicy.c | 106 ++++++++++++++++++++++++++++++++++++++----------- > 1 file changed, 83 insertions(+), 23 deletions(-) > > diff --git a/mm/mempolicy.c b/mm/mempolicy.c > index 988575f29c53..9aa884107f4c 100644 > --- a/mm/mempolicy.c > +++ b/mm/mempolicy.c > @@ -113,6 +113,7 @@ > #include > #include > #include > +#include > > #include "internal.h" > > @@ -3421,6 +3422,7 @@ struct iw_node_attr { > > struct sysfs_wi_group { > struct kobject wi_kobj; > + struct mutex kobj_lock; > struct iw_node_attr *nattrs[]; > }; > > @@ -3470,13 +3472,24 @@ static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr, > > static void sysfs_wi_node_delete(int nid) > { > - if (!wi_group->nattrs[nid]) > + struct iw_node_attr *attr; > + > + if (nid < 0 || nid >= nr_node_ids) > + return; > + > + mutex_lock(&wi_group->kobj_lock); > + attr = wi_group->nattrs[nid]; > + if (!attr) { > + mutex_unlock(&wi_group->kobj_lock); > return; > + } > + > + wi_group->nattrs[nid] = NULL; > + mutex_unlock(&wi_group->kobj_lock); > > - sysfs_remove_file(&wi_group->wi_kobj, > - &wi_group->nattrs[nid]->kobj_attr.attr); > - kfree(wi_group->nattrs[nid]->kobj_attr.attr.name); > - kfree(wi_group->nattrs[nid]); > + sysfs_remove_file(&wi_group->wi_kobj, &attr->kobj_attr.attr); > + kfree(attr->kobj_attr.attr.name); > + kfree(attr); > } > > static void sysfs_wi_release(struct kobject *wi_kobj) > @@ -3495,35 +3508,77 @@ static const struct kobj_type wi_ktype = { > > static int sysfs_wi_node_add(int nid) > { > - struct iw_node_attr *node_attr; > + int ret = 0; > char *name; > + struct iw_node_attr *new_attr = NULL; > > - node_attr = kzalloc(sizeof(*node_attr), GFP_KERNEL); > - if (!node_attr) > + if (nid < 0 || nid >= nr_node_ids) { > + pr_err("Invalid node id: %d\n", nid); > + return -EINVAL; > + } > + > + new_attr = kzalloc(sizeof(struct iw_node_attr), GFP_KERNEL); > + if (!new_attr) > return -ENOMEM; > > name = kasprintf(GFP_KERNEL, "node%d", nid); > if (!name) { > - kfree(node_attr); > + kfree(new_attr); > return -ENOMEM; > } > > - sysfs_attr_init(&node_attr->kobj_attr.attr); > - node_attr->kobj_attr.attr.name = name; > - node_attr->kobj_attr.attr.mode = 0644; > - node_attr->kobj_attr.show = node_show; > - node_attr->kobj_attr.store = node_store; > - node_attr->nid = nid; > + mutex_lock(&wi_group->kobj_lock); > + if (wi_group->nattrs[nid]) { > + mutex_unlock(&wi_group->kobj_lock); > + pr_info("Node [%d] already exists\n", nid); > + kfree(new_attr); > + kfree(name); > + return 0; > + } > + wi_group->nattrs[nid] = new_attr; > > - if (sysfs_create_file(&wi_group->wi_kobj, &node_attr->kobj_attr.attr)) { > - kfree(node_attr->kobj_attr.attr.name); > - kfree(node_attr); > - pr_err("failed to add attribute to weighted_interleave\n"); > - return -ENOMEM; > + sysfs_attr_init(&wi_group->nattrs[nid]->kobj_attr.attr); > + wi_group->nattrs[nid]->kobj_attr.attr.name = name; > + wi_group->nattrs[nid]->kobj_attr.attr.mode = 0644; > + wi_group->nattrs[nid]->kobj_attr.show = node_show; > + wi_group->nattrs[nid]->kobj_attr.store = node_store; > + wi_group->nattrs[nid]->nid = nid; > + > + ret = sysfs_create_file(&wi_group->wi_kobj, > + &wi_group->nattrs[nid]->kobj_attr.attr); > + if (ret) { > + kfree(wi_group->nattrs[nid]->kobj_attr.attr.name); > + kfree(wi_group->nattrs[nid]); > + wi_group->nattrs[nid] = NULL; > + pr_err("Failed to add attribute to weighted_interleave: %d\n", ret); > } > + mutex_unlock(&wi_group->kobj_lock); > > - wi_group->nattrs[nid] = node_attr; > - return 0; > + return ret; > +} > + > +static int wi_node_notifier(struct notifier_block *nb, > + unsigned long action, void *data) > +{ > + int err; > + struct memory_notify *arg = data; > + int nid = arg->status_change_nid; > + > + if (nid < 0) > + return NOTIFY_OK; > + > + switch(action) { > + case MEM_ONLINE: > + err = sysfs_wi_node_add(nid); > + if (err) > + pr_err("failed to add sysfs [node%d]\n", nid); > + break; > + case MEM_OFFLINE: > + sysfs_wi_node_delete(nid); > + break; > + } > + > + return NOTIFY_OK; > } > > static int __init add_weighted_interleave_group(struct kobject *mempolicy_kobj) > @@ -3534,13 +3589,17 @@ static int __init add_weighted_interleave_group(struct kobject *mempolicy_kobj) > GFP_KERNEL); > if (!wi_group) > return -ENOMEM; > + mutex_init(&wi_group->kobj_lock); > > err = kobject_init_and_add(&wi_group->wi_kobj, &wi_ktype, mempolicy_kobj, > "weighted_interleave"); > if (err) > goto err_put_kobj; > > - for_each_node_state(nid, N_POSSIBLE) { > + for_each_online_node(nid) { > + if (!node_state(nid, N_MEMORY)) > + continue; > + > err = sysfs_wi_node_add(nid); > if (err) { > pr_err("failed to add sysfs [node%d]\n", nid); > @@ -3548,6 +3607,7 @@ static int __init add_weighted_interleave_group(struct kobject *mempolicy_kobj) > } > } > > + hotplug_memory_notifier(wi_node_notifier, DEFAULT_CALLBACK_PRI); > return 0; > > err_del_kobj: > -- > 2.34.1 Sent using hkml (https://github.com/sjp38/hackermail)