From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B457C32772 for ; Tue, 23 Aug 2022 10:21:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 56EDE8D0002; Tue, 23 Aug 2022 06:21:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 51ED08D0001; Tue, 23 Aug 2022 06:21:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BF8C8D0002; Tue, 23 Aug 2022 06:21:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2E6148D0001 for ; Tue, 23 Aug 2022 06:21:39 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 05B9D1C5CA0 for ; Tue, 23 Aug 2022 10:21:39 +0000 (UTC) X-FDA: 79830465918.03.C01D365 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf24.hostedemail.com (Postfix) with ESMTP id 8BB0C18005C for ; Tue, 23 Aug 2022 10:21:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1661250097; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pQgsmCPlb+zxinxjCoRmOdMYgNutUmbY+EPSFFWc23Q=; b=QA6ZrHyuAYVemUXsHEylARWhd6GO3mLmFBER69hmf4GNpkGGoUwZsOjyCdTvvJUumMNM2D 3dvGgx32epcMG8nySdrzrWlCeLmbIaGNzvo5PjeCz69dwwT3kRYmTgaeSQFkupsnUyjbui vTcxdfv1KYOxs4u1PySEkdGyvnHLXQc= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-92-9lY_27XpOiSrQ4QelY44Xw-1; Tue, 23 Aug 2022 06:21:36 -0400 X-MC-Unique: 9lY_27XpOiSrQ4QelY44Xw-1 Received: by mail-wm1-f71.google.com with SMTP id j22-20020a05600c485600b003a5e4420552so10033820wmo.8 for ; Tue, 23 Aug 2022 03:21:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc; bh=pQgsmCPlb+zxinxjCoRmOdMYgNutUmbY+EPSFFWc23Q=; b=RsqJHm/ImI2w1JeFkzKr0hQ1sxZjsUzdbpJgMoUxW8vz/Np1Ueu3BruDcJO5Jl5nlR XjCcRnk4gcjg/vSnzJjhanbM2uV56yyjZu6t+zVXQdjKLbPadhGtOyIqZXDgmm3JZjrL qI8WU034RuMfGrl01Vc/Z6+rOkuGHLrsdRb7DNEtJr+eFhYQCKDAQ9LuwnhTx666tVZ/ /ENWL+jpiNgKzl0VUPH6ot0TvmwlhP2BXvPoCAX11jcbCyK7h/BVzNZKqemVFre95vco cen8wXI3Upx0wA9CR5xpNceSseYO9TpD94V/6T+0qZrlAq6tESz0TgdcUrX/nfG7MlZ1 pzGw== X-Gm-Message-State: ACgBeo0YVtjk4/jiBB1L27EXQeHjjPysLFMY7ZPeuhoE1lnv9n91SQZr kFl366QhhbiLCSePfrqnQAwVcXoRb8uGhEHWK9sk0cDT4FyY4NUbKP16V9TP3V5mC01rlErsBZF 26yXT3yXSpVE= X-Received: by 2002:a1c:7c18:0:b0:3a5:aaae:d203 with SMTP id x24-20020a1c7c18000000b003a5aaaed203mr1753539wmc.2.1661250095732; Tue, 23 Aug 2022 03:21:35 -0700 (PDT) X-Google-Smtp-Source: AA6agR5jY1+l93jubKbrfBeLVufXcUK74eww7Bkgb5AljLD0KmIpk2K/goe1cipCs3utuQsv7ZHgIg== X-Received: by 2002:a1c:7c18:0:b0:3a5:aaae:d203 with SMTP id x24-20020a1c7c18000000b003a5aaaed203mr1753517wmc.2.1661250095401; Tue, 23 Aug 2022 03:21:35 -0700 (PDT) Received: from ?IPV6:2003:cb:c70b:1600:c48b:1fab:a330:5182? (p200300cbc70b1600c48b1faba3305182.dip0.t-ipconnect.de. [2003:cb:c70b:1600:c48b:1fab:a330:5182]) by smtp.gmail.com with ESMTPSA id t185-20020a1c46c2000000b003a673055e68sm5361666wma.0.2022.08.23.03.21.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 23 Aug 2022 03:21:34 -0700 (PDT) Message-ID: <60933ffc-b850-976c-78a0-0ee6e0ea9ef0@redhat.com> Date: Tue, 23 Aug 2022 12:21:34 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 To: Muchun Song , gregkh@linuxfoundation.org, rafael@kernel.org, mike.kravetz@oracle.com, akpm@linux-foundation.org, osalvador@suse.de, andi@firstfloor.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, muchun.song@linux.dev References: <20220819080029.12241-1-songmuchun@bytedance.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH RESEND] mm: hugetlb: simplify per-node sysfs creation and removal In-Reply-To: <20220819080029.12241-1-songmuchun@bytedance.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661250098; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pQgsmCPlb+zxinxjCoRmOdMYgNutUmbY+EPSFFWc23Q=; b=Ycpl+3amTIU9JMGqFe91S+CfYmVpR0iVAcl8WwsgPqhwIdOM+wikJdP1YkrXnJ7stOG6Yz 9KE1WZDkrIttJiaagFaDm9hvhjmdjFxr5Mf5zCFDg6y6TQPv6O/ximihoBb4bhxgEE5gcQ bmtcCHqgCImVdhD6KpVT84QD8Nxb4Kw= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QA6ZrHyu; spf=pass (imf24.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661250098; a=rsa-sha256; cv=none; b=bdGrQWJJ3J+BVCZIUEeqjU4o4BlLfof3BPiuF8psxldXntjrx+JJsdseeGtx+SjUT5kduK hPEXmZJNWR6Lp/D/Lr2MuJeTewX71uXG8/rpQxlfYWhNEaPX/AAqaPYYwXGFAk9WO5gocf 0SBVATyMa782fwIYgm/o4jtHgbLVzJQ= Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QA6ZrHyu; spf=pass (imf24.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspam-User: X-Stat-Signature: zbfxwxxx4a1d17xjysxyu1j7bc8jtttw X-Rspamd-Queue-Id: 8BB0C18005C X-Rspamd-Server: rspam10 X-HE-Tag: 1661250098-534101 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 19.08.22 10:00, Muchun Song wrote: > The following commit offload per-node sysfs creation and removal to a kworker and > did not say why it is needed. And it also said "I don't know that this is > absolutely required". It seems like the author was not sure as well. Since it > only complicates the code, this patch will revert the changes to simplify the code. > > 39da08cb074c ("hugetlb: offload per node attribute registrations") > > We could use memory hotplug notifier to do per-node sysfs creation and removal > instead of inserting those operations to node registration and unregistration. > Then, it can reduce the code coupling between node.c and hugetlb.c. Also, it can > simplify the code. > > Signed-off-by: Muchun Song [...] > @@ -683,7 +626,6 @@ static int register_node(struct node *node, int num) > void unregister_node(struct node *node) > { > compaction_unregister_node(node); > - hugetlb_unregister_node(node); /* no-op, if memoryless node */ > node_remove_accesses(node); > node_remove_caches(node); > device_unregister(&node->dev); > @@ -905,74 +847,8 @@ void register_memory_blocks_under_node(int nid, unsigned long start_pfn, > (void *)&nid, func); > return; > } [...] > /* > * Create all node devices, which will properly link the node > * to applicable memory block devices and already created cpu devices. > diff --git a/include/linux/node.h b/include/linux/node.h > index 40d641a8bfb0..ea817b507f54 100644 > --- a/include/linux/node.h > +++ b/include/linux/node.h > @@ -2,15 +2,15 @@ > /* > * include/linux/node.h - generic node definition > * > - * This is mainly for topological representation. We define the > - * basic 'struct node' here, which can be embedded in per-arch > + * This is mainly for topological representation. We define the > + * basic 'struct node' here, which can be embedded in per-arch > * definitions of processors. > * > * Basic handling of the devices is done in drivers/base/node.c > - * and system devices are handled in drivers/base/sys.c. > + * and system devices are handled in drivers/base/sys.c. > * > * Nodes are exported via driverfs in the class/node/devices/ > - * directory. > + * directory. Unrelated changes. > */ > #ifndef _LINUX_NODE_H_ > #define _LINUX_NODE_H_ > @@ -18,7 +18,6 @@ > #include > #include > #include > -#include > > /** > * struct node_hmem_attrs - heterogeneous memory performance attributes > @@ -84,10 +83,6 @@ static inline void node_set_perf_attrs(unsigned int nid, > struct node { > struct device dev; > struct list_head access_list; > - > -#if defined(CONFIG_MEMORY_HOTPLUG) && defined(CONFIG_HUGETLBFS) > - struct work_struct node_work; > -#endif > #ifdef CONFIG_HMEM_REPORTING > struct list_head cache_attrs; > struct device *cache_dev; > @@ -96,7 +91,6 @@ struct node { > > struct memory_block; > extern struct node *node_devices[]; > -typedef void (*node_registration_func_t)(struct node *); > > #if defined(CONFIG_MEMORY_HOTPLUG) && defined(CONFIG_NUMA) > void register_memory_blocks_under_node(int nid, unsigned long start_pfn, > @@ -144,11 +138,6 @@ extern void unregister_memory_block_under_nodes(struct memory_block *mem_blk); > extern int register_memory_node_under_compute_node(unsigned int mem_nid, > unsigned int cpu_nid, > unsigned access); > - > -#ifdef CONFIG_HUGETLBFS > -extern void register_hugetlbfs_with_node(node_registration_func_t doregister, > - node_registration_func_t unregister); > -#endif > #else > static inline void node_dev_init(void) > { > @@ -176,11 +165,6 @@ static inline int unregister_cpu_under_node(unsigned int cpu, unsigned int nid) > static inline void unregister_memory_block_under_nodes(struct memory_block *mem_blk) > { > } > - > -static inline void register_hugetlbfs_with_node(node_registration_func_t reg, > - node_registration_func_t unreg) > -{ > -} > #endif > > #define to_node(device) container_of(device, struct node, dev) > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index 536a52c29035..9a72499486c1 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -33,6 +33,7 @@ > #include > #include > #include > +#include > > #include > #include > @@ -3967,19 +3968,19 @@ static void hugetlb_unregister_node(struct node *node) > * Register hstate attributes for a single node device. > * No-op if attributes already registered. > */ > -static void hugetlb_register_node(struct node *node) > +static int hugetlb_register_node(struct node *node) > { > struct hstate *h; > struct node_hstate *nhs = &node_hstates[node->dev.id]; > int err; > > if (nhs->hugepages_kobj) > - return; /* already allocated */ > + return 0; /* already allocated */ > > nhs->hugepages_kobj = kobject_create_and_add("hugepages", > &node->dev.kobj); > if (!nhs->hugepages_kobj) > - return; > + return -ENOMEM; > > for_each_hstate(h) { > err = hugetlb_sysfs_add_hstate(h, nhs->hugepages_kobj, > @@ -3989,9 +3990,28 @@ static void hugetlb_register_node(struct node *node) > pr_err("HugeTLB: Unable to add hstate %s for node %d\n", > h->name, node->dev.id); > hugetlb_unregister_node(node); > - break; > + return -ENOMEM; > } > } > + return 0; > +} > + > +static int __meminit hugetlb_memory_callback(struct notifier_block *self, > + unsigned long action, void *arg) > +{ > + int ret = 0; > + struct memory_notify *mnb = arg; > + int nid = mnb->status_change_nid; > + > + if (nid == NUMA_NO_NODE) > + return NOTIFY_DONE; > + > + if (action == MEM_GOING_ONLINE) > + ret = hugetlb_register_node(node_devices[nid]); > + else if (action == MEM_CANCEL_ONLINE || action == MEM_OFFLINE) > + hugetlb_unregister_node(node_devices[nid]); > + > + return notifier_from_errno(ret); > } > > /* > @@ -4003,18 +4023,11 @@ static void __init hugetlb_register_all_nodes(void) > { > int nid; > > - for_each_node_state(nid, N_MEMORY) { > - struct node *node = node_devices[nid]; > - if (node->dev.id == nid) > - hugetlb_register_node(node); > - } > - > - /* > - * Let the node device driver know we're here so it can > - * [un]register hstate attributes on node hotplug. > - */ > - register_hugetlbfs_with_node(hugetlb_register_node, > - hugetlb_unregister_node); > + get_online_mems(); > + hotplug_memory_notifier(hugetlb_memory_callback, 0); > + for_each_node_state(nid, N_MEMORY) > + hugetlb_register_node(node_devices[nid]); > + put_online_mems(); > } > #else /* !CONFIG_NUMA */ Do we really *need* the memory hotplug notifier and the added complexity due for handling memory-less nodes? Why can't we simply register/unregister sysfs entries in register_node/unregister_node and call it a day? TBH, we should just have sysfs entries for memory-less nodes and not care about such (corner) cases. -- Thanks, David / dhildenb