From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACAEEC36010 for ; Fri, 4 Apr 2025 08:47:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 71FDD6B0007; Fri, 4 Apr 2025 04:47:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6CF656B0008; Fri, 4 Apr 2025 04:47:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56EB96B000A; Fri, 4 Apr 2025 04:47:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 357E96B0007 for ; Fri, 4 Apr 2025 04:47:57 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E9A911423A8 for ; Fri, 4 Apr 2025 08:47:56 +0000 (UTC) X-FDA: 83295733752.11.E31524D Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf08.hostedemail.com (Postfix) with ESMTP id A3F9F160005 for ; Fri, 4 Apr 2025 08:47:54 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=KnrrVpi7; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=ZR2y1HsE; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=KnrrVpi7; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=ZR2y1HsE; dmarc=none; spf=pass (imf08.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743756474; a=rsa-sha256; cv=none; b=phCKrsdcyiDh8GvK7yunVNHwPfmyK21Rdio/6bECH9cRIEcWrg8UR+YK228HyzmDYSlzkD oQZFZM1mMPm1nZ43jqR9F4VmUf/FTqmBCrjXVnw1SeqYi45qubKOyLkqmksFVswhWpWM9S J60ItR6haLBJ7x8Y6AeXCmj3JXX7e1g= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=KnrrVpi7; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=ZR2y1HsE; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=KnrrVpi7; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=ZR2y1HsE; dmarc=none; spf=pass (imf08.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743756474; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EIE8nJnyuWIo9mF7+tfr0Mdf07sXKWR7IdaYTIIfhRw=; b=pncNzv3xgV3sWLt9iILGaEeRZpbY1SFSyqBL1zm0/EWwXATXZJV2nJ+oBMNIoF6dVvrE5d ujhTJvXAecn2ZrlA2ZAAA9tUv78zfu1/UbIYgTwaY/m6nxgT0Ck1xR3p3qbIn5NfjlRVQU i0tO++UdCYhOI5ckM2T6V5IV/hgOTyg= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B88C821184; Fri, 4 Apr 2025 08:47:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1743756472; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EIE8nJnyuWIo9mF7+tfr0Mdf07sXKWR7IdaYTIIfhRw=; b=KnrrVpi7yhvcOZBUNlBE3GrgW3poaQc1opRACCSfcN1XugNt2gbhRlNqq/a5wUrJp3jTlR W1pNwSZzQzg4dPR0wO4qdj+cWEOzrGhM25+WKsZ/m/TLrZRA30ll4d8ZBocROKD5iBg/3C X8aFPEP4pQ/UlBzkgRTk+OJ1c6i+lE0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1743756472; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EIE8nJnyuWIo9mF7+tfr0Mdf07sXKWR7IdaYTIIfhRw=; b=ZR2y1HsEoIoH3jyowMbZWArJeK+EURcoDbKUBd/ci7y/B9kH/rEqvwBK8KyioxXNO1D5Uh b0iWfz8ssWD1tEAA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1743756472; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EIE8nJnyuWIo9mF7+tfr0Mdf07sXKWR7IdaYTIIfhRw=; b=KnrrVpi7yhvcOZBUNlBE3GrgW3poaQc1opRACCSfcN1XugNt2gbhRlNqq/a5wUrJp3jTlR W1pNwSZzQzg4dPR0wO4qdj+cWEOzrGhM25+WKsZ/m/TLrZRA30ll4d8ZBocROKD5iBg/3C X8aFPEP4pQ/UlBzkgRTk+OJ1c6i+lE0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1743756472; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EIE8nJnyuWIo9mF7+tfr0Mdf07sXKWR7IdaYTIIfhRw=; b=ZR2y1HsEoIoH3jyowMbZWArJeK+EURcoDbKUBd/ci7y/B9kH/rEqvwBK8KyioxXNO1D5Uh b0iWfz8ssWD1tEAA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 9928B13691; Fri, 4 Apr 2025 08:47:52 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id YaqbIric72eCfgAAD6G6ig (envelope-from ); Fri, 04 Apr 2025 08:47:52 +0000 Message-ID: <21b36caa-68d8-4ae4-a290-cff2e7e3411f@suse.cz> Date: Fri, 4 Apr 2025 10:47:52 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 0/2] Implement numa node notifier To: David Hildenbrand , Oscar Salvador Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Hyeonggon Yoo <42.hyeyoo@gmail.com>, mkoutny@suse.com, Dan Williams , Jonathan Cameron References: <20250401092716.537512-1-osalvador@suse.de> <78c976ba-1eaf-47b7-a310-b8a99a3882e2@suse.cz> Content-Language: en-US From: Vlastimil Babka In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Action: no action X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A3F9F160005 X-Stat-Signature: jbg7gr8u94rfg9fgwg8afdb5jtfinsio X-Rspam-User: X-HE-Tag: 1743756474-32866 X-HE-Meta: U2FsdGVkX18u1lnG2rUnMRZCDQJieaOpKq2qXOBhQIuLJzjltNGMqGLSU3TIFXXtX3M1uaedUyNkaTSIel9HV1oVNFQEBAGxgJSGJMpDRqgpRlGyDVMSXmw6DeX1y25APhT2YHiHkL9L8SSvj5SQyR0ympdrs9ZJXehylgwoT43uvfuSDG5OW41HWTC6bUrt2se3neF0BG2PjjGo3kzPtix5XK69ofzGnnH9/S2y3aA1kGJWtVaXNOxxzySSxmGa6/zkdIxsdt8gA3rVdhDK7VEX0imcQMSurNQmtuXSgkzxKuEFYkOQgoouWaXRsaEb+cF5tRn6sC8zEMylt1cZXVVvHObmALhpbHQmjQct3OLy6tkM4QazIgQby0FjeDnnFjVwXPiVZvzGp1y5OSbko/Ayac8XFw/TTcwrgqlLMEECQn9DBD0eR3vbo0n12+51GUyQA+3MAktRVLx99Pou0SLA8OYjtMTpFg1eDORuqFWwvIVGFLmbzndmq6h2smNShM2Lg7l2q3b5vuePXlWR+Y9f3IPjdtnidv0Qef/tUJQFriAas9tA9W9QqIAtojhlLQ6jfhfF0e777QJdlex4czN/dlg00MNmVNbsgyA+kOcA9HKWjxhY0QjCOMuE3u4z86V8aJpzjAU+XX3xiK0dEUMsCIHqzDNWVVf39Lgruc9jakOs6ZqCrA9zrTObThE7WXwgIWRSTOGcoJHk9mtzbBIdHpCxpyOv73XhY5y23GhVxnqAuCvmZ9BAXX42yXnQOylmjyCnqpR8OthHj/pxXDGplwhJgobCr+cGC8Zx0YKg6392AgjlqrQpivlzRUXJJVmWCAQzzR8Z1nWgMb3dQv9XiJpySFe0iC/OBwWaIpbMpnnkF6fiXaypF7jSqmJHLh7bbJAyjhZZ78fl9TI0UgnplNvnl/RbugzlnqIT8MlU2+kQ6QqUU2g/E7ca9qMszR1X3tDaWl2X8epo1mJ Vg9xouu8 YSG7fKYRfEREUPUKkOlKaRxZ3E0pfxRv4ym/GWaEVfIRVqe/kVG/jKBT4FYNXC1u/UQwxgwKrgCPQkYwRvORKo+mG3etDJzv0zVLoyWfL1Xx/GOTbUFrLfpx7FOTi5vGVyYUzDykPb4vBN2zqYQdNrywp+u7CfZ5W8m8iA2qLjDB4noZd4SiYNZkCx3bu/zlzmvikvNkM7Q3tEk4Oe6h4RBAXI6v4TQHRV8JXj4HPjua2eNJP6yy2EhiYu9Befpndk939iU5BEBEK9EkJErd1gfiboe8MFIi8XNTe5T/xy13GNAGr9yoWda/omRsUDXhHkB/yRuCbEv/OUUx8XCKQBVl3Qew+j/J9iigwiBg2ULiDxGlaUcu3Y8J9uO96nQm9y1ia X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 4/3/25 15:08, David Hildenbrand wrote: > On 03.04.25 15:02, David Hildenbrand wrote: >> On 02.04.25 19:03, Oscar Salvador wrote: >>> On Wed, Apr 02, 2025 at 06:06:51PM +0200, Vlastimil Babka wrote: >>>> What if we had two chains: >>>> >>>> register_node_notifier() >>>> register_node_normal_notifier() >>>> >>>> I think they could have shared the state #defines and struct node_notify >>>> would have just one nid and be always >= 0. >>>> >>>> Or would it add too much extra boilerplate and only slab cares? >>> >>> We could indeed go on that direction to try to decouple >>> status_change_nid from status_change_nid_normal. >>> >>> Although as you said, slub is the only user of status_change_nid_normal >>> for the time beign, so I am not sure of adding a second chain for only >>> one user. >>> >>> Might look cleaner though, and the advantatge is that slub would not get >>> notified for nodes adquiring only ZONE_MOVABLE. >>> >>> Let us see what David thinks about it. >> >> I'd hope we'd be able to get rid of the _normal stuff completely, it's seems >> way to specialized. >> >> We added that in >> >> commit b9d5ab2562eceeada5e4837a621b6260574dd11d >> Author: Lai Jiangshan >> Date: Tue Dec 11 16:01:05 2012 -0800 >> >> slub, hotplug: ignore unrelated node's hot-adding and hot-removing >> >> SLUB only focuses on the nodes which have normal memory and it ignores the >> other node's hot-adding and hot-removing. >> >> Aka: if some memory of a node which has no onlined memory is online, but >> this new memory onlined is not normal memory (for example, highmem), we >> should not allocate kmem_cache_node for SLUB. >> >> And if the last normal memory is offlined, but the node still has memory, >> we should remove kmem_cache_node for that node. (The current code delays >> it when all of the memory is offlined) >> >> So we only do something when marg->status_change_nid_normal > 0. >> marg->status_change_nid is not suitable here. >> >> The same problem doesn't exist in SLAB, because SLAB allocates kmem_list3 >> for every node even the node don't have normal memory, SLAB tolerates >> kmem_list3 on alien nodes. SLUB only focuses on the nodes which have >> normal memory, it don't tolerate alien kmem_cache_node. The patch makes >> SLUB become self-compatible and avoids WARNs and BUGs in rare conditions. >> >> >> How "bad" would it be if we do the slab_mem_going_online_callback() etc even >> for completely-movable nodes? I assume one kmem_cache_alloc() per slab_caches. Yes. >> slab_mem_going_offline_callback() only does shrinking, #dontcare >> >> Looking at slab_mem_offline_callback(), we never even free the caches either >> way when offlining. So the implication would be that we would have movable-only nodes >> set in slab_nodes. Yes. >> We don't expect many such nodes, so ... do we care? Right, the memory waste should be negligible in the big picture. So Oscar can you change this as part of your series? > BTW, isn't description of slab_nodes wrong? > > "Tracks for which NUMA nodes we have kmem_cache_nodes allocated." -- but > as there is no freeing done in slab_mem_offline_callback(), isn't it > always kept allocated? Hm yes right now it's "is it in slab_nodes => it's allocated" and not equivalent. I guess we could remove slab_mem_offline_callback() completely and thus stop clearing from slab_nodes and the comment will be true again? That would mean new caches created after node hotremove will allocate for that node but that's even more negligible than the waste we're doing already. > (probably I am missing something) >