From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F076C369AB for ; Mon, 21 Apr 2025 16:33:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 801666B0005; Mon, 21 Apr 2025 12:33:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7B0826B0007; Mon, 21 Apr 2025 12:33:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 651446B0008; Mon, 21 Apr 2025 12:33:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 477F06B0005 for ; Mon, 21 Apr 2025 12:33:45 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 36EB61A0905 for ; Mon, 21 Apr 2025 16:33:46 +0000 (UTC) X-FDA: 83358597252.11.EF8FEDD Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf05.hostedemail.com (Postfix) with ESMTP id 0792B100005 for ; Mon, 21 Apr 2025 16:33:43 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=zY3rXpA5; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=MZdTM1qP; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=zY3rXpA5; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=MZdTM1qP; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf05.hostedemail.com: domain of pfalcato@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=pfalcato@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745253224; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gn+F6t0UpvOZaMj0EZexJvJLHkd2FPUuvN+AXOFjojs=; b=mdrT+YNzhHyPRwajxe9XAE3Tj0lLRXGfhonXRhc72j8VAeYZ3nbuJDyyXBLnFIw0bDY+qk G7dE4Ud0RqEue1CeW8gBWnSwL5tYR32vt6MCqBXBSxWJmL1frdJbV34XldyV88UBW4/EN1 wFC1vuexar0zTFTFIdGON35+bdznKSg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745253224; a=rsa-sha256; cv=none; b=bvPm3L2EM+URWbOrpIdnMT6ni5Bzg7ZRJDi7KJ2rmDPHS1fMWxgSgP7ExLqYkdHJm1ZGSI z4AfM639/q6ENMs5M8nD9GHnCg20SXMFjgFosi/PG8/VqDwtVWppYafnpkwC1LHZe4iOgw 2zQHSsOtmgGcg9N7BB4d6EkdS6/LzAs= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=zY3rXpA5; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=MZdTM1qP; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=zY3rXpA5; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=MZdTM1qP; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf05.hostedemail.com: domain of pfalcato@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=pfalcato@suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 10B6521242; Mon, 21 Apr 2025 16:33:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1745253222; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gn+F6t0UpvOZaMj0EZexJvJLHkd2FPUuvN+AXOFjojs=; b=zY3rXpA56aVwAtVvEaTBFVhlJFR/prwQJuSm84SmbKfYBjqS+pi+l/T1zARXJw1QR4mYyA K1s6HMQWf00+Uuzo3LQ27K7IZuPTtdEB77RGGt7zbbiuvxcRYGpsO/FfqD4b4c4PcEeLHp DAkRqpIZXSqqD1BxK6dn4wPEH4IzcGo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1745253222; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gn+F6t0UpvOZaMj0EZexJvJLHkd2FPUuvN+AXOFjojs=; b=MZdTM1qPao1kB2OHEfnqjsEsoVWN2a+gVrEOHA5ZkCLA6BWg42CWclOAllPqEN2v99+dEy fYN7ONcgBR6wPqBg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1745253222; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gn+F6t0UpvOZaMj0EZexJvJLHkd2FPUuvN+AXOFjojs=; b=zY3rXpA56aVwAtVvEaTBFVhlJFR/prwQJuSm84SmbKfYBjqS+pi+l/T1zARXJw1QR4mYyA K1s6HMQWf00+Uuzo3LQ27K7IZuPTtdEB77RGGt7zbbiuvxcRYGpsO/FfqD4b4c4PcEeLHp DAkRqpIZXSqqD1BxK6dn4wPEH4IzcGo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1745253222; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gn+F6t0UpvOZaMj0EZexJvJLHkd2FPUuvN+AXOFjojs=; b=MZdTM1qPao1kB2OHEfnqjsEsoVWN2a+gVrEOHA5ZkCLA6BWg42CWclOAllPqEN2v99+dEy fYN7ONcgBR6wPqBg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 916B013A3D; Mon, 21 Apr 2025 16:33:40 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id mSTFHWRzBmg5XwAAD6G6ig (envelope-from ); Mon, 21 Apr 2025 16:33:40 +0000 Date: Mon, 21 Apr 2025 17:33:38 +0100 From: Pedro Falcato To: Harry Yoo Cc: Christoph Lameter , David Rientjes , Andrew Morton , Roman Gushchin , "Tobin C. Harding" , Alexander Viro , Matthew Wilcox , Vlastimil Babka , Dave Chinner , Rik van Riel , Andrea Arcangeli , "Liam R. Howlett" , Lorenzo Stoakes , Jann Horn , David Hildenbrand , Oscar Salvador , Michal Hocko , Byungchul Park , linux-mm@kvack.org Subject: Re: [DISCUSSION] Revisiting Slab Movable Objects Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Action: no action X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 0792B100005 X-Rspam-User: X-Stat-Signature: pjmzd6xnonk55c5amk4y9fpmca5imkhm X-HE-Tag: 1745253223-919465 X-HE-Meta: U2FsdGVkX1+8GoQap6prqzAD9jaC0U/G63YfhN4oHXjZnvIZ0RUd/Amg0p/a+g0EBfRx58N4SyicmhxXuQDaz/n764u7Ieow/SOOH983lTtMnPjTUev/bHRzqG7lwRrcu5zK5Lq+/XY6fqi/ah0ePFNgho2m52mB0zS/sr7fXcUyFcHFajLYm8WbN9UuXvc5u4tyrI2rfOD1l+qIJgThVHt0mKlhMOKa84rMJFFD/ynhOu6Bm6rtbcW1GneJwI32+TcNQrYpVBODA4vtzVzhUyYKy1N6CCoFUqwMh11ep6a9ucHOXQlJUvsxA7497IITyG3mPKnWVTXvrGSsjgP2KXziihFKwykwqAg8NUixRBr6NKKQIaM1GsFZtHsR3U46wYuaH8m7xMYAjUSn7ZMb+5cW8WRm2l0AEkXHHsexHk3v46TNlsx4fN16Ivk7LC5KnG8gRD8KrCo5/xmDM6NzHLoNTQQBlI2Dkv3pTO3sZ9cM9zNbZy2bp2IlaSvQ+ol2YYvVrYdmYUbtGa9XkvWmxd75gstGgt1fgBoPi/iCYCFASlRzIsBG3SvH+WPNJohB8LPTgOKMrMbnmYhacZE2EEKpZUP39ZWzWwsT7TGwlTCrCsJOLWU1jcSgJSaYiUcoBnsU0zts0eqP0sOGDBZVMCtZILMxU52wfbA5CGX6wGT4krGXnITa/VJnkgk6w/pBwIY/rV2PIs0NJapuvW19IfrZCaPNZ24b03/dpeOB0zZXHGyCz6Y9OnR3tMVye0YAlkwNW2U7jL92SLkwzHGkRH1bKRIlNakLXIr+3UlkKHZuhTgwaHiK6Pn/PMm+oYAeft5xjDr7KbUkHv+5r2SFy6fNNl8LU221WYmk3HPXzKo6rrted/a0CEukkCHcO/qhiqVQK2lPnttDPLDaAtNouZC15AorHQQsxZsZdVdbamHfJ3SRZ0G99haJrRkhNFjsJvPuYWaMwgEPqTDv6GV ozN2x92M NSI2uyIIJc624POYYfCbgINydpNChHrD6QHE81PkOYQZW661F0BLYam07np7lyCdkyC62NKk3WSAAgthr/FudsRWACRUFLUJU1mdtK/dJpQxqLsQid1fVUYDvWVmoOSPoZUc5xWlDUGkeaxeJjVpTCs4vJ3u73G6XM/5O3/vAUfHHhsurJd+Ri2Y0646/ztLNi5XvTOJX7kFVx6vD7mN+uHkpJG8z7ryFV/dvyBg4OnYaKfz1Gv721otGN5PhXZofgh+gR0ODa4MzNlGEKgsi4gHznNRW3tHpC208PWVxo5vqaU4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Apr 21, 2025 at 10:47:39PM +0900, Harry Yoo wrote: > Hi folks, > Hi Harry, Some passing thoughts... > As a long term project, I'm starting to look into resurrecting > Slab Movable Objects. The goal is to make certain types of slab memory > movable and thus enable targeted reclamation, migration, and > defragmentation. > > The main purpose of this posting is to briefly review what's been tried > in the past, ask people why prior efforts have stalled (due to lack of > time or insufficient justification for additional complexity?), > and discuss what's feasible today. > > Please add anyone I may have missed to Cc. :) > > Previous Work on Slab Movable Objects > ===================================== > > Christoph Lameter, Slab Defragmentation Reduction, 2007-2017 (V16: [2]): > Christoph Lameter, Slab object migration for xarray, 2017-2018 (V2: [3]): > Christoph's long-standing effort (since 2007) aiming to defragment > slab memory in cases where sparsely populated slabs occupy excessive > amount of memory. > > Early versions of the work focused on defragmenting slab caches > for filesystem data structures such as inode, dentry, and buffer head. > updatedb was suggested as the standard way to trigger for generating > sparsely populated slabs on file servers. > > However, defragmenting slabs for filesystem data structures has proven > to be very difficult to fully solve, because inodes and dentries are > neither reclaimable nor migratable, limiting the effectiveness of > defragmentation. > > In late 2018, the effort was revived with a new focus on migrating > XArray nodes. However, it appears the work was discontinued after > V2 [3]? > > Tobin C. Harding, Slab Movable Objects, 2019 (First Non-RFC: [5]) > - Tobin C. Harding revived Christoph's earlier work and introduced > a few enhancements, including partial shrinking of dentries, moving > objects to and from a specific NUMA node, and balancing objects across > all NUMA nodes. > > Also appears to be discontinued after the first non-RFC version [5]? > > At LSFMM 2017, Andrea Arcangeli suggested [6] virtually mapped slabs, > which might be useful since migrating them does not require changing the > address of objects. But as Rik van Riel pointed out at that time, it > isn't really useful for defragmentation. Andrea Arcangeli responded > that it can be beneficial for memory hotplug, compaction and out-of-memory > avoidance. > > The exact mechanism wasn't described in [6], but I assume it'll involve > 1) unmap a slab (and page faults after unmap need to wait for migration > to complete), 2) copy objects to a new slab, and 3) map the new slab? > But the idea hasn't gained enough attention for anyone to actually > implement it. I don't think this is a silver bullet. It opens a whole separate can of worms while maintaining similar issues. But instead of worrying about updating pointers, you're worrying about locking out _any_ sort of access, which would involve stop_machine(). You can't even atomically replace a PTE without running into issues (arm requires BBM, thus this doesn't work) so it cannot be applied to anything that can't page fault (xarray and the maple tree are used in IRQ paths, if I'm not mistaken). > > Potential Candidates of SMO > =========================== > > Basic Rules > ----------- > > - Slab memory can only be reclaimed or migrated if the user of the slab > provides a way to isolate / migrate objects. > - If objects can be reclaimed, it makes sense to simply reclaim them > instead of migrating them (unless we know it's better to keep that > object in memory). In any case I think you want to give subsystems the power to decide between {RECLAIMED, MIGRATED, SKIPPED}. > - Some objects can't be reclaimed, but migrating them is (if possible) > still useful for defragmentation and compaction. > - However it is not always feasible > > Potential candidates include (but not limited to): > -------------------------------------------------- > > - XArray nodes can be migrated (can't be reclaimed as they're being used) > - Can be reclaimed if it only includes shadow entries. > - Maple tree nodes (if without external locking) and VMAs can be migrated > and obviously can't be reclaimed. > - Negative dentry should be reclaimed, instead of being migrated. > - Only unused dentries can be reclaimed without high cost. Unused dentries can also have a high cost if they're accessed in the future (and the dentry LRU has some difficulty in... LRU'ing - thus the negative dentry problem). In any case, it would be interesting to see if the existing shrinker interface could be used for this stuff. We already best-effort-reclaim objects, maybe we could best-effort-migrate objects too? The problem of reclaiming is adjacent to migrating, and we already have infrastructure for it... -- Pedro