From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79FA9C02193 for ; Tue, 4 Feb 2025 16:44:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F4AA6B007B; Tue, 4 Feb 2025 11:44:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 07DCC6B0085; Tue, 4 Feb 2025 11:44:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E60596B0088; Tue, 4 Feb 2025 11:44:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C3CA36B007B for ; Tue, 4 Feb 2025 11:44:48 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5D8D6C020A for ; Tue, 4 Feb 2025 16:44:48 +0000 (UTC) X-FDA: 83082836256.19.44F7D50 Received: from out-187.mta0.migadu.com (out-187.mta0.migadu.com [91.218.175.187]) by imf05.hostedemail.com (Postfix) with ESMTP id 6DCE9100002 for ; Tue, 4 Feb 2025 16:44:46 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=mieaxx+9; spf=pass (imf05.hostedemail.com: domain of yosry.ahmed@linux.dev designates 91.218.175.187 as permitted sender) smtp.mailfrom=yosry.ahmed@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738687486; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7DurDhckPAU6pBt3/2ZFvS48rDpKGWKJwEozIXca7v0=; b=K730JjpiNWFwE5jmwXfQ5RuIWPXE3pT+0RHrQYYCPDZRP3+xi12EBeG+ZxBo+rs9KaXRvg NYxZ5prsr1N/ZlRWRl7E4VP/bfw4QESTy507XW3Fz6/BVlXIFbbs5C7XBC7am44NkDbsez 53LTY0pxfxdlrCq5GFJFjePwng56PRQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738687486; a=rsa-sha256; cv=none; b=BISup89zMlWXsGeci+d+HDVxYj6RvJNtM2sn1AtnUUbMWe5rdk870zU7rOclj+t4/J61dx Zy7eOh94wkC2PqWQpdJsjilHZu49vAL4oRxpP1nE5jxNYNhuseZAS6O6EEnrEYrtso/iuI OT0nuBndAMlgf7FugyoESejTKErrmK0= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=mieaxx+9; spf=pass (imf05.hostedemail.com: domain of yosry.ahmed@linux.dev designates 91.218.175.187 as permitted sender) smtp.mailfrom=yosry.ahmed@linux.dev; dmarc=pass (policy=none) header.from=linux.dev Date: Tue, 4 Feb 2025 16:44:33 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1738687479; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=7DurDhckPAU6pBt3/2ZFvS48rDpKGWKJwEozIXca7v0=; b=mieaxx+9Ok/z6Ii2u94poTjManCQW2n1vCWpPRuk0zDBTRFEn5oVqQvUTBaGxtHwRrD4IB RiDtLEqH3y5uKae0aRBtULfQ2J3LjfdrNonsNw+mke9sfkwNeDANahUEuRMLXLXsfmFGdc OnP+ZIEfUOs2ZOg/glN0qjFs8gIr5nQ= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Yosry Ahmed To: Kairui Song Cc: lsf-pc@lists.linux-foundation.org, linux-mm , Andrew Morton , Chris Li , Johannes Weiner , Chengming Zhou , Shakeel Butt , Hugh Dickins , Matthew Wilcox , Barry Song <21cnbao@gmail.com>, Nhat Pham , Usama Arif , Ryan Roberts , "Huang, Ying" Subject: Re: [LSF/MM/BPF TOPIC] Integrate Swap Cache, Swap Maps with Swap Allocator Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 6DCE9100002 X-Stat-Signature: 8xekaxhbuas875t69qapqxscejg17jy1 X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1738687486-514328 X-HE-Meta: U2FsdGVkX19mL4h8iSDF/lGzZQbIutjwxS8USr5Xll2vN1i7DCqYPn6i0eAjAtC+W4Rg5/zzAvg3s92SQTYOVjIOiwCT8gTzVAAvoLH2kat+WataPP6Ly6v1fqSeCJIjzTTWYhaCqn4eXAXdHp8BFBE/O5tMZKCXfQ6jOxiQKHJyO4I+AKJ0cx3cCkpAwIx0LKNS7rG0rx96fZJcAYoD3THhk2/ApKFN0nPLBZOdXjp4xCCv+m1TW4Rh57RTlak1oHYADTWQjKNEYAPvC0FlOa8+ALeXoFqffRRiSs1PJn2VTWWKN+fQfdVqfcxKHLjFke9fr+rH2o0EweGM5xfnPMtGVkwhVOpfnE8U7ikMgVz/+kigbhd210MTDkcv+eD0zrmz0/FZzPl+T8FoT9qAqvM6msSkZ/XjLRQ8GhscPoMTk2tqYZMA2ixa86polL3t4pDyLknekwhnTkBw2s5Ix20KNRUyp6PEhobrK5I0RfvZw5yXsJxCEfhgOLozosfd9OVY9Ong+A66UK5TZUrPW4RA/DVLLmeBKCOkKRpKRjwbYEcZkRS0BPGVYmqETTWl0YGgld+7c9JWsHjRUuH474xqePqf7pw4KHQ+HVAAz0mo3OzYX6A29GvzEGANfskZZzFaeueSDaj4fAN3WwPQlNZMFVRSqKKvlmmIIlfDsCxQxxY74mJIN+7qbUM7+qdxN9vNl9J9mrWffZvToTaoSO8MNtBYA2fNXJ9k6rdk+7w63BYnSn3i1bF+GGb4QuA6FY5Okl7KBgZY9A87OT7PibwdL7NLu7OmRONXdbKrwsJPiufPL3dQSdBQc2dteEqafm3oOFv8tmjrjSxELMMf6YwKQB5pGKPhqSwbf+saGHClffayxzjedHSAJAJYVq/yIABD4+FkChLV1XQeAsPer3zOEaqDSTbEe/visHoF/pvoHwtTU9p7w42tOIiNGNBvlFjVso2vO3HJ+/PIpbW UHkSFvEe DMZDINZlZFzhrFDHDNf0zlPgLMNc/eKGt7qE+Tv0yjLjDR/xssWRgAqm9WJObyIiV+gz0KDSnFPLNzbEWLCieiP5OUF0BZSXNf43R8L1s89z9+Mpml3ANhaaQxh84pXBv1fd9y/ZYL2qIBoFgpFYachCjAfeiY+6R+AFeOFfBvMIcgXB7jILVtnL4WWCqXMM63wlD4xN9L/HdDxWaRKhaKRmS52uGhem1IZNMtwXpfIg6akZXb6pvGEPT1irhzI2gR9NyOTnffPwnyiJ0jxWItnv6NaWpJTIegWWyldvV4teTbzCoADg3L4did6u1THyqkImUc6TJoVBd/GoV5l958qPNchT3SM4W4L6fIwgpK2HGxxQ51qghGT7uJHXPZzAfQyjY6J6K3qAkTHHG0Ot7Uq5/yLdxdqNk8eIJKICkjXL5MAShwQUb6yYws5FfcEcsElAgOJ0xStg+v4RXA/wDzGXHnI+d2R0Z8vFlOnHG6DFknMI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Feb 04, 2025 at 07:44:46PM +0800, Kairui Song wrote: > Hi all, sorry for the late submission. > > Following previous work and topics with the SWAP allocator > [1][2][3][4], this topic would propose a way to redesign and integrate > multiple swap data into the swap allocator, which should be a > future-proof design, achieving following benefits: > - Even lower memory usage than the current design > - Higher performance (Remove HAS_CACHE pin trampoline) > - Dynamic allocation and growth support, further reducing idle memory usage > - Unifying the swapin path for a more maintainable code base (Remove SYNC_IO) > - More extensible, provide a clean bedrock for implementing things > like discontinuous swapout, readahead based mTHP swapin and more. > > People have been complaining about the SWAP management subsystem [5]. > Many incremental workarounds and optimizations are added, but causes > many other problems eg. [6][7][8][9] and making implementing new > features more difficult. One reason is the current design almost has > the minimal memory usage (1 byte swap map) with acceptable > performance, so it's hard to beat with incremental changes. But > actually as more code and features are added, there are already lots > of duplicated parts. So I'm proposing this idea to overhaul whole SWAP > slot management from a different aspect, as the following work on the > SWAP allocator [2]. > > Chris's topic "Swap abstraction" at LSFMM 2024 [1] raised the idea of > unifying swap data, we worked together to implement the short term > solution first: The swap allocator was the bottleneck for performance > and fragmentation issues. The new cluster allocator solved these > issues, and turned the cluster into a basic swap management unit. > It also removed slot cache freeing path, and I'll post another series > soon to remove the slot cache allocation path, so folios will always > interact with the SWAP allocator directly, preparing for this long > term goal: I believe this was first raised in some form in LSFMM 2023 [1] :) The approach described here is different, as it's cluster-based, which is interesting. I am interested to know how this helps separate the swap core layer from the underlying backing, as Johannes asked. In all cases, Nhat is also working on something similar, so we need some coordination here to avoid duplicated/conflicting work. Thanks! [1]https://lwn.net/Articles/932077/ > > A brief intro of the new design > =============================== > > It will first be a drop-in replacement for swap cache, using a per > cluster table to handle all things required for SWAP management. > Compared to the previous attempt to unify swap cache [11], this will > have lower overhead with more features achievable: > > struct swap_cluster_info { > spinlock_t lock; > u16 count; > u8 flags; > u8 order; > + void *table; /* 512 entries */ > struct list_head list; > }; > > The table itself can have variants of format, but for basic usage, > each void* could be in one of the following type: > > /* > * a NULL: | ----------- 0 ------------| - Empty slot > * a Shadow: | SWAP_COUNT |---- Shadow ----|XX1| - Swaped out > * a PFN: | SWAP_COUNT |------ PFN -----|X10| - Cached > * a Pointer: |----------- Pointer ---------|100| - Reserved / Unused yet > * SWAP_COUNT is still 8 bits. > */ > > Clearly it can hold both cache and swap count. The shadow still has > enough for distance (using 16M as buckets for 52 bit VA) or gen > counting. For COUNT_CONTINUED, it can simply allocate another 512 > atomics for one cluster. > > The table is protected by ci->lock, which has little to none contention. > It also gets rid of the "HAS_CACHE bit setting vs Cache Insert", > "HAS_CACHE pin as trampoline" issue, deprecating SWP_SYNCHRONOUS_IO. > And remove the "multiple smaller file in one bit swapfile" design. > > It will further remove the swap cgroup map. Cached folio (stored as > PFN) or shadow can provide such info. Some careful audit and workflow > redesign might be needed. > > Each entry will be 8 bytes, smaller than current (8 bytes cache) + (2 > bytes cgroup map) + (1 bytes SWAP Map) = 11 bytes. > > Shadow reclaim and high order storing are still doable too, by > introducing dense cluster tables formats. We can even optimize it > specially for shmem to have 1 bit per entry. And empty clusters can > have their table freed. This part might be optional. > > And it can have more types for supporting things like entry migrations > or virtual swapfile. The example formats above showed four types. Last > three or more bits can be used as a type indicator, as HAS_CACHE and > COUNT_CONTINUED will be gone. > > Issues > ====== > There are unresolved problems or issues that may be worth getting some > addressing: > - Is workingset node reclaim really worth doing? We didn't do that > until 5649d113ffce in 2023. Especially considering fragmentation of > slab and the limited amount of SWAP compared to file cache. > - Userspace API change? This new design will allow dynamic growth of > swap size (especially for non physical devices like ZRAM or a > virtual/ghost swapfile). This may be worth thinking about how to be > used. > - Advanced usage and extensions for issues like "Swap Min Order", > "Discontinuous swapout". For example the "Swap Min Order" issue might > be solvable by allocating only specific order using the new cluster > allocator, then having an abstract / virtual file as a batch layer. > This layer may use some "redirection entries" in its table, with a > very low overhead and be optional in real world usage. Details are yet > to be decided. > - Noticed that this will allow all swapin to no longer bypass swap > cache (just like previous series) with better performance. This may > provide an opportunity to implement a tunable readahead based large > folio swapin. [12] > > [1] https://lwn.net/Articles/974587/ > [2] https://lpc.events/event/18/contributions/1769/ > [3] https://lwn.net/Articles/984090/ > [4] https://lwn.net/Articles/1005081/ > [5] https://lwn.net/Articles/932077/ > [6] https://lore.kernel.org/linux-mm/20240206182559.32264-1-ryncsn@gmail.com/ > [7] https://lore.kernel.org/lkml/20240324210447.956973-1-hannes@cmpxchg.org/ > [8] https://lore.kernel.org/lkml/20240926211936.75373-1-21cnbao@gmail.com/ > [9] https://lore.kernel.org/all/CAMgjq7ACohT_uerSz8E_994ZZCv709Zor+43hdmesW_59W1BWw@mail.gmail.com/ > [10] https://lore.kernel.org/lkml/20240326185032.72159-1-ryncsn@gmail.com/ > [11] https://lwn.net/Articles/966845/ > [12] https://lore.kernel.org/lkml/874j7zfqkk.fsf@yhuang6-desk2.ccr.corp.intel.com/ >