From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45FC6C54E5D for ; Thu, 14 Mar 2024 08:52:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C62058008E; Thu, 14 Mar 2024 04:52:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BEA0380073; Thu, 14 Mar 2024 04:52:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A636E8008E; Thu, 14 Mar 2024 04:52:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 924D580073 for ; Thu, 14 Mar 2024 04:52:39 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5D76CC09E9 for ; Thu, 14 Mar 2024 08:52:39 +0000 (UTC) X-FDA: 81895028838.04.9DB7966 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf26.hostedemail.com (Postfix) with ESMTP id E5BC8140002 for ; Thu, 14 Mar 2024 08:52:36 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Zo3fjWDk; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=cLKsHToY; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Zo3fjWDk; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=cLKsHToY; dmarc=none; spf=pass (imf26.hostedemail.com: domain of jack@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710406357; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oGUsCY8vsPZ7B1LpGm5h1QHc5TrYPfvxHvWc2YdmiCU=; b=LBy7lkFMDXk87aXbUeFSJc0IVpw3tFEoxxWYEQ4j7YNYumDEXOrqmqlpiAV5WYqZzmHoeT id0LgypN2mJe4WXE9b/FGDU8ymLYE7MrzBbSrrzRbXfIn0BDiD6hSO6m0mghabzkA1fEcb 4LQkfGAbPZNp5Df9RjSLJs+7T4MSd6A= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Zo3fjWDk; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=cLKsHToY; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Zo3fjWDk; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=cLKsHToY; dmarc=none; spf=pass (imf26.hostedemail.com: domain of jack@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710406357; a=rsa-sha256; cv=none; b=vLiIG0vkNVJVfiPZv9J0PifOupAPk9WP4vs0YXXtWYwQ25hwloaerVJjRZWk86hbxUQFz3 fpiCl3KXuOE4TXi4UFKmamNS2NQ/V37vHvSruOr+O5Mn0K2252Prn36RwfL8TVw/UzzeiF mBF4Eny1uOA/F8fzn7xPHIMtFD7quZc= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 29FEB1F823; Thu, 14 Mar 2024 08:52:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1710406355; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oGUsCY8vsPZ7B1LpGm5h1QHc5TrYPfvxHvWc2YdmiCU=; b=Zo3fjWDkwNWbhSYZF5VWJZxv8DYK+qkRPoq6cusZYn1SUa6cewL/9+2jXNH0u6spA8wODY eVJAwKgY1JDDVFJ/AQy8hUUONgvsh7hH74tbf1YIBbd/mC/eyXol0xsB8Lp7XUlzUSRsMc y+BDrxnIuPObfoHatYp4JiWyBLCer+A= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1710406355; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oGUsCY8vsPZ7B1LpGm5h1QHc5TrYPfvxHvWc2YdmiCU=; b=cLKsHToYQYg7OnmLQaQVR2DhmStTxb+seLLaReV8yGoDieU5kdqbAMkwtgoCV2I2DwtP3T M8KiRegg27WbFYBA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1710406355; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oGUsCY8vsPZ7B1LpGm5h1QHc5TrYPfvxHvWc2YdmiCU=; b=Zo3fjWDkwNWbhSYZF5VWJZxv8DYK+qkRPoq6cusZYn1SUa6cewL/9+2jXNH0u6spA8wODY eVJAwKgY1JDDVFJ/AQy8hUUONgvsh7hH74tbf1YIBbd/mC/eyXol0xsB8Lp7XUlzUSRsMc y+BDrxnIuPObfoHatYp4JiWyBLCer+A= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1710406355; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oGUsCY8vsPZ7B1LpGm5h1QHc5TrYPfvxHvWc2YdmiCU=; b=cLKsHToYQYg7OnmLQaQVR2DhmStTxb+seLLaReV8yGoDieU5kdqbAMkwtgoCV2I2DwtP3T M8KiRegg27WbFYBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1BCA41386E; Thu, 14 Mar 2024 08:52:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QEqCBtO68mXkLAAAD6G6ig (envelope-from ); Thu, 14 Mar 2024 08:52:35 +0000 Received: by quack3.suse.cz (Postfix, from userid 1000) id B0431A07D9; Thu, 14 Mar 2024 09:52:30 +0100 (CET) Date: Thu, 14 Mar 2024 09:52:30 +0100 From: Jan Kara To: Jared Hulbert Cc: Jan Kara , Chuanhua Han , Chris Li , linux-mm , lsf-pc@lists.linux-foundation.org, ryan.roberts@arm.com, 21cnbao@gmail.com, david@redhat.com Subject: Re: [Lsf-pc] [LSF/MM/BPF TOPIC] Swap Abstraction "the pony" Message-ID: <20240314085230.hisynexjn2o3dr6d@quack3> References: <039190fb-81da-c9b3-3f33-70069cdb27b0@oppo.com> <20240307140344.4wlumk6zxustylh6@quack3> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspam-User: X-Stat-Signature: asiccy6tpboxpetxbw4b7kdzi7ibxmff X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: E5BC8140002 X-HE-Tag: 1710406356-138776 X-HE-Meta: U2FsdGVkX1/rK87+/fzjig31FnTAFLP2a/nzkKmxwgzGalLWc7olktS/zHfVCj9gEeCCTUUsb7C9BASsoeWjEgvPyFykoe815BeCYP+5NdXa4nHqwH0YMZ+mPEQoiejEHCh+iImOiRFWkbR0PjUtgXoJ/Ked08RaFKEs8Rl3uvreKsx6xhPWlXfV+EUdM6RXTgvanzCP+2pqhwImpDXFSJppj8rL7TNXoQazlb8tqR+9Jv4uG4P/1tI2QbgKdOeY46j9EonX83s1GNxL6KwmOwkkS//T9JF1yseHYR5OuV79KBZkySkh22Es6usEFc6ctpukpdrwV+HBlbndorTlQwHdBdxs6fvTdTHmxBv7NidsL3h0lQ7Q9nuYDYQ/w+jHBdhzQoVHj4+bgqw4xAScMkyDg4GzfixPOoCI6isSUh3VDHssIBzlLpWxIBdxyo+uFYU9xDy/2AhzPlCTKpW1pLZyPQlN5ddOr5YLYGQavbmytK8n7PJur5W5G10lDDvhHZZ6StrNLKzCBCQlWk8w64u1PRx+zlahgG6J1zm9mmrVgqu9ibq3VmLFQyrIgDGorxSXNk9jlVuWsQj9qQf7VrBCLngjUCKLhJKYo6XEcniFvw4e3SXD3ItsedcRhQhFCV3F/k7Av6U0qnFwUsVa9q0Ng/3pp91XxKpY+DkT8sivO2srP/FdQj+kiI2ILY80rqECMOXfQI12FLkbGCBbCqecuJqe5ODPuNLGR1ruTZzKKHMdjsv371IVrDF7kT2p2G84FZfuw4phr5Lx6/ydwSMnSztT4addwQFACSVPmgvTE4t51FFZ4mfDlC6qM5h59U7pXjYUdMY87gxw3s1acGg9xQ51RynR31h2wsWhTnJcJ8TOq+i/G373/sxJq0ZkOZcgrrara44= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu 07-03-24 13:06:27, Jared Hulbert wrote: > On Thu, Mar 7, 2024 at 9:35 AM Jan Kara wrote: > > > > Well, but then if you fill in space of a particular order and need to swap > > out a page of that order what do you do? Return ENOSPC prematurely? > > > > Frankly as I'm reading the discussions here, it seems to me you are trying > > to reinvent a lot of things from the filesystem space :) Like block > > allocation with reasonably efficient fragmentation prevention, transparent > > data compression (zswap), hierarchical storage management (i.e., moving > > data between different backing stores), efficient way to get from > > VMA+offset to the place on disk where the content is stored. Sure you still > > don't need a lot of things modern filesystems do like permissions,> directory structure (or even more complex namespacing stuff), all the stuff > > achieving fs consistency after a crash, etc. But still what you need is a > > notable portion of what filesystems do. > > > > So maybe it would be time to implement swap as a proper filesystem? Or even > > better we could think about factoring out these bits out of some existing > > filesystem to share code? > > Yes. Thank you. I've been struggling to communicate this. > > I'm thinking you can just use existing filesystems as a first step > with a modest glue layer. See the branch of this thread where I'm > babbling on to Chris about this. > > "efficient way to get from VMA+offset to place on the disk where > content is stored" > You mean treat swapped pages like they were mmap'ed files and use the > same code paths? How big of a project is that? That seems either > deceptively easy or really hard... I've been away too long and was > never really good enough to have a clear vision of the scale. Well, conceptually it is easy to consider anonymous VMA as a mapping of some file (anon pages are the page cache of this file) and swapout is just writeback event. But I suspect the details are going to get hairy with this concept - filesystems are generally optimized for handling large contiguous blocks where as anon memory is much more random access, also filesystem code does not expect to be run from reclaim context so locking and memory demands might be a problem. So although the unification of anon and file backed memory is intriguing, I didn't mean to go *this* far :) I rather meant we could export some functional blocks like block allocator from some filesystem as a library swap code could use. Honza -- Jan Kara SUSE Labs, CR