From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4C2CC54E49 for ; Thu, 7 Mar 2024 08:57:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5034B6B0129; Thu, 7 Mar 2024 03:57:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B35E6B012E; Thu, 7 Mar 2024 03:57:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 37CA16B012F; Thu, 7 Mar 2024 03:57:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2229F6B012E for ; Thu, 7 Mar 2024 03:57:32 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id CA8C41C1315 for ; Thu, 7 Mar 2024 08:57:31 +0000 (UTC) X-FDA: 81869639502.02.32CCC29 Received: from mail-oa1-f45.google.com (mail-oa1-f45.google.com [209.85.160.45]) by imf29.hostedemail.com (Postfix) with ESMTP id 496BD120008 for ; Thu, 7 Mar 2024 08:57:30 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=AKepStNw; spf=pass (imf29.hostedemail.com: domain of jaredeh@gmail.com designates 209.85.160.45 as permitted sender) smtp.mailfrom=jaredeh@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709801850; a=rsa-sha256; cv=none; b=fa4Nh/psVwb4ELYzHODXav5hqp0HQ+nEEGXG2xVRxZBe4pcQlJsC6ZXDeyd/sr44yWguay ywWE0bihDibAVhRIZOAgRwmGNlqmYu6rqOL5IXz3wcacJ77ZxIWeNlzc2pcA3NPATpN5Ad UDn0P8HsU88I5GJmpcF3jewtSfM6jlk= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=AKepStNw; spf=pass (imf29.hostedemail.com: domain of jaredeh@gmail.com designates 209.85.160.45 as permitted sender) smtp.mailfrom=jaredeh@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709801850; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hMeRrLk3y0QffUoC+P3pm92d9SNpteUBXqpn/NW98tQ=; b=INvj7vi11IcylDo3p02xzl+w6CPgxVdDmTNP5UubazNdhSxnuK4UDvYTrgQTwrN9uAr8i6 2efj3I2iIcNMbosGgcjls/3UG0rBmK2ZQ7VTnrkSSh1ZyhwweRFm/0+wtzr6T0l+Deh4BY E/0jF4SBLnBFTFK9ANaQjAy9bf8dNR8= Received: by mail-oa1-f45.google.com with SMTP id 586e51a60fabf-221822a7dc7so31670fac.3 for ; Thu, 07 Mar 2024 00:57:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709801849; x=1710406649; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=hMeRrLk3y0QffUoC+P3pm92d9SNpteUBXqpn/NW98tQ=; b=AKepStNwYe9HRKpmaAl8VtzHTdCmeQw1nvavkpPXBUGcTXRv6Qn8sNIlyHmx0I1+Aa XZy9WbUzLQou+bDiCBO+Mf3Y0e+66dx2uXwEmk4YIXJoPYpKXBFiQFpe++wXigK3XnPu 334jDbk4wTQXqP26V8M5sUwWpDsKi4yehP7N39euV6Q3bx4T3+4D53FO0i0iYbT7ICTO 0f+wsEzQoMyADF5XpdA9u8NpdX7RA1m7RqDjgREiqLbF4N0PsikETbgztBeSOZ7Ul9YA XGblxXK539drcST4TtCk86o3ORafhOum/VcJjbiYcXgGB9iM8pGjBQPeUv+bmCDmMvgk z8Qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709801849; x=1710406649; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hMeRrLk3y0QffUoC+P3pm92d9SNpteUBXqpn/NW98tQ=; b=kFoNMBK4YmbuPrRuEoZM8zWbW+fWEOnJtmFOUnl0KCL8iZ9u2qdN+jB5/8T9FVGwZL Cix34Rl0hQ0UUzUbQc4pEJY2PmGMp0MwyG+Y1m4U/R/6DGYgNefrokQ8SjHucdW1Dnee 5ARRk9UjQ5iEFnLcP4lHUf4aTbyWRibVDJhr8tQMF/9SC1FVD+42j7CgJz0tpm+R8LHC ZaYd2IeHwnPD8HaIAaYsgUyXQO+COMoB2Zxjpifg/n70Nk0dqNAXhMSpH1WGahFb77p2 +yozFzE5UCV58LDR2Bop/OxnBfqZCH0Je2DW3I3RAe6t89XZky49XsS+z/kbVCkBRD8/ 5yEg== X-Gm-Message-State: AOJu0Yzf7u2FGo34aBSI7obkg6ECnSzeQcUdUJDNVtNnoHL3afHTeyLk GCTW4e5fFbrEde/Nl9LWpbuosy5NKEEbpYN95yTQXQ57ke3I/eH+FHM/7QtJECHmSTLl8MFh7lU ouXemHGb/kPdEX9oILIH69EStvGI= X-Google-Smtp-Source: AGHT+IH17KO/IT843We8rzAGVeopV6EDwZEJlfuSHHpvO44VPO4iD7OJ9LTXM/jSYL5cXJqqzvnyEuJUgasHFKGyGfk= X-Received: by 2002:a05:6870:1650:b0:21e:a8d2:7a8e with SMTP id c16-20020a056870165000b0021ea8d27a8emr7876106oae.17.1709801849292; Thu, 07 Mar 2024 00:57:29 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Jared Hulbert Date: Thu, 7 Mar 2024 00:57:17 -0800 Message-ID: Subject: Re: [LSF/MM/BPF TOPIC] Swap Abstraction "the pony" To: Chris Li Cc: linux-mm Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 496BD120008 X-Stat-Signature: p867974qhk8dcdhx9oquaqadp5rrspxi X-Rspam-User: X-HE-Tag: 1709801850-291780 X-HE-Meta: U2FsdGVkX1+4dbBPRa4oQ48FB+8soMjb3eHiwYOYpvXjYlW8Yv3b6+KcZPgZ/vrf+IyzqIqWFNGBSPU9NnnFNIYk8ZYZ48h/FprhqiImt9WVmkqdu3eGfv5M1/ppOBwsjNXpwgswwCnGtofJVBDkwkJFeMfYtpuAWD7EjEcYaYq8nhZr4QM0H7OD4c5kFmdSlYJEyrzWMLJb0I0hDDw0bGD/Zru8tqpYDRYVlWkoNOEFAesuXNpyYMqyqvx67S5XDy49FU7mn2wIl3kvXuhiBKukKuvVEAeckK3KKe7EsZOx2KyCo1GsxepPwxLKXTIMyonQHXs8ijbVmftlUDaju7gyV7xIGqJeCoDpqQIHGG/Ida+8cZ+WyOsku9R20zwTPAdQvt1fkTsQtJgX9AgS7/4Z/YbmdIu5l43ELdw6CAY40wG/QNRzHgJraGAT6hQJ1vOJC8zOqMnhMEguJD0oFaBo0rBPjWdByhgEPfrAqlrD6LxlRJExlA7sxb6wRD94teYK1hCHPYH2Vs5J9vWtzf5RqwGaLn05+EnnF0uzgfLAsLJ2SfS2lO3ShO3ozDxTSpPAZHPZnbXJbJl0/5VWQxx5yfsIfxQbO3xMynTL7KDmiDTWFgK1vfJVmKIGKje5VomOqdlCwYlJeBv4fNCemev9rlBnbXKFTCgu+x/dq+w/z13ekd41tTSN/X/ZWw0/7bvX0WrLjIfpg4m7Kq5iVAaXRBYIRoxI40f28+trUajwbSog9m2TVl3nspCZeUoTKV/vlYM0T3e1QQleSQlDfmhv4/m2lker8ZQ19TJRgZsEwUXvB9M2TW2n64jnSRfLyiCUBtFjhOlZN5x7E/EvFefPB8n/6na0RlBLZndOQD94K0610m9ZgGWJPGyqYekxnZ4SUlyfEyUjGOcMYhLtKWJfgQ0j2DoBoKxK28J5GfasAyj1kuC+3y+lCSbXdVIgRYvIs/gobzx4gdPD544 VHPe58oD VCY8fUtqSq8fPGYM8QmXqV5xDlg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.031337, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Mar 6, 2024 at 4:46=E2=80=AFPM Chris Li wrote: > > OK, you are suggesting not using file inodes for 4K swap pages. > Also not design our own data structure to manage swap entry allocation. > > Then how do you allocate swap entries using this file system or database? > More detail on how swap entries map into the large files offsets can > help me understand what you are trying to do. > > Swap file support exists in the kernel. You can block IO on the swap > device with a given offset. The block device API exists. That is how > the swap back end works right now. I am not sure I understand your > question. To apply the database model to the problems of fragmentation and mTHP you could have a file for every page size. All your offsets would be aligned. Similar to what Chuanhua Han is proposing in the swap device on another subthread. Here is an example of how filesystems would make this all so easy. Let's assume you have a 20GB filesystem so you set it up with a 10GB file for 4KB pages and 10GB for mTHP. Then overtime workloads change and the 4KB is only using 2GB while the mTHP needs more space so you decide to add 5GB to the mTHP taking it from the 4KB. However, while the 4KB is largely unutilized there is a valid entry at the last offset, you can't truncate the file without moving entries. If you fallocate(FALLOC_FL_PUNCH_HOLE) when you free entries then you end up with a sparse file and can easily grow the mTHP file to 15GB. You end up with 25GB of logical space on the 20GB disk no problem.