From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C35AC27C55 for ; Mon, 10 Jun 2024 18:48:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 891436B008A; Mon, 10 Jun 2024 14:48:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8410F6B0092; Mon, 10 Jun 2024 14:48:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 707B96B0096; Mon, 10 Jun 2024 14:48:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 52DB36B008A for ; Mon, 10 Jun 2024 14:48:05 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id BF68F81056 for ; Mon, 10 Jun 2024 18:48:04 +0000 (UTC) X-FDA: 82215863688.15.D9BB3B8 Received: from mail-vs1-f44.google.com (mail-vs1-f44.google.com [209.85.217.44]) by imf16.hostedemail.com (Postfix) with ESMTP id 05DA018000C for ; Mon, 10 Jun 2024 18:48:02 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=4hzSPX8v; spf=pass (imf16.hostedemail.com: domain of yosryahmed@google.com designates 209.85.217.44 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718045283; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EuGUa7CVchxCQzZnDrxry9gBHUUsdKMsQsZqXLfKacE=; b=FG+VYqMciuiJQVT0XlcmO4W9BoSF+1fxY/bMKhyV7cvmz5ul3JXYh+NKHyagRpBGo1gmpk XZN3vHshSDi0AaPqiImCkyx20u5NDE/+YJRSCiRX7k+nI60R9+pmqJQ9k6gLU0v93OxOGE XKgOfBHv5dXUoQz0UT1Oxc4dQA05mxw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718045283; a=rsa-sha256; cv=none; b=6vH5+uwcWO4K9apEShWUxZefr1z/R/4el5uWL5OHLsKg6l5Yv7nr+blTcVoHWTF9K051dS dcdjbDHA/kRIP+OEYQc8FGqf5nepr/ZPw2V5TUAeYkuYNYfHZHIN5KF5D3nnHi/oR5F3G+ OqbTC0m1vgOrkFBqJfcHwpS/7Yl6jfE= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=4hzSPX8v; spf=pass (imf16.hostedemail.com: domain of yosryahmed@google.com designates 209.85.217.44 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-vs1-f44.google.com with SMTP id ada2fe7eead31-48c4c15e57fso381435137.3 for ; Mon, 10 Jun 2024 11:48:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1718045282; x=1718650082; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=EuGUa7CVchxCQzZnDrxry9gBHUUsdKMsQsZqXLfKacE=; b=4hzSPX8vGFBumxrqNwl5HKzN8TBWDzKa3sOeZM7Jmoq00FgxvcpyCPREnnOe7ogjeU GqWMn2wKZw0PRHZKkDvWQaGd1+W69zu4OAv7coLFar5eV/3ez2oxzk8PYMBqfqmD995f sAHEMTgoHBRj9+prJDPAnnRAletBvXRLhveYUiJ+a4GIpEvTDtdkodwhe7+UH+PScBzB JnRRnFkhpPGm+XzMoFzH9AvBwtD++68aC6NFNor6x4DdhATwHUH8Y1YLcu/xFCvoD9c7 UGNFbDGNQci0Mt120MwX3L6ZYjasZjd/EuJqxBY8tHII84n2E8hnplG4+3TrvpKbRQEe iIVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718045282; x=1718650082; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EuGUa7CVchxCQzZnDrxry9gBHUUsdKMsQsZqXLfKacE=; b=YgMqwxW/jtDWclSNr5g6822/UHE0Tn1zr+qvFHsxoXKKQVa7k/l07cMFtg3281yRhY HY+yrJOvVdEobj4Xu7IM56peTv8rrWROJ5MNQHZLtb984x6ssXzaYCWAlIRgi2dH6rko Xdp5tleyfQspq8Igs5ziL2tAIY7QGzihWO2QxfxaQWe1cNmRK+6uxVL3KK48iyECF4rf S9qdhjDEWTzILwY76ygnlvnw5bMroNvWIIeDuLeUWf9WiJmtDjTeorjoPgGEWppzrqYI B25EB2J1+do7dA1gELrbrvHaXKgZ3EeqKkdbMjp4K8RSqjjoLyFqf0JidHvqAB4zhbAs eZ/A== X-Forwarded-Encrypted: i=1; AJvYcCVZcFA7WoJEIjMXLQb4Z766X8AB1gRa2tfVX4ribpJuubBsBkB5CMqG3pir/MwPSdyi8M//3BxsqLVVmnThD1yfRfE= X-Gm-Message-State: AOJu0YxwD1Om9k7WlLFOrHVuFS36PhY0MRuHjuZcTAmpXHYf1Q2egBxV uZDkw/FzdyEAymlarxjHvTJQS0oC4iHWJIzi+vK/MzA7vV4H6ztwBu/OnLkvQlXDe8yY0k0XfsU WQECg+C+ezplCj/SdHbg2zIRPlNSRAXpeGPOI X-Google-Smtp-Source: AGHT+IFYp+hxMPPq6Tw4jfx8m1UKvQOQde5VyRSxdjv6ntcwDcAYoaAaXX+ZUFxeuqo7QHBu5dD5nketLn2jRf8xyEI= X-Received: by 2002:a67:fbda:0:b0:47a:231a:3856 with SMTP id ada2fe7eead31-48c275deb29mr11694458137.16.1718045281792; Mon, 10 Jun 2024 11:48:01 -0700 (PDT) MIME-Version: 1.0 References: <20240610121820.328876-1-usamaarif642@gmail.com> <20240610121820.328876-2-usamaarif642@gmail.com> In-Reply-To: From: Yosry Ahmed Date: Mon, 10 Jun 2024 11:47:24 -0700 Message-ID: Subject: Re: [PATCH v3 1/2] mm: store zero pages to be swapped out in a bitmap To: Usama Arif Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, david@redhat.com, ying.huang@intel.com, hughd@google.com, willy@infradead.org, nphamcs@gmail.com, chengming.zhou@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 9rj3mkn1eion1t561yj95qfxwjciafmd X-Rspamd-Queue-Id: 05DA018000C X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1718045282-289122 X-HE-Meta: U2FsdGVkX1+0950985GkIQ1owsZAw0NtnyP976GtzjB9P2J4zFMbO5px1gJc4jdwzfYT7DRWD+I5KVdCkKJdtSZ2+7YUeOZf7hIpRsHqyipaE6D/Gskx9J07BveKZK+OxoWFNoz0F7zilmf1PbiHp1xP1mvyB536IZnHkrS3quqIlQzpsJ8vX2WdPcaDq0M9NN4wd1R7eE+ukqeJTW18dInBPFpdsPCisR/utQ8P9RBIF2+w4EiMcmXaFaygoVq6B+LW9byX1TxOUMRt4TgFfX2xs30/uW04WiPcUyzZjBCCZSlvHgJnp+FDkp9rx81Y866ZYJDlbWVE5uUN7CByL/rXhg8tGI6xlGgS1oFFvfPbbRBi5BQb6XsLd0nixuyvdp7TpXt7w1cFzbiMLemdA5o8X5iPC67zJyvWEf2cIbhHeSA70y2F4zps18u0ytgH+jh8d0NwqQ4t3sa25UuHbodogsc0tAjJ0CqEZ5O7kKJ/j2gigS3zjZMn7Masoms+J5l9hjpNOiGljQKDFv86pzPwy/XF/K7PKdf1FiqIqXTf5FvjgADza38gyru0Sgaf4f9/CSWhS44eq99gINACUG5R39w0tZwgEoMSmvR2CUU25yfUj9rK+VH7kw11/bpr/Do2UiUfpfWhLON0k9Ufbbb4cFEp1fC7QmAfqU57fXoSHIoAd1QhLtO611gIrFGj1qzomd6OH4LdalWR/LvsKcgdsnZLajESVJItezx3sxTfbuyxHl7aGxeD9vEqJ+qxJTfJ5i61iTtzzdw9LOD7F14lpib5DZSNGLawymANpc/CTXP48+efTF4Nuyy1RXQmXQ83NusvbAHKBweAoFhLP6LoCXVATvgGwiOt99g6ijFnwzO1PiF6sHvjzdz8G+Ot5CwI9PadP8/Hn0hKGmS5tjcnz4/rgn62pAjHiGU3h00y1igwiqPsZx2bPpOp9d1NW/gGWtrhAhiVkRClO2E QeRNIFbm 9EBbS+/n+Cg/t1geKWHuyOcznmP8svsu4FdM9L591ivhPleVBJYNMeYrY8nhoxn86TZxydAd3B0uUvGPdzrY0acLMUhC1K1WoTi8rGZZdxSJGJesFBvGkyHznOiJI9R1OlCaciO9PXV7NDq6Mkmg5aNaCbQus25sa0YYHHNxXWJePrN0U/TpG8JauBjq7OxG+wRWYlTkpDN3TyCUPVFMt3em4fecWi0/H48IJaa7+4S92muZhox8SsrUQPGHZB+h+RmXGvMPsIrDXHPnw0nC5zHlh6A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Jun 10, 2024 at 11:36=E2=80=AFAM Usama Arif wrote: > > > On 10/06/2024 18:57, Yosry Ahmed wrote: > > On Mon, Jun 10, 2024 at 5:18=E2=80=AFAM Usama Arif wrote: > >> Approximately 10-20% of pages to be swapped out are zero pages [1]. > >> Rather than reading/writing these pages to flash resulting > >> in increased I/O and flash wear, a bitmap can be used to mark these > >> pages as zero at write time, and the pages can be filled at > >> read time if the bit corresponding to the page is set. > >> With this patch, NVMe writes in Meta server fleet decreased > >> by almost 10% with conventional swap setup (zswap disabled). > >> > >> [1]https://lore.kernel.org/all/20171018104832epcms5p1b2232e2236258de3d= 03d1344dde9fce0@epcms5p1/ > >> > >> Signed-off-by: Usama Arif > >> --- > >> include/linux/swap.h | 1 + > >> mm/page_io.c | 92 ++++++++++++++++++++++++++++++++++++++++++= +- > >> mm/swapfile.c | 21 +++++++++- > >> 3 files changed, 111 insertions(+), 3 deletions(-) > >> > >> diff --git a/include/linux/swap.h b/include/linux/swap.h > >> index a11c75e897ec..e88563978441 100644 > >> --- a/include/linux/swap.h > >> +++ b/include/linux/swap.h > >> @@ -299,6 +299,7 @@ struct swap_info_struct { > >> signed char type; /* strange name for an index = */ > >> unsigned int max; /* extent of the swap_map */ > >> unsigned char *swap_map; /* vmalloc'ed array of usage = counts */ > >> + unsigned long *zeromap; /* vmalloc'ed bitmap to track = zero pages */ > >> struct swap_cluster_info *cluster_info; /* cluster info. Only= for SSD */ > >> struct swap_cluster_list free_clusters; /* free clusters list= */ > >> unsigned int lowest_bit; /* index of first free in swa= p_map */ > >> diff --git a/mm/page_io.c b/mm/page_io.c > >> index a360857cf75d..2cac1e11fb85 100644 > >> --- a/mm/page_io.c > >> +++ b/mm/page_io.c > >> @@ -172,6 +172,82 @@ int generic_swapfile_activate(struct swap_info_st= ruct *sis, > >> goto out; > >> } > >> > >> +static bool is_folio_page_zero_filled(struct folio *folio, int i) > >> +{ > >> + unsigned long *data; > >> + unsigned int pos, last_pos =3D PAGE_SIZE / sizeof(*data) - 1; > >> + bool ret =3D false; > >> + > >> + data =3D kmap_local_folio(folio, i * PAGE_SIZE); > >> + if (data[last_pos]) > >> + goto out; > >> + for (pos =3D 0; pos < PAGE_SIZE / sizeof(*data); pos++) { > >> + if (data[pos]) > >> + goto out; > >> + } > >> + ret =3D true; > >> +out: > >> + kunmap_local(data); > >> + return ret; > >> +} > >> + > >> +static bool is_folio_zero_filled(struct folio *folio) > >> +{ > >> + unsigned int i; > >> + > >> + for (i =3D 0; i < folio_nr_pages(folio); i++) { > >> + if (!is_folio_page_zero_filled(folio, i)) > >> + return false; > >> + } > >> + return true; > >> +} > > Is there any benefit to iterating on the folio in pages (i.e. have > > both is_folio_zero_filled() and is_folio_page_zero_filled())? Why > > don't we just kmap the entire folio and check it all at once? > > Is there an API to kmap an entire folio? I thought kmap_local_folio() takes in a 'size' parameter for some reason, my bad. > > I could just do data =3D page_address(&folio->page) in above and iterate > through folio_nr_pages(folio) * PAGE_SIZE, and do it all in one > function, but this just looks much nicer and much more readable? Yeah if we need to map each page individually, the current code is better. > > In other places in the kernel, the folio iteration is through pages as we= ll: > > https://elixir.bootlin.com/linux/latest/source/arch/csky/abiv2/cacheflush= .c#L29 > > https://elixir.bootlin.com/linux/latest/source/arch/mips/mm/cache.c#L162 > > and others as well. [..] > >> @@ -515,8 +600,11 @@ void swap_read_folio(struct folio *folio, bool sy= nchronous, > >> psi_memstall_enter(&pflags); > >> } > >> delayacct_swapin_start(); > >> - > >> - if (zswap_load(folio)) { > >> + if (swap_zeromap_folio_test(folio)) { > >> + folio_zero_fill(folio); > >> + folio_mark_uptodate(folio); > >> + folio_unlock(folio); > > We don't currently support swapping in large folios, but it is a work > > in progress, and this will break once we have it. > > swap_zeromap_folio_test() will return false even if parts of the folio > > are in fact zero-filled. Then, we will go read those from disk swap, > > essentially corrupting data. > > So yes, with this patch I tested swap out of large zero folio, but when > swapping in it was page by page. My assumption was that swap_read_folio > (when support is added) would only pass a large folio that was earlier > swapped out as a large folio. So if a zero filled large folio was > swapped out, the zeromap will be set for all the pages in that folio, > and at large folio swap in (when it is supported), it will see that all > the bits in the zeromap for that folio are set, and will just > folio_zero_fill. > > If even a single page in large folio has non-zero data then zeromap will > not store it and it will go to either zswap or disk, and at read time as > all the bits in zeromap are not set, it will still goto either zswap or > disk, so I think this works? > > Is my assumption wrong that only large folios can be swapped in only if > they were swapped out as large? I think this code works in that case. I think the assumption is incorrect. I think we would just check if contiguous PTEs have contiguous swap entries and swapin the folio as a large folio in this case. It is likely that the swap entries are contiguous because it was swapped out as a large folio, but I don't think it's guaranteed. For example, here is a patch that implements large swapin support for the synchronous swapin case, and I think it just checks that the PTEs have contiguous swap entries: https://lore.kernel.org/linux-mm/20240304081348.197341-6-21cnbao@gmail.com/ This makes a lot of sense because otherwise you'd have to keep track of how the folios were composed at the time of swapout, to be able to swap the same folios back in.