From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 773AFCD37AE for ; Wed, 4 Sep 2024 05:55:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE01D8D01E4; Wed, 4 Sep 2024 01:55:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C8F748D0003; Wed, 4 Sep 2024 01:55:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B2FEC8D01E4; Wed, 4 Sep 2024 01:55:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8F0468D0003 for ; Wed, 4 Sep 2024 01:55:36 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 317161C4F21 for ; Wed, 4 Sep 2024 05:55:36 +0000 (UTC) X-FDA: 82525993872.27.80D5B4B Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf11.hostedemail.com (Postfix) with ESMTP id 477DE40007 for ; Wed, 4 Sep 2024 05:55:34 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BL2PabIg; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725429310; a=rsa-sha256; cv=none; b=7RzQPy6FwMW5sxUTnAL/BsId09B6B7vVUWVGSEaGAIVDeeYofw4aTJbg7PkRMxXkwHBKJ5 BNu15ImlFgZLmV1I0hqkOK8rDHjYAwxcUUBaRq8MV87/GtU5V3TOpXkwmNuowDyQuVKfCT ejXlNiptxzIOQg36jVeLyqe9JLzTLLw= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BL2PabIg; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725429310; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JoFH4FMt+wXQqq7eQ8+QSpnLM+4TMCJr3a9E4w8Z+hE=; b=BxSWmfPj7SV3xuTiGJcPA3x+sBqNLMad5V7xf151yRrV0qEOatszxUw02HWgB8LFA5DIhg q6P3426qdcvny13a1iqaMk/pQDkz3qlrgzX5jsqdnaJ9IyI+PGNhWZ6VNZtquPIdRQiguT nIjE5sgAChbE4QG+olcrytbLAS7MeHM= Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-2d8abac30ddso2498810a91.0 for ; Tue, 03 Sep 2024 22:55:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725429333; x=1726034133; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JoFH4FMt+wXQqq7eQ8+QSpnLM+4TMCJr3a9E4w8Z+hE=; b=BL2PabIgAJ+7swFZC4XQBbj5c6nN6bGSVY+IdWVGDwoOIhVNK0IfqnAFpIib1NWfmX J2odyX1Ek+htbMdbYR6jLrF+vhgPMNWVFdp7LeWlg5fUs6xFsJeUDqSyL/TV6mQOFAcE LEsSFfY0AftBMK0cUThX8nWbOjB2Mxn/sH4E1rh3Ym0iESNe3jHdwgDJYnlSmqYqdypB jW7U/2dqOPKLDH09LOxyGkZBPmG4QRozYrNOMSUkbELm9QgHi9/J1t9qfLnurTx6jzsF 4OB0+3PSplI9YEsYAv8faYsk3YKa993Fjd0xvzCugeGtT8CyqMWzy31mA/W8U+xkBzKB 450g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725429333; x=1726034133; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JoFH4FMt+wXQqq7eQ8+QSpnLM+4TMCJr3a9E4w8Z+hE=; b=GS+/mZR1EQL2KH+cx0col87V7DeYWMIpk56ZhlHzVyH2lBAHdue/UI8Bza1TmXeVEl O4c9pgVN/8qmTKgI4e+uuVBDyNmTGgnDDCwRWT/mxj7UILIAK7WXG5QJBDQZuSlGtPxb 3rWnao8qp2qtysLH1FyXv7GALd7bDmRNSSJM2e85qOyk9DQpRhbR5kWoCm2apKFMAPta brXLOqJT/T8l+wNIa9RO+DfVAo+oyA5KUtJ5Z070+NhRqNhh/bAgFng927ngFktPwxuu jbxIfpVN0J9xYYM2hInjUNcU8i1FcZNOQTdg52oBdCZfe4EIKBBOivH5Uf6zD7l31EBI LiOA== X-Forwarded-Encrypted: i=1; AJvYcCUaM5Bh0zNPQAVSUW9xwyvU3TjTqAsrGHp/0FxWpNp5chubwXqHLPJdY+VAJ4M6p2Wyce06gCFetQ==@kvack.org X-Gm-Message-State: AOJu0YwXpGfBNlLcFDZ4c9fqU/ntHVAhYhoEPbVM7Ehbjs9+oTHiUWjn +JKmtuKsCJn9XaxpkJ0xi7vMYrS2yBX/kvY5qYAgR6Pk3gjsxjLs X-Google-Smtp-Source: AGHT+IEctEsdGjIZiTg5BvQ9VLZNiSdEqaY9shFXt0y07cPPwA4g/hKJgUr5PrKitSIQj7/q1WURRw== X-Received: by 2002:a05:6a21:39b:b0:1cc:ef31:da6d with SMTP id adf61e73a8af0-1ccef31dc1emr16410973637.52.1725429332700; Tue, 03 Sep 2024 22:55:32 -0700 (PDT) Received: from Barrys-MBP.hub ([2407:7000:8942:5500:3dfb:fdbe:2749:2520]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae9505acsm6732365ad.66.2024.09.03.22.55.26 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 03 Sep 2024 22:55:31 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: usamaarif642@gmail.com Cc: akpm@linux-foundation.org, chengming.zhou@linux.dev, david@redhat.com, hannes@cmpxchg.org, hughd@google.com, kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, nphamcs@gmail.com, shakeel.butt@linux.dev, willy@infradead.org, ying.huang@intel.com, yosryahmed@google.com, hanchuanhua@oppo.com Subject: Re: [PATCH v4 1/2] mm: store zero pages to be swapped out in a bitmap Date: Wed, 4 Sep 2024 17:55:22 +1200 Message-Id: <20240904055522.2376-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240612124750.2220726-2-usamaarif642@gmail.com> References: <20240612124750.2220726-2-usamaarif642@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Queue-Id: 477DE40007 X-Rspamd-Server: rspam01 X-Stat-Signature: t8aboy8r197o9nn5ajntmeg3984rdsst X-HE-Tag: 1725429334-552917 X-HE-Meta: U2FsdGVkX1+V2mNoOJTmjBxoZSacw6vkOSzn/JRv7XnySx2uynAiX5xrhz5oJx9PcBsGFhMRqoMP6syYi8hY7+t4x4YePh5HdOTQupXTr5gDMuO8mRPnoKDAq8YkhhLzbqTOcAfNahxOb4rnHJ65GuY3yaT+lAKwOSmxwhIq8wgd/h2YOzCnd2fNGC9vs4Zam7XpTu/Rc2FNiHo9cSm3L7sT8flHvShiR5S0dajBavxKoS0Ah+U6N7NE7Q9rpryAD3pqfBdIpZVfBeTOYiGtKjtl06NNVwOwAGyfGXoEgq6AikgDQB2tKVjCzYjrkCy4fkZAhZlH8zl0NQQ2Ls8rYUjxZZ4znaGS9J8MkU8+cYHDs1Vj1WSJsKINUel/2Y+bsqcdaKQrxSV76uN6TsJQvaWqGqurnlSyiqkgBpIbvy57hz/UZDCFzX2WRZuaFhTFm8oxf5LjlXqylWNgvPHIRo54AlLLL+lal6n2NPxSJViqKKmrEMjTpauwszdVwJN02+CmTbfnXJANj0VotnDKMcDNLSWMurZ8vvzgXXoGXjewiCGCQVLpqvnkWUVQ1RRGdmycjj/G1USA2DzACnf0zlf0okNiTxfCd3uJGw7PpeoKgDr42AvK37PT+yLBU3C/yIPVA2rzeO0DU+EExPznVRHOQQfVcTxfYolqzRrG3yJUDfBwj4LZm9R9qjxe4KX8jz0Pns6PrUy6LDgyha0bUhFt0yiuAUd1LVFpWomxOrz9iAC/XOKiwtcwInSv+AErK2zzjLhjouNFFtoZCb5SGII1nnh0RD9lEarcFP5/bPylUkmrBEjpf3vf+BqBDXZCrCzMfqYxj2frFmTGdZKrKw5kWac2Q3p9vbPy/lVhgOF/iLq/ZIYCN9lS002aPBoo3zJzd5YjAhVDixRDHUgqt0DhqpsZ1RcDA13iMke7KHoIgbU8trSiu7EF9hc4p3tPMteWUMqawwsfqgUVNDt C87afBsM hZ8X5jB8jOL6mTjV8U7mmMmGx73oP7puhoPpQUhvxizqeYimIA+22WuUFFnKPQrHelSFB4jqwSb0EzT2b+0/aU7ob5PSsqHSWRPwOEin8Ey582Ku3jfP+EvuWYzYDaP2oym6SsJyDgU7DW4YxL85IMhxzauAAlTW1E4gD/1IRPE3VGvbCx1wKbY4k9tprMog/PlrK1iu2wc2N5+UkUca38Uu2mVGW06uyqFYzXlQcsyFbxg3w1TYaJgsE3zDzoMvCr+1anCG8Yh/A7KucS0W23VKivmjoKfge6jvg1r1aYaW7dUuYCf2G/12dXqK3AyN06kM7PHzi30EDj1U+P5Uej/A+JMZ+L+5LhFj+BTxy/2SIJXJN0cqWP/v0tT3UJvXPqXpPruvOfETNk0NZzI4JOWXqWEEVlxmW7jkxL441uESeKGrj23vh8Q7Yu6majVaORdOD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jun 13, 2024 at 12:48 AM Usama Arif wrote: > > Approximately 10-20% of pages to be swapped out are zero pages [1]. > Rather than reading/writing these pages to flash resulting > in increased I/O and flash wear, a bitmap can be used to mark these > pages as zero at write time, and the pages can be filled at > read time if the bit corresponding to the page is set. > With this patch, NVMe writes in Meta server fleet decreased > by almost 10% with conventional swap setup (zswap disabled). > > [1] https://lore.kernel.org/all/20171018104832epcms5p1b2232e2236258de3d03d1344dde9fce0@epcms5p1/ > > Signed-off-by: Usama Arif > --- >  include/linux/swap.h |   1 + >  mm/page_io.c         | 114 ++++++++++++++++++++++++++++++++++++++++++- >  mm/swapfile.c        |  24 ++++++++- >  3 files changed, 136 insertions(+), 3 deletions(-) > > diff --git a/include/linux/swap.h b/include/linux/swap.h > index a11c75e897ec..e88563978441 100644 > --- a/include/linux/swap.h > +++ b/include/linux/swap.h > @@ -299,6 +299,7 @@ struct swap_info_struct { >         signed char     type;           /* strange name for an index */ >         unsigned int    max;            /* extent of the swap_map */ >         unsigned char *swap_map;        /* vmalloc'ed array of usage counts */ > +       unsigned long *zeromap;         /* vmalloc'ed bitmap to track zero pages */ >         struct swap_cluster_info *cluster_info; /* cluster info. Only for SSD */ >         struct swap_cluster_list free_clusters; /* free clusters list */ >         unsigned int lowest_bit;        /* index of first free in swap_map */ > diff --git a/mm/page_io.c b/mm/page_io.c > index a360857cf75d..39fc3919ce15 100644 > --- a/mm/page_io.c > +++ b/mm/page_io.c > @@ -172,6 +172,88 @@ int generic_swapfile_activate(struct swap_info_struct *sis, >         goto out; >  } > > +static bool is_folio_page_zero_filled(struct folio *folio, int i) > +{ > +       unsigned long *data; > +       unsigned int pos, last_pos = PAGE_SIZE / sizeof(*data) - 1; > +       bool ret = false; > + > +       data = kmap_local_folio(folio, i * PAGE_SIZE); > +       if (data[last_pos]) > +               goto out; > +       for (pos = 0; pos < PAGE_SIZE / sizeof(*data); pos++) { > +               if (data[pos]) > +                       goto out; > +       } > +       ret = true; > +out: > +       kunmap_local(data); > +       return ret; > +} > + > +static bool is_folio_zero_filled(struct folio *folio) > +{ > +       unsigned int i; > + > +       for (i = 0; i < folio_nr_pages(folio); i++) { > +               if (!is_folio_page_zero_filled(folio, i)) > +                       return false; > +       } > +       return true; > +} > + > +static void folio_zero_fill(struct folio *folio) > +{ > +       unsigned int i; > + > +       for (i = 0; i < folio_nr_pages(folio); i++) > +               clear_highpage(folio_page(folio, i)); > +} > + > +static void swap_zeromap_folio_set(struct folio *folio) > +{ > +       struct swap_info_struct *sis = swp_swap_info(folio->swap); > +       swp_entry_t entry; > +       unsigned int i; > + > +       for (i = 0; i < folio_nr_pages(folio); i++) { > +               entry = page_swap_entry(folio_page(folio, i)); > +               set_bit(swp_offset(entry), sis->zeromap); > +       } > +} > + > +static void swap_zeromap_folio_clear(struct folio *folio) > +{ > +       struct swap_info_struct *sis = swp_swap_info(folio->swap); > +       swp_entry_t entry; > +       unsigned int i; > + > +       for (i = 0; i < folio_nr_pages(folio); i++) { > +               entry = page_swap_entry(folio_page(folio, i)); > +               clear_bit(swp_offset(entry), sis->zeromap); > +       } > +} > + > +/* > + * Return the index of the first subpage which is not zero-filled > + * according to swap_info_struct->zeromap. > + * If all pages are zero-filled according to zeromap, it will return > + * folio_nr_pages(folio). > + */ > +static unsigned int swap_zeromap_folio_test(struct folio *folio) > +{ > +       struct swap_info_struct *sis = swp_swap_info(folio->swap); > +       swp_entry_t entry; > +       unsigned int i; > + > +       for (i = 0; i < folio_nr_pages(folio); i++) { > +               entry = page_swap_entry(folio_page(folio, i)); > +               if (!test_bit(swp_offset(entry), sis->zeromap)) > +                       return i; > +       } > +       return i; > +} > + >  /* >   * We may have stale swap cache pages in memory: notice >   * them here and get rid of the unnecessary final write. > @@ -195,6 +277,13 @@ int swap_writepage(struct page *page, struct writeback_control *wbc) >                 folio_unlock(folio); >                 return ret; >         } > + > +       if (is_folio_zero_filled(folio)) { > +               swap_zeromap_folio_set(folio); > +               folio_unlock(folio); > +               return 0; > +       } > +       swap_zeromap_folio_clear(folio); >         if (zswap_store(folio)) { >                 folio_start_writeback(folio); >                 folio_unlock(folio); > @@ -426,6 +515,26 @@ static void sio_read_complete(struct kiocb *iocb, long ret) >         mempool_free(sio, sio_pool); >  } > > +static bool swap_read_folio_zeromap(struct folio *folio) > +{ > +       unsigned int idx = swap_zeromap_folio_test(folio); > + > +       if (idx == 0) > +               return false; > + > +       /* > +        * Swapping in a large folio that is partially in the zeromap is not > +        * currently handled. Return true without marking the folio uptodate so > +        * that an IO error is emitted (e.g. do_swap_page() will sigbus). > +        */ > +       if (WARN_ON_ONCE(idx < folio_nr_pages(folio))) > +               return true; Hi Usama, Yosry, I feel the warning is wrong as we could have the case where idx==0 is not zeromap but idx=1 is zeromap. idx == 0 doesn't necessarily mean we should return false. What about the below change which both fixes the warning and unblocks large folios swap-in? diff --git a/mm/page_io.c b/mm/page_io.c index 4bc77d1c6bfa..7d7ff7064e2b 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -226,26 +226,6 @@ static void swap_zeromap_folio_clear(struct folio *folio) } } -/* - * Return the index of the first subpage which is not zero-filled - * according to swap_info_struct->zeromap. - * If all pages are zero-filled according to zeromap, it will return - * folio_nr_pages(folio). - */ -static unsigned int swap_zeromap_folio_test(struct folio *folio) -{ - struct swap_info_struct *sis = swp_swap_info(folio->swap); - swp_entry_t entry; - unsigned int i; - - for (i = 0; i < folio_nr_pages(folio); i++) { - entry = page_swap_entry(folio_page(folio, i)); - if (!test_bit(swp_offset(entry), sis->zeromap)) - return i; - } - return i; -} - /* * We may have stale swap cache pages in memory: notice * them here and get rid of the unnecessary final write. @@ -524,9 +504,10 @@ static void sio_read_complete(struct kiocb *iocb, long ret) static bool swap_read_folio_zeromap(struct folio *folio) { - unsigned int idx = swap_zeromap_folio_test(folio); + unsigned int nr_pages = folio_nr_pages(folio); + unsigned int nr = swap_zeromap_entries_count(folio->swap, nr_pages); - if (idx == 0) + if (nr == 0) return false; /* @@ -534,7 +515,7 @@ static bool swap_read_folio_zeromap(struct folio *folio) * currently handled. Return true without marking the folio uptodate so * that an IO error is emitted (e.g. do_swap_page() will sigbus). */ - if (WARN_ON_ONCE(idx < folio_nr_pages(folio))) + if (WARN_ON_ONCE(nr < nr_pages)) return true; folio_zero_range(folio, 0, folio_size(folio)); diff --git a/mm/swap.h b/mm/swap.h index f8711ff82f84..2d59e9d89e95 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -80,6 +80,32 @@ static inline unsigned int folio_swap_flags(struct folio *folio) { return swp_swap_info(folio->swap)->flags; } + +/* + * Return the number of entries which are zero-filled according to + * swap_info_struct->zeromap. It isn't precise if the return value + * is larger than 0 and smaller than nr to avoid extra iterations, + * In this case, it means entries haven't consistent zeromap. + */ +static inline unsigned int swap_zeromap_entries_count(swp_entry_t entry, int nr) +{ + struct swap_info_struct *sis = swp_swap_info(entry); + unsigned long offset = swp_offset(entry); + unsigned int type = swp_type(entry); + unsigned int n = 0; + + for (int i = 0; i < nr; i++) { + entry = swp_entry(type, offset + i); + if (test_bit(offset + i, sis->zeromap)) { + if (i != n) + return i; + n++; + } + } + + return n; +} + #else /* CONFIG_SWAP */ struct swap_iocb; static inline void swap_read_folio(struct folio *folio, struct swap_iocb **plug) @@ -171,6 +197,11 @@ static inline unsigned int folio_swap_flags(struct folio *folio) { return 0; } + +static inline unsigned int swap_zeromap_entries_count(swp_entry_t entry, int nr) +{ + return 0; +} #endif /* CONFIG_SWAP */ #endif /* _MM_SWAP_H */ > + > +       folio_zero_fill(folio); > +       folio_mark_uptodate(folio); > +       return true; > +} > + >  static void swap_read_folio_fs(struct folio *folio, struct swap_iocb **plug) >  { >         struct swap_info_struct *sis = swp_swap_info(folio->swap); > @@ -515,8 +624,9 @@ void swap_read_folio(struct folio *folio, bool synchronous, >                 psi_memstall_enter(&pflags); >         } >         delayacct_swapin_start(); > - > -       if (zswap_load(folio)) { > +       if (swap_read_folio_zeromap(folio)) { > +               folio_unlock(folio); > +       } else if (zswap_load(folio)) { >                 folio_mark_uptodate(folio); >                 folio_unlock(folio); >         } else if (data_race(sis->flags & SWP_FS_OPS)) { > diff --git a/mm/swapfile.c b/mm/swapfile.c > index f1e559e216bd..48d8dca0b94b 100644 > --- a/mm/swapfile.c > +++ b/mm/swapfile.c > @@ -453,6 +453,8 @@ static unsigned int cluster_list_del_first(struct swap_cluster_list *list, >  static void swap_cluster_schedule_discard(struct swap_info_struct *si, >                 unsigned int idx) >  { > +       unsigned int i; > + >         /* >          * If scan_swap_map_slots() can't find a free cluster, it will check >          * si->swap_map directly. To make sure the discarding cluster isn't > @@ -461,6 +463,13 @@ static void swap_cluster_schedule_discard(struct swap_info_struct *si, >          */ >         memset(si->swap_map + idx * SWAPFILE_CLUSTER, >                         SWAP_MAP_BAD, SWAPFILE_CLUSTER); > +       /* > +        * zeromap can see updates from concurrent swap_writepage() and swap_read_folio() > +        * call on other slots, hence use atomic clear_bit for zeromap instead of the > +        * non-atomic bitmap_clear. > +        */ > +       for (i = 0; i < SWAPFILE_CLUSTER; i++) > +               clear_bit(idx * SWAPFILE_CLUSTER + i, si->zeromap); > >         cluster_list_add_tail(&si->discard_clusters, si->cluster_info, idx); > > @@ -482,7 +491,7 @@ static void __free_cluster(struct swap_info_struct *si, unsigned long idx) >  static void swap_do_scheduled_discard(struct swap_info_struct *si) >  { >         struct swap_cluster_info *info, *ci; > -       unsigned int idx; > +       unsigned int idx, i; > >         info = si->cluster_info; > > @@ -498,6 +507,8 @@ static void swap_do_scheduled_discard(struct swap_info_struct *si) >                 __free_cluster(si, idx); >                 memset(si->swap_map + idx * SWAPFILE_CLUSTER, >                                 0, SWAPFILE_CLUSTER); > +               for (i = 0; i < SWAPFILE_CLUSTER; i++) > +                       clear_bit(idx * SWAPFILE_CLUSTER + i, si->zeromap); >                 unlock_cluster(ci); >         } >  } > @@ -1059,9 +1070,12 @@ static void swap_free_cluster(struct swap_info_struct *si, unsigned long idx) >  { >         unsigned long offset = idx * SWAPFILE_CLUSTER; >         struct swap_cluster_info *ci; > +       unsigned int i; > >         ci = lock_cluster(si, offset); >         memset(si->swap_map + offset, 0, SWAPFILE_CLUSTER); > +       for (i = 0; i < SWAPFILE_CLUSTER; i++) > +               clear_bit(offset + i, si->zeromap); >         cluster_set_count_flag(ci, 0, 0); >         free_cluster(si, idx); >         unlock_cluster(ci); > @@ -1336,6 +1350,7 @@ static void swap_entry_free(struct swap_info_struct *p, swp_entry_t entry) >         count = p->swap_map[offset]; >         VM_BUG_ON(count != SWAP_HAS_CACHE); >         p->swap_map[offset] = 0; > +       clear_bit(offset, p->zeromap); >         dec_cluster_info_page(p, p->cluster_info, offset); >         unlock_cluster(ci); > > @@ -2597,6 +2612,7 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile) >         free_percpu(p->cluster_next_cpu); >         p->cluster_next_cpu = NULL; >         vfree(swap_map); > +       bitmap_free(p->zeromap); >         kvfree(cluster_info); >         /* Destroy swap account information */ >         swap_cgroup_swapoff(p->type); > @@ -3123,6 +3139,12 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags) >                 goto bad_swap_unlock_inode; >         } > > +       p->zeromap = bitmap_zalloc(maxpages, GFP_KERNEL); > +       if (!p->zeromap) { > +               error = -ENOMEM; > +               goto bad_swap_unlock_inode; > +       } > + >         if (p->bdev && bdev_stable_writes(p->bdev)) >                 p->flags |= SWP_STABLE_WRITES; > > -- > 2.43.0 > >