From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B686FE6FE49 for ; Fri, 6 Sep 2024 21:59:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A5D16B007B; Fri, 6 Sep 2024 17:59:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 454936B0083; Fri, 6 Sep 2024 17:59:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F51E6B0085; Fri, 6 Sep 2024 17:59:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0B1C46B007B for ; Fri, 6 Sep 2024 17:59:01 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 833051C4D5A for ; Fri, 6 Sep 2024 21:59:00 +0000 (UTC) X-FDA: 82535679240.25.C8BA29D Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) by imf01.hostedemail.com (Postfix) with ESMTP id 890E040004 for ; Fri, 6 Sep 2024 21:58:58 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dMZd7neG; spf=pass (imf01.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.215.171 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725659841; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fFjzAfcWlZGdLMpuzA2ujYuM7IQsh3EHmDmz8Nw4EAk=; b=dsAz0yE/lbeKJapzFd7pKa8atahM1FHFv7jSIDvbCf6EfdK57+yqIX5M+ErH/6eYQYN+u4 Qc2rDyjKPurMxFgcMV975BJL8rbyRzrKx4ZYCDf8opYNKReF+BO5ziHGN4yTbNLv+Oc67x nRqJW4xfl706Z7IN9LBY9MbHzsXL95M= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725659841; a=rsa-sha256; cv=none; b=emElNKD7YCPUeU+zhX1duxqmsLGg8l3/hAYWZfcCwMqqlpnfb9whnzBWUq4AH9tEAdctWi vGWZMZG/jxPnqrt3EliOX9pjnTK/DKRbSV8fYyfHxgObM+AmGMgfFLM48rkQbCukZnalSa z6mHpdr3SsFgxHmoim7/WFdu7Lbx/NQ= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dMZd7neG; spf=pass (imf01.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.215.171 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pg1-f171.google.com with SMTP id 41be03b00d2f7-7d4f9e39c55so1803505a12.2 for ; Fri, 06 Sep 2024 14:58:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725659937; x=1726264737; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fFjzAfcWlZGdLMpuzA2ujYuM7IQsh3EHmDmz8Nw4EAk=; b=dMZd7neG7O4z22NoAMC9zrrHAPVZRqLnuKhqHiad5/ln9iaXiIrRR/j0QSVAgFgLv2 Pbir3RzWmFqSBksHHQTfbFYQkV4DxoHMczmiK5rU5cLlOeAme4ob8NmldPcmAsqh7z3/ Y+IwqBdvmPau9I3/SgVJ5MoFHbM1xSf/qrQaHvB94SjMAMPUtmtco8PfSuFR92i38Hnk 8NPy1xj5d0qwZHWy55AYj9hD063TLG8xTF1AMEZ8lL8x3rzurxMvOirV+R0Kp5q7vasJ kiEYjCo7Gr1M/SRWNAk/KXdy9NjZyrcAAZPBdUWpweGWBLEy7sVzhNAM2FUska0yIPoK khZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725659937; x=1726264737; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fFjzAfcWlZGdLMpuzA2ujYuM7IQsh3EHmDmz8Nw4EAk=; b=TCYAAjIOpxpgpLlMf73/aDCWpSll7sCKVt1MuOAFWFkbX0hHfO/yTVWUiMW19dk+5b 3ikDdudsbzwpMimFk6w2SONG+fqjdLkbLbGim7xaFkqHzTFc63ibGsJtxYickqL9QRXb kTYCmj2RIq202YDN6h784Kl9hJIPbrJWjNZAQk3r+qPu11XiRl1ZAnGvQ1vRb3FXnSxi VOQksgo41aQylBMcP1swTJE9cOLL+8vJOuz4GkeYvCZdoy1Zj4Xm7LxjfESFaAPu8ayE Vbr8SWy92CUsuk7nrvbdMu1k+PokHhRl23mkE3iTS988hvo0Qy7FHyqD5lXtFOyiMvuT PjPg== X-Forwarded-Encrypted: i=1; AJvYcCULiL4gmZ4eka40Pyw6HpHjpqB4NKVqx6nbJCl8SlA+rCYkOUQKj/HVQ3Q7sH0kzaH6PLDVlbxACQ==@kvack.org X-Gm-Message-State: AOJu0Ywmf1zLUxTbgXIaN7TI1T2q/76IHfBFn4ViU0H4Q/L6AmFlhiBb xqy/k8OqsPx+cppMguxBevmMCRWnaMm/kqC7q9ndTMhGywJTO/+P X-Google-Smtp-Source: AGHT+IGUtSJX3ipBuRp9WBTDy3e7+ahjOPjujOomQp1mfFxRuQsnbQauT+2gnJ0gJeWh/ZyqvG9yAQ== X-Received: by 2002:a05:6a20:d524:b0:1ce:d483:fc4d with SMTP id adf61e73a8af0-1cf2acd74eemr496435637.32.1725659937003; Fri, 06 Sep 2024 14:58:57 -0700 (PDT) Received: from Barrys-MBP.hub ([2407:7000:8942:5500:f164:3736:44c7:42e5]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7178b5660adsm3456275b3a.89.2024.09.06.14.58.47 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 06 Sep 2024 14:58:56 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: yosryahmed@google.com Cc: 21cnbao@gmail.com, akpm@linux-foundation.org, baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hanchuanhua@oppo.com, hannes@cmpxchg.org, hch@infradead.org, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, mhocko@suse.com, minchan@kernel.org, nphamcs@gmail.com, ryan.roberts@arm.com, ryncsn@gmail.com, senozhatsky@chromium.org, shakeel.butt@linux.dev, shy828301@gmail.com, surenb@google.com, usamaarif642@gmail.com, v-songbaohua@oppo.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com Subject: Re: [PATCH v8 1/3] mm: Fix swap_read_folio_zeromap() for large folios with partial zeromap Date: Sat, 7 Sep 2024 09:58:42 +1200 Message-Id: <20240906215842.1603-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 890E040004 X-Stat-Signature: amjpn4gcwmxtabxrp3xfcuir9xn94pdz X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1725659938-893674 X-HE-Meta: U2FsdGVkX1+0CgZVYobAVrowFfqDXngOF5oCz84u7mAh2MpAC3dcRm1WXM3MwZgijy8ZcQ2dFLl/JDQDgdTyUmTliGbAQnOBAa8dPfBA62LeBQeIjtSsapojbfrUmtghae6NrOSVDah+TTT1zPGJ+s4IBxlcBaeWbdpIYDffcjV5GDx4t4xfpZUp4Mahd9qqqONg54+WrAYDegFL1N0sm1NAu83T+86WL4vDnfR2+z1F+t8RF5BvV2Wa8qZL87ICrI2e39BcecHpXGQUEhxiuzJJ97NJuUjaZcF9vSKn0PFH2YEGjGUuOT2zWSDtyTz03nrjAAFzpDXi9Qa3ZNntsaA3siHM2n331rfzddP6EaSQ6DYnfVBxL9t2nW6NiEZWqEZ1hLKTJf1pf1hgdgO8GXmPEfd+YOb8/k0D3CYopYW5Coa02Zu8HP9yGGysikEYEGjmWA+4llMFpxhMJ+9LnkeTUMMT5F3WdDopA3ZqZvFet/7MmtosgVZSO/ZzgdtpxhBm9k6ELmx7y6kudou4Owy2dx4TiAOCuVuH1wRs1StBXyObnUKCbTYI+EBWfkxDy1DovM6X1MVBEnvMc9q9P+6ssK21Yj3TDaQLD40pEQWlU99UrrzuGRBDdHHfpgnrtFMwufYXMGIBpRuiFC1rYb7x9gqoOlSk0FyOw4Is+1GGvMFxBEO5XZxEDUYwuHajlJgSUDLnUfxsW+XU+yfU2wKJZqMQaNe9tYF9plfpGLdannGFFkXJIZSVrVK11w7/R+/GbPWx/hX1dOykngQnnkovM7F8skMv746AArmnMGLATTPwxHSQzc7GeVR0EapJDAySJJ/JQeu2Zo3aUx/J5EwEALrZ8Zqo+j+GR/fpYJd70twSYOTsXtwdTdPmznwyqrBI86Cbwo8YkpKJ3WKKtb7PdD21LM08kp9WEy2d3ozqfY3aFCPjS09AWONcxyEKXrR/+i86yPVtVI8Yzvm F+zXS5Dd 3LyxlBlaq5G7mVN7pzRCc2N28Pp8fAYYOvMv5tQPqqo6YZoAcQp/ZdzEFyDuEuJ2Q0Y1VPh2Thl7Jebc/p5b1ee/yeC9Ovx6c7hEz3k5SDwDer2tOLU6VUSDCjthSiq2/z5G5Wqy4mOnB1ApQNd1NeTI8a+CZBM27x+qTGdkOIaXn8V9e1P9tHqn7G5XhtHpImGjI+GvssfrJwXEROhFmQUxSviHaJinLaasXCTTq+ASAZGwseRjdU6scrkOZ+zElCcyFnTXV0OFpBkprFOmgzWLKqzoj30aR6DQjOeMuiWz+eLVrMtwX7ZaQXvhZQtlJ4EE/uJRdCiI9qDWIUGO/tOVDb8Jnb5oPG2CMK8YRVL1QMjmhn5lD4n36qxlAp7qO35r2/1mBoLg2X1WXKEUTBHVfx7BHD71j5jLONkqEebSBY1xZQfhPMFz/wDc00lAbhQvo6k/qmFtTEvYkw/zzKSLQFn9aelzgOA+sU6+HumgskIFtqqNN6xNxxL0YgXewdMGrXlYHMOBHA6M= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, Sep 7, 2024 at 6:32 AM Yosry Ahmed wrote: > > On Thu, Sep 5, 2024 at 5:11 PM Barry Song <21cnbao@gmail.com> wrote: > > > > From: Barry Song > > > > There could be a corner case where the first entry is non-zeromap, > > but a subsequent entry is zeromap. In this case, we should not > > let swap_read_folio_zeromap() return false since we will still > > read corrupted data. > > > > Additionally, the iteration of test_bit() is unnecessary and > > can be replaced with bitmap operations, which are more efficient. > > > > We can adopt the style of swap_pte_batch() and folio_pte_batch() to > > introduce swap_zeromap_batch() which seems to provide the greatest > > flexibility for the caller. This approach allows the caller to either > > check if the zeromap status of all entries is consistent or determine > > the number of contiguous entries with the same status. > > > > Since swap_read_folio() can't handle reading a large folio that's > > partially zeromap and partially non-zeromap, we've moved the code > > to mm/swap.h so that others, like those working on swap-in, can > > access it. > > > > Fixes: 0ca0c24e3211 ("mm: store zero pages to be swapped out in a bitmap") > > Cc: Usama Arif > > Cc: Yosry Ahmed > > Signed-off-by: Barry Song > > --- > >  mm/page_io.c | 32 +++++++------------------------- > >  mm/swap.h    | 33 +++++++++++++++++++++++++++++++++ > >  2 files changed, 40 insertions(+), 25 deletions(-) > > > > diff --git a/mm/page_io.c b/mm/page_io.c > > index 4bc77d1c6bfa..2dfe2273a1f1 100644 > > --- a/mm/page_io.c > > +++ b/mm/page_io.c > > @@ -226,26 +226,6 @@ static void swap_zeromap_folio_clear(struct folio *folio) > >         } > >  } > > > > -/* > > - * Return the index of the first subpage which is not zero-filled > > - * according to swap_info_struct->zeromap. > > - * If all pages are zero-filled according to zeromap, it will return > > - * folio_nr_pages(folio). > > - */ > > -static unsigned int swap_zeromap_folio_test(struct folio *folio) > > -{ > > -       struct swap_info_struct *sis = swp_swap_info(folio->swap); > > -       swp_entry_t entry; > > -       unsigned int i; > > - > > -       for (i = 0; i < folio_nr_pages(folio); i++) { > > -               entry = page_swap_entry(folio_page(folio, i)); > > -               if (!test_bit(swp_offset(entry), sis->zeromap)) > > -                       return i; > > -       } > > -       return i; > > -} > > - > >  /* > >   * We may have stale swap cache pages in memory: notice > >   * them here and get rid of the unnecessary final write. > > @@ -524,19 +504,21 @@ static void sio_read_complete(struct kiocb *iocb, long ret) > > > >  static bool swap_read_folio_zeromap(struct folio *folio) > >  { > > -       unsigned int idx = swap_zeromap_folio_test(folio); > > - > > -       if (idx == 0) > > -               return false; > > +       int nr_pages = folio_nr_pages(folio); > > +       bool is_zeromap; > > +       int nr_zeromap = swap_zeromap_batch(folio->swap, nr_pages, &is_zeromap); > > swap_zeromap_batch() reads to me like the number of entries that are > in the zeromap (i.e. bits are set), not the number of contiguous equal > bits. I can't think of a better name though :/ We now have swap_pte_batch() and folio_pte_batch(), both of which return the number of entries sharing the same attribute as the first entry. These functions are frequently used in the memory management code with conditions like if (swap_pte_batch() != nr) and if (folio_pte_batch() != nr). Given this, it seems we could adopt a consistent approach for handling entries in the same manner as the first one :-) > > The local variable is not adding much value here either. It's > reinforcing the misunderstanding I point out above, if anything. You > can just drop that. > well, I feel I can remove this local variable by: diff --git a/mm/page_io.c b/mm/page_io.c index 2dfe2273a1f1..bc1183299a7d 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -506,14 +506,14 @@ static bool swap_read_folio_zeromap(struct folio *folio) { int nr_pages = folio_nr_pages(folio); bool is_zeromap; - int nr_zeromap = swap_zeromap_batch(folio->swap, nr_pages, &is_zeromap); /* * Swapping in a large folio that is partially in the zeromap is not * currently handled. Return true without marking the folio uptodate so * that an IO error is emitted (e.g. do_swap_page() will sigbus). */ - if (WARN_ON_ONCE(nr_zeromap != nr_pages)) + if (WARN_ON_ONCE(swap_zeromap_batch(folio->swap, nr_pages, + &is_zeromap) != nr_pages)) return true; if (!is_zeromap) > > > >         /* > >          * Swapping in a large folio that is partially in the zeromap is not > >          * currently handled. Return true without marking the folio uptodate so > >          * that an IO error is emitted (e.g. do_swap_page() will sigbus). > >          */ > > -       if (WARN_ON_ONCE(idx < folio_nr_pages(folio))) > > +       if (WARN_ON_ONCE(nr_zeromap != nr_pages)) > >                 return true; > > > > +       if (!is_zeromap) > > +               return false; > > + > >         folio_zero_range(folio, 0, folio_size(folio)); > >         folio_mark_uptodate(folio); > >         return true; > > diff --git a/mm/swap.h b/mm/swap.h > > index f8711ff82f84..1cc56a02fb5f 100644 > > --- a/mm/swap.h > > +++ b/mm/swap.h > > @@ -80,6 +80,32 @@ static inline unsigned int folio_swap_flags(struct folio *folio) > >  { > >         return swp_swap_info(folio->swap)->flags; > >  } > > + > > +/* > > + * Return the count of contiguous swap entries that share the same > > + * zeromap status as the starting entry. If is_zeromap is not NULL, > > + * it will return the zeromap status of the starting entry. > > + */ > > +static inline int swap_zeromap_batch(swp_entry_t entry, int max_nr, > > +               bool *is_zeromap) > > +{ > > +       struct swap_info_struct *sis = swp_swap_info(entry); > > +       unsigned long start = swp_offset(entry); > > +       unsigned long end = start + max_nr; > > +       bool start_entry_zeromap; > > + > > +       start_entry_zeromap = test_bit(start, sis->zeromap); > > first_bit is probably a better name. > yep, might be. I am glad to rename if it makes the code easier to understand: diff --git a/mm/swap.h b/mm/swap.h index 1cc56a02fb5f..e0397a197620 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -92,15 +92,15 @@ static inline int swap_zeromap_batch(swp_entry_t entry, int max_nr, struct swap_info_struct *sis = swp_swap_info(entry); unsigned long start = swp_offset(entry); unsigned long end = start + max_nr; - bool start_entry_zeromap; + bool first_bit; - start_entry_zeromap = test_bit(start, sis->zeromap); - if (is_zeromap) - *is_zeromap = start_entry_zeromap; + first_bit = test_bit(start, sis->zeromap); + if (is_zeromap) + *is_zeromap = first_bit; if (max_nr <= 1) return max_nr; - if (start_entry_zeromap) + if (first_bit) return find_next_zero_bit(sis->zeromap, end, start) - start; else return find_next_bit(sis->zeromap, end, start) - start; > > +       if (is_zeromap) > > +               *is_zeromap = start_entry_zeromap; > > + > > +       if (max_nr <= 1) > > +               return max_nr; > > +       if (start_entry_zeromap) > > +               return find_next_zero_bit(sis->zeromap, end, start) - start; > > +       else > > +               return find_next_bit(sis->zeromap, end, start) - start; > > The usage of these functions look correct to me, although > FIND_NEXT_BIT is not really easy for me to parse :) Yep :-) with the above two changes, the patch becomes: >From 272c04cb758b8062eaa96a52b855ff79c8afdf6a Mon Sep 17 00:00:00 2001 From: Barry Song Date: Thu, 5 Sep 2024 11:56:03 +1200 Subject: [PATCH v8 1/3] mm: Fix swap_read_folio_zeromap() for large folios with partial zeromap There could be a corner case where the first entry is non-zeromap, but a subsequent entry is zeromap. In this case, we should not let swap_read_folio_zeromap() return false since we will still read corrupted data. Additionally, the iteration of test_bit() is unnecessary and can be replaced with bitmap operations, which are more efficient. We can adopt the style of swap_pte_batch() and folio_pte_batch() to introduce swap_zeromap_batch() which seems to provide the greatest flexibility for the caller. This approach allows the caller to either check if the zeromap status of all entries is consistent or determine the number of contiguous entries with the same status. Since swap_read_folio() can't handle reading a large folio that's partially zeromap and partially non-zeromap, we've moved the code to mm/swap.h so that others, like those working on swap-in, can access it. Fixes: 0ca0c24e3211 ("mm: store zero pages to be swapped out in a bitmap") Cc: Usama Arif Cc: Yosry Ahmed Signed-off-by: Barry Song --- mm/page_io.c | 32 +++++++------------------------- mm/swap.h | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 25 deletions(-) diff --git a/mm/page_io.c b/mm/page_io.c index 4bc77d1c6bfa..bc1183299a7d 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -226,26 +226,6 @@ static void swap_zeromap_folio_clear(struct folio *folio) } } -/* - * Return the index of the first subpage which is not zero-filled - * according to swap_info_struct->zeromap. - * If all pages are zero-filled according to zeromap, it will return - * folio_nr_pages(folio). - */ -static unsigned int swap_zeromap_folio_test(struct folio *folio) -{ - struct swap_info_struct *sis = swp_swap_info(folio->swap); - swp_entry_t entry; - unsigned int i; - - for (i = 0; i < folio_nr_pages(folio); i++) { - entry = page_swap_entry(folio_page(folio, i)); - if (!test_bit(swp_offset(entry), sis->zeromap)) - return i; - } - return i; -} - /* * We may have stale swap cache pages in memory: notice * them here and get rid of the unnecessary final write. @@ -524,19 +504,21 @@ static void sio_read_complete(struct kiocb *iocb, long ret) static bool swap_read_folio_zeromap(struct folio *folio) { - unsigned int idx = swap_zeromap_folio_test(folio); - - if (idx == 0) - return false; + int nr_pages = folio_nr_pages(folio); + bool is_zeromap; /* * Swapping in a large folio that is partially in the zeromap is not * currently handled. Return true without marking the folio uptodate so * that an IO error is emitted (e.g. do_swap_page() will sigbus). */ - if (WARN_ON_ONCE(idx < folio_nr_pages(folio))) + if (WARN_ON_ONCE(swap_zeromap_batch(folio->swap, nr_pages, + &is_zeromap) != nr_pages)) return true; + if (!is_zeromap) + return false; + folio_zero_range(folio, 0, folio_size(folio)); folio_mark_uptodate(folio); return true; diff --git a/mm/swap.h b/mm/swap.h index f8711ff82f84..e0397a197620 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -80,6 +80,32 @@ static inline unsigned int folio_swap_flags(struct folio *folio) { return swp_swap_info(folio->swap)->flags; } + +/* + * Return the count of contiguous swap entries that share the same + * zeromap status as the starting entry. If is_zeromap is not NULL, + * it will return the zeromap status of the starting entry. + */ +static inline int swap_zeromap_batch(swp_entry_t entry, int max_nr, + bool *is_zeromap) +{ + struct swap_info_struct *sis = swp_swap_info(entry); + unsigned long start = swp_offset(entry); + unsigned long end = start + max_nr; + bool first_bit; + + first_bit = test_bit(start, sis->zeromap); + if (is_zeromap) + *is_zeromap = first_bit; + + if (max_nr <= 1) + return max_nr; + if (first_bit) + return find_next_zero_bit(sis->zeromap, end, start) - start; + else + return find_next_bit(sis->zeromap, end, start) - start; +} + #else /* CONFIG_SWAP */ struct swap_iocb; static inline void swap_read_folio(struct folio *folio, struct swap_iocb **plug) @@ -171,6 +197,13 @@ static inline unsigned int folio_swap_flags(struct folio *folio) { return 0; } + +static inline int swap_zeromap_batch(swp_entry_t entry, int max_nr, + bool *has_zeromap) +{ + return 0; +} + #endif /* CONFIG_SWAP */ #endif /* _MM_SWAP_H */ -- 2.34.1 Thanks Barry