From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9254E6FE3D for ; Fri, 6 Sep 2024 18:32:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DA6A06B007B; Fri, 6 Sep 2024 14:32:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D56346B0083; Fri, 6 Sep 2024 14:32:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF6BC6B0085; Fri, 6 Sep 2024 14:32:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A19596B007B for ; Fri, 6 Sep 2024 14:32:47 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0B658140EDA for ; Fri, 6 Sep 2024 18:32:47 +0000 (UTC) X-FDA: 82535159574.20.40CF708 Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) by imf28.hostedemail.com (Postfix) with ESMTP id 2BB8BC0003 for ; Fri, 6 Sep 2024 18:32:44 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=VoMTFWjQ; spf=pass (imf28.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.46 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725647466; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/xZR5pZidPrjdEX8X4OXt++VFgXgiFZPMi5tniCrHIM=; b=nCtAY8PQoQAPk++ydasBIWvpg219TzFcWPyst1Paw+mXzwmv/jOz0THDkGu+po4PZdkXBt z5lEmZqMAUPD7c1TE1sOMUfIsoS5qqQydKbBVG+8TLQ4ncts4r80OTFF8neZFLgGyzUreL bIvwoYi0qOfRK3wJM35XY23DfNSHC6U= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725647466; a=rsa-sha256; cv=none; b=liaalqIf5ViShXbKW5N58Kwv+QZd18/SWu71+2MHZVm2ToeN3tMtytvrXESY5Zritw5qVt bnDIhDA8vVlVA/sju8rmM8Zjz7WSZVFsgU5mIIl4pcLKpeGMd8eZXmbiGR/s4D6rVoFxo0 Bk0R8ZpV2Gkgre0PVpLTrVCWf5q2/MA= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=VoMTFWjQ; spf=pass (imf28.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.46 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-ej1-f46.google.com with SMTP id a640c23a62f3a-a8d0d0aea3cso71179566b.3 for ; Fri, 06 Sep 2024 11:32:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725647563; x=1726252363; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=/xZR5pZidPrjdEX8X4OXt++VFgXgiFZPMi5tniCrHIM=; b=VoMTFWjQ6/C4KxdKxK6PdIbneTyftSNBfOFsZEDQcqRIrEXyR2HfaOwZz8Zt+RsYLN AW7FiRraefo/nanOThMjbFxAiHVPwpHXramDc1MSGNA0Qm2TgQaOvRjCNSDvI18EEaeQ i+SkgsP4KD2dlTGlYAOZ/b2S+UJ5lPyp5IEXAKDvZRMfDoLhomCAFtdISRkZwDhRqL0L 4d21/LpSL0tgnHCtJ5n3aYZeaT52IJVF66NfUghpKUbiHwXutHEOe5Yu6+lwDNfEsEy4 NJsEnjt02LIGmyWkOWK6vnzk44fUtXbzmFw/+ykWJ7jVCohckceRY6A+rGFE8KvsFzMJ Jh3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725647563; x=1726252363; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/xZR5pZidPrjdEX8X4OXt++VFgXgiFZPMi5tniCrHIM=; b=Ze5YTK4b2qJDfilaaQ06FTfC/XEROQFoWGN2l2HI1vHj8qXkuEZK5hxBnYjVen9hkB 04DBZksTT9RA9vAzHrzYF1G7EUnEK6hyc27D/OWLzBKdHBYIziecneNuliTBS0QjBlkr /AP0onfGzlJD3aX1bTcdTbrQBqYfMqVqFGlXn4IynZgMcp2+jEwTNgS9gmR5r0M4CFqV khLzpWgify5Q23hcrrLYUz4aMCjdAGOe//jG6NwfhyduWrS3tD+j9HkG5bMhF959C5l4 lSvgW/HMvXziRw+jgnY+IbJ8gbNz3k/n0N8BDxJOsgrp0jpqWCY0xwfX5NLt/3hrdCvp vkyg== X-Forwarded-Encrypted: i=1; AJvYcCWIGTWWFtkZAA+eE6ji8aXLfSI+3U59B7gPhqsUK1Igi0JqY9Kog6y6zQsC2ENEAJxZPN/OIgzMbQ==@kvack.org X-Gm-Message-State: AOJu0YyckIoGpCsilocmCGnFk4R7h1xJ56YzdGGlrQ7x1gScgUMhzspR yXBa+d//FhKBgUAfgdK4folTEzjQsRIsakIR3EOdFUaW2OYPNv8p24TD95cQb1tMrVHmAwIuQDC pm0S4Ge6y7xTCxzk1vkWPIya3vDdekhFPJtt5 X-Google-Smtp-Source: AGHT+IEOp32+s6cAncKwanaXicGS4uaeek9IlbcC59IcvMCUON/lvRWwH4y+J7v1GgByoXLuw4V5pVULf5mqMBnDTZw= X-Received: by 2002:a17:907:36c7:b0:a86:ab90:6af6 with SMTP id a640c23a62f3a-a8a885f9ca6mr259888966b.16.1725647562713; Fri, 06 Sep 2024 11:32:42 -0700 (PDT) MIME-Version: 1.0 References: <20240906001047.1245-1-21cnbao@gmail.com> <20240906001047.1245-2-21cnbao@gmail.com> In-Reply-To: <20240906001047.1245-2-21cnbao@gmail.com> From: Yosry Ahmed Date: Fri, 6 Sep 2024 11:32:06 -0700 Message-ID: Subject: Re: [PATCH v8 1/3] mm: Fix swap_read_folio_zeromap() for large folios with partial zeromap To: Barry Song <21cnbao@gmail.com> Cc: akpm@linux-foundation.org, linux-mm@kvack.org, hanchuanhua@oppo.com, baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hannes@cmpxchg.org, hch@infradead.org, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, mhocko@suse.com, minchan@kernel.org, nphamcs@gmail.com, ryan.roberts@arm.com, ryncsn@gmail.com, senozhatsky@chromium.org, shakeel.butt@linux.dev, shy828301@gmail.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com, Usama Arif Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 2BB8BC0003 X-Stat-Signature: dz33poa5o6xj15pyncey1oh7mhx957hc X-HE-Tag: 1725647564-574977 X-HE-Meta: U2FsdGVkX197Tz3KG5danTqYIfYZRhTudnx/+u0g3KOZ78kQx0qHAsEd9PSldS1S5GBB11qNWf3myX71C4i3bRhjKDwCrw7W6PjHn3Ump/R0SlqvbckwOieZv5ExEcBQOQ1njWdjACMa1l/TKfifPuBGySFXDI+ByLqUHgXWaaxTgClySCI4zDls4ktTFQGc5pfK+ppm9jI1FgcQRrUE4i0Y86g9PRgsalbNBTSUlYcGqr7G1mTtEfUa0IDkIRNxCiLhh6pJ2m72PtiSdDgSW9EXGJKAikn9HHQT0qOU+UROMXIbI4MttTzQbDIEAJZCtYhjfzsWz0w5ptmwOzihUL/JZgq3j5V2QldKXeoUioL6hsnZGMSfwcpfunukddgRFrod65ZVIXBXTpHCNxI+JV+mzLIVV2KPAnJz/F67F58pZgoDZXqDXxVl/PkPF2T18p19sNNuIksIWAAu3sxLfsT2FpBFSwdF6mewE0Tbl3yWrB85kOjip9bAEw0QBbnb7jhOvxwHAhpH2nMIQtdeRS2+Z+8KPpZmA+KPoq80tHsgaYUm2f4HE3PnEAsD7WSsvVF/uRYk0d62mUzcMN2YHX/wcFqBx2KnvGB8kFcj3NrVjnkpMBtiH2tZIte1a5EDozkEZObfBLa+4WGMhu7d8yVsDRt593i7RUZDuuTexh8sGKnZ8ccYOC/7v3ksq1NL0PeMz8z+xMj+pkA0M/oHxtiKxI+w1x/DHcKhNt/0+URALaqr9Pug8JXxCYA8V26yAxepu4EIh1Zyx+xsOA7FnSgrnTWX0cG53ALr+aEM7/HL9SG94eL9M8OCZ+Q3A+3ifYr6VkpDp9FWbVmOlCQgRFyoycjtPwRyDi+wa60LifSLQIjiK8MFdbbmx57L7IOrVbP7Lxkf4fXM7fGbdikPbNjc8wkxIRVOK86s54glCDd0DDJOmDINAdr1w+Dh7ifc2xMr1LMQYnyQaVqfO6w UgUsULFY kHKzDEkvoZPaUe9HEIdjha9eMwxh+tcieHWDg+b2M+Vgq1eBw8aS+/qtAxS6u0yH7d3QpuLtNbRutYE7TdBnlLSMB0xL4pytz0aBLb+2G2ZlCPhHMUdufJMdmQpgYDWcDxK3PIwY0MvSGMovOuqRB0uedtZ2COuunCjYeJTdUehao14KeSyBZ+tlpvi6VtEV2xZLDmn0by1eV3vg50/UCTjjtI5GlXCmLJvzfIEf1iAOFnynCWWhXy3zGxfMo0c3RNJGYqznV48q4Gpjd4S/tyHAaL1yxgVzG+GFRa3XoqAuuV5W5/18uyPJl/w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Sep 5, 2024 at 5:11=E2=80=AFPM Barry Song <21cnbao@gmail.com> wrote= : > > From: Barry Song > > There could be a corner case where the first entry is non-zeromap, > but a subsequent entry is zeromap. In this case, we should not > let swap_read_folio_zeromap() return false since we will still > read corrupted data. > > Additionally, the iteration of test_bit() is unnecessary and > can be replaced with bitmap operations, which are more efficient. > > We can adopt the style of swap_pte_batch() and folio_pte_batch() to > introduce swap_zeromap_batch() which seems to provide the greatest > flexibility for the caller. This approach allows the caller to either > check if the zeromap status of all entries is consistent or determine > the number of contiguous entries with the same status. > > Since swap_read_folio() can't handle reading a large folio that's > partially zeromap and partially non-zeromap, we've moved the code > to mm/swap.h so that others, like those working on swap-in, can > access it. > > Fixes: 0ca0c24e3211 ("mm: store zero pages to be swapped out in a bitmap"= ) > Cc: Usama Arif > Cc: Yosry Ahmed > Signed-off-by: Barry Song > --- > mm/page_io.c | 32 +++++++------------------------- > mm/swap.h | 33 +++++++++++++++++++++++++++++++++ > 2 files changed, 40 insertions(+), 25 deletions(-) > > diff --git a/mm/page_io.c b/mm/page_io.c > index 4bc77d1c6bfa..2dfe2273a1f1 100644 > --- a/mm/page_io.c > +++ b/mm/page_io.c > @@ -226,26 +226,6 @@ static void swap_zeromap_folio_clear(struct folio *f= olio) > } > } > > -/* > - * Return the index of the first subpage which is not zero-filled > - * according to swap_info_struct->zeromap. > - * If all pages are zero-filled according to zeromap, it will return > - * folio_nr_pages(folio). > - */ > -static unsigned int swap_zeromap_folio_test(struct folio *folio) > -{ > - struct swap_info_struct *sis =3D swp_swap_info(folio->swap); > - swp_entry_t entry; > - unsigned int i; > - > - for (i =3D 0; i < folio_nr_pages(folio); i++) { > - entry =3D page_swap_entry(folio_page(folio, i)); > - if (!test_bit(swp_offset(entry), sis->zeromap)) > - return i; > - } > - return i; > -} > - > /* > * We may have stale swap cache pages in memory: notice > * them here and get rid of the unnecessary final write. > @@ -524,19 +504,21 @@ static void sio_read_complete(struct kiocb *iocb, l= ong ret) > > static bool swap_read_folio_zeromap(struct folio *folio) > { > - unsigned int idx =3D swap_zeromap_folio_test(folio); > - > - if (idx =3D=3D 0) > - return false; > + int nr_pages =3D folio_nr_pages(folio); > + bool is_zeromap; > + int nr_zeromap =3D swap_zeromap_batch(folio->swap, nr_pages, &is_= zeromap); swap_zeromap_batch() reads to me like the number of entries that are in the zeromap (i.e. bits are set), not the number of contiguous equal bits. I can't think of a better name though :/ The local variable is not adding much value here either. It's reinforcing the misunderstanding I point out above, if anything. You can just drop that. > > /* > * Swapping in a large folio that is partially in the zeromap is = not > * currently handled. Return true without marking the folio uptod= ate so > * that an IO error is emitted (e.g. do_swap_page() will sigbus). > */ > - if (WARN_ON_ONCE(idx < folio_nr_pages(folio))) > + if (WARN_ON_ONCE(nr_zeromap !=3D nr_pages)) > return true; > > + if (!is_zeromap) > + return false; > + > folio_zero_range(folio, 0, folio_size(folio)); > folio_mark_uptodate(folio); > return true; > diff --git a/mm/swap.h b/mm/swap.h > index f8711ff82f84..1cc56a02fb5f 100644 > --- a/mm/swap.h > +++ b/mm/swap.h > @@ -80,6 +80,32 @@ static inline unsigned int folio_swap_flags(struct fol= io *folio) > { > return swp_swap_info(folio->swap)->flags; > } > + > +/* > + * Return the count of contiguous swap entries that share the same > + * zeromap status as the starting entry. If is_zeromap is not NULL, > + * it will return the zeromap status of the starting entry. > + */ > +static inline int swap_zeromap_batch(swp_entry_t entry, int max_nr, > + bool *is_zeromap) > +{ > + struct swap_info_struct *sis =3D swp_swap_info(entry); > + unsigned long start =3D swp_offset(entry); > + unsigned long end =3D start + max_nr; > + bool start_entry_zeromap; > + > + start_entry_zeromap =3D test_bit(start, sis->zeromap); first_bit is probably a better name. > + if (is_zeromap) > + *is_zeromap =3D start_entry_zeromap; > + > + if (max_nr <=3D 1) > + return max_nr; > + if (start_entry_zeromap) > + return find_next_zero_bit(sis->zeromap, end, start) - sta= rt; > + else > + return find_next_bit(sis->zeromap, end, start) - start; The usage of these functions look correct to me, although FIND_NEXT_BIT is not really easy for me to parse :)