From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E2B4EE49A4 for ; Tue, 22 Aug 2023 17:00:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B64D028004C; Tue, 22 Aug 2023 13:00:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AEE19280046; Tue, 22 Aug 2023 13:00:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9681928004C; Tue, 22 Aug 2023 13:00:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 81581280046 for ; Tue, 22 Aug 2023 13:00:46 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 29B661A0412 for ; Tue, 22 Aug 2023 17:00:46 +0000 (UTC) X-FDA: 81152354892.15.FF5B4E9 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by imf27.hostedemail.com (Postfix) with ESMTP id D6F1240021 for ; Tue, 22 Aug 2023 17:00:42 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=4jRci1y6; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692723643; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JZUbfAzYGz3v36gjFaurJR1ubEKPGQnTkTa8Ar2a45A=; b=5IWxu3mbBXFA+22yrjtAl+cOFSucUdtCXbeSvrxnWvBE+a8cNLq373AgZ3/Q3s7grCNgmj qrfvupIWSjB/q01Uew/4ckhYYkVDPS89CI6zaYraJ8+UKfG+M4zH077sl2Ar0MNgMFrQVn pv08UKjtFzUB7qV7+omx2k0hu4keEUk= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=4jRci1y6; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692723643; a=rsa-sha256; cv=none; b=5DhFampBcRR+DsmhA1b0dKce2nqvc3fHzvPDbQl7rby/ph0m86yN7k/cpVRXsqGYKALmZN iUJ29V0qt0RlSyaEE7iM3OhbMVi0UAFlSR+WUCnixvio5zP/lg7W+PRvydYrAgp3Ig7EsI XQvB3Tcn4fSkEovfjS8EZ0pFKlIFK7g= Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-9a1869f2c06so602292866b.0 for ; Tue, 22 Aug 2023 10:00:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1692723641; x=1693328441; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=JZUbfAzYGz3v36gjFaurJR1ubEKPGQnTkTa8Ar2a45A=; b=4jRci1y6zV56tlbGQj++H4N/tMfMYeo2ZUkjZ14qcMQ7vkC9P2hTD4IPaFHUlD3AAh EvzpVCoSREUfiEYuVJKI0DzzAFJ/Pr/ms4uacWVN50slkdF20vA6qZby/UhzWdDYDERl 7+iYP+SgrPkRWdc4Y4v8gODontaJK2CNI9ng8TyirwsXb3SR/fz5tGCmDyl55RsqVQJQ vrQQVswoIUM2v7N5nkRoAmLjXYjy0XM328untXhHzz9Ix4QWUnGl224gZ8HNdsMD1+xH 5ZoIIjyyudAf+BtEd+S0I3hJl4/7PR9abCc/PbWNGHghKg58EqJcQRw4rfFVISjmtVC6 Oz8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692723641; x=1693328441; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JZUbfAzYGz3v36gjFaurJR1ubEKPGQnTkTa8Ar2a45A=; b=ZZOMNenXpbQ46hQMqiTaSjVJDRBuJoGEaeOvH7ezDlOb0fgY7ItdhCQQHaV/rF45yS w22ZUOqjqC8MJNmMH5qqk16WZ9Ofzzu83yjTZo+fyUxhLOxhLHFw2LJxjFfzMKvuu2cp xvA0KWBBpQzSmLpTxx+85DbA954IKUkRp9U45S/Lu41kwtkZ/4y5QDcdjopnXbuaabFp ptYVyN723AfwVDA/G4k5U6XyBfQbk9ahJ0GntM3hAHExMz5tcCOFEoh9ihwv0JBsn/5N rpQHOMFwGOVSG0k7v+Qcu3g1gzK5hOeU3ke2IuEQT1UfDLG+beKKU5bf3i+3BEfKX6E1 hFTg== X-Gm-Message-State: AOJu0YyVbKD9cTtnaj4Kt+rpqIEmbc7thW6Jjk4iwVmFF+9og0zEcxPx UuKqEj5jMbt7BcymMF4mIWf02k+cqtupRSpqtINsSw== X-Google-Smtp-Source: AGHT+IECpjSEhdUBwmCMsY3jOPY46f7tYrmZ9nJqd197KnPIyXeHnd1w5NuXfiXFAhAlTBsskMGCDfBvFk5nAZcvARY= X-Received: by 2002:a17:906:2249:b0:9a1:cbe5:758a with SMTP id 9-20020a170906224900b009a1cbe5758amr472967ejr.18.1692723641082; Tue, 22 Aug 2023 10:00:41 -0700 (PDT) MIME-Version: 1.0 References: <20230821160849.531668-1-david@redhat.com> <20230821160849.531668-2-david@redhat.com> In-Reply-To: <20230821160849.531668-2-david@redhat.com> From: Yosry Ahmed Date: Tue, 22 Aug 2023 10:00:04 -0700 Message-ID: Subject: Re: [PATCH mm-unstable v1 1/4] mm/swap: stop using page->private on tail pages for THP_SWAP To: David Hildenbrand Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Andrew Morton , Matthew Wilcox , Peter Xu , Catalin Marinas , Will Deacon , Hugh Dickins , Seth Jennings , Dan Streetman , Vitaly Wool Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D6F1240021 X-Stat-Signature: zhfjum6t5jcy7uo595xiriep6crotgq3 X-Rspam-User: X-HE-Tag: 1692723642-649426 X-HE-Meta: U2FsdGVkX1+f2i5cmRLubONmzNRtP+9xuzGkL9+wZso72gMTj7GCy2Qs5yc1Qe4WderE2mQpbV1UKn3UV8Eoh8JK6h0hB5n6DsE2Lnh4PHptA2Z3aMm/nDDCHS+jmtk0b0j03EKk6YrrEBDwe84dPh+p6WwY0Jd51+B5fTqXjSwC2MxD8bH24E4ikeUGjjmZDbgLfjTwgCRMPaP8rN2oQFzYucWsaPzqeUO2nqAnkyO/BGu99oE2BWK0q4YdwpkWHKPXdkdLd0UV/CGeMe04IQYafHXFoML/qr1BXnSt1mKKL1U9xawjbmvzGL+m8PH8VgLlSeuSa8l7UBGReqOjpwdnsCd34XS6Xtub52T0wQAj5PhWmoqbUWKyhh1j9T0P7k52J3evnK05sgMURC4tjrHAYFeugpkl/z6iLVedZagfZqogAtK7xbpGgzrsXr89LlvaQjHX9gLikyw0CXy88P7+5FiZltWrIL3yo+yDvGPJgKkOuEewN57lGWJygttZ58ardVmQURkBT2Esr/RUAc33MGjmUySeXmUkOIEHLYm9jGD4koH4YGzEDFEQvrvPBbN29w3YxH9iLqLXLEdQh7lAuZ9MLPb7c6HB+okYoyK6hFd7J61wQo6ebkGe+HejDF1QGZjEmpXg1K6TQVnVorOO8m0r10bw8bD9mxVSscWD6a199vI5jilgWQjxrBvmOD3exPD2I+N2IoCK+o4QHBTQpCTnWpAvu2AchPLETwLZD7R/X0g0uqRc78NBg2sEI/Wx0xbtGqQ8/Tw6O2A3EWkMGlZemwBo5vQltAgQ41KLo2FY8IuVUDMwPJydyxQqJF+c+ZZOR4LapTRJF5PtYw12X0v/T3lu2eLMJM8U6k7kZL0xTX4LPhZBQ2zZRoOnA99rhe1/jJ6FfbXlImsy2is+2sbzfmBRloqkHuAUPBoqYstEeZN3Vw+OXgb9SlfRMC81ZLyjkHboZS2V50e +ZCIeskB RTOjJiGDOy4VFOzR2TPcFeC9rKJ+APazW/I6jD+Qre3ZdELv/4m4Emffk4BpCf9Pwhj8BNIsp5HVyB4FUD/avlKsEox/7AX1GJ+6zF5jM2lRaGFnJe0ysaEjR3Qflc4BshLTKVzfYdGTbsJS+AbvX5JJMXBt+kCaPl7S+E3UXm/javlux1LfTjednTAn1NrtKrnO7Xl72k0uUdjUSch3/QZ/r/pHMTU1H5NydttXPWrF1rvifW9xvqAmok6rCcemW/YCublhRKwKNTHb/XGP0YM9FNqgVNRtYjCQWs8yQprHd0yg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Aug 21, 2023 at 9:09=E2=80=AFAM David Hildenbrand wrote: > > Let's stop using page->private on tail pages, making it possible to > just unconditionally reuse that field in the tail pages of large folios. > > The remaining usage of the private field for THP_SWAP is in the THP > splitting code (mm/huge_memory.c), that we'll handle separately later. > > Update the THP_SWAP documentation and sanity checks in mm_types.h and > __split_huge_page_tail(). > > Signed-off-by: David Hildenbrand > --- > arch/arm64/mm/mteswap.c | 5 +++-- > include/linux/mm_types.h | 12 +----------- > include/linux/swap.h | 9 +++++++++ > mm/huge_memory.c | 15 ++++++--------- > mm/memory.c | 2 +- > mm/rmap.c | 2 +- > mm/swap_state.c | 5 +++-- > mm/swapfile.c | 4 ++-- > 8 files changed, 26 insertions(+), 28 deletions(-) > > diff --git a/arch/arm64/mm/mteswap.c b/arch/arm64/mm/mteswap.c > index cd508ba80ab1..a31833e3ddc5 100644 > --- a/arch/arm64/mm/mteswap.c > +++ b/arch/arm64/mm/mteswap.c > @@ -33,8 +33,9 @@ int mte_save_tags(struct page *page) > > mte_save_page_tags(page_address(page), tag_storage); > > - /* page_private contains the swap entry.val set in do_swap_page *= / > - ret =3D xa_store(&mte_pages, page_private(page), tag_storage, GFP= _KERNEL); > + /* lookup the swap entry.val from the page */ > + ret =3D xa_store(&mte_pages, page_swap_entry(page).val, tag_stora= ge, > + GFP_KERNEL); > if (WARN(xa_is_err(ret), "Failed to store MTE tags")) { > mte_free_tag_storage(tag_storage); > return xa_err(ret); > diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h > index b9b6c88875b9..61361f1750c3 100644 > --- a/include/linux/mm_types.h > +++ b/include/linux/mm_types.h > @@ -333,11 +333,8 @@ struct folio { > atomic_t _pincount; > #ifdef CONFIG_64BIT > unsigned int _folio_nr_pages; > - /* 4 byte gap here */ > - /* private: the union with struct page is transitional */ > - /* Fix THP_SWAP to not use tail->private */ > - unsigned long _private_1; > #endif > + /* private: the union with struct page is transitional */ > }; > struct page __page_1; > }; > @@ -358,9 +355,6 @@ struct folio { > /* public: */ > struct list_head _deferred_list; > /* private: the union with struct page is transitional */ > - unsigned long _avail_2a; > - /* Fix THP_SWAP to not use tail->private */ > - unsigned long _private_2a; > }; > struct page __page_2; > }; > @@ -385,9 +379,6 @@ FOLIO_MATCH(memcg_data, memcg_data); > offsetof(struct page, pg) + sizeof(struct page)) > FOLIO_MATCH(flags, _flags_1); > FOLIO_MATCH(compound_head, _head_1); > -#ifdef CONFIG_64BIT > -FOLIO_MATCH(private, _private_1); > -#endif > #undef FOLIO_MATCH > #define FOLIO_MATCH(pg, fl) \ > static_assert(offsetof(struct folio, fl) =3D=3D = \ > @@ -396,7 +387,6 @@ FOLIO_MATCH(flags, _flags_2); > FOLIO_MATCH(compound_head, _head_2); > FOLIO_MATCH(flags, _flags_2a); > FOLIO_MATCH(compound_head, _head_2a); > -FOLIO_MATCH(private, _private_2a); > #undef FOLIO_MATCH > > /** > diff --git a/include/linux/swap.h b/include/linux/swap.h > index bb5adc604144..84fe0e94f5cd 100644 > --- a/include/linux/swap.h > +++ b/include/linux/swap.h > @@ -339,6 +339,15 @@ static inline swp_entry_t folio_swap_entry(struct fo= lio *folio) > return entry; > } > > +static inline swp_entry_t page_swap_entry(struct page *page) > +{ > + struct folio *folio =3D page_folio(page); > + swp_entry_t entry =3D folio_swap_entry(folio); > + > + entry.val +=3D page - &folio->page; Would it be better to use folio_page_idx() here? > + return entry; > +} > + > static inline void folio_set_swap_entry(struct folio *folio, swp_entry_t= entry) > { > folio->private =3D (void *)entry.val; > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > index cc2f65f8cc62..c04702ae71d2 100644 > --- a/mm/huge_memory.c > +++ b/mm/huge_memory.c > @@ -2446,18 +2446,15 @@ static void __split_huge_page_tail(struct page *h= ead, int tail, > page_tail->index =3D head->index + tail; > > /* > - * page->private should not be set in tail pages with the excepti= on > - * of swap cache pages that store the swp_entry_t in tail pages. > - * Fix up and warn once if private is unexpectedly set. > - * > - * What of 32-bit systems, on which folio->_pincount overlays > - * head[1].private? No problem: THP_SWAP is not enabled on 32-bi= t, and > - * pincount must be 0 for folio_ref_freeze() to have succeeded. > + * page->private should not be set in tail pages. Fix up and warn= once > + * if private is unexpectedly set. > */ > - if (!folio_test_swapcache(page_folio(head))) { > - VM_WARN_ON_ONCE_PAGE(page_tail->private !=3D 0, page_tail= ); > + if (unlikely(page_tail->private)) { > + VM_WARN_ON_ONCE_PAGE(true, page_tail); > page_tail->private =3D 0; > } > + if (PageSwapCache(head)) > + set_page_private(page_tail, (unsigned long)head->private = + tail); > > /* Page flags must be visible before we make the page non-compoun= d. */ > smp_wmb(); > diff --git a/mm/memory.c b/mm/memory.c > index d003076b218d..ff13242c1589 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -3882,7 +3882,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) > * changed. > */ > if (unlikely(!folio_test_swapcache(folio) || > - page_private(page) !=3D entry.val)) > + page_swap_entry(page).val !=3D entry.val)) > goto out_page; > > /* > diff --git a/mm/rmap.c b/mm/rmap.c > index 1f04debdc87a..ec7f8e6c9e48 100644 > --- a/mm/rmap.c > +++ b/mm/rmap.c > @@ -1647,7 +1647,7 @@ static bool try_to_unmap_one(struct folio *folio, s= truct vm_area_struct *vma, > */ > dec_mm_counter(mm, mm_counter(&folio->page)); > } else if (folio_test_anon(folio)) { > - swp_entry_t entry =3D { .val =3D page_private(sub= page) }; > + swp_entry_t entry =3D page_swap_entry(subpage); > pte_t swp_pte; > /* > * Store the swap location in the pte. > diff --git a/mm/swap_state.c b/mm/swap_state.c > index 01f15139b7d9..2f2417810052 100644 > --- a/mm/swap_state.c > +++ b/mm/swap_state.c > @@ -100,6 +100,7 @@ int add_to_swap_cache(struct folio *folio, swp_entry_= t entry, > > folio_ref_add(folio, nr); > folio_set_swapcache(folio); > + folio_set_swap_entry(folio, entry); > > do { > xas_lock_irq(&xas); > @@ -113,7 +114,6 @@ int add_to_swap_cache(struct folio *folio, swp_entry_= t entry, > if (shadowp) > *shadowp =3D old; > } > - set_page_private(folio_page(folio, i), entry.val = + i); > xas_store(&xas, folio); > xas_next(&xas); > } > @@ -154,9 +154,10 @@ void __delete_from_swap_cache(struct folio *folio, > for (i =3D 0; i < nr; i++) { > void *entry =3D xas_store(&xas, shadow); > VM_BUG_ON_PAGE(entry !=3D folio, entry); > - set_page_private(folio_page(folio, i), 0); > xas_next(&xas); > } > + entry.val =3D 0; > + folio_set_swap_entry(folio, entry); > folio_clear_swapcache(folio); > address_space->nrpages -=3D nr; > __node_stat_mod_folio(folio, NR_FILE_PAGES, -nr); > diff --git a/mm/swapfile.c b/mm/swapfile.c > index d46933adf789..bd9d904671b9 100644 > --- a/mm/swapfile.c > +++ b/mm/swapfile.c > @@ -3369,7 +3369,7 @@ struct swap_info_struct *swp_swap_info(swp_entry_t = entry) > > struct swap_info_struct *page_swap_info(struct page *page) > { > - swp_entry_t entry =3D { .val =3D page_private(page) }; > + swp_entry_t entry =3D page_swap_entry(page); > return swp_swap_info(entry); > } > > @@ -3384,7 +3384,7 @@ EXPORT_SYMBOL_GPL(swapcache_mapping); > > pgoff_t __page_file_index(struct page *page) > { > - swp_entry_t swap =3D { .val =3D page_private(page) }; > + swp_entry_t swap =3D page_swap_entry(page); > return swp_offset(swap); > } > EXPORT_SYMBOL_GPL(__page_file_index); > -- > 2.41.0 > >