From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DB78C04A6A for ; Wed, 16 Aug 2023 13:33:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D7BF280016; Wed, 16 Aug 2023 09:33:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 787BB8D0038; Wed, 16 Aug 2023 09:33:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65138280016; Wed, 16 Aug 2023 09:33:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 52E3D8D0038 for ; Wed, 16 Aug 2023 09:33:40 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 14BB5120F60 for ; Wed, 16 Aug 2023 13:33:40 +0000 (UTC) X-FDA: 81130060200.24.32A4863 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 372964000A for ; Wed, 16 Aug 2023 13:33:36 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=azXIyNg2; spf=pass (imf04.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692192817; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Sa0+x7EXRi4IXTjVM59CpmyQzx6tOI8j4jqQQJalqrw=; b=0+iCQ1kG6Ii/wlE+EOZsn2/43buPPaRO5wMIOXq7CnD5T2wuAj0URgwCzIh7t27tsq0TQJ Ral2zwg8bdapTOzmF5xECzUxCA42ah4iEV/7ZJPNLiWQoW/5G3/njuA+qURRqsQewtVz5u rAtiB4ZsgtHh0d7+0Xi3HiTqTzWOh0w= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692192817; a=rsa-sha256; cv=none; b=E6OuRHdPy0DqN5gehzhXDBCq95XJ778PZzdk0Q2PNsRpjJpR2+K2E5G2jjIcP14d0Enn/C IaQofkWfd9ugdVOGMfd50p9Qyx4JlZ3a0ahY0o+Mkfr17zKGknJZwPvi7Yp4zzTxkO+TFm sZv3r/BRkiHP4gHWlAnhT2Y6t4Fn1W4= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=azXIyNg2; spf=pass (imf04.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692192816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Sa0+x7EXRi4IXTjVM59CpmyQzx6tOI8j4jqQQJalqrw=; b=azXIyNg2K7vzgXODhdx6Xik2vM8Ge/WfavTaaeZJFrVGcLFgH/u+g5OvM4C5Ikos4MtVsG Kn+nKtUB2oyp+UilxGsPNJ5OCy2I1sivOi0Nls7XPVphrOaZNPWEPh+636MQ6nzSxYEmdB LasCJtdKCh8qjT435es+Z613UVQ5Qho= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-644-2YU4d3xLOL61waiqvIeYgA-1; Wed, 16 Aug 2023 09:33:34 -0400 X-MC-Unique: 2YU4d3xLOL61waiqvIeYgA-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3f42bcef2acso40103015e9.2 for ; Wed, 16 Aug 2023 06:33:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692192813; x=1692797613; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Sa0+x7EXRi4IXTjVM59CpmyQzx6tOI8j4jqQQJalqrw=; b=XlZlg4S0pO5G6yUWerOgMi9UhMD1bBt06Zan8l8Wd3ao0C5mSqUoIx3LvT3FUtAMFi r+ODWHcjTqWRZ79RV4oM/4ud9RP+JLQETg+1xQi9l4nqp8gOkLA8lh2vSXDXxc18IYeY Bzszd3wWpoiwESk/DEpU5Zsrk8nDkEesu+p3XGI9HnRGfXHjDhJymi7Tj/MWQssjbaxe QvZaVPBfk9Iofm+vizW4tdeb6vveq0ue8SIOvNBxS9JHViEiPJZoYPBxC98Oc07BqBOE eSNJ0ZwI9hfCEPbxPmGhzEo7S6JiMJpxjtlTR1/1yj7EegMFWwowoNr8Hr1hoFreZlfn zGHg== X-Gm-Message-State: AOJu0Yyq2OUzcjrONTY3cWcEwFZpmjtcTlrja1d/etL/Xn1jASW9flgC /m5d+lGkua52VFPRxeTEyDIHF9mOP47U9AdRkYpg3erQASaFmpLeRY2QOm4q7IfwpaUgGzRRFDL tXQ0KX71yZSI= X-Received: by 2002:a05:600c:3784:b0:3fb:dbd0:a7ea with SMTP id o4-20020a05600c378400b003fbdbd0a7eamr1465985wmr.37.1692192813395; Wed, 16 Aug 2023 06:33:33 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFjNJST7TXDwcOWPe2Ad/HweRVZ3L40JuHAZHXd+y6Yqt71pQM3nkvyNCJOZx/ehNZO6aLqqw== X-Received: by 2002:a05:600c:3784:b0:3fb:dbd0:a7ea with SMTP id o4-20020a05600c378400b003fbdbd0a7eamr1465966wmr.37.1692192812977; Wed, 16 Aug 2023 06:33:32 -0700 (PDT) Received: from ?IPV6:2003:cb:c74b:8b00:5520:fa3c:c527:592f? (p200300cbc74b8b005520fa3cc527592f.dip0.t-ipconnect.de. [2003:cb:c74b:8b00:5520:fa3c:c527:592f]) by smtp.gmail.com with ESMTPSA id i9-20020a05600c290900b003fe539b83f2sm24286719wmd.42.2023.08.16.06.33.31 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Aug 2023 06:33:32 -0700 (PDT) Message-ID: Date: Wed, 16 Aug 2023 15:33:30 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 To: Peter Xu , linux-kernel@vger.kernel.org, Matthew Wilcox , linux-mm@kvack.org Cc: Hugh Dickins , "Kirill A . Shutemov" , Randy Dunlap , Mike Kravetz , Yu Zhao , Ryan Roberts , Andrew Morton , Yang Shi References: <20230815212547.431693-1-peterx@redhat.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH RFC v3] mm: Proper document tail pages fields for folio In-Reply-To: <20230815212547.431693-1-peterx@redhat.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 372964000A X-Rspam-User: X-Stat-Signature: 7bhnc7ra9oban5fxqskn9yegy59awdg5 X-Rspamd-Server: rspam03 X-HE-Tag: 1692192816-988183 X-HE-Meta: U2FsdGVkX1+PifrIjIY7DUB7ePmeKl7nfwXyZRcufMQ1qMxJ1X6qgM0fj/mL3C8T8pt+kr2J55+HjWmwBD7zgQfHbjJPtlHd+6cTzpyZN35dHn4hBAxOrkUYA9vMI3wIGTzGT+Av2zDRG3njtVHw1m2gJkDh4vXzFNj1uUQzesoboRUuwUrk2yn0KUi6OuyvKS9wU5ZZwq7GHSJ5J8qJxhoZAF3GW+qbyjTO2sEJ4zWRXEr8MvquBSJjvftPleg8iCNRikAoxhGGt2AS7fuAf0R+f/6hV+h3xZQhnSI24tJI/dtOASUQckRcJlEvHYfDiO+NmW495WeZBGq8m7EymeAinIg1Zk7FwTyLxqtD8cum7cVCLVRZQ6ftpsU/72Q9iVXdzQObC4PRllERONUuFPt+tdhP4X7IIFkWaKV/0i1guhwJ3t5T7EyliAWbsoUChpjsm1cNK9e/wgPWlNxtU3DbyCepOL0C8oXre+Wxo/Cw6CGI59T/3SI27V8oASHtJcsD0eLJeJUKp6fJMtiqWKMVwwZg9tg7xiXltoUP/dZmx3gmiEaHulfonI/VVMKniWT/KtSkg3/XY8h8Qf1Vwlntixqv778R5iRXJteN5zx3NT+qttrpu9AXZaldKOejzOjSYcH7wSlkIA8z9Tzxgnpf41EnGs7azUTr+I1xx7JG+39Ui8y3lDjAXqxpT8RPg3Gibo0HmVfhNFZBIB/sLL6Y85hWZRJ/JdoXaX5/YAHs5arKBTpJXEaQdficUry4eV7BS0V5rP6YKaRFQZd80nAMZDo2+eZFjaugQC6fsVgRcKd6Y1maaQRiytDcggEs40BI1lJGVBVd4P508gqbjDLL+cHKhYukOZtuB0iRRmRFh/7txYBqINtI4/rxNZRKajZCFzjkomKrzQHLpLjZXCPNW3JU21dS/mQmeHyr8sHRCl0FYc5gpewRVBchdVWeDtyM7gapR6zFAI4vzyl HhBdgJof d9DI0Aew5LOpcVZX/hi2FRD4yhKSEmhyLSsHy7ejRiFtEyRPDA/MQ/6bjwUaa4I6TyJvlJ7ovdCxvuN+UZnT43xMSGXxc1Gax0DsBkmm0zzZLo982peCoYddtlTeOu/y3/MM/7rBu2L29rYD9k3+uB88RjXNr5//vvFgZ3ZCz0OY+Hah7BOMkl9cyAspY2NX8Q070nj0fuU6eXB+5M3/2NMqcDPY4b7d01h0L5oTRZDNLP2SRkVqe71g3vojjiX0ufUFABOXXltZR+B3AeiRNwYNic1LmmqWtf1IxZUyNqJqHsuHtmLho7TcUnf/CjVUgMwviQyu61iUca9IUylBt1Nf37qhi22lXOo8xb5hK7+FVpmLtmgboem1Y1V7isFVE9P9Fn11EoxkikjFVgwRbsgH0xExCowJdbHGD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 15.08.23 23:25, Peter Xu wrote: > Tail page struct reuse is over-comlicated. Not only because we have It is complicated, agreed. With the ->private for THP_SWAP gone, we would have to document less. Stating that 4*4byte / 4*8 byte are available after flags+head would be sufficient and I'd even drop the table. > implicit uses of tail page fields (mapcounts, or private for thp swap > support, etc., that we may still use in the page structs, Instead of documenting that thp swap should no longer touch the private field of tail pages, maybe we can indeed fix that quite easily. My simple tests passed so far. If there isn't something obvious missing, I can do more testing and send this as an official patch. From ec0f8b0dd8fb81c316b6a4c5fc9ae7563e625404 Mon Sep 17 00:00:00 2001 From: David Hildenbrand Date: Wed, 16 Aug 2023 13:14:45 +0200 Subject: [PATCH] mm/swap: stop using page->private on tail pages for THP_SWAP Let's stop using page->private on tail pages, making it possible to just unconditionally reuse that field in the tail pages of large folios. Signed-off-by: David Hildenbrand --- arch/arm64/mm/mteswap.c | 5 +++-- include/linux/swap.h | 9 +++++++++ mm/huge_memory.c | 15 ++++++--------- mm/memory.c | 2 +- mm/rmap.c | 2 +- mm/swap_state.c | 4 ++-- mm/swapfile.c | 4 ++-- 7 files changed, 24 insertions(+), 17 deletions(-) diff --git a/arch/arm64/mm/mteswap.c b/arch/arm64/mm/mteswap.c index cd508ba80ab1..a31833e3ddc5 100644 --- a/arch/arm64/mm/mteswap.c +++ b/arch/arm64/mm/mteswap.c @@ -33,8 +33,9 @@ int mte_save_tags(struct page *page) mte_save_page_tags(page_address(page), tag_storage); - /* page_private contains the swap entry.val set in do_swap_page */ - ret = xa_store(&mte_pages, page_private(page), tag_storage, GFP_KERNEL); + /* lookup the swap entry.val from the page */ + ret = xa_store(&mte_pages, page_swap_entry(page).val, tag_storage, + GFP_KERNEL); if (WARN(xa_is_err(ret), "Failed to store MTE tags")) { mte_free_tag_storage(tag_storage); return xa_err(ret); diff --git a/include/linux/swap.h b/include/linux/swap.h index bb5adc604144..84fe0e94f5cd 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -339,6 +339,15 @@ static inline swp_entry_t folio_swap_entry(struct folio *folio) return entry; } +static inline swp_entry_t page_swap_entry(struct page *page) +{ + struct folio *folio = page_folio(page); + swp_entry_t entry = folio_swap_entry(folio); + + entry.val += page - &folio->page; + return entry; +} + static inline void folio_set_swap_entry(struct folio *folio, swp_entry_t entry) { folio->private = (void *)entry.val; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 0b709d2c46c6..f7e04cbcb063 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2451,18 +2451,15 @@ static void __split_huge_page_tail(struct page *head, int tail, page_tail->index = head->index + tail; /* - * page->private should not be set in tail pages with the exception - * of swap cache pages that store the swp_entry_t in tail pages. - * Fix up and warn once if private is unexpectedly set. - * - * What of 32-bit systems, on which folio->_pincount overlays - * head[1].private? No problem: THP_SWAP is not enabled on 32-bit, and - * pincount must be 0 for folio_ref_freeze() to have succeeded. + * page->private should not be set in tail pages. Fix up and warn once + * if private is unexpectedly set. */ - if (!folio_test_swapcache(page_folio(head))) { - VM_WARN_ON_ONCE_PAGE(page_tail->private != 0, page_tail); + if (unlikely(page_tail->private)) { + VM_WARN_ON_ONCE_PAGE(true, page_tail); page_tail->private = 0; } + if (PageSwapCache(head)) + set_page_private(page_tail, (unsigned long)head->private + tail); /* Page flags must be visible before we make the page non-compound. */ smp_wmb(); diff --git a/mm/memory.c b/mm/memory.c index d003076b218d..ff13242c1589 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3882,7 +3882,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) * changed. */ if (unlikely(!folio_test_swapcache(folio) || - page_private(page) != entry.val)) + page_swap_entry(page).val != entry.val)) goto out_page; /* diff --git a/mm/rmap.c b/mm/rmap.c index 1f04debdc87a..ec7f8e6c9e48 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1647,7 +1647,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, */ dec_mm_counter(mm, mm_counter(&folio->page)); } else if (folio_test_anon(folio)) { - swp_entry_t entry = { .val = page_private(subpage) }; + swp_entry_t entry = page_swap_entry(subpage); pte_t swp_pte; /* * Store the swap location in the pte. diff --git a/mm/swap_state.c b/mm/swap_state.c index 01f15139b7d9..450819934e34 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -100,6 +100,7 @@ int add_to_swap_cache(struct folio *folio, swp_entry_t entry, folio_ref_add(folio, nr); folio_set_swapcache(folio); + folio_set_swap_entry(folio, entry); do { xas_lock_irq(&xas); @@ -113,7 +114,6 @@ int add_to_swap_cache(struct folio *folio, swp_entry_t entry, if (shadowp) *shadowp = old; } - set_page_private(folio_page(folio, i), entry.val + i); xas_store(&xas, folio); xas_next(&xas); } @@ -154,9 +154,9 @@ void __delete_from_swap_cache(struct folio *folio, for (i = 0; i < nr; i++) { void *entry = xas_store(&xas, shadow); VM_BUG_ON_PAGE(entry != folio, entry); - set_page_private(folio_page(folio, i), 0); xas_next(&xas); } + folio->private = 0; folio_clear_swapcache(folio); address_space->nrpages -= nr; __node_stat_mod_folio(folio, NR_FILE_PAGES, -nr); diff --git a/mm/swapfile.c b/mm/swapfile.c index d46933adf789..bd9d904671b9 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -3369,7 +3369,7 @@ struct swap_info_struct *swp_swap_info(swp_entry_t entry) struct swap_info_struct *page_swap_info(struct page *page) { - swp_entry_t entry = { .val = page_private(page) }; + swp_entry_t entry = page_swap_entry(page); return swp_swap_info(entry); } @@ -3384,7 +3384,7 @@ EXPORT_SYMBOL_GPL(swapcache_mapping); pgoff_t __page_file_index(struct page *page) { - swp_entry_t swap = { .val = page_private(page) }; + swp_entry_t swap = page_swap_entry(page); return swp_offset(swap); } EXPORT_SYMBOL_GPL(__page_file_index); -- 2.41.0 -- Cheers, David / dhildenb