From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E17CC3DA4A for ; Tue, 20 Aug 2024 19:35:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D37716B0083; Tue, 20 Aug 2024 15:35:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CC0A36B0085; Tue, 20 Aug 2024 15:35:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B60656B0088; Tue, 20 Aug 2024 15:35:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 8E2306B0083 for ; Tue, 20 Aug 2024 15:35:19 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 3033B1A04BD for ; Tue, 20 Aug 2024 19:35:19 +0000 (UTC) X-FDA: 82473627558.29.F5B9749 Received: from mail-vk1-f182.google.com (mail-vk1-f182.google.com [209.85.221.182]) by imf29.hostedemail.com (Postfix) with ESMTP id 4A226120009 for ; Tue, 20 Aug 2024 19:35:17 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Pf5Sn6q0; spf=pass (imf29.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.221.182 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724182478; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JhdEj+sJpVIC9ERYexH59LfqXSJtceBbJDRCMReoFXM=; b=NOL+QtCVUdjdqiFsS7LBF/uH8O6djKbLhz+RTkpmgE7aH5waorpYfZOjt8NIKON5EGY1+k W9riUFecWARViCvZwJfJGcXFDM/N5gJ+2ggEV7ALb+5Z8hUU4um1t6KBZ9zPBG4esfJ6y5 un/oy53EEhf0ldLwI5rAOi4H5MMTldQ= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Pf5Sn6q0; spf=pass (imf29.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.221.182 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724182478; a=rsa-sha256; cv=none; b=by6vzu0i+JwXmPgLktfcEXk6cnGL5OS1mzvrk3HZXPB3a2GGr1wAbUCz3p+uhKoLzuMKrW VkL55d0FrtvmNYYX4raczmPqvUqAyIhdhJHqTB7mIwI2bvBp0QPN9ZtJEILfwaHOPOdR0N Pj15VJIfvzhboMcH6Lo8TUxeEQc98yo= Received: by mail-vk1-f182.google.com with SMTP id 71dfb90a1353d-4f511fddfcfso2136584e0c.1 for ; Tue, 20 Aug 2024 12:35:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724182516; x=1724787316; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=JhdEj+sJpVIC9ERYexH59LfqXSJtceBbJDRCMReoFXM=; b=Pf5Sn6q0gB8IUCyYtNMPurOLJ9dzeD3nmF7g+bOQaUBVVoRHbzvoMgIcCxbVsW5mQD vMtgzEl7Azy3i5r1Cl6uvLp8yIXQkek9eeZ3bG/8t+Vp2VOdq3+tYYbJRSZymQSi+IxB d0EcUer4Niih95y1Lp3mFnhce57EfkFblyv+xZe7Dt2LDSkF221OFIHixqS0hjoljfzh JCdA/0zF/mcWE9cXpInIbJ+CiZWpucahH5XCFmzkovxN4tMnAC5P2W/HEQ/aulTAYxUM rnNtWNLmcvgho+r9+rGbvZE1Zu0jFAvf3rS59383YYsppqb6avPDk3S7lOGE/Gsnwg4w d+yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724182516; x=1724787316; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JhdEj+sJpVIC9ERYexH59LfqXSJtceBbJDRCMReoFXM=; b=UY2pd++OYeeddTUt2hhb2WpWpsOOE6S/61ij1B6lU5zBjyozPe5fa5/TU2Y3yon7RZ lTwENdokd1f4CO8Wt3QMGNHZaSbI0Mhe9ol7sxrA7tMw5j+shYdVItd8SVoHnmQEztK5 08QH6lFN6Z3s2o5vkyKVVBvkdN4abq6PenLdG0Co/64+pWv5VTMlanbZRBT9cbWAi0n1 BnraZROTwNWrqBWOqncpAdITgAXcu+9/hf2HfFaENsjxyn+7ijORL3alZ9hOcJW2jz0W 8rLFy05lYerTUBTqTmEkPvzX+X7AAFaNCJ8LvMEipNZjr8QjsxxGPpxgJuLcnMb+xd3Y TjZg== X-Gm-Message-State: AOJu0Yy/2FmatKWWUg2RrHz0rV8W1mnAzEqLdZ5PJzfxaurGWNtQOBMX O789+PyAUNy7KDCd52pFeuJCWMY4T7cEV7TiZQpyud0Bsijp7iFI X-Google-Smtp-Source: AGHT+IF+YL/VmQ7OVyY18Cts466NyKsFBNZ/U/RLmdXiO1mMkPReytJ40J1jROEAHVhtKbmvuEq/hA== X-Received: by 2002:a05:6122:3bd2:b0:4e4:e90f:6749 with SMTP id 71dfb90a1353d-4fcf39a13cfmr42828e0c.10.1724182516133; Tue, 20 Aug 2024 12:35:16 -0700 (PDT) Received: from ?IPV6:2a03:83e0:1145:4:1409:786c:cb1d:c3fb? ([2620:10d:c091:500::4:4b1f]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4536a07667fsm51564551cf.93.2024.08.20.12.35.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 20 Aug 2024 12:35:15 -0700 (PDT) Message-ID: <953d398d-58be-41c6-bf30-4c9df597de77@gmail.com> Date: Tue, 20 Aug 2024 15:35:14 -0400 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 4/6] mm: Introduce a pageflag for partially mapped folios To: Barry Song , akpm@linux-foundation.org Cc: linux-mm@kvack.org, hannes@cmpxchg.org, riel@surriel.com, shakeel.butt@linux.dev, roman.gushchin@linux.dev, yuzhao@google.com, david@redhat.com, ryan.roberts@arm.com, rppt@kernel.org, willy@infradead.org, cerasuolodomenico@gmail.com, ryncsn@gmail.com, corbet@lwn.net, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kernel-team@meta.com References: <20240819023145.2415299-1-usamaarif642@gmail.com> <20240819023145.2415299-5-usamaarif642@gmail.com> <9a58e794-2156-4a9f-a383-1cdfc07eee5e@gmail.com> Content-Language: en-US From: Usama Arif In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 4A226120009 X-Stat-Signature: gsuxbyfqh5s1838b65j363bczu99gips X-HE-Tag: 1724182517-148444 X-HE-Meta: U2FsdGVkX18Lh2wqlmBABMfpWxKzoX+7uJCp326cr/aT1EJuhPXt3CcdxzAR9YVEZqgspMX5dkQSc3/88V8UPw0SSSq6eb+7LaGylsZ/aCwSBzhn17ipH/dnPkX3XSFgfBTLlUJq+zppWVu1SSwvU4SjAnsGwKx8q0gKpeILOMoNS4jWy45vIrc2OiMa6AkLo59Sek7APn6QA40iAGWjt/i3kVAF2DjKoDHm5UjY/Nzvih0B4GevIH32ErwzRvoCARILbfyvoXCpVGfCk3+KHvYbVfYhRvAWZC0wgfDhw88DcnYFG3hcgGzsN0kOEfOV3LWLmnN58b/SDGNUDTKteP4CzXDFfCbs/+mzE/coOEEV9BdVwQMnF0Psa4tWuvDlY2QvjPRVBJ7td9GlwEPi6kOjRHRHlfUz8WWBjcddFmmJzOy4XhaM3qD8Sd9F5y79/Ijkk84CnN0lcDF2jpOMuBO5PqVdhNrU0K9HTC48HBWAjOI9UITxQg5UCrxJWEtoqhqYAh8qlbrKpFfp5/SA8LzHrnqjG5MFbsjmmMSLfcqGwMjxTErR5UPRZv5mcYur6JNCA2HJ7TYSj9TFj1W6AzTMfwrGabELQhev35ht0Tp7vJlBKo82Cc8XMFDrxoLCUZc9jF50hVFxIDiUXrMOlyncUlQ6ii3y5DOonKFIFadNIZKx0z+EajgKorsyVPGzT40Os1YH9Y5e/WcT7mhZvTyP/ea6A+CzEdkHb/BGvIm3faURb02uQE22OkcFmNj0SrOn6XQckHF+xYJOCmPox1NVG8ljQenuvBZRbHy88qNVOD40yK3vCzAfQNadRxpe5I87fCPdYlgHafZ+QpTzUkJUm8GcKdak4H/9xNn3nI1zVYOjEnAYTLA/gL5g6lhvHD5SZscdJBL8J3y/R2Fr7SqwPED0VxTf9b9k9S+je3fxlpI3ruy98EX5XPhVReHI1gR86HurcVh5Ay7gRAQ 9pxcy2Np X538hckUcvFUteQTiwKMeHE1f6z9I2H/wgbbRBgNff8IdE1G0wzvw7WIwqbzTxf7QQQA3zfBi4jbk5CbbW3Qc05qYy1QxBsPtmfewva5t+Uulm7ATDkni+QTRnLPXUSFjtD6FPEe5l4x1JQ6hB54r5ptb3PG78MSC0BQYJc5p06sPTESJ6mCBDTF35xRHIOZ3CO7llO08ARC3mKB5LvJPS+S7Hq4j17e6JmjivqdRZimEwLKGKK52IJaLureck+WemB7CpxZBSU7G46PMrPSZQhOhNjq1cyiYVP8lIDPiaAx1gnCo9UXNpWrRFmIjmWebjSXcHBP4+SLAh5wWjG7tkrSYnLySnu7+7T61QmhxlNA8E0Fbeazy6iW9MwXWyRCoqphB0uQJSlzTTxorEGr7n3y76i5UyO9raY4YTtAnwmBxGm7XPoy9tadwlYDNuckfs4MhvCMpJiOmeB3dJwQHHqYD6b9o5+0oLEpoUJcBqsLWX55IXAv4ga9xHA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 19/08/2024 22:55, Barry Song wrote: > On Tue, Aug 20, 2024 at 9:34 AM Barry Song wrote: >> >> On Tue, Aug 20, 2024 at 8:16 AM Usama Arif wrote: >>> >>> >>> >>> On 19/08/2024 20:00, Barry Song wrote: >>>> On Tue, Aug 20, 2024 at 2:17 AM Usama Arif wrote: >>>>> >>>>> >>>>> >>>>> On 19/08/2024 09:29, Barry Song wrote: >>>>>> Hi Usama, >>>>>> >>>>>> I feel it is much better now! thanks! >>>>>> >>>>>> On Mon, Aug 19, 2024 at 2:31 PM Usama Arif wrote: >>>>>>> >>>>>>> Currently folio->_deferred_list is used to keep track of >>>>>>> partially_mapped folios that are going to be split under memory >>>>>>> pressure. In the next patch, all THPs that are faulted in and collapsed >>>>>>> by khugepaged are also going to be tracked using _deferred_list. >>>>>>> >>>>>>> This patch introduces a pageflag to be able to distinguish between >>>>>>> partially mapped folios and others in the deferred_list at split time in >>>>>>> deferred_split_scan. Its needed as __folio_remove_rmap decrements >>>>>>> _mapcount, _large_mapcount and _entire_mapcount, hence it won't be >>>>>>> possible to distinguish between partially mapped folios and others in >>>>>>> deferred_split_scan. >>>>>>> >>>>>>> Eventhough it introduces an extra flag to track if the folio is >>>>>>> partially mapped, there is no functional change intended with this >>>>>>> patch and the flag is not useful in this patch itself, it will >>>>>>> become useful in the next patch when _deferred_list has non partially >>>>>>> mapped folios. >>>>>>> >>>>>>> Signed-off-by: Usama Arif >>>>>>> --- >>>>>>> include/linux/huge_mm.h | 4 ++-- >>>>>>> include/linux/page-flags.h | 11 +++++++++++ >>>>>>> mm/huge_memory.c | 23 ++++++++++++++++------- >>>>>>> mm/internal.h | 4 +++- >>>>>>> mm/memcontrol.c | 3 ++- >>>>>>> mm/migrate.c | 3 ++- >>>>>>> mm/page_alloc.c | 5 +++-- >>>>>>> mm/rmap.c | 5 +++-- >>>>>>> mm/vmscan.c | 3 ++- >>>>>>> 9 files changed, 44 insertions(+), 17 deletions(-) >>>>>>> >>>>>>> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h >>>>>>> index 4c32058cacfe..969f11f360d2 100644 >>>>>>> --- a/include/linux/huge_mm.h >>>>>>> +++ b/include/linux/huge_mm.h >>>>>>> @@ -321,7 +321,7 @@ static inline int split_huge_page(struct page *page) >>>>>>> { >>>>>>> return split_huge_page_to_list_to_order(page, NULL, 0); >>>>>>> } >>>>>>> -void deferred_split_folio(struct folio *folio); >>>>>>> +void deferred_split_folio(struct folio *folio, bool partially_mapped); >>>>>>> >>>>>>> void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, >>>>>>> unsigned long address, bool freeze, struct folio *folio); >>>>>>> @@ -495,7 +495,7 @@ static inline int split_huge_page(struct page *page) >>>>>>> { >>>>>>> return 0; >>>>>>> } >>>>>>> -static inline void deferred_split_folio(struct folio *folio) {} >>>>>>> +static inline void deferred_split_folio(struct folio *folio, bool partially_mapped) {} >>>>>>> #define split_huge_pmd(__vma, __pmd, __address) \ >>>>>>> do { } while (0) >>>>>>> >>>>>>> diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h >>>>>>> index a0a29bd092f8..c3bb0e0da581 100644 >>>>>>> --- a/include/linux/page-flags.h >>>>>>> +++ b/include/linux/page-flags.h >>>>>>> @@ -182,6 +182,7 @@ enum pageflags { >>>>>>> /* At least one page in this folio has the hwpoison flag set */ >>>>>>> PG_has_hwpoisoned = PG_active, >>>>>>> PG_large_rmappable = PG_workingset, /* anon or file-backed */ >>>>>>> + PG_partially_mapped = PG_reclaim, /* was identified to be partially mapped */ >>>>>>> }; >>>>>>> >>>>>>> #define PAGEFLAGS_MASK ((1UL << NR_PAGEFLAGS) - 1) >>>>>>> @@ -861,8 +862,18 @@ static inline void ClearPageCompound(struct page *page) >>>>>>> ClearPageHead(page); >>>>>>> } >>>>>>> FOLIO_FLAG(large_rmappable, FOLIO_SECOND_PAGE) >>>>>>> +FOLIO_TEST_FLAG(partially_mapped, FOLIO_SECOND_PAGE) >>>>>>> +/* >>>>>>> + * PG_partially_mapped is protected by deferred_split split_queue_lock, >>>>>>> + * so its safe to use non-atomic set/clear. >>>>>>> + */ >>>>>>> +__FOLIO_SET_FLAG(partially_mapped, FOLIO_SECOND_PAGE) >>>>>>> +__FOLIO_CLEAR_FLAG(partially_mapped, FOLIO_SECOND_PAGE) >>>>>>> #else >>>>>>> FOLIO_FLAG_FALSE(large_rmappable) >>>>>>> +FOLIO_TEST_FLAG_FALSE(partially_mapped) >>>>>>> +__FOLIO_SET_FLAG_NOOP(partially_mapped) >>>>>>> +__FOLIO_CLEAR_FLAG_NOOP(partially_mapped) >>>>>>> #endif >>>>>>> >>>>>>> #define PG_head_mask ((1UL << PG_head)) >>>>>>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c >>>>>>> index 2d77b5d2291e..70ee49dfeaad 100644 >>>>>>> --- a/mm/huge_memory.c >>>>>>> +++ b/mm/huge_memory.c >>>>>>> @@ -3398,6 +3398,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, >>>>>>> * page_deferred_list. >>>>>>> */ >>>>>>> list_del_init(&folio->_deferred_list); >>>>>>> + __folio_clear_partially_mapped(folio); >>>>>>> } >>>>>>> spin_unlock(&ds_queue->split_queue_lock); >>>>>>> if (mapping) { >>>>>>> @@ -3454,11 +3455,13 @@ void __folio_undo_large_rmappable(struct folio *folio) >>>>>>> if (!list_empty(&folio->_deferred_list)) { >>>>>>> ds_queue->split_queue_len--; >>>>>>> list_del_init(&folio->_deferred_list); >>>>>>> + __folio_clear_partially_mapped(folio); >>>>>> >>>>>> is it possible to make things clearer by >>>>>> >>>>>> if (folio_clear_partially_mapped) >>>>>> __folio_clear_partially_mapped(folio); >>>>>> >>>>>> While writing without conditions isn't necessarily wrong, adding a condition >>>>>> will improve the readability of the code and enhance the clarity of my mTHP >>>>>> counters series. also help decrease smp cache sync if we can avoid >>>>>> unnecessary writing? >>>>>> >>>>> >>>>> Do you mean if(folio_test_partially_mapped(folio))? >>>>> >>>>> I don't like this idea. I think it makes the readability worse? If I was looking at if (test) -> clear for the first time, I would become confused why its being tested if its going to be clear at the end anyways? >>>> >>>> In the pmd-order case, the majority of folios are not partially mapped. >>>> Unconditional writes will trigger cache synchronization across all >>>> CPUs (related to the MESI protocol), making them more costly. By >>>> using conditional writes, such as "if(test) write," we can avoid >>>> most unnecessary writes, which is much more efficient. Additionally, >>>> we only need to manage nr_split_deferred when the condition >>>> is met. We are carefully evaluating all scenarios to determine >>>> if modifications to the partially_mapped flag are necessary. >>>> >>> >>> >>> Hmm okay, as you said its needed for nr_split_deferred anyways. Something like below is ok to fold in? >>> >>> commit 4ae9e2067346effd902b342296987b97dee29018 (HEAD) >>> Author: Usama Arif >>> Date: Mon Aug 19 21:07:16 2024 +0100 >>> >>> mm: Introduce a pageflag for partially mapped folios fix >>> >>> Test partially_mapped flag before clearing it. This should >>> avoid unnecessary writes and will be needed in the nr_split_deferred >>> series. >>> >>> Signed-off-by: Usama Arif >>> >>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c >>> index 5d67d3b3c1b2..ccde60aaaa0f 100644 >>> --- a/mm/huge_memory.c >>> +++ b/mm/huge_memory.c >>> @@ -3479,7 +3479,8 @@ void __folio_undo_large_rmappable(struct folio *folio) >>> if (!list_empty(&folio->_deferred_list)) { >>> ds_queue->split_queue_len--; >>> list_del_init(&folio->_deferred_list); >>> - __folio_clear_partially_mapped(folio); >>> + if (folio_test_partially_mapped(folio)) >>> + __folio_clear_partially_mapped(folio); >>> } >>> spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); >>> } >>> @@ -3610,7 +3611,8 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, >>> } else { >>> /* We lost race with folio_put() */ >>> list_del_init(&folio->_deferred_list); >>> - __folio_clear_partially_mapped(folio); >>> + if (folio_test_partially_mapped(folio)) >>> + __folio_clear_partially_mapped(folio); >>> ds_queue->split_queue_len--; >>> } >>> if (!--sc->nr_to_scan) >>> >> >> Do we also need if (folio_test_partially_mapped(folio)) in >> split_huge_page_to_list_to_order()? >> >> I recall that in Yu Zhao's TAO, there’s a chance of splitting (shattering) >> non-partially-mapped folios. To be future-proof, we might want to handle >> both cases equally. > > we recall we also have a real case which can split entirely_mapped > folio: > > mm: huge_memory: enable debugfs to split huge pages to any order > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=fc4d182316bd5309b4066fd9ef21529ea397a7d4 > >> >> By the way, we might not need to clear the flag for a new folio. This differs >> from the init_list, which is necessary. If a new folio has the partially_mapped >> flag, it indicates that we failed to clear it when freeing the folio to >> the buddy system, which is a bug we need to fix in the free path. >> >> Thanks >> Barry I believe the below fixlet should address all concerns: >From 95492a51b1929ea274b4e5b78fc74e7736645d58 Mon Sep 17 00:00:00 2001 From: Usama Arif Date: Mon, 19 Aug 2024 21:07:16 +0100 Subject: [PATCH] mm: Introduce a pageflag for partially mapped folios fix Test partially_mapped flag before clearing it. This should avoid unnecessary writes and will be needed in the nr_split_deferred series. Also no need to clear partially_mapped prepping compound head, as it should start with already being cleared. Signed-off-by: Usama Arif --- include/linux/page-flags.h | 2 +- mm/huge_memory.c | 9 ++++++--- mm/internal.h | 4 +--- 3 files changed, 8 insertions(+), 7 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index c3bb0e0da581..f1602695daf2 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -1182,7 +1182,7 @@ static __always_inline void __ClearPageAnonExclusive(struct page *page) */ #define PAGE_FLAGS_SECOND \ (0xffUL /* order */ | 1UL << PG_has_hwpoisoned | \ - 1UL << PG_large_rmappable) + 1UL << PG_large_rmappable | 1UL << PG_partially_mapped) #define PAGE_FLAGS_PRIVATE \ (1UL << PG_private | 1UL << PG_private_2) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 5d67d3b3c1b2..402b9d933de0 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3422,7 +3422,8 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, * page_deferred_list. */ list_del_init(&folio->_deferred_list); - __folio_clear_partially_mapped(folio); + if (folio_test_partially_mapped(folio)) + __folio_clear_partially_mapped(folio); } spin_unlock(&ds_queue->split_queue_lock); if (mapping) { @@ -3479,7 +3480,8 @@ void __folio_undo_large_rmappable(struct folio *folio) if (!list_empty(&folio->_deferred_list)) { ds_queue->split_queue_len--; list_del_init(&folio->_deferred_list); - __folio_clear_partially_mapped(folio); + if (folio_test_partially_mapped(folio)) + __folio_clear_partially_mapped(folio); } spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); } @@ -3610,7 +3612,8 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, } else { /* We lost race with folio_put() */ list_del_init(&folio->_deferred_list); - __folio_clear_partially_mapped(folio); + if (folio_test_partially_mapped(folio)) + __folio_clear_partially_mapped(folio); ds_queue->split_queue_len--; } if (!--sc->nr_to_scan) diff --git a/mm/internal.h b/mm/internal.h index 27cbb5365841..52f7fc4e8ac3 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -662,10 +662,8 @@ static inline void prep_compound_head(struct page *page, unsigned int order) atomic_set(&folio->_entire_mapcount, -1); atomic_set(&folio->_nr_pages_mapped, 0); atomic_set(&folio->_pincount, 0); - if (order > 1) { + if (order > 1) INIT_LIST_HEAD(&folio->_deferred_list); - __folio_clear_partially_mapped(folio); - } } static inline void prep_compound_tail(struct page *head, int tail_idx) -- 2.43.5