From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50F0CC54791 for ; Wed, 13 Mar 2024 11:37:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C2F7694001E; Wed, 13 Mar 2024 07:37:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BDFCC940010; Wed, 13 Mar 2024 07:37:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA60394001E; Wed, 13 Mar 2024 07:37:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 96390940010 for ; Wed, 13 Mar 2024 07:37:27 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 61DC11611AC for ; Wed, 13 Mar 2024 11:37:27 +0000 (UTC) X-FDA: 81891815334.29.6E8DFBF Received: from mail-vs1-f41.google.com (mail-vs1-f41.google.com [209.85.217.41]) by imf25.hostedemail.com (Postfix) with ESMTP id 88922A000C for ; Wed, 13 Mar 2024 11:37:24 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="Lj1/iVcI"; spf=pass (imf25.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.217.41 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710329844; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AjRnjtzpV+uCo3l3VhygApoQp+hOFij2LJqpdk2jAWo=; b=Izy/xmF5XDXX7vA4QOLU1BepYGjEB4La7tfnMjsQUDfo8kQqK7/cqJrKs3ulGUQcYUUvkI +AFEhPiVx8cpt/DafY/sm8J2nhVJip0hIundBB/4KU3ASnbUv+RtUkPijbhSuatbftUkuH nnNHV1cRH7d8V5xtR9UMfQTt84w2qfc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710329844; a=rsa-sha256; cv=none; b=pydQCPXVPc0Q2IorgPPEOcdJYftO+Tq9skHeCk9touPfiwO5yLVdM7G8XZOuIKZws83WH+ fRRWDwzYu0HqzeKmrIxUZk85kP0sM/5ZWclns46OkpI9w5Rsw+w0XUXKlQPhLSmV5WXFv2 58/3tc0YVvbRjz3M91z0OXhNkK89HwY= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="Lj1/iVcI"; spf=pass (imf25.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.217.41 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-vs1-f41.google.com with SMTP id ada2fe7eead31-474d39c0c49so177955137.2 for ; Wed, 13 Mar 2024 04:37:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710329843; x=1710934643; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=AjRnjtzpV+uCo3l3VhygApoQp+hOFij2LJqpdk2jAWo=; b=Lj1/iVcI1bg5to1w7Whx3mEl1mN/fYjEGGQiYF+LpmqdvQ2+Q64tjXZR7NcBneGW9Y 7qvMVKb9KRUfTFXE1LSwwugz2JFYTc2+lg2eMr4RN5O7z4RrdPHuk1uP9K/fUYuXNp21 MajWzJt4uFNHMviAeIEsJn5XuVj0g+JkuFGfZtKreItO7M6MeNwJP2bBVcGCpfhsmM3R sTMQEGBzDiLxkIyVotPgZzgZABfpObb6rtALmjSnj7tW8rC6cTYgM6rU9ypXC7MrkMK5 tnFT1Yd1+lDDR4g3vcM9VVZgtHS6tR+t9GQ4iUx2pOOFsjcaWWx3fK52++2Dwan/7N/y RzBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710329843; x=1710934643; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AjRnjtzpV+uCo3l3VhygApoQp+hOFij2LJqpdk2jAWo=; b=kdeLJKFiEDjK+8k6BZlplTStqTMMIMxZH93bSwx1RnzEcmvT9VkyiYdy2ovyKrybn/ N3Iv6XUanf2XNokk/L55mXVRHUwkQawS9npIjwF0oyjQVYlpn4D+/My+T8l70q4rcZvq aGKEzkWaxptfNBaEASHQOQzTB2M6bWm1erO5Ak1LiciY+LMl0wlmItbxTKQyf6+0PnJO uEllr/tTJNaf9A7kqUeU0fswwex/JWeca7WwzpKEbLgcILPpy/kuloZ/LqcxoCs9ZtkO hS2FAvZCozQ41duZueSmnZjRINNkOcBGUEdp8C0kiaexGldvizr9a6o7kjnlQylPrp+4 nv9w== X-Forwarded-Encrypted: i=1; AJvYcCXQGxw95NKq3qCrzvuILQykOQQpXEq9pvpdF+vYpfliTijj/mvP1bnbrRiM6Nfsin4JUW+DN3s7kr4swjZYvzAFI5U= X-Gm-Message-State: AOJu0Yz882Q32LdfFVoSt5xzSBela3DuGdRUuf17umtNFlvSThnES6f6 VnHHTWBfNpx7NMpJnARquIpmsk0bY5Ri6VJSmEJP47uHKHwREka25uIHIUv3jznxPwlTzYlSYxu 03avzBwcd+SV2s+JyPFeToPnKm68= X-Google-Smtp-Source: AGHT+IEl5DAH9ljE8P51bblxPb7xw1TbJLdWxSKuLlfolN++w7AIkRLli11onb8TUzBGv3OmsbVK/l5XKJWhAT1xdlw= X-Received: by 2002:a05:6102:1981:b0:473:1f3d:4cc9 with SMTP id jm1-20020a056102198100b004731f3d4cc9mr2432714vsb.23.1710329843355; Wed, 13 Mar 2024 04:37:23 -0700 (PDT) MIME-Version: 1.0 References: <20240311150058.1122862-1-ryan.roberts@arm.com> <20240311150058.1122862-7-ryan.roberts@arm.com> <00a3ba1d-98e1-409b-ae6e-7fbcbdcd74d5@arm.com> In-Reply-To: From: Barry Song <21cnbao@gmail.com> Date: Wed, 13 Mar 2024 19:37:11 +0800 Message-ID: Subject: Re: [PATCH v4 6/6] mm: madvise: Avoid split during MADV_PAGEOUT and MADV_COLD To: Ryan Roberts Cc: Lance Yang , Andrew Morton , David Hildenbrand , Matthew Wilcox , Huang Ying , Gao Xiang , Yu Zhao , Yang Shi , Michal Hocko , Kefeng Wang , Chris Li , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 88922A000C X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 8qdg3hqfrqgdkmwwm194qphccjmc8ob3 X-HE-Tag: 1710329844-200493 X-HE-Meta: U2FsdGVkX19A2DEOuTAzRyBUnoPoFprXsvB25q89q+EViBdIkCKWAXkV3ZYjnQJPaCGuwHPwtC5YkpBxgLe/BRx8CUhfKQr9sWzfAuJBbLnF9Gx9B26zTPoc8xJ30LFVgHdUIU1s7VMkIT2dwugPzEyvSjfuIdjnuLlchZnx3urY/lrKU5XosbGk2ltJNOmC8SplzcJvDaYIwavJchVmYsnTJcXeB9vJoHYkBr/IYmPMrSTcWkDSAPjAvqMZwLczCZsb6Q/ZO4wHrj/LQrB5VBpWPiNix8u25Va95UZIuXwFMeth6RgCEAPJ4u/ENLM+ZVkJlKs9/bTawxBQzTqzWvgvpTpkkpc8TTAfHHmRxIIZTPa4CD3WiNRFKsBiRj6SFz1aRDPrtf8/y+5IC3p+9gS1cv0IZyRsShYeAsqJRibI0rsdLmyICMlB/waa4LnPxbMi0q13oRWYQOQ/x060mtNaoc8mj/c4YITtWrqTNVinQNImQIe7X7BuBQVqClbz3uUGSQn23HhiuBndiR/7x6qZAFZbc75vouXnKHbXsdyDpI5JMdVNfCIEsgkBHaTqukB1D7mi6t92cdhK+fCSHB26YVHEAZHDkLxsWE5U6uTsU8HFYM7KixiZY5FKznQjgAcm8O4A6NpZilx10UqdW3WeRm3U0ckfh8d6FryuLpWUcYYsrDDWDI8a8iblhTjR4LuqtGRMArl12uH+3cCiz/0aec9maXxKZx35bnu8LS5GZd82rEtqqfFbnDUU8fGDzTapoJ44z9jtwgbNGatpo2XNBVgqCIuLEvJ/lRIlD6+wVHTvelfp7O/C9XR04EcxS3pK1Xb8J8ooI725rPnXz7GU4Nlxf7bHq7Y8e1U4Uow2qg45cE5pHWIvCZKWB0ZTCsY3UPwTzG9GuEoIwD/dm+Pr1/ChYlgg4+KZzFhruj2/RwN9f3VCmVqtLMwE3zla0Ksp58YnC/FGKGtfIAb F48JutiN u/O8TQC/gm8JRrVJhetyXjxAoR4Rb6FE8iEhxV1b8s5c8MFBL3Lf6bU9NbqEpu+UI66gImnA97OputhPf0Ge8BRww35ngQZpbTWSFmXBciOvwfJD25fPnlfnek89RP2siyk/pCwPK0bY7AgyhvkZ1t3ct/IzzBInFsJ1tXPR0BSAkbAZFkZf826PSHqQRjXJIiYkV54NMmY0e8fszLjIttFIcE1HrEpc8nJozatWh7SJ4ouXf/BvhpN7uIDDuxG0e35pHl63A8co/Ne+uYtz0AIZBkCPWNIhlN7/N8Z1QBszF3vtY1CTn0vsMzH3JBo0QtI4ylPoVBmKE1dq7Vz60ql2mGFS6EaV33F+Dv5kArBqg1E+2m95y3Djv77fJCePKQgzb79qMTgb5PzZKsl223Gmw04rItrNtnVK0NCmlyx7b3eQSjOQNmqkhkwl7qUfUB5ectu4+gz4c/Pug8ujTFixP52/oACjdHyDCYT39FW0IQs1k0CSXuxdSfSjM5Auta8USCHAYusISFK8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Mar 13, 2024 at 7:08=E2=80=AFPM Ryan Roberts = wrote: > > On 13/03/2024 10:37, Barry Song wrote: > > On Wed, Mar 13, 2024 at 10:36=E2=80=AFPM Ryan Roberts wrote: > >> > >> On 13/03/2024 09:16, Barry Song wrote: > >>> On Wed, Mar 13, 2024 at 10:03=E2=80=AFPM Ryan Roberts wrote: > >>>> > >>>> On 13/03/2024 07:19, Barry Song wrote: > >>>>> On Tue, Mar 12, 2024 at 4:01=E2=80=AFAM Ryan Roberts wrote: > >>>>>> > >>>>>> Rework madvise_cold_or_pageout_pte_range() to avoid splitting any = large > >>>>>> folio that is fully and contiguously mapped in the pageout/cold vm > >>>>>> range. This change means that large folios will be maintained all = the > >>>>>> way to swap storage. This both improves performance during swap-ou= t, by > >>>>>> eliding the cost of splitting the folio, and sets us up nicely for > >>>>>> maintaining the large folio when it is swapped back in (to be cove= red in > >>>>>> a separate series). > >>>>>> > >>>>>> Folios that are not fully mapped in the target range are still spl= it, > >>>>>> but note that behavior is changed so that if the split fails for a= ny > >>>>>> reason (folio locked, shared, etc) we now leave it as is and move = to the > >>>>>> next pte in the range and continue work on the proceeding folios. > >>>>>> Previously any failure of this sort would cause the entire operati= on to > >>>>>> give up and no folios mapped at higher addresses were paged out or= made > >>>>>> cold. Given large folios are becoming more common, this old behavi= or > >>>>>> would have likely lead to wasted opportunities. > >>>>>> > >>>>>> While we are at it, change the code that clears young from the pte= s to > >>>>>> use ptep_test_and_clear_young(), which is more efficent than > >>>>>> get_and_clear/modify/set, especially for contpte mappings on arm64= , > >>>>>> where the old approach would require unfolding/refolding and the n= ew > >>>>>> approach can be done in place. > >>>>>> > >>>>>> Signed-off-by: Ryan Roberts > >>>>> > >>>>> This looks so much better than our initial RFC. > >>>>> Thank you for your excellent work! > >>>> > >>>> Thanks - its a team effort - I had your PoC and David's previous bat= ching work > >>>> to use as a template. > >>>> > >>>>> > >>>>>> --- > >>>>>> mm/madvise.c | 89 ++++++++++++++++++++++++++++++-----------------= ----- > >>>>>> 1 file changed, 51 insertions(+), 38 deletions(-) > >>>>>> > >>>>>> diff --git a/mm/madvise.c b/mm/madvise.c > >>>>>> index 547dcd1f7a39..56c7ba7bd558 100644 > >>>>>> --- a/mm/madvise.c > >>>>>> +++ b/mm/madvise.c > >>>>>> @@ -336,6 +336,7 @@ static int madvise_cold_or_pageout_pte_range(p= md_t *pmd, > >>>>>> LIST_HEAD(folio_list); > >>>>>> bool pageout_anon_only_filter; > >>>>>> unsigned int batch_count =3D 0; > >>>>>> + int nr; > >>>>>> > >>>>>> if (fatal_signal_pending(current)) > >>>>>> return -EINTR; > >>>>>> @@ -423,7 +424,8 @@ static int madvise_cold_or_pageout_pte_range(p= md_t *pmd, > >>>>>> return 0; > >>>>>> flush_tlb_batched_pending(mm); > >>>>>> arch_enter_lazy_mmu_mode(); > >>>>>> - for (; addr < end; pte++, addr +=3D PAGE_SIZE) { > >>>>>> + for (; addr < end; pte +=3D nr, addr +=3D nr * PAGE_SIZE) = { > >>>>>> + nr =3D 1; > >>>>>> ptent =3D ptep_get(pte); > >>>>>> > >>>>>> if (++batch_count =3D=3D SWAP_CLUSTER_MAX) { > >>>>>> @@ -447,55 +449,66 @@ static int madvise_cold_or_pageout_pte_range= (pmd_t *pmd, > >>>>>> continue; > >>>>>> > >>>>>> /* > >>>>>> - * Creating a THP page is expensive so split it on= ly if we > >>>>>> - * are sure it's worth. Split it if we are only ow= ner. > >>>>>> + * If we encounter a large folio, only split it if= it is not > >>>>>> + * fully mapped within the range we are operating = on. Otherwise > >>>>>> + * leave it as is so that it can be swapped out wh= ole. If we > >>>>>> + * fail to split a folio, leave it in place and ad= vance to the > >>>>>> + * next pte in the range. > >>>>>> */ > >>>>>> if (folio_test_large(folio)) { > >>>>>> - int err; > >>>>>> - > >>>>>> - if (folio_estimated_sharers(folio) > 1) > >>>>>> - break; > >>>>>> - if (pageout_anon_only_filter && !folio_tes= t_anon(folio)) > >>>>>> - break; > >>>>>> - if (!folio_trylock(folio)) > >>>>>> - break; > >>>>>> - folio_get(folio); > >>>>>> - arch_leave_lazy_mmu_mode(); > >>>>>> - pte_unmap_unlock(start_pte, ptl); > >>>>>> - start_pte =3D NULL; > >>>>>> - err =3D split_folio(folio); > >>>>>> - folio_unlock(folio); > >>>>>> - folio_put(folio); > >>>>>> - if (err) > >>>>>> - break; > >>>>>> - start_pte =3D pte =3D > >>>>>> - pte_offset_map_lock(mm, pmd, addr,= &ptl); > >>>>>> - if (!start_pte) > >>>>>> - break; > >>>>>> - arch_enter_lazy_mmu_mode(); > >>>>>> - pte--; > >>>>>> - addr -=3D PAGE_SIZE; > >>>>>> - continue; > >>>>>> + const fpb_t fpb_flags =3D FPB_IGNORE_DIRTY= | > >>>>>> + FPB_IGNORE_SOFT_DI= RTY; > >>>>>> + int max_nr =3D (end - addr) / PAGE_SIZE; > >>>>>> + > >>>>>> + nr =3D folio_pte_batch(folio, addr, pte, p= tent, max_nr, > >>>>>> + fpb_flags, NULL); > >>>>> > >>>>> I wonder if we have a quick way to avoid folio_pte_batch() if users > >>>>> are doing madvise() on a portion of a large folio. > >>>> > >>>> Good idea. Something like this?: > >>>> > >>>> if (pte_pfn(pte) =3D=3D folio_pfn(folio) > >>> > >>> what about > >>> > >>> "If (pte_pfn(pte) =3D=3D folio_pfn(folio) && max_nr >=3D nr_pages)" > >>> > >>> just to account for cases where the user's end address falls within > >>> the middle of a large folio? > >> > >> yes, even better. I'll add this for the next version. > >> > >>> > >>> > >>> BTW, another minor issue is here: > >>> > >>> if (++batch_count =3D=3D SWAP_CLUSTER_MAX) { > >>> batch_count =3D 0; > >>> if (need_resched()) { > >>> arch_leave_lazy_mmu_mode(); > >>> pte_unmap_unlock(start_pte, ptl); > >>> cond_resched(); > >>> goto restart; > >>> } > >>> } > >>> > >>> We are increasing 1 for nr ptes, thus, we are holding PTL longer > >>> than small folios case? we used to increase 1 for each PTE. > >>> Does it matter? > >> > >> I thought about that, but the vast majority of the work is per-folio, = not > >> per-pte. So I concluded it would be best to continue to increment per-= folio. > > > > Okay. The original patch commit b2f557a21bc8 ("mm/madvise: add > > cond_resched() in madvise_cold_or_pageout_pte_range()") > > primarily addressed the real-time wake-up latency issue. MADV_PAGEOUT > > and MADV_COLD are much less critical compared > > to other scenarios where operations like do_anon_page or do_swap_page > > necessarily need PTL to progress. Therefore, adopting > > an approach that relatively aggressively releases the PTL seems to > > neither harm MADV_PAGEOUT/COLD nor disadvantage > > others. > > > > We are slightly increasing the duration of holding the PTL due to the > > iteration of folio_pte_batch() potentially taking longer than > > the case of small folios, which do not require it. > > If we can't scan all the PTEs in a page table without dropping the PTL > intermittently we have bigger problems. This all works perfectly fine in = all the > other PTE iterators; see zap_pte_range() for example. I've no doubt about folio_pte_batch(). was just talking about the original rt issue it might affect. > > > However, compared > > to operations like folio_isolate_lru() and folio_deactivate(), > > this increase seems negligible. Recently, we have actually removed > > ptep_test_and_clear_young() for MADV_PAGEOUT, > > which should also benefit real-time scenarios. Nonetheless, there is a > > small risk with large folios, such as 1 MiB mTHP, where > > we may need to loop 256 times in folio_pte_batch(). > > As I understand it, RT and THP are mutually exclusive. RT can't handle th= e extra > latencies THPs can cause in allocation path, etc. So I don't think you wi= ll see > a problem here. I was actually taking a different approach on the phones as obviously we have some UX(user-experience)/UI/audio related tasks which cannot tolerate allocation latency. with a TAO-similar optimization(we did it by ext_migratetype for some pageblocks= ), we actually don't push buddy to do compaction or reclamation for forming 64KiB folio. We immediately fallback to small folios if a zero-latency 64KiB allocation can't be obtained from some kinds of pools - ext_migratetype pageblocks. > > > > > I would vote for increasing 'nr' or maybe max(log2(nr), 1) rather than > > 1 for two reasons: > > > > 1. We are not making MADV_PAGEOUT/COLD worse; in fact, we are > > improving them by reducing the time taken to put the same > > number of pages into the reclaim list. > > > > 2. MADV_PAGEOUT/COLD scenarios are not urgent compared to others that > > genuinely require the PTL to progress. Moreover, > > the majority of time spent on PAGEOUT is actually reclaim_pages(). > > I understand your logic. But I'd rather optimize for fewer lock acquisiti= ons for > the !RT+THP case, since RT+THP is not supported. Fair enough. I agree we can postpone this until RT and THP become an available option. For now, keeping this patch simpler seems to be better. > > > > >>> > >>>> nr =3D folio_pte_batch(folio, addr, pte, ptent, max_= nr, > >>>> fpb_flags, NULL); > >>>> > >>>> If we are not mapping the first page of the folio, then it can't be = a full > >>>> mapping, so no need to call folio_pte_batch(). Just split it. > >>>> > >>>>> > >>>>>> + > >>>>>> + if (nr < folio_nr_pages(folio)) { > >>>>>> + int err; > >>>>>> + > >>>>>> + if (folio_estimated_sharers(folio)= > 1) > >>>>>> + continue; > >>>>>> + if (pageout_anon_only_filter && !f= olio_test_anon(folio)) > >>>>>> + continue; > >>>>>> + if (!folio_trylock(folio)) > >>>>>> + continue; > >>>>>> + folio_get(folio); > >>>>>> + arch_leave_lazy_mmu_mode(); > >>>>>> + pte_unmap_unlock(start_pte, ptl); > >>>>>> + start_pte =3D NULL; > >>>>>> + err =3D split_folio(folio); > >>>>>> + folio_unlock(folio); > >>>>>> + folio_put(folio); > >>>>>> + if (err) > >>>>>> + continue; > >>>>>> + start_pte =3D pte =3D > >>>>>> + pte_offset_map_lock(mm, pm= d, addr, &ptl); > >>>>>> + if (!start_pte) > >>>>>> + break; > >>>>>> + arch_enter_lazy_mmu_mode(); > >>>>>> + nr =3D 0; > >>>>>> + continue; > >>>>>> + } > >>>>>> } > >>>>>> > >>>>>> /* > >>>>>> * Do not interfere with other mappings of this fo= lio and > >>>>>> - * non-LRU folio. > >>>>>> + * non-LRU folio. If we have a large folio at this= point, we > >>>>>> + * know it is fully mapped so if its mapcount is t= he same as its > >>>>>> + * number of pages, it must be exclusive. > >>>>>> */ > >>>>>> - if (!folio_test_lru(folio) || folio_mapcount(folio= ) !=3D 1) > >>>>>> + if (!folio_test_lru(folio) || > >>>>>> + folio_mapcount(folio) !=3D folio_nr_pages(foli= o)) > >>>>>> continue; > >>>>> > >>>>> This looks so perfect and is exactly what I wanted to achieve. > >>>>> > >>>>>> > >>>>>> if (pageout_anon_only_filter && !folio_test_anon(f= olio)) > >>>>>> continue; > >>>>>> > >>>>>> - VM_BUG_ON_FOLIO(folio_test_large(folio), folio); > >>>>>> - > >>>>>> - if (!pageout && pte_young(ptent)) { > >>>>>> - ptent =3D ptep_get_and_clear_full(mm, addr= , pte, > >>>>>> - tlb->fullm= m); > >>>>>> - ptent =3D pte_mkold(ptent); > >>>>>> - set_pte_at(mm, addr, pte, ptent); > >>>>>> - tlb_remove_tlb_entry(tlb, pte, addr); > >>>>>> + if (!pageout) { > >>>>>> + for (; nr !=3D 0; nr--, pte++, addr +=3D P= AGE_SIZE) { > >>>>>> + if (ptep_test_and_clear_young(vma,= addr, pte)) > >>>>>> + tlb_remove_tlb_entry(tlb, = pte, addr); > >>>>>> + } > >>>>> > >>>>> This looks so smart. if it is not pageout, we have increased pte > >>>>> and addr here; so nr is 0 and we don't need to increase again in > >>>>> for (; addr < end; pte +=3D nr, addr +=3D nr * PAGE_SIZE) > >>>>> > >>>>> otherwise, nr won't be 0. so we will increase addr and > >>>>> pte by nr. > >>>> > >>>> Indeed. I'm hoping that Lance is able to follow a similar pattern fo= r > >>>> madvise_free_pte_range(). > >>>> > >>>> > >>>>> > >>>>> > >>>>>> } > >>>>>> > >>>>>> /* > >>>>>> -- > >>>>>> 2.25.1 > >>>>>> > >>>>> > >>>>> Overall, LGTM, > >>>>> > >>>>> Reviewed-by: Barry Song > >>>> > >>>> Thanks! > >>>> > > > > Thanks > > Barry >