From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E35EEB64DD for ; Wed, 26 Jul 2023 03:27:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC7076B0071; Tue, 25 Jul 2023 23:27:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A776C6B0074; Tue, 25 Jul 2023 23:27:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93FD06B0075; Tue, 25 Jul 2023 23:27:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 8565D6B0071 for ; Tue, 25 Jul 2023 23:27:33 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 54188C010A for ; Wed, 26 Jul 2023 03:27:33 +0000 (UTC) X-FDA: 81052327986.06.70158A4 Received: from mail-qt1-f171.google.com (mail-qt1-f171.google.com [209.85.160.171]) by imf20.hostedemail.com (Postfix) with ESMTP id 8A74A1C0011 for ; Wed, 26 Jul 2023 03:27:31 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=MC43VpmA; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.171 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690342051; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=U6xj4wxFn0rypQoF1PGqlJBlsHAu3fI3DlYhEQlJySM=; b=Z+v69Ig/AdGTirVtJdQbzbUr8mW8twZn/Um8ScsyoMHKOqlKUcsetptM64yYdRAjp7pYpv qqdiXG7iAzRoJr1LI6q7lUFJ9X1Akv0az0STBg5n6niT1KsIoc6LNsVgrl1lbkZMmQ3XsQ bd3dIv2c7Q0ZYECNWVztLlwV2kAMbcw= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=MC43VpmA; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.171 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690342051; a=rsa-sha256; cv=none; b=0sy9TrVpO49zJWBfeC0KT/dEMUGiVQeMAe3AZedRdOhzsvZIqfH/uKUGhsKPTScpJwIZ2S m7Ajk7FzmXloPbcjphyb3V7iWMp2ZkK1XBnDlt7HoAUspnfmAWQ5UU/RMWv5owO1rHUe+I 7JhhrK4EuR/jCBkKx3tr7kiXQbdMz4g= Received: by mail-qt1-f171.google.com with SMTP id d75a77b69052e-40540a8a3bbso135691cf.1 for ; Tue, 25 Jul 2023 20:27:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690342050; x=1690946850; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=U6xj4wxFn0rypQoF1PGqlJBlsHAu3fI3DlYhEQlJySM=; b=MC43VpmA9s5GMATDplIXFmcJYSGXim3w1RXDdeAWzYf1BVnVkWEdmq8neZAzx9h9uI afOsinbCBFeIWwnJJZzGKH3JQ62MWlvCycZ1KpNTTBmbfokMx0qjNFrj58EsvO1Fk1Wy Oyejq9HJZ3XwSq3YEkPetcQcLkPqPe4yKyHsA1XK1pWD10eK4V+vfq7k53UGGMqJ9yh1 Gt8IidEtFQtMrs4G30Hb204rwjBIzzvJuMgflkzScapjW3QnXxb9BMe3jnoVjq5ScV+n bUwjgZQQ4xeq62+1s49FMyEQF0aRBqShRZ7CZ4BBqqaWXlm6ONAt/hyFirDqEk7g5cqO Fdjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690342050; x=1690946850; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U6xj4wxFn0rypQoF1PGqlJBlsHAu3fI3DlYhEQlJySM=; b=LF3c+zmWko0dfHdeo0BfQdm4RjGpKJGwmraKtF3K/d2IpbLAFWdri0ofNcvUZ25y8u su0RAqauLwqDYXtkguKXE+1kKGkuOp8Y48U66W5J78qRAwmUwtv2UmtSuCFlLLhHM8hB Jy6A1wOFYaNh2VlTYxZl7HCMzFcj5lljQFoWwpMRS72Yjw1GmjFZZ5VNOqMjFSRwCoUd ljs6Kw0+YM5gVlIb2S69z7Cl8BmYCpZofiMJH+dMLoFsleBnGURdP2ajjYraYlL2gdwz 90djyVXNZ4PKj0KSWt2yvA9oybjsw6/AR4kprCZe5Hg/MJyDBlnpFvBn+xnCXnGOSvU6 DHlA== X-Gm-Message-State: ABy/qLbKXwc4ziHVKQLGG8Cf+vwZU5L/xVW68gJGLL5a1YRR1S6km6TL Jh9uJCJz3NOnypdo+v/+5IjIrdgM/zC3NxxSu5NKWw== X-Google-Smtp-Source: APBJJlFdoxX78Vk/RXp8V+I4N+9q9PbH0elQr7zNdlT4hfEMCNNjyi+LpianjrSduzy/W259yQruvpYdAXsXV30pwCg= X-Received: by 2002:ac8:7d55:0:b0:403:f3f5:1a8 with SMTP id h21-20020ac87d55000000b00403f3f501a8mr412509qtb.12.1690342050506; Tue, 25 Jul 2023 20:27:30 -0700 (PDT) MIME-Version: 1.0 References: <20230721094043.2506691-1-fengwei.yin@intel.com> <20230721094043.2506691-3-fengwei.yin@intel.com> <05bc90b6-4954-b945-f0d8-373f565c1248@intel.com> In-Reply-To: <05bc90b6-4954-b945-f0d8-373f565c1248@intel.com> From: Yu Zhao Date: Tue, 25 Jul 2023 21:26:54 -0600 Message-ID: Subject: Re: [RFC PATCH v2 2/4] madvise: Use notify-able API to clear and flush page table entries To: Yin Fengwei Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, minchan@kernel.org, willy@infradead.org, david@redhat.com, ryan.roberts@arm.com, shy828301@gmail.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 8A74A1C0011 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: k3ko1g39d3m54f3k9jjf9k39q74riypy X-HE-Tag: 1690342051-534758 X-HE-Meta: U2FsdGVkX1/y71UD4MfVfw3I5k/uGz9HtTEEEX299DUnAdQqNb9I+y4V+opsAyfhcYZpwXGhDx5ajemIGFBCsG0dU1YqnRvC7fg6Ah4YfqoY7529EoNDiKFx7WrGq2U446RPBwVyYgx+X6ZznrC2hxFiqGyUR6/9VC65+ThgDIJTfHE5qcUcC2WFAOdqi+kR0oi2L8McexQojexDpahV7raVOwv2OrI64jEodXun/1oZUmbr2dMZT3xjk+woR89yG+PLF0TQvMK0MG4pJY/J9pf9uuiX3Djtd+4VVQfWKicIUPl0H2JmAKNTWZlrsbt3/b6apw4E38PriFn0BdPj8EnzlqLrzrbiSu4IKwg5OdIjaF2bnwF73HzdhBrJtVjvrv1YEsZWIn56Q9tiR6ELAaUaPZSf/l9F0wQ8EMufn6YeCCNSNeFuqj4OlASY2gcIQki/9jTqdSQKfTJD1MZUkgspR98cVabqeYX9DIc9sRpz43OFKhCKZtLBpe7nWp7Il8qNRHGANMhdJlhIhhtQIT68fn58PrkBiW3eHCYYOUAjPuUmNW1lCCmMMI//ouTnb0rgI0awAcOrH7XuM6y8AmWq8q9TiPXbfvIJ0rTADJQ5cOE75ZgjhbApDfp3fjuKtnc+eARR7mPch/40WuSytju6W4zWIfYBtHagXZOcmSKHMSGCFQzj4/gapRdHhzuhvEU3JXmhQCFtkykFgF4CjFgCcgj/KMuzM6kaMJLxJvrW6+R6mKjQ5mc+9gtwISSdtK7qoy5edxxE41nCW3Gt1n5OejR1vtavFrhzjoub93K32lOYcv9TZNcAqSsd3Zv8GBWO1fHAURxJzzgDM0TflLtxayoUneR7r9hOg/B+2HMatdeToPMLHK2BXjTEZzrnLwacqeW/tey6/Tai18vfqgWxl/0AG/KZlhhQmNrhMeUrybdfDlEyNAySsIxTQ3bgJkOCTWFzEuEzhpgxwa1 iRnuaK1u w1STgKogZbGZTd790Rza5j/FP3TdHORA9mujy4F67iCYMEf2KAylc0r7Ad4IeCShQ2kecSzgrWgxkxKZLWZWcWA2vRJgcLQW6ilr77WGUBBdHNL7tTFomRhHBbSXjjrPpmLBpqrtyiMBd0NENEpm1DqKJREp7HE8mIchz1zKxJw1SnABr529IcNe9cg79ek/BWa1czBwblITEjqNvpghpwLly5iebdHmONZL5BzRDbwsvcXgBFYkZB7Wicw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jul 25, 2023 at 8:49=E2=80=AFPM Yin Fengwei = wrote: > > > On 7/25/23 13:55, Yu Zhao wrote: > > On Fri, Jul 21, 2023 at 3:41=E2=80=AFAM Yin Fengwei wrote: > >> > >> Currently, in function madvise_cold_or_pageout_pte_range(), the > >> young bit of pte/pmd is cleared notify subscripter. > >> > >> Using notify-able API to make sure the subscripter is signaled about > >> the young bit clearing. > >> > >> Signed-off-by: Yin Fengwei > >> --- > >> mm/madvise.c | 18 ++---------------- > >> 1 file changed, 2 insertions(+), 16 deletions(-) > >> > >> diff --git a/mm/madvise.c b/mm/madvise.c > >> index f12933ebcc24..b236e201a738 100644 > >> --- a/mm/madvise.c > >> +++ b/mm/madvise.c > >> @@ -403,14 +403,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_= t *pmd, > >> return 0; > >> } > >> > >> - if (pmd_young(orig_pmd)) { > >> - pmdp_invalidate(vma, addr, pmd); > >> - orig_pmd =3D pmd_mkold(orig_pmd); > >> - > >> - set_pmd_at(mm, addr, pmd, orig_pmd); > >> - tlb_remove_pmd_tlb_entry(tlb, pmd, addr); > >> - } > >> - > >> + pmdp_clear_flush_young_notify(vma, addr, pmd); > >> folio_clear_referenced(folio); > >> folio_test_clear_young(folio); > >> if (folio_test_active(folio)) > >> @@ -496,14 +489,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_= t *pmd, > >> > >> VM_BUG_ON_FOLIO(folio_test_large(folio), folio); > >> > >> - if (pte_young(ptent)) { > >> - ptent =3D ptep_get_and_clear_full(mm, addr, pt= e, > >> - tlb->fullmm); > >> - ptent =3D pte_mkold(ptent); > >> - set_pte_at(mm, addr, pte, ptent); > >> - tlb_remove_tlb_entry(tlb, pte, addr); > >> - } > >> - > >> + ptep_clear_flush_young_notify(vma, addr, pte); > > > > These two places are tricky. > > > > I agree there is a problem here, i.e., we are not consulting the mmu > > notifier. In fact, we do pageout on VMs on ChromeOS, and it's been a > > known problem to me for a while (not a high priority one). > > > > tlb_remove_tlb_entry() is batched flush, ptep_clear_flush_young() is > > not. But, on x86, we might see a performance improvement since > > ptep_clear_flush_young() doesn't flush TLB at all. On ARM, there might > > be regressions though. > > > > I'd go with ptep_clear_young_notify(), but IIRC, Minchan mentioned he > > prefers flush. So I'll let him chime in. > I am OK with either way even no flush way here is more efficient for > arm64. Let's wait for Minchan's comment. Yes, and I don't think there would be any "negative" consequences without tlb flushes when clearing the A-bit. > > If we do end up with ptep_clear_young_notify(), please remove > > mmu_gather -- it should have been done in this patch. > > I suppose "remove mmu_gather" means to trigger flush tlb operation in > batched way to make sure no stale data in TLB for long time on arm64 > platform. In madvise_cold_or_pageout_pte_range(), we only need struct mmu_gather *tlb because of tlb_remove_pmd_tlb_entry(), i.e., flushing tlb after clearing the A-bit. There is no correction, e.g., potential data corruption, involved there.