From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD86AD637D8 for ; Thu, 14 Nov 2024 03:09:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42E076B008C; Wed, 13 Nov 2024 22:09:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3DD7E6B0092; Wed, 13 Nov 2024 22:09:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 27E036B0098; Wed, 13 Nov 2024 22:09:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 097286B008C for ; Wed, 13 Nov 2024 22:09:17 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id AEAE7A0F08 for ; Thu, 14 Nov 2024 03:09:16 +0000 (UTC) X-FDA: 82783218210.23.6288438 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by imf27.hostedemail.com (Postfix) with ESMTP id 573D140196 for ; Thu, 14 Nov 2024 03:08:29 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=GfoMYVon; spf=pass (imf27.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731553610; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ic6E1MfCrTLiottFhJNcjdiSZfVmBufK3+/Ihufuuuc=; b=hLbrMh0i+4Cc6FXf6duAJ5w1frxArJ4FBHN6MQYRDvZ8em7Qx6y9692bHQCg62Eou3FZBn AwYygrr32KcBsnl8SXHYZPR0s2rohGlQ5oxae5jW55QmTkmjNd39vwlMVRzGkI0AOhjqfv wCdcbjWq1O2gSe+x/UXAz42OQPtXdBc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731553610; a=rsa-sha256; cv=none; b=Gf1UMf/Qch7O176aYKmu8jCm13gI40NSqyXYR/DiqpmvJ6eaJFA4nKI8lk8OSvCSIjdjX2 NzjlLQysp5qmyxE1mKkVp5eNw9WtHD2Wo1t1+4FjBu3A6dQ04pjpeZXGClmNrFUj9p1/gh xk5c6lEciOV7HfHWZF2lqavCVfuHDZs= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=GfoMYVon; spf=pass (imf27.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pg1-f175.google.com with SMTP id 41be03b00d2f7-7f3da2c2cb5so69888a12.2 for ; Wed, 13 Nov 2024 19:09:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1731553752; x=1732158552; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=Ic6E1MfCrTLiottFhJNcjdiSZfVmBufK3+/Ihufuuuc=; b=GfoMYVonI3AzYRZ9NCCzmzMGHALbCOFwdwNdQc52T3Aqoe7MZCwtNOVefLCPgqr6ZP KrbnXivKZ67gTlkvuxlLScwcVJnGWDawtYKjBM/UE8E/vIX5L6rHfg9Kh50oYDSpGDnj fwojTdnuPLXhTVwH/LvXxpXtTgZX9O7t5oyamtxWbgWN9Y8zkZjr3CPs1OIgjo01CeTR 51ma9nHIYvJ8IZlsTUTtTSUOLDX8PxZ9rUStbhiH9F8WHv19np9PDWQqiFfkBnjKhxuI 05smgO9BV3gJaU9VD8YqfbtXEWpordZhUHEJWfYt9ZlOyckG4p0DwTVyjRmqsvY3hMkZ QUYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731553752; x=1732158552; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Ic6E1MfCrTLiottFhJNcjdiSZfVmBufK3+/Ihufuuuc=; b=Eb88i4CaPZtFFw/mtL8uT6ptgW1m1c/IlWQ9Cawt6+v8R6httBGfAMo3I+AKZPpNWS bGFCM4f2CP5MMACoK9LfixNxyrs13m8GSOJH3ov2KexoWVGdP0zgMPjkJIi7AdQcmZY1 t/4BF0QYmesCqv4mlcwzFcrg2lrXf+tOyEp3zMlEuSfpePi4RHV6RsSZdRpZFZZ/qunu tXCDgFk1iXgVlgpPxXTvv3sVabGGJfhRUIaO83tniyRQMhGr1J/JJ7hHZXkG2X7+od+G 6Bnx6qs4B+kLWO5/EaSDY0p5tx8ol8BezuiDLtDdvXg6edrDYX6DsB9De4LQv6qHEakQ Phbg== X-Forwarded-Encrypted: i=1; AJvYcCUvbxV7AxppDllB+C35CjbwDTXGa+NyDx6QHkJZr+U+ATcotT6hD0nDbQe3afWxYkrGFQf3+yhiNQ==@kvack.org X-Gm-Message-State: AOJu0YxGfVQ/CD5R7tRHrqGq+hnXv28LC/zJvy1mqZFQZexXSM2x+E9M i948twmWFB8PRg40OsHNUSkzo8hAHx3O8gqan9sG8q73VMYislljEOggvwPK46g= X-Google-Smtp-Source: AGHT+IHsDh5m1SqaRs7selWdid+JYC0KnSB5eclPMEkrNblU32XA/gt2P8JTKPvbhc91KQh/ctYRmw== X-Received: by 2002:a17:903:234c:b0:20b:6188:fc5e with SMTP id d9443c01a7336-21183d66a42mr317181895ad.28.1731553752212; Wed, 13 Nov 2024 19:09:12 -0800 (PST) Received: from [10.84.149.95] ([203.208.167.148]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-211c7d0627esm744865ad.202.2024.11.13.19.09.06 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 13 Nov 2024 19:09:11 -0800 (PST) Message-ID: <6c2c4e04-13da-4072-8d63-d9bea0168214@bytedance.com> Date: Thu, 14 Nov 2024 11:09:04 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 3/7] mm: introduce do_zap_pte_range() Content-Language: en-US To: David Hildenbrand Cc: jannh@google.com, hughd@google.com, willy@infradead.org, mgorman@suse.de, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, zokeefe@google.com, rientjes@google.com, peterx@redhat.com, catalin.marinas@arm.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, x86@kernel.org References: <1639ac32194f2b2590852f410fd3ce3595eb730b.1730360798.git.zhengqi.arch@bytedance.com> <2fd11f54-8c0c-401d-8635-e54ebf7facc2@bytedance.com> <6e9c649f-5fc9-4fcc-928c-c4f46a74ca66@redhat.com> From: Qi Zheng In-Reply-To: <6e9c649f-5fc9-4fcc-928c-c4f46a74ca66@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 573D140196 X-Stat-Signature: ab4dm9ijdungdbfzqxtwq7u66wms5czh X-HE-Tag: 1731553709-768954 X-HE-Meta: U2FsdGVkX18EcsJO1SvFSMHn+cutr8TuBmJzncdpq4D4IrWIj2NsrKSv1gGmqXVtkS+v9F/ce9o+RvZOf6OK/rkgi/vZ4VsLGJI9U9dvCoBkf5kstjlX78VRoGIjCZyHsnG6Da8AZe+K+pbOBcJFpPXV4NMmVyUv2HAHPbsj7RfXq/tJ0oYYzqk0LW142UjyxM6iTbWveQ5ZpDioHfK0NwlidmxAm9yWKx2QVgcdF1lP6N10/bavRoanVGfcS+f0d20FE0Tz7RlWvxWfj2hW4pCIfWOFfEnHpRE1CT0t85Qz58wE5Zntedie5Vn4XjtPvBLbYkemoO7QGSHNLgLHM4JJAuRzR5/Eiq22XYH+FKtQlxdwC3xqGO+oDWVAaBg+zgqt94j6V8Xl6qlOFfaWRFISrNSZ2Rir4lavt3kxjcAzpre4SGe3/F/KJcNv0LpQ2tQkOMvRvaC1aCMiU6hfZAOKnN0kAhEgxsEinsgW/nbcIerxoB6HOcAMK+/SZNyrsWwa3X6OujdeV5XmCHXawAgDbwtDcoOAR6VM6ONAXSpfsWULLm06qlMGyAwUZMDNTbWkbfZuj9GkdFwXzgTJOrLmn5/cmas66H4HsU0OxFBL/rgerd7MQMCe6opGFdeASE+n16AVc2pojdhuZI/upAGdNJxPCg3zF5xxhj4HNSxEAOGA+bZxQgvputbqPqT7P4yTuPy5uq/oP/TCbjmGqzu2IAZxd6F8kz3HbhV2zbn0BmuYV0VRwe8ITBmG9W3CSJXSW4HYTO1a2gbI2wRnYa5Oqz78GgMLIvEl1svjTs87F4W4Br3O3Mar7Yygvwz5/0h2qYnTs4Us3ceqa3WaWrjucSJISTt/kok7P6nJHwWwbsp9yLU497lJqec085SOBc63HIzJRrmcPsXN9aY/LPymuJc7ksDKMGys5HM4oLVnCKj2HOndWJSnMolNlb98NJ1uSBSqQvys3GUzyMt iI6mVq/q Tv8/Lx0MTYkN35R4n5D11bfwZkYRI74ajdYogo8t+YE8K9geUYDn2tL3l66sAHuc4zHdb1eiLKzh1zw46PJgCqg6xiT+NIN5OlDokCUhzA1hDSDMlBroFXq8UWfVmVsnW04M0t8eSBUzTGZgY+4cY90t+NIgXmq+NttBE5WY86IsIciZc5vnx5BdAF4ul9RtSb0FbL/MQPlXYnopnskSLCRuedieFPWjztj/q2LN5LHpPEpjiyt/ECNDNSqlZxJFrXEWgYmBBlt0f9a2pWFyzWH/o6PY2OE0guWFbvKfhbhQSOiCVxLl7jI5aQsQJqoXk4e0Njf6YsY44t711weAa9G/x4/9Tz5k5PyBHeNiX+IAHquY4PKh5Y8PXhB6IShKUEpNV+HbIrE/BOLEHZvHHA4ixNmUnzaESi2dUylvKzokxSQtGLKbHB+0uyw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/11/13 19:43, David Hildenbrand wrote: > On 13.11.24 03:40, Qi Zheng wrote: >> >> >> On 2024/11/13 01:00, David Hildenbrand wrote: >>> On 31.10.24 09:13, Qi Zheng wrote: >>>> This commit introduces do_zap_pte_range() to actually zap the PTEs, >>>> which >>>> will help improve code readability and facilitate secondary checking of >>>> the processed PTEs in the future. >>>> >>>> No functional change. >>>> >>>> Signed-off-by: Qi Zheng >>>> --- >>>>    mm/memory.c | 45 ++++++++++++++++++++++++++------------------- >>>>    1 file changed, 26 insertions(+), 19 deletions(-) >>>> >>>> diff --git a/mm/memory.c b/mm/memory.c >>>> index bd9ebe0f4471f..c1150e62dd073 100644 >>>> --- a/mm/memory.c >>>> +++ b/mm/memory.c >>>> @@ -1657,6 +1657,27 @@ static inline int zap_nonpresent_ptes(struct >>>> mmu_gather *tlb, >>>>        return nr; >>>>    } >>>> +static inline int do_zap_pte_range(struct mmu_gather *tlb, >>>> +                   struct vm_area_struct *vma, pte_t *pte, >>>> +                   unsigned long addr, unsigned long end, >>>> +                   struct zap_details *details, int *rss, >>>> +                   bool *force_flush, bool *force_break) >>>> +{ >>>> +    pte_t ptent = ptep_get(pte); >>>> +    int max_nr = (end - addr) / PAGE_SIZE; >>>> + >>>> +    if (pte_none(ptent)) >>>> +        return 1; >>> >>> Maybe we should just skip all applicable pte_none() here directly. >> >> Do you mean we should keep pte_none() case in zap_pte_range()? Like >> below: >> > > No rather an addon patch that will simply skip over all > consecutive pte_none, like: > > if (pte_none(ptent)) { >     int nr; > >     for (nr = 1; nr < max_nr; nr++) { >         ptent = ptep_get(pte + nr); >         if (pte_none(ptent)) >             continue; >     } > >     max_nr -= nr; >     if (!max_nr) >         return nr; >     addr += nr * PAGE_SIZE; >     pte += nr; > } I tend to hand over the pte/addr increments here to the loop outside do_zap_pte_range(), like this: diff --git a/mm/memory.c b/mm/memory.c index bd9ebe0f4471f..2367a1c19edd6 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1657,6 +1657,36 @@ static inline int zap_nonpresent_ptes(struct mmu_gather *tlb, return nr; } +static inline int do_zap_pte_range(struct mmu_gather *tlb, + struct vm_area_struct *vma, pte_t *pte, + unsigned long addr, unsigned long end, + struct zap_details *details, int *rss, + bool *force_flush, bool *force_break) +{ + pte_t ptent = ptep_get(pte); + int max_nr = (end - addr) / PAGE_SIZE; + + if (pte_none(ptent)) { + int nr = 1; + + for (; nr < max_nr; nr++) { + ptent = ptep_get(pte + nr); + if (!pte_none(ptent)) + break; + } + + return nr; + } + + if (pte_present(ptent)) + return zap_present_ptes(tlb, vma, pte, ptent, max_nr, + addr, details, rss, force_flush, + force_break); + + return zap_nonpresent_ptes(tlb, vma, pte, ptent, max_nr, addr, + details, rss); +} + static unsigned long zap_pte_range(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr, unsigned long end, @@ -1679,28 +1709,14 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, flush_tlb_batched_pending(mm); arch_enter_lazy_mmu_mode(); do { - pte_t ptent = ptep_get(pte); - int max_nr; - - nr = 1; - if (pte_none(ptent)) - continue; - if (need_resched()) break; - max_nr = (end - addr) / PAGE_SIZE; - if (pte_present(ptent)) { - nr = zap_present_ptes(tlb, vma, pte, ptent, max_nr, - addr, details, rss, &force_flush, - &force_break); - if (unlikely(force_break)) { - addr += nr * PAGE_SIZE; - break; - } - } else { - nr = zap_nonpresent_ptes(tlb, vma, pte, ptent, max_nr, - addr, details, rss); + nr = do_zap_pte_range(tlb, vma, pte, addr, end, details, + rss, &force_flush, &force_break); + if (unlikely(force_break)) { + addr += nr * PAGE_SIZE; + break; } } while (pte += nr, addr += PAGE_SIZE * nr, addr != end); > > Assuming that it's likely more common to have larger pte_none() holes > that single ones, optimizing out the > need_resched()+force_break+incremental pte/addr increments etc. >