From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9076FC36014 for ; Tue, 1 Apr 2025 10:45:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F38C280002; Tue, 1 Apr 2025 06:45:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 77A8B280001; Tue, 1 Apr 2025 06:45:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 61C12280002; Tue, 1 Apr 2025 06:45:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 406A9280001 for ; Tue, 1 Apr 2025 06:45:15 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 76BAFB7210 for ; Tue, 1 Apr 2025 10:45:15 +0000 (UTC) X-FDA: 83285142990.02.3FB8614 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf05.hostedemail.com (Postfix) with ESMTP id D40D1100003 for ; Tue, 1 Apr 2025 10:45:13 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743504314; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1EOdvYJkFKVm1CqeC3uoFOzZ7wtsQLqjaPy+XxoMkaY=; b=N1AHm66MKDOLaGTBuoAYtAFJjeMGOk/3AWTmf59fF/b99R1TWPCbSgcBxdVqW3l/sWPIh8 WGC86cOFhUMXbSU8AlK+nM+evwNK9Ck4hkF9YUyfUnNqDHJFJUG9Hielg7I6/VVPFxTAJ9 cdMBM91wn1oQEwh1ffQ+5570Fds1d1U= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743504314; a=rsa-sha256; cv=none; b=DFKQotUgpWd7lgWjhLIl6rOwtwAEifWQ7pAwbmwZWAA2leqvQasOMYVnyK62slb5SSJta7 BkrrVn6O3HtzQB32ESDcVsNaPfGSyXMED0uY9EqkaZunJA7YWxBpL+3THlsTd446bGjUHp I27lz2J5m9GQ+nFmPmrp0nLieyRjabU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 68CD214BF; Tue, 1 Apr 2025 03:45:16 -0700 (PDT) Received: from [10.1.28.189] (XHFQ2J9959.cambridge.arm.com [10.1.28.189]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8A8A13F63F; Tue, 1 Apr 2025 03:45:11 -0700 (PDT) Message-ID: <145ec273-7223-45b8-a7f6-4e593a3cc8ee@arm.com> Date: Tue, 1 Apr 2025 11:45:09 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 2/2] mm: mincore: use folio_pte_batch() to batch process large folios Content-Language: en-GB To: Baolin Wang , akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ziy@nvidia.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <7ad05bc9299de5d954fb21a2da57f46dd6ec59d0.1742960003.git.baolin.wang@linux.alibaba.com> <54886038-3707-4ea0-bd84-00a8f4a19a6a@arm.com> From: Ryan Roberts In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam01 X-Stat-Signature: w7f1zrmqybndtxf16gmtah8ykd44jw1k X-Rspam-User: X-Rspamd-Queue-Id: D40D1100003 X-HE-Tag: 1743504313-93234 X-HE-Meta: U2FsdGVkX1+Fx2x0lddQ5WMmVhAaq/q7Nt4wI0OmFhJKhrFMQrBOXAunJP8TIavkByw6jXOMmn+Md0tY2HeN+Nb6IECJX6nnZVhYs0oF0uFl8w96NPeKznhRx7EcTGoy6QS1dlu11d4JMYF0rVF/oKhI0D5pdDArHI9rmHAyikNvcROMstH78SU+5xaoMN+rKeYf3aH45dSB8dnSFaNScnnqX+9WGBX2m20/jyOl7QD4nDSAMo4BRd/qkjbxsU7jsRvtIflSqmpQk956+IghR1ILudFlIbauakyKQhzDGPqHL3dsLErwhIzZLCnY351+M3HsLzh5mlKTa5shuQ5AKCYS+X0V8eHl4uHGxgzw4yhG4wtJbdDY0/v4iNu5z4Wifr+GiiapcwoyVDmEwXq5FKG9SfLmJraD4zngOSyZTkjnLJaceCEUHSilTq4bETnPeljn/rSyeuWiA04EwSuzGyoF9ou/XECCSYrBsuVSornpawPQHh/S26uV4XAUWejOY6s3/Kp5agK3GDMcTsEdAZBPsABnOx0LWeU77bqC4Kq50p+Ylsy3NOCHRGxFfyUnqSCcNQX8BH8tS5OQbJMCTD98Injx7PjqBjt02EgNHFTxvnMAMf4AtCDIXvzUJbfoTNjUWcciYol4GKmE8kMEI8dp4HGWvxeec/CdxrU4IxCprd2Eo6GwT6Uv06e4JZVWKjhDlLVSRHnmfQMxuJIRBROWfSmB/tgPq+9sqcY0xYdBJOuZgPiB79CQL4mMqe3vQZJLdZW0LzMI0g18W8wlFFVRxOt6Fn+Eas+dKiHBgxHyOOIR7hZd4ebZ+HWXq9ARG5P1WqXY26KFljcEgQ2r7RM5UvPCAS/+fOemygf8bksKDWrQoPSqVXUayHzDzP8Kpr/Ls0wMr6I= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 30/03/2025 15:57, Baolin Wang wrote: > > > On 2025/3/27 22:08, Ryan Roberts wrote: >> On 25/03/2025 23:38, Baolin Wang wrote: >>> When I tested the mincore() syscall, I observed that it takes longer with >>> 64K mTHP enabled on my Arm64 server. The reason is the mincore_pte_range() >>> still checks each PTE individually, even when the PTEs are contiguous, >>> which is not efficient. >>> >>> Thus we can use folio_pte_batch() to get the batch number of the present >>> contiguous PTEs, which can improve the performance. I tested the mincore() >>> syscall with 1G anonymous memory populated with 64K mTHP, and observed an >>> obvious performance improvement: >>> >>> w/o patch        w/ patch        changes >>> 6022us            1115us            +81% >>> >>> Moreover, I also tested mincore() with disabling mTHP/THP, and did not >>> see any obvious regression. >>> >>> Signed-off-by: Baolin Wang >>> --- >>>   mm/mincore.c | 27 ++++++++++++++++++++++----- >>>   1 file changed, 22 insertions(+), 5 deletions(-) >>> >>> diff --git a/mm/mincore.c b/mm/mincore.c >>> index 832f29f46767..88be180b5550 100644 >>> --- a/mm/mincore.c >>> +++ b/mm/mincore.c >>> @@ -21,6 +21,7 @@ >>>     #include >>>   #include "swap.h" >>> +#include "internal.h" >>>     static int mincore_hugetlb(pte_t *pte, unsigned long hmask, unsigned long >>> addr, >>>               unsigned long end, struct mm_walk *walk) >>> @@ -105,6 +106,7 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long >>> addr, unsigned long end, >>>       pte_t *ptep; >>>       unsigned char *vec = walk->private; >>>       int nr = (end - addr) >> PAGE_SHIFT; >>> +    int step, i; >>>         ptl = pmd_trans_huge_lock(pmd, vma); >>>       if (ptl) { >>> @@ -118,16 +120,31 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long >>> addr, unsigned long end, >>>           walk->action = ACTION_AGAIN; >>>           return 0; >>>       } >>> -    for (; addr != end; ptep++, addr += PAGE_SIZE) { >>> +    for (; addr != end; ptep += step, addr += step * PAGE_SIZE) { >>>           pte_t pte = ptep_get(ptep); >>>   +        step = 1; >>>           /* We need to do cache lookup too for pte markers */ >>>           if (pte_none_mostly(pte)) >>>               __mincore_unmapped_range(addr, addr + PAGE_SIZE, >>>                            vma, vec); >>> -        else if (pte_present(pte)) >>> -            *vec = 1; >>> -        else { /* pte is a swap entry */ >>> +        else if (pte_present(pte)) { >>> +            if (pte_batch_hint(ptep, pte) > 1) { >>> +                struct folio *folio = vm_normal_folio(vma, addr, pte); >>> + >>> +                if (folio && folio_test_large(folio)) { >>> +                    const fpb_t fpb_flags = FPB_IGNORE_DIRTY | >>> +                                FPB_IGNORE_SOFT_DIRTY; >>> +                    int max_nr = (end - addr) / PAGE_SIZE; >>> + >>> +                    step = folio_pte_batch(folio, addr, ptep, pte, >>> +                            max_nr, fpb_flags, NULL, NULL, NULL); >>> +                } >>> +            } >> >> You could simplify to the following, I think, to avoid needing to grab the folio >> and call folio_pte_batch(): >> >>             else if (pte_present(pte)) { >>                 int max_nr = (end - addr) / PAGE_SIZE; >>                 step = min(pte_batch_hint(ptep, pte), max_nr); >>             } ... >> >> I expect the regression you are seeing here is all due to calling ptep_get() for >> every pte in the contpte batch, which will cause 16 memory reads per pte (to >> gather the access/dirty bits). For small folios its just 1 read per pte. > > Right. > >> pte_batch_hint() will skip forward in blocks of 16 so you now end up with the >> same number as for the small folio case. You don't need all the fancy extras >> that folio_pte_batch() gives you here. > > Sounds reasonable. Your suggestion looks simple, but my method can batch the > whole large folio (such as large folios containing more than 16 contiguous PTEs) > at once. Sure but folio_pte_batch() just implements that with another loop that calls pte_batch_hint(), so it all amounts to the same thing. In fact there are some extra checks in folio_pte_batch() that you don't need so it might be a bit slower. Thanks, Ryan > Anyway, let me do some performance measurements for your suggestion. > Thanks.