From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAC09C28B20 for ; Sun, 30 Mar 2025 19:57:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 12C6C280002; Sun, 30 Mar 2025 15:57:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0B5B1280001; Sun, 30 Mar 2025 15:57:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9927280002; Sun, 30 Mar 2025 15:57:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C7D78280001 for ; Sun, 30 Mar 2025 15:57:31 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A429758ECF for ; Sun, 30 Mar 2025 19:57:33 +0000 (UTC) X-FDA: 83279277186.06.C2B4645 Received: from out30-124.freemail.mail.aliyun.com (out30-124.freemail.mail.aliyun.com [115.124.30.124]) by imf07.hostedemail.com (Postfix) with ESMTP id 1FA5440003 for ; Sun, 30 Mar 2025 19:57:30 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=BPgobB6+; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf07.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.124 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743364652; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ni9PDOb25HWqXvjusXB3wdjsKAybc+x8kFq39soa9YU=; b=fklFzB6ge58C03Or0i9PnQAkb8n519kfk3F9ELXCflkmlYF830sjF06gSFbcq2K743rXDA yuTdSCU0ESTAxMoajFhGYcCU3gOEuB3wh/fj9r7h3O/Kty0Y6jH2yxg8lw3dQeBPECbWlq 6JUGqKnHRmD3wXL7dsaK1T5fZOHS1aQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743364652; a=rsa-sha256; cv=none; b=QZ9hZ+78v6b5B7tIa34v1LEOtisUHQTh05VfM8bHk4FUnn6Omjqr/W7IkzXlYtqEeg+iav iwGMylrDwI2bdItNCTD33ZNasbzKjEAbDFvOnb6+CFvpnwVlD6Lno1vDyzpYWNr5La2rh4 998OAaTewvlvgvA1eDb+nxGRO0u/cLE= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=BPgobB6+; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf07.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.124 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1743364648; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type; bh=ni9PDOb25HWqXvjusXB3wdjsKAybc+x8kFq39soa9YU=; b=BPgobB6+6olZPYNQEEbvGsOawFKYaDKUcX3c5AGZOanqJ5Otkuuw2dfeZjz2yKfshr8B2SIS8bEJ1T9NSxx/qd/OGvY0E2G3nit0Nv7hFZQTDnkJHtThP9VeRn+bsYz8Vrt0bTiCAEpj2nOar4S3SjP08oHY3suPmXsdX4eMQuk= Received: from 30.58.211.89(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WTNZI5i_1743364641 cluster:ay36) by smtp.aliyun-inc.com; Mon, 31 Mar 2025 03:57:26 +0800 Message-ID: Date: Sun, 30 Mar 2025 20:57:19 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 2/2] mm: mincore: use folio_pte_batch() to batch process large folios To: Ryan Roberts , akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, 21cnbao@gmail.com, ziy@nvidia.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <7ad05bc9299de5d954fb21a2da57f46dd6ec59d0.1742960003.git.baolin.wang@linux.alibaba.com> <54886038-3707-4ea0-bd84-00a8f4a19a6a@arm.com> From: Baolin Wang In-Reply-To: <54886038-3707-4ea0-bd84-00a8f4a19a6a@arm.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1FA5440003 X-Stat-Signature: k3ct54niaymkxufnmne3fxsdp5k5grpe X-Rspam-User: X-HE-Tag: 1743364650-612908 X-HE-Meta: U2FsdGVkX1/XyQmWl8mCzL6hIyKH24RIrs95ovi8OVoEFGgqe76t5ZpVQ1kyGPGrB+EHfXYptDiy59FoQOIpdnHTrlVe2AE/W7ZB//+C08WPzTtN55IRh2QY6wRWcZUh2ccrG9HOUvk5k7GZoMKvGXeC2q3wgEoabWsr2KShkQX3ih5jMzfY5eGz2TNj3wxjeQ7c05robsgeuIpZsRgHhASNmSf+rdfXiv8/XHecqKCPXinNEgmrvLpQrgZNVXWyihhThSxHDkYgB5PE43XefUdMLybqVKobNVgT1bjKLX+AT8STYJ/294ffGSkfGx8rzDsSkhj5k9nREn9j3dUC6J/abXIgxpjSGVuO1x0V695bSHra+3QJbPBq/fjYupHmZRMSRO05TwIMzWFnJvICb/mCUNVV88tXtq5MzdAh6eip9u2wtwYWiSVdmZ3v17TtaSs92dkT0gkvdYCrbMVz543ZbGpJWA85XXwbQmLWPot9CQs8hyZruiR9dOZlD9AfpzWGqbLtjt3ndJDDkG7LcHLJoTQlOrw+7V/+sD9sNEOgpia7eG1cDkfJEjqCjRTUghvAqHS/Amg/LjLPwyM4WNxxQOomGl3VvUM1sfi0Pd7FdUIZTMYnMbYXVBzYeCegXtT2zvQa+bNdvrKvAGbz/ZK8ugl7lza1Pr/8fvPNXK9Jf5H8sSbJEzrtq2mTXbuOr1tmBu+JWNN25dnndSi9YUCBkTFEsLYfLS0BOVHOhdICfNVbeYodRCAOPcOdTCvQw+FKG17qSttSxwXR+srLlkxZtyob7zvJbifFlrhAh5eOqT4sHA8Z6bXhZCEuPgNLXrfZKlhuoqGQuFkiJDXgP40Wczr0rwO6RI9rxEiNoFkjxQ2+qPqk05iHVxpAzMJN5X2+iLzGXyWNUx0J/AjzYhObnhLX3csjsfFRg/8IGaP9AOenGSG+TVMRxWNx+Mgi8agZlQ9tj+F05RlgIta X6ZnFgEM MKnbwnZ5pRjRRaYa6g8iz+O2H2aVA3kBZ+AMMC+qrLp3rqKTcW0PMJP/i5wTbazP506sKJhH8AXpiZCbKr1gKaZnnjnmhFndeyZfFvFaxglfAHkIUSwY15azaYpomSPoSJZAFsoAOuIcRhRVDZA/P5Nr+ZmPckuVmprfnj1HXQLRkiKYFr1nhPcvMomTio0lMrTHzb8qDHbxZ4+YjWTFVlyn+dsFq7LD+UqL+WFO7Kdcj+INiGAXxUzOLLO0JNJwE24kb04R6Eq7a26LA3oT53x+aqMXCKAFxZ9z0prH41Nd4s7UBwDoNPNSeplOwmZly9ynv X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2025/3/27 22:08, Ryan Roberts wrote: > On 25/03/2025 23:38, Baolin Wang wrote: >> When I tested the mincore() syscall, I observed that it takes longer with >> 64K mTHP enabled on my Arm64 server. The reason is the mincore_pte_range() >> still checks each PTE individually, even when the PTEs are contiguous, >> which is not efficient. >> >> Thus we can use folio_pte_batch() to get the batch number of the present >> contiguous PTEs, which can improve the performance. I tested the mincore() >> syscall with 1G anonymous memory populated with 64K mTHP, and observed an >> obvious performance improvement: >> >> w/o patch w/ patch changes >> 6022us 1115us +81% >> >> Moreover, I also tested mincore() with disabling mTHP/THP, and did not >> see any obvious regression. >> >> Signed-off-by: Baolin Wang >> --- >> mm/mincore.c | 27 ++++++++++++++++++++++----- >> 1 file changed, 22 insertions(+), 5 deletions(-) >> >> diff --git a/mm/mincore.c b/mm/mincore.c >> index 832f29f46767..88be180b5550 100644 >> --- a/mm/mincore.c >> +++ b/mm/mincore.c >> @@ -21,6 +21,7 @@ >> >> #include >> #include "swap.h" >> +#include "internal.h" >> >> static int mincore_hugetlb(pte_t *pte, unsigned long hmask, unsigned long addr, >> unsigned long end, struct mm_walk *walk) >> @@ -105,6 +106,7 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, >> pte_t *ptep; >> unsigned char *vec = walk->private; >> int nr = (end - addr) >> PAGE_SHIFT; >> + int step, i; >> >> ptl = pmd_trans_huge_lock(pmd, vma); >> if (ptl) { >> @@ -118,16 +120,31 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, >> walk->action = ACTION_AGAIN; >> return 0; >> } >> - for (; addr != end; ptep++, addr += PAGE_SIZE) { >> + for (; addr != end; ptep += step, addr += step * PAGE_SIZE) { >> pte_t pte = ptep_get(ptep); >> >> + step = 1; >> /* We need to do cache lookup too for pte markers */ >> if (pte_none_mostly(pte)) >> __mincore_unmapped_range(addr, addr + PAGE_SIZE, >> vma, vec); >> - else if (pte_present(pte)) >> - *vec = 1; >> - else { /* pte is a swap entry */ >> + else if (pte_present(pte)) { >> + if (pte_batch_hint(ptep, pte) > 1) { >> + struct folio *folio = vm_normal_folio(vma, addr, pte); >> + >> + if (folio && folio_test_large(folio)) { >> + const fpb_t fpb_flags = FPB_IGNORE_DIRTY | >> + FPB_IGNORE_SOFT_DIRTY; >> + int max_nr = (end - addr) / PAGE_SIZE; >> + >> + step = folio_pte_batch(folio, addr, ptep, pte, >> + max_nr, fpb_flags, NULL, NULL, NULL); >> + } >> + } > > You could simplify to the following, I think, to avoid needing to grab the folio > and call folio_pte_batch(): > > else if (pte_present(pte)) { > int max_nr = (end - addr) / PAGE_SIZE; > step = min(pte_batch_hint(ptep, pte), max_nr); > } ... > > I expect the regression you are seeing here is all due to calling ptep_get() for > every pte in the contpte batch, which will cause 16 memory reads per pte (to > gather the access/dirty bits). For small folios its just 1 read per pte. Right. > pte_batch_hint() will skip forward in blocks of 16 so you now end up with the > same number as for the small folio case. You don't need all the fancy extras > that folio_pte_batch() gives you here. Sounds reasonable. Your suggestion looks simple, but my method can batch the whole large folio (such as large folios containing more than 16 contiguous PTEs) at once. Anyway, let me do some performance measurements for your suggestion. Thanks.