From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0576D7494E for ; Wed, 30 Oct 2024 01:29:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 303118D0002; Tue, 29 Oct 2024 21:29:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2B1F18D0001; Tue, 29 Oct 2024 21:29:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 103938D0002; Tue, 29 Oct 2024 21:29:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id DDE938D0001 for ; Tue, 29 Oct 2024 21:29:42 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4CC9F1202A9 for ; Wed, 30 Oct 2024 01:29:42 +0000 (UTC) X-FDA: 82728536142.11.8751041 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) by imf26.hostedemail.com (Postfix) with ESMTP id 4EF38140002 for ; Wed, 30 Oct 2024 01:29:20 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=FMj1dPDs; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf26.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.18 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730251662; a=rsa-sha256; cv=none; b=tCsXhnSasjt49l+Y7st1tm9BGLbEBFJJtLVcbj20v8ca98YftRg4L3lUX8qY6YsY6l2m2G iitAD+kZ5PQJI+LOyChHXlOA4/RPKy4x8px7B2JXLPtP9yqvnPMQt/coCTc6nJUi8JdPdT GJLR1n/iGee2U//d9gRwGJhvq0YhOrA= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=FMj1dPDs; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf26.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.18 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730251662; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qCkxQduFOsBH+QrnIW7v5WW8s0l8O5ZIpyQm5Tf8E7U=; b=JwEXAHf7ja5mf+Ipkz/hXzK0vJyRf2YiZsZ28GFxr3SFqYEEL2VySIhfxi6h60SZVyE8wN I8dY3uohCIL8naXQLVEug3EdJvZupSt4nzwS0GW/s8YVT6kPt2KS45elGucuMZz3a5+Tqa bCq15eQV5PjIN4xiuhH7MenMU6YRGQE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730251780; x=1761787780; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version:content-transfer-encoding; bh=SYCSJ0g602I6InVLRWPb4JxvBmwfjQId3qOLZ85mYeU=; b=FMj1dPDsAgz/JfLzuAOPO4YzUyQft2vfE/gNc3B0NRtJq+byW2qUvNpU 2EOmtIH/24unBaThKIAwvDZLZ6BQCB9Cb+56vGB0YFBtZNH/RmGVWB5Eb HlfXY+v/7XoyHWK9DHpiA9/VH1yqy1Iv5N8TMoHXbhCxqdbvWSzBD+qu5 q6rfxseydQonwyUUcJynyXUQVqcnmFCQ5x+nqiW7Y82oHd4VT8wc7mlho jC2j+CdncajNsiBlxlv3OAKMoUR3ULwp0HTwD8bFAFnMS4hmKjl19vZ8h nNkiuHdAiGRqlF+vM7W+WdQahvF/PWu9Qay4wHuO3QGY79iSZC2w6uKEW Q==; X-CSE-ConnectionGUID: bU90m0ixQn6ZYZitok6nzA== X-CSE-MsgGUID: PpXn8zteRXSQI2syRXPc8w== X-IronPort-AV: E=McAfee;i="6700,10204,11240"; a="29357181" X-IronPort-AV: E=Sophos;i="6.11,243,1725346800"; d="scan'208";a="29357181" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Oct 2024 18:29:38 -0700 X-CSE-ConnectionGUID: YKDdiC1LTRC+VlFI3oJb4g== X-CSE-MsgGUID: 9EWqZVDoSXiUrVF3InOD6g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,243,1725346800"; d="scan'208";a="112985825" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Oct 2024 18:29:35 -0700 From: "Huang, Ying" To: Kefeng Wang Cc: Barry Song <21cnbao@gmail.com>, , , , , , Subject: Re: [PATCH] mm: shmem: convert to use folio_zero_range() In-Reply-To: <2524689c-08f5-446c-8cb9-924f9db0ee3a@huawei.com> (Kefeng Wang's message of "Mon, 28 Oct 2024 19:41:55 +0800") References: <06d99b89-17ad-447e-a8f1-8e220b5688ac@huawei.com> <20241022225603.10491-1-21cnbao@gmail.com> <31afe958-91eb-484a-90b9-91114991a9a2@huawei.com> <87iktg3n2i.fsf@yhuang6-desk2.ccr.corp.intel.com> <5ccb295c-6ec9-4d00-8236-e3ba19221f40@huawei.com> <875xpg39q5.fsf@yhuang6-desk2.ccr.corp.intel.com> <1a37d4a9-eef8-4fe0-aeb0-fa95c33b305a@huawei.com> <871q042x1h.fsf@yhuang6-desk2.ccr.corp.intel.com> <86f9f4e8-9c09-4333-ae4f-f51a71c3aca7@huawei.com> <87ttcx0x4j.fsf@yhuang6-desk2.ccr.corp.intel.com> <9318769a-0d51-4c03-a808-fc3a3f09d492@huawei.com> <2524689c-08f5-446c-8cb9-924f9db0ee3a@huawei.com> Date: Wed, 30 Oct 2024 09:26:03 +0800 Message-ID: <874j4uwfdg.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4EF38140002 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 5tspsapxgn4zdx4gfp4ucjdzcaro54nh X-HE-Tag: 1730251760-440600 X-HE-Meta: U2FsdGVkX1+V4DPnrSTJBXpdEYDVz82o1FzAsxH7WSxg5X+4gwuW4SIYQ2/k/WUXOAxwi/D6gEtFpDegqhITAHerMWpPEVQVOdeYSWGepLCCKDGk9+eMbQjXR0UXkAcf08xzGSock08exrecJLt7p69yDXhxBNS8YAURz5Po4Dl/6vcEIJWrGLXdd3MIThOLf9wG2o9ESwZmqdj4CxM23IUkatEAG4eumR1mEhzLFfiOclbSdTnmFuFsTBQDfNJB/5Yd2/iIJEPEUYk4jtWC5ZbXzUHg5+q8p40m+mKHP/8LLW87XDJk48de8oaUWZn50Ztda5fqthYaHOZ6j7/LDfaZ7Aac3w6sUbyaC6v61EGYDLMHRUUI/3j/3uokCq3MQ11avvvmRMjEwkEdhXC5NLMcoLQKVEuxumTkBcnjrNMg7e8xmylnQEepxRuCmdSnh4cq7NaW9zdQVSAb4co3EnkBu9iap+B1CqnYuY+fR10VhOhNfprb1IZRs3tT5fzGPYtedKOlSK/b0FclnP4Kn1PoSoAcYSnGFisZExYVtvISOi4s4i40V5Uqxl1ra5Jl4Bizq0u2Csr40Tl80nG0K4rp/fzJbdtZXOJErAhnV78x49ZIEN84B9wAvRNsdu1vSrYXuiJQJyVeI/KjxGeOrdE/5yE0lQR7RLJNiGFo1C9t/xvg7gGQ2UbqAWBxxZbXHSkBQ1j7uu9EymLHvrN1wFg5i9Q0VStbf+wyU77XFYZHaUcKwtPwcV7c7UMv+QQCNMsBm2xz8g1APoZ1gGfq2UuROgR9vChP4WZ98b/NC0r0xMz4G6QF5MQNKcxbRoPEgh/+j3HmpIH0+zeD7MKH4sqMmGhJhGoNgYgz9JG36pBwbzhVsoSDmeEfYBIacMLoVKvlshaXM3b+LlMbNL/JkdK9w0NOAGksp4pcVIcdRRwoIptfNxTX1GnhybW8VOW1KOAAGq5vXDdDEoUB1ni jARDQtjA eU4RaNw1uDr+pEvU7AXoqJIhtdqDrdO3SATsqbBEVqaAdvTwGbDM8h+EcpSxs8QtsWiDM94ICBwK75mAlxqPyW4nRIXkiZOmsBmxJMZroiz2HRZsx4bhEcg1js6ZYHx1+4ajIiUXAs26F0qLJlX+OPoSYxalVi9Sm2XIbJ5T+IBx8faYe83p0LiPqgHI2u+t9RGaOvRJCRM+9BIYqf7XTyQjB52SJDzAL/YpfDOYM1oeS3/I= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi, Kefeng, Sorry for late reply, the email is buried in my inbox, just dig it out. Kefeng Wang writes: > On 2024/10/28 14:37, Kefeng Wang wrote: >> On 2024/10/28 10:39, Huang, Ying wrote: >>> Kefeng Wang writes: >>> >>>> On 2024/10/25 20:21, Huang, Ying wrote: >>>>> Kefeng Wang writes: >>>>> >>>>>> On 2024/10/25 15:47, Huang, Ying wrote: >>>>>>> Kefeng Wang writes: >>>>>>> >>>>>>>> On 2024/10/25 10:59, Huang, Ying wrote: >>>>>>>>> Hi, Kefeng, >>>>>>>>> Kefeng Wang writes: >>>>>>>>> >>>>>>>>>> +CC Huang Ying, >>>>>>>>>> >>>>>>>>>> On 2024/10/23 6:56, Barry Song wrote: >>>>>>>>>>> On Wed, Oct 23, 2024 at 4:10=E2=80=AFAM Kefeng Wang >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> ... >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 2024/10/17 23:09, Matthew Wilcox wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 17, 2024 at 10:25:04PM +0800, >>>>>>>>>>>>>>>>>>>>>>>>>>> Kefeng Wang wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> Directly use folio_zero_range() to cleanup cod= e. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Are you sure there's no performance >>>>>>>>>>>>>>>>>>>>>>>>>>> regression introduced by this? >>>>>>>>>>>>>>>>>>>>>>>>>>> clear_highpage() is often optimised in ways >>>>>>>>>>>>>>>>>>>>>>>>>>> that we can't optimise for >>>>>>>>>>>>>>>>>>>>>>>>>>> a plain memset(). =C2=A0On the other hand, if >>>>>>>>>>>>>>>>>>>>>>>>>>> the folio is large, maybe a >>>>>>>>>>>>>>>>>>>>>>>>>>> modern CPU will be able to do better than >>>>>>>>>>>>>>>>>>>>>>>>>>> clear-one-page-at-a-time. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Right, I missing this, clear_page might be >>>>>>>>>>>>>>>>>>>>>>>>>> better than memset, I change >>>>>>>>>>>>>>>>>>>>>>>>>> this one when look at the shmem_writepage(), >>>>>>>>>>>>>>>>>>>>>>>>>> which already convert to >>>>>>>>>>>>>>>>>>>>>>>>>> use folio_zero_range() from >>>>>>>>>>>>>>>>>>>>>>>>>> clear_highpage(), also I grep >>>>>>>>>>>>>>>>>>>>>>>>>> folio_zero_range(), there are some other to >>>>>>>>>>>>>>>>>>>>>>>>>> use folio_zero_range(). >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> fs/bcachefs/fs-io-buffered.c: >>>>>>>>>>>>>>>>>>>>>>>>>> folio_zero_range(folio, 0, >>>>>>>>>>>>>>>>>>>>>>>>>> folio_size(folio)); >>>>>>>>>>>>>>>>>>>>>>>>>> fs/bcachefs/fs-io-buffered.c: >>>>>>>>>>>>>>>>>>>>>>>>>> folio_zero_range(f, >>>>>>>>>>>>>>>>>>>>>>>>>> 0, folio_size(f)); >>>>>>>>>>>>>>>>>>>>>>>>>> fs/bcachefs/fs-io-buffered.c: >>>>>>>>>>>>>>>>>>>>>>>>>> folio_zero_range(f, >>>>>>>>>>>>>>>>>>>>>>>>>> 0, folio_size(f)); >>>>>>>>>>>>>>>>>>>>>>>>>> fs/libfs.c: =C2=A0 =C2=A0 folio_zero_range(folio= , 0, >>>>>>>>>>>>>>>>>>>>>>>>>> folio_size(folio)); >>>>>>>>>>>>>>>>>>>>>>>>>> fs/ntfs3/frecord.c: >>>>>>>>>>>>>>>>>>>>>>>>>> folio_zero_range(folio, 0, >>>>>>>>>>>>>>>>>>>>>>>>>> folio_size(folio)); >>>>>>>>>>>>>>>>>>>>>>>>>> mm/page_io.c: =C2=A0 folio_zero_range(folio, 0, >>>>>>>>>>>>>>>>>>>>>>>>>> folio_size(folio)); >>>>>>>>>>>>>>>>>>>>>>>>>> mm/shmem.c: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 >>>>>>>>>>>>>>>>>>>>>>>>>> folio_zero_range(folio, 0, >>>>>>>>>>>>>>>>>>>>>>>>>> folio_size(folio)); >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> IOW, what performance testing have you done >>>>>>>>>>>>>>>>>>>>>>>>>>> with this patch? >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> No performance test before, but I write a >>>>>>>>>>>>>>>>>>>>>>>>>> testcase, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> 1) allocate N large folios >>>>>>>>>>>>>>>>>>>>>>>>>> (folio_alloc(PMD_ORDER)) >>>>>>>>>>>>>>>>>>>>>>>>>> 2) then calculate the diff(us) when clear >>>>>>>>>>>>>>>>>>>>>>>>>> all N folios >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 >>>>>>>>>>>>>>>>>>>>>>>>>> clear_highpage/folio_zero_range/ >>>>>>>>>>>>>>>>>>>>>>>>>> folio_zero_user >>>>>>>>>>>>>>>>>>>>>>>>>> 3) release N folios >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> the result(run 5 times) shown below on my machin= e, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> N=3D1, >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 clear_highpage >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0folio_zero_range =C2=A0 =C2=A0folio_zero_u= ser >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A01 =C2=A0 =C2=A0 =C2=A069 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 74 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= 177 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A02 =C2=A0 =C2=A0 =C2=A057 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 62 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= 168 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A03 =C2=A0 =C2=A0 =C2=A054 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 58 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= 234 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A04 =C2=A0 =C2=A0 =C2=A054 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 58 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= 157 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A05 =C2=A0 =C2=A0 =C2=A056 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 62 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= 148 >>>>>>>>>>>>>>>>>>>>>>>>>> avg =C2=A0 =C2=A0 =C2=A0 58 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 62.8 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0 176.8 >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> N=3D100 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 clear_highpage >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0folio_zero_range =C2=A0 =C2=A0folio_zero_u= ser >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A01 =C2=A0 =C2=A011015 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 >>>>>>>>>>>>>>>>>>>>>>>>>> 11309 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 32833 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A02 =C2=A0 =C2=A010385 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 >>>>>>>>>>>>>>>>>>>>>>>>>> 11110 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 49751 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A03 =C2=A0 =C2=A010369 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 >>>>>>>>>>>>>>>>>>>>>>>>>> 11056 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 33095 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A04 =C2=A0 =C2=A010332 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 >>>>>>>>>>>>>>>>>>>>>>>>>> 11017 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 33106 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A05 =C2=A0 =C2=A010483 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 >>>>>>>>>>>>>>>>>>>>>>>>>> 11000 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 49032 >>>>>>>>>>>>>>>>>>>>>>>>>> avg =C2=A0 =C2=A0 10516.8 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 11098.4 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0 39563.4 >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> N=3D512 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 clear_highpage >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0folio_zero_range =C2=A0 folio_zero_user >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A01 =C2=A0 =C2=A055560 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 >>>>>>>>>>>>>>>>>>>>>>>>>> 60055 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0156876 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A02 =C2=A0 =C2=A055485 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 >>>>>>>>>>>>>>>>>>>>>>>>>> 60024 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0157132 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A03 =C2=A0 =C2=A055474 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 >>>>>>>>>>>>>>>>>>>>>>>>>> 60129 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0156658 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A04 =C2=A0 =C2=A055555 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 >>>>>>>>>>>>>>>>>>>>>>>>>> 59867 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0157259 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A05 =C2=A0 =C2=A055528 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 >>>>>>>>>>>>>>>>>>>>>>>>>> 59932 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0157108 >>>>>>>>>>>>>>>>>>>>>>>>>> avg =C2=A0 =C2=A0 55520.4 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 60001.4 >>>>>>>>>>>>>>>>>>>>>>>>>> =C2=A0157006.6 >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> folio_zero_user with many cond_resched(), so >>>>>>>>>>>>>>>>>>>>>>>>>> time fluctuates a lot, >>>>>>>>>>>>>>>>>>>>>>>>>> clear_highpage is better folio_zero_range as >>>>>>>>>>>>>>>>>>>>>>>>>> you said. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Maybe add a new helper to convert all >>>>>>>>>>>>>>>>>>>>>>>>>> folio_zero_range(folio, 0, >>>>>>>>>>>>>>>>>>>>>>>>>> folio_size(folio)) >>>>>>>>>>>>>>>>>>>>>>>>>> to use clear_highpage + flush_dcache_folio? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> If this also improves performance for other >>>>>>>>>>>>>>>>>>>>>>>>> existing callers of >>>>>>>>>>>>>>>>>>>>>>>>> folio_zero_range(), then that's a positive outcom= e. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> hi Kefeng, >>>>>>>>>>>>>>>>>>>>>> what's your point? providing a helper like >>>>>>>>>>>>>>>>>>>>>> clear_highfolio() or similar? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Yes, from above test, using clear_highpage/ >>>>>>>>>>>>>>>>>>>>> flush_dcache_folio is better >>>>>>>>>>>>>>>>>>>>> than using folio_zero_range() for folio >>>>>>>>>>>>>>>>>>>>> zero(especially for large >>>>>>>>>>>>>>>>>>>>> folio), so I'd like to add a new helper, maybe >>>>>>>>>>>>>>>>>>>>> name it folio_zero() >>>>>>>>>>>>>>>>>>>>> since it zero the whole folio. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> we already have a helper like folio_zero_user()? >>>>>>>>>>>>>>>>>>>> it is not good enough? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Since it is with many cond_resched(), the >>>>>>>>>>>>>>>>>>> performance is worst... >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Not exactly? It should have zero cost for a >>>>>>>>>>>>>>>>>> preemptible kernel. >>>>>>>>>>>>>>>>>> For a non-preemptible kernel, it helps avoid >>>>>>>>>>>>>>>>>> clearing the folio >>>>>>>>>>>>>>>>>> from occupying the CPU and starving other processes, >>>>>>>>>>>>>>>>>> right? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> --- a/mm/shmem.c >>>>>>>>>>>>>>>>> +++ b/mm/shmem.c >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> @@ -2393,10 +2393,7 @@ static int >>>>>>>>>>>>>>>>> shmem_get_folio_gfp(struct inode >>>>>>>>>>>>>>>>> *inode, pgoff_t index, >>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 * it now, lest undo on failure >>>>>>>>>>>>>>>>> cancel our earlier guarantee. >>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 */ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0if (sgp !=3D SGP_WRITE && ! >>>>>>>>>>>>>>>>> folio_test_uptodate(folio)) { >>>>>>>>>>>>>>>>> - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 long i= , n =3D folio_nr_pages(folio); >>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>> - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 for (i= =3D 0; i < n; i++) >>>>>>>>>>>>>>>>> - =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 >>>>>>>>>>>>>>>>> clear_highpage(folio_page(folio, i)); >>>>>>>>>>>>>>>>> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 folio_= zero_user(folio, vmf->address); >>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0flush_dcache_folio(folio); >>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0folio_mark_uptodate(folio); >>>>>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0} >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Do we perform better or worse with the following? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Here is for SGP_FALLOC, vmf =3D NULL, we could use >>>>>>>>>>>>>>>> folio_zero_user(folio, >>>>>>>>>>>>>>>> 0), I think the performance is worse, will retest once >>>>>>>>>>>>>>>> I can access >>>>>>>>>>>>>>>> hardware. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Perhaps, since the current code uses >>>>>>>>>>>>>>> clear_hugepage(). Does using >>>>>>>>>>>>>>> index << PAGE_SHIFT as the addr_hint offer any benefit? >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> when use folio_zero_user(), the performance is vary bad >>>>>>>>>>>>>> with above >>>>>>>>>>>>>> fallocate test(mount huge=3Dalways), >>>>>>>>>>>>>> >>>>>>>>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 folio_z= ero_range =C2=A0 clear_highpage >>>>>>>>>>>>>> folio_zero_user >>>>>>>>>>>>>> real =C2=A0 =C2=A00m1.214s =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 0m1.111s =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00m3.159s >>>>>>>>>>>>>> user =C2=A0 =C2=A00m0.000s =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 0m0.000s =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00m0.000s >>>>>>>>>>>>>> sys =C2=A0 =C2=A0 0m1.210s =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 0m1.109s =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00m3.152s >>>>>>>>>>>>>> >>>>>>>>>>>>>> I tried with addr_hint =3D 0/index << PAGE_SHIFT, no >>>>>>>>>>>>>> obvious different. >>>>>>>>>>>>> >>>>>>>>>>>>> Interesting. Does your kernel have preemption disabled or >>>>>>>>>>>>> preemption_debug enabled? >>>>>>>>>>>> >>>>>>>>>>>> ARM64 server, CONFIG_PREEMPT_NONE=3Dy >>>>>>>>>>> this explains why the performance is much worse. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> If not, it makes me wonder whether folio_zero_user() in >>>>>>>>>>>>> alloc_anon_folio() is actually improving performance as >>>>>>>>>>>>> expected, >>>>>>>>>>>>> compared to the simpler folio_zero() you plan to implement. := -) >>>>>>>>>>>> >>>>>>>>>>>> Yes, maybe, the folio_zero_user(was clear_huge_page) is from >>>>>>>>>>>> 47ad8475c000 ("thp: clear_copy_huge_page"), so original >>>>>>>>>>>> clear_huge_page >>>>>>>>>>>> is used in HugeTLB, clear PUD size maybe spend many time, >>>>>>>>>>>> but for PMD or >>>>>>>>>>>> other size of large folio, cond_resched is not necessary >>>>>>>>>>>> since we >>>>>>>>>>>> already have some folio_zero_range() to clear large folio, >>>>>>>>>>>> and no issue >>>>>>>>>>>> was reported. >>>>>>>>>>> probably worth an optimization. calling cond_resched() for >>>>>>>>>>> each page >>>>>>>>>>> seems too aggressive and useless. >>>>>>>>>> >>>>>>>>>> After some test, I think the cond_resched() is not the root caus= e, >>>>>>>>>> no performance gained with batched cond_resched(), even I kill >>>>>>>>>> cond_resched() from process_huge_page, no improvement. >>>>>>>>>> >>>>>>>>>> But when I unconditionally use clear_gigantic_page() in >>>>>>>>>> folio_zero_user(patched), there is big improvement with above >>>>>>>>>> fallocate on tmpfs(mount huge=3Dalways), also I test some >>>>>>>>>> other testcase, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 1) case-anon-w-seq-mt: (2M PMD THP) >>>>>>>>>> >>>>>>>>>> base: >>>>>>>>>> real=C2=A0=C2=A0=C2=A0 0m2.490s=C2=A0=C2=A0=C2=A0 0m2.254s=C2=A0= =C2=A0=C2=A0 0m2.272s >>>>>>>>>> user=C2=A0=C2=A0=C2=A0 1m59.980s=C2=A0=C2=A0 2m23.431s=C2=A0=C2= =A0 2m18.739s >>>>>>>>>> sys=C2=A0=C2=A0=C2=A0=C2=A0 1m3.675s=C2=A0=C2=A0=C2=A0 1m15.462s= =C2=A0=C2=A0 1m15.030s >>>>>>>>>> >>>>>>>>>> patched: >>>>>>>>>> real=C2=A0=C2=A0=C2=A0 0m2.234s=C2=A0=C2=A0=C2=A0 0m2.225s=C2=A0= =C2=A0=C2=A0 0m2.159s >>>>>>>>>> user=C2=A0=C2=A0=C2=A0 2m56.105s=C2=A0=C2=A0 2m57.117s=C2=A0=C2= =A0 3m0.489s >>>>>>>>>> sys=C2=A0=C2=A0=C2=A0=C2=A0 0m17.064s=C2=A0=C2=A0 0m17.564s=C2= =A0=C2=A0 0m16.150s >>>>>>>>>> >>>>>>>>>> Patched kernel win on sys and bad in user, but real is >>>>>>>>>> almost same, >>>>>>>>>> maybe a little better than base. >>>>>>>>> We can find user time difference.=C2=A0 That means the original >>>>>>>>> cache hot >>>>>>>>> behavior still applies on your system. >>>>>>>>> However, it appears that the performance to clear page from end to >>>>>>>>> begin >>>>>>>>> is really bad on your system. >>>>>>>>> So, I suggest to revise the current implementation to use >>>>>>>>> sequential >>>>>>>>> clearing as much as possible. >>>>>>>>> >>>>>>>> >>>>>>>> I test case-anon-cow-seq-hugetlb for copy_user_large_folio() >>>>>>>> >>>>>>>> base: >>>>>>>> real=C2=A0=C2=A0=C2=A0 0m6.259s=C2=A0=C2=A0=C2=A0 0m6.197s=C2=A0= =C2=A0=C2=A0 0m6.316s >>>>>>>> user=C2=A0=C2=A0=C2=A0 1m31.176s=C2=A0=C2=A0 1m27.195s=C2=A0=C2=A0= 1m29.594s >>>>>>>> sys=C2=A0=C2=A0=C2=A0=C2=A0 7m44.199s=C2=A0=C2=A0 7m51.490s=C2=A0= =C2=A0 8m21.149s >>>>>>>> >>>>>>>> patched(use copy_user_gigantic_page for 2M hugetlb too) >>>>>>>> real=C2=A0=C2=A0=C2=A0 0m3.182s=C2=A0=C2=A0=C2=A0 0m3.002s=C2=A0= =C2=A0=C2=A0 0m2.963s >>>>>>>> user=C2=A0=C2=A0=C2=A0 1m19.456s=C2=A0=C2=A0 1m3.107s=C2=A0=C2=A0= =C2=A0 1m6.447s >>>>>>>> sys=C2=A0=C2=A0=C2=A0=C2=A0 2m59.222s=C2=A0=C2=A0 3m10.899s=C2=A0= =C2=A0 3m1.027s >>>>>>>> >>>>>>>> and sequential copy is better than the current implementation, >>>>>>>> so I will use sequential clear and copy. >>>>>>> Sorry, it appears that you misunderstanding my suggestion.=C2=A0 I >>>>>>> suggest to >>>>>>> revise process_huge_page() to use more sequential memory clearing a= nd >>>>>>> copying to improve its performance on your platform. >>>>>>> -- Best Regards, >>>>>>> Huang, Ying >>>>>>> >>>>>>>>>> 2) case-anon-w-seq-hugetlb:(2M PMD HugeTLB) >>>>>>>>>> >>>>>>>>>> base: >>>>>>>>>> real=C2=A0=C2=A0=C2=A0 0m5.175s=C2=A0=C2=A0=C2=A0 0m5.117s=C2=A0= =C2=A0=C2=A0 0m4.856s >>>>>>>>>> user=C2=A0=C2=A0=C2=A0 5m15.943s=C2=A0=C2=A0 5m7.567s=C2=A0=C2= =A0=C2=A0 4m29.273s >>>>>>>>>> sys=C2=A0=C2=A0=C2=A0=C2=A0 2m38.503s=C2=A0=C2=A0 2m21.949s=C2= =A0=C2=A0 2m21.252s >>>>>>>>>> >>>>>>>>>> patched: >>>>>>>>>> real=C2=A0=C2=A0=C2=A0 0m4.966s=C2=A0=C2=A0=C2=A0 0m4.841s=C2=A0= =C2=A0=C2=A0 0m4.561s >>>>>>>>>> user=C2=A0=C2=A0=C2=A0 6m30.123s=C2=A0=C2=A0 6m9.516s=C2=A0=C2= =A0=C2=A0 5m49.733s >>>>>>>>>> sys=C2=A0=C2=A0=C2=A0=C2=A0 0m58.503s=C2=A0=C2=A0 0m47.847s=C2= =A0=C2=A0 0m46.785s >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This case is similar to the case1. >>>>>>>>>> >>>>>>>>>> 3) fallocate hugetlb 20G (2M PMD HugeTLB) >>>>>>>>>> >>>>>>>>>> base: >>>>>>>>>> real=C2=A0=C2=A0=C2=A0 0m3.016s=C2=A0=C2=A0=C2=A0 0m3.019s=C2=A0= =C2=A0=C2=A0 0m3.018s >>>>>>>>>> user=C2=A0=C2=A0=C2=A0 0m0.000s=C2=A0=C2=A0=C2=A0 0m0.000s=C2=A0= =C2=A0=C2=A0 0m0.000s >>>>>>>>>> sys=C2=A0=C2=A0=C2=A0=C2=A0 0m3.009s=C2=A0=C2=A0=C2=A0 0m3.012s= =C2=A0=C2=A0=C2=A0 0m3.010s >>>>>>>>>> >>>>>>>>>> patched: >>>>>>>>>> >>>>>>>>>> real=C2=A0=C2=A0=C2=A0 0m1.136s=C2=A0=C2=A0=C2=A0 0m1.136s=C2=A0= =C2=A0=C2=A0 0m1.136s >>>>>>>>>> user=C2=A0=C2=A0=C2=A0 0m0.000s=C2=A0=C2=A0=C2=A0 0m0.000s=C2=A0= =C2=A0=C2=A0 0m0.004s >>>>>>>>>> sys=C2=A0=C2=A0=C2=A0=C2=A0 0m1.133s=C2=A0=C2=A0=C2=A0 0m1.133s= =C2=A0=C2=A0=C2=A0 0m1.129s >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> There is big win on patched kernel, and it is similar to >>>>>>>>>> above tmpfs >>>>>>>>>> test, so maybe we could revert the commit c79b57e462b5 ("mm: >>>>>>>>>> hugetlb: >>>>>>>>>> clear target sub-page last when clearing huge page"). >>>>>> >>>>>> I tried the following changes, >>>>>> diff --git a/mm/memory.c b/mm/memory.c >>>>>> index 66cf855dee3f..e5cc75adfa10 100644 >>>>>> --- a/mm/memory.c >>>>>> +++ b/mm/memory.c >>>>>> @@ -6777,7 +6777,7 @@ static inline int process_huge_page( >>>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 base =3D 0; >>>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 l =3D n; >>>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Process subpages at the end of huge= page */ >>>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 for (i =3D nr_pages - 1; i >=3D 2 * n; i--) { >>>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 for (i =3D 2 * n; i < nr_pages; i++) { >>>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 cond_resched(); >>>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 ret =3D process_subpage(addr + i * >>>>>> PAGE_SIZE, i, >>>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 arg); >>>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 if (ret) >>>>>> >>>>>> Since n =3D 0, so the copying is from start to end now, but not >>>>>> improvement for case-anon-cow-seq-hugetlb, >>>>>> >>>>>> and if use copy_user_gigantic_pager, the time reduced from 6s to 3s >>>>>> >>>>>> diff --git a/mm/memory.c b/mm/memory.c >>>>>> index fe21bd3beff5..2c6532d21d84 100644 >>>>>> --- a/mm/memory.c >>>>>> +++ b/mm/memory.c >>>>>> @@ -6876,10 +6876,7 @@ int copy_user_large_folio(struct folio *dst, >>>>>> struct folio *src, >>>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 .vma =3D vma, >>>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 }; >>>>>> >>>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (unlikely(nr_pages > MAX_OR= DER_NR_PAGES)) >>>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 return copy_user_gigantic_page(dst, src, addr_hint, >>>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 vma, nr_pages); >>>>>> - >>>>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return process_huge_page(addr_= hint, nr_pages, >>>>>> copy_subpage, &arg); >>>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return copy_user_gigantic_page= (dst, src, addr_hint, vma, >>>>>> nr_pages); >>>>>> =C2=A0=C2=A0 } >>>>> It appears that we have code generation issue here.=C2=A0 Can you che= ck >>>>> it? >>>>> Whether code is inlined in the same way? >>>>> >>>> >>>> No different, and I checked the asm, both process_huge_page and >>>> copy_user_gigantic_page are inlined, it is strange... >>> >>> It's not inlined in my configuration.=C2=A0 And __always_inline below c= hanges >>> it for me. >>> >>> If it's already inlined and the code is actually almost same, why >>> there's difference?=C2=A0 Is it possible for you to do some profile or >>> further analysis? >> Yes, will continue to debug this. > > My bad, I has some refactor patch before using copy_user_large_folio(), > > ba3fda2a7b08 mm: use copy_user_large_page // good performance > a88666ae8f4d mm: call might_sleep() in folio_zero/copy_user() > 3ab7d4d405e9 mm: calculate the base address in the folio_zero/copy_user() > 7b240664c07d mm: convert to folio_copy_user() // I made a mistake > which use dst instead of src in copy_user_gigantic_page() > 1a951e310aa9 mm: use aligned address in copy_user_gigantic_page() > e095ce052607 mm: use aligned address in clear_gigantic_page() > > so please ignore the copy test result (case-anon-cow-seq-hugetlb) > > In summary: > 1) for copying, no obvious different between > copy_user_large_folio/process_huge_page(copying from last to start or > copying from start to last) > > 2) for clearing, clear_gigantic_page is better than process_huge_page > from my machine, and after clearing page from start to last(current, > it process page from last to first), the performance is same to the > clear_gigantic_page. Can you show detailed data, at least user/sys/real time? Previously, we can find user time reduction and sys time increment. -- Best Regards, Huang, Ying >>=20 >>> >>>>> Maybe we can start with >>>>> modified=C2=A0=C2=A0 mm/memory.c >>>>> @@ -6714,7 +6714,7 @@ EXPORT_SYMBOL(__might_fault); >>>>> =C2=A0=C2=A0=C2=A0 * operation.=C2=A0 The target subpage will be proc= essed last to keep its >>>>> =C2=A0=C2=A0=C2=A0 * cache lines hot. >>>>> =C2=A0=C2=A0=C2=A0 */ >>>>> -static inline int process_huge_page( >>>>> +static __always_inline int process_huge_page( >>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 unsigned long addr_hint, unsigne= d int nr_pages, >>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int (*process_subpage)(unsigned = long addr, int idx, void *arg), >>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 void *arg) >>> >>> -- Best Regards, >>> Huang, Ying >>> >>=20