From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A14AC3DA6E for ; Wed, 20 Dec 2023 15:05:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0C8676B0087; Wed, 20 Dec 2023 10:05:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 077376B0088; Wed, 20 Dec 2023 10:05:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E80ED6B0089; Wed, 20 Dec 2023 10:05:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id D698C6B0087 for ; Wed, 20 Dec 2023 10:05:13 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 90C63C0C2C for ; Wed, 20 Dec 2023 15:05:13 +0000 (UTC) X-FDA: 81587519706.13.35B5FF8 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf15.hostedemail.com (Postfix) with ESMTP id 4CADCA001C for ; Wed, 20 Dec 2023 15:05:11 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; spf=pass (imf15.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703084711; a=rsa-sha256; cv=none; b=vE/V5tdSz/lngfef/8RqXQPIAM6ISBOPcH2DcI4cd43O4YyFV1xApWiZ4rJPLRPR7kRUV9 lXUBXpHOCoqhl+pGGc45TXFMyQtCsy5qUYJTK2KDO74nyE6RCPa+5GthFV4Tf0rfrW48+7 7n5McP1wGSGkK6fMQ1dJtasMQbx57V8= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; spf=pass (imf15.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703084711; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h7BGnrYwWm4ECJgPSco10cNCr3ei3qZXena4UlpVSqU=; b=FmxpMCqFDX3hMG9vRuzZBeh023z8NXD2lmRbNOKa7czEl9Rw4HTjIAYqOryB3elwna5wm3 0597fMtW6CFHGhVN5qs3cDGALFhji/jrhYA0awgwScP+SNO1lIwim8uglxFz7VcsSST6JM 3D3dHrM/QNuKztCJOlTqmOG7x1qSc/k= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 095B41FB; Wed, 20 Dec 2023 07:05:55 -0800 (PST) Received: from [10.57.75.247] (unknown [10.57.75.247]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 93AB03F64C; Wed, 20 Dec 2023 07:05:06 -0800 (PST) Message-ID: <396cae5d-70e4-449f-af6c-2348b720d3a3@arm.com> Date: Wed, 20 Dec 2023 15:05:05 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 02/16] mm: Batch-copy PTE ranges during fork() To: David Hildenbrand , Catalin Marinas , Will Deacon , Ard Biesheuvel , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Andrew Morton , Anshuman Khandual , Matthew Wilcox , Yu Zhao , Mark Rutland , Kefeng Wang , John Hubbard , Zi Yan , Barry Song <21cnbao@gmail.com>, Alistair Popple , Yang Shi Cc: linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20231218105100.172635-1-ryan.roberts@arm.com> <7c0236ad-01f3-437f-8b04-125d69e90dc0@redhat.com> <9a58b1a2-2c13-4fa0-8ffa-2b3d9655f1b6@arm.com> <28968568-f920-47ac-b6fd-87528ffd8f77@redhat.com> <10b0b562-c1c0-4a66-9aeb-a6bff5c218f6@arm.com> <8f8023cb-3c31-4ead-a9e6-03a10e9490c6@redhat.com> <699cb1db-51eb-460e-9ceb-1ce08ca03050@redhat.com> <2a8c5b6c-f5ae-43b2-99aa-6d10e79b76e1@redhat.com> <3194b8a5-3f72-4d9e-a267-fbdad32ad864@redhat.com> <9f99a3ca-051e-4b1b-81e9-8456d8e422ad@redhat.com> <5fcbf405-7e62-4b38-acc4-a9dd8cc91214@redhat.com> Content-Language: en-GB From: Ryan Roberts In-Reply-To: <5fcbf405-7e62-4b38-acc4-a9dd8cc91214@redhat.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 4CADCA001C X-Stat-Signature: r8cr7x55dys5488i1ka4gtsxybeqft7h X-HE-Tag: 1703084711-85137 X-HE-Meta: U2FsdGVkX19dBwkMS1mNxHdDGnaySG65G6jNWMazwlaK3INX8K3ZDzL35dY2Qq73gzr7ZzV3/iyX0LzQeuJciEeu65auE6y2DiGeTIQ1B2ThzcjiASCe/B6+IAEpGZMSY9ztQ0btFyn9JBH0zjL5YggNkUevTS1pC8fcU9u9PgUV6qHcWCrIe3mvc2xWTS8KiczT2O1wuEnr+MYhALt3GnagT3lr0Yah1sC4tEIqV9wk5Crs0P2kHiYqxGywaRq6xUuNAB68MJW0WPHpd9y30ESvwhbLvBvUB0O+TWKBS0+bIE3XvmQdhHPb/MIQqBjRI5w7wxft7LjaVzop4wtfzTLTJIUlluwcgon1gKApO37rJIIjRkrY6KKJSDCH/o3L0T4TQlkrebaG0sUBfK2lw2iG1S+c/DEstKdn8zn3HhA0VS2VqTBkTNZqrtxhFQBvvoXLPUApsuaMS9rVBiKLcki01/h00NuGGm9nImEAvOLrssHKzD3dzJv6w290pvR6TK2j+tbhjKo1rrpVivCgH9ge4KfgBFOgL+7P06t3mjcxJiJqbMq7v/j5WYwUeD5HZSlPoTUuzZMvGjn6RhJqiXOVCBHBGAISdAGkV5/NMeqvMOPrvSXdPxjqQwUcOXtRRxQr/96cvJdFFTqq8CQ5BSHDgEz3RnzFXlBQXk8mB7Qd9eZ8q0SWo9VUOs4cq+EwDGI5ZFOkWuJC2mdFfyf0U8I4htJAbkgdJ1H3SYmO6gfR8Nx5EvF0oD6Ganwj7zoJ6J6aQQ3wzMLTujUo7q4a3SG4ViolKQ30UkcBPiaTUGhab9HhR1Pjeb8Hsxr1+RPHAcIuw0PuxtTiHxY8aDFSNCpfTAtfveLB9uCnKw5aYF1ql3xjfkdAN7YL2cCFQD3yD/ToIfupIEaRuAy1EjXJsvbpVQWoAKkRjh3CrlBPE6Q= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 20/12/2023 14:00, David Hildenbrand wrote: > [...] > >>>> >>> >>> gcc version 13.2.1 20231011 (Red Hat 13.2.1-4) (GCC) >>> >>>  From Fedora 38. So "a bit" newer :P >>> >> >> I'll retry with newer toolchain. >> >> FWIW, with the code fix and the original compiler: >> >> Fork, order-0, Apple M2: >> | kernel                |   mean_rel |   std_rel | >> |:----------------------|-----------:|----------:| >> | mm-unstable           |       0.0% |      0.8% | >> | hugetlb-rmap-cleanups |       1.3% |      2.0% | >> | fork-batching         |       4.3% |      1.0% | >> >> Fork, order-9, Apple M2: >> | kernel                |   mean_rel |   std_rel | >> |:----------------------|-----------:|----------:| >> | mm-unstable           |       0.0% |      0.8% | >> | hugetlb-rmap-cleanups |       0.9% |      0.9% | >> | fork-batching         |     -37.3% |      1.0% | >> >> Fork, order-0, Ampere Altra: >> | kernel                |   mean_rel |   std_rel | >> |:----------------------|-----------:|----------:| >> | mm-unstable           |       0.0% |      0.7% | >> | hugetlb-rmap-cleanups |       3.2% |      0.7% | >> | fork-batching         |       5.5% |      1.1% | >> >> Fork, order-9, Ampere Altra: >> | kernel                |   mean_rel |   std_rel | >> |:----------------------|-----------:|----------:| >> | mm-unstable           |       0.0% |      0.1% | >> | hugetlb-rmap-cleanups |       0.5% |      0.1% | >> | fork-batching         |     -10.4% |      0.1% | >> > > I just gave it another quick benchmark run on that Intel system. > > hugetlb-rmap-cleanups -> fork-batching > > order-0: 0.014114 -> 0.013848 > > -1.9% > > order-9: 0.014262 -> 0.009410 > > -34% > > Note that I disable SMT and turbo, and pin the test to one CPU, to make the > results as stable as possible. My kernel config has anything related to > debugging disabled. > And with gcc 13.2 on arm64: Fork, order-0, Apple M2 VM: | kernel | mean_rel | std_rel | |:----------------------|-----------:|----------:| | mm-unstable | 0.0% | 1.5% | | hugetlb-rmap-cleanups | -3.3% | 1.1% | | fork-batching | -3.6% | 1.4% | Fork, order-9, Apple M2 VM: | kernel | mean_rel | std_rel | |:----------------------|-----------:|----------:| | mm-unstable | 0.0% | 1.8% | | hugetlb-rmap-cleanups | -5.8% | 1.3% | | fork-batching | -38.1% | 2.3% | Fork, order-0, Ampere Altra: | kernel | mean_rel | std_rel | |:----------------------|-----------:|----------:| | mm-unstable | 0.0% | 1.3% | | hugetlb-rmap-cleanups | -0.1% | 0.4% | | fork-batching | -0.4% | 0.5% | Fork, order-9, Ampere Altra: | kernel | mean_rel | std_rel | |:----------------------|-----------:|----------:| | mm-unstable | 0.0% | 0.1% | | hugetlb-rmap-cleanups | -0.1% | 0.1% | | fork-batching | -13.9% | 0.1% | So all looking good. Compiler was the issue. Sorry for the noise. So please go ahead with you rmap v2 stuff, and I'll wait for you to post the fork and zap batching patches properly, then rebase my arm64 contpte stuff on top and remeasure everything. Thanks, Ryan