From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5CA7D41C3E for ; Wed, 13 Nov 2024 12:56:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3A7716B0093; Wed, 13 Nov 2024 07:56:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 309416B00A4; Wed, 13 Nov 2024 07:56:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D06D6B00A6; Wed, 13 Nov 2024 07:56:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id EF1EF6B0093 for ; Wed, 13 Nov 2024 07:56:31 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6E9FC80A0A for ; Wed, 13 Nov 2024 12:56:31 +0000 (UTC) X-FDA: 82781069364.12.B498314 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf05.hostedemail.com (Postfix) with ESMTP id 6279B100028 for ; Wed, 13 Nov 2024 12:55:08 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731502413; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=paG34W9RuUFqjynCcqmF9vylDQ8hE8NgKGEWL4OoefM=; b=G4qRDi1K5fwKByd3vsC1O7RmJYcozR/r9DMRcAuE0lyO1ujkF+SkI8P6G1PIgnypruapoN qi6M201Q/IXcWOIWbo4JuM/MGMvSWH33OV3xzTBul5eIe2EDiozatX1NaNFCRrqRiJiKpP eaCRkTNDlWGPWo/V5l7Wrf0yeNsv2dI= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731502413; a=rsa-sha256; cv=none; b=G05MsbFB4OUeGgxCEBYuGIxxG5aMChR+Rav4jl3kZILONAjaZIKOcyionQ/TbEnC7gWPqB YIH2D2oj6YCwNx6MSBneV14bF+97MB2jKiUZYQ/R+AAsfQ7Z5O/FwQ42MYZPToSattGzEC CDs9cL8/B8LMnGVd62Y5kOWhF8umnDk= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 738261655; Wed, 13 Nov 2024 04:56:58 -0800 (PST) Received: from [10.1.38.177] (XHFQ2J9959.cambridge.arm.com [10.1.38.177]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D103C3F66E; Wed, 13 Nov 2024 04:56:25 -0800 (PST) Message-ID: Date: Wed, 13 Nov 2024 12:56:24 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH v1 00/57] Boot-time page size selection for arm64 Content-Language: en-GB To: Petr Tesarik Cc: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241017142752.17f2c816@mordecai.tesarici.cz> <20241111131442.51738a30@mordecai.tesarici.cz> <046ce0ae-b4d5-4dbd-ad9d-eb8de1bba1b8@arm.com> <20241112104544.574dd733@mordecai.tesarici.cz> <5a041e51-a43b-4878-ab68-4757d3141889@arm.com> <20241112115039.41993e4b@mordecai.tesarici.cz> <20241113134038.5843ab73@mordecai.tesarici.cz> From: Ryan Roberts In-Reply-To: <20241113134038.5843ab73@mordecai.tesarici.cz> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 6279B100028 X-Stat-Signature: moiahx8qef8ycd8ss1f1gg9qsyk95nkw X-Rspam-User: X-HE-Tag: 1731502508-742380 X-HE-Meta: U2FsdGVkX18StNmkLPrJXXAFpXuaswteqlBTtkfdux2rNL997HHZCbBRI0EQRlRxS90uq86kmVe+axnngvWUKYiiC43whK9CnLUzbVhsun2BbqvblbPuIon7sd8ZE5vno+VBcRiASgcQQUP3fKgeL7v9ct/TBUaq7c+lA8Ut3Tylf4FqtnTeaA+P70kh61P2xsk6/fiJxuUzx2e04xfVT5BFl80A5VIZj85nuo5azHtFYfqt387BBBcMt9BuLXzECDXOvwI9I2xmn42rcQricIU8S6rOpG2vrbjUIqQ8RX0v1tWaeQ8ooOSGnDmQNcJkrMkdG64yGFbk+tXBcqVEN6e1anZG42nwWrNmFvH2NiuV/6+hMOVvB6Ip+o0EJVYualYEZnZznuty7TRBj7i4USYHIZFhIZBIuFcXts84Rv7duhvWNIAxixoBLzO8I/JricbU6xj7N3j3SrYRyZACns6NuPM+F60JeGtdpH5jtKC0W41BwZrecfLPGgrfUpeHYZud84TRV6vDg3q49IKv1PoYoEHuGpWLF9vnE+CvCMZk5qZB2SG9N69KlCK6bKAxFA3BSm+ik8LD2vCMTXRoVavsVO+WtXqBgpkICqurS5rZBa2x8OVH36ggBsGfFCMWIzJNguCbEoApHZMO3Ga1LXvWydN9FKT0sNnblKVzah+cKvtoV23fmP4V4SjXSX3LuHuwPqJADEle15ffcD2aSKCbQ3KTSEzACRrNfbf8cENXyYmtLty/pMkAfFO1XANmfCFRBiOJFNz8ywQ/PVp/cF/9V6OIPVusPH+QiodxSOnrf7s37N6aPJ8Pxh4V+g0kx8zTjzsUAxAIZx1WjVkkwGgvrxuTunBA3SPegrRvSIOCF+W4aJODbA+G11rtDHC+t1vSoLvcw6aKiXrpXrpXTHjAIUDV0WlrMxyf7OIV3OYPDTmyYmT12X6f/YLIHfPRaljJaFbZEtE00lADmgL eMSYxQ6r ljrR2N3APrtMEZGqZcSIzAA31tkQIm+mtPPmmu/qDlOc4Z1qL78snSWNyr722zxQUKEM1ceEkKnhuKgBkjHXBVGbIN4j+imDRtGDdR+3QiSlC4PIdXxeI740PP81/O7Xjxvr1oGwf2HdTo/LKEctqh66a7sUUksrAxICDO4a3aM1zIWWqQPNckYsdSdtfJ+3diuLc1ADucJlYApTi1SQTR8drKeGOhrl8UxvOH1dZEiMOcsIcRPQvpZa+79kL/4nWULKegCb086K26aQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 13/11/2024 12:40, Petr Tesarik wrote: > On Tue, 12 Nov 2024 11:50:39 +0100 > Petr Tesarik wrote: > >> On Tue, 12 Nov 2024 10:19:34 +0000 >> Ryan Roberts wrote: >> >>> On 12/11/2024 09:45, Petr Tesarik wrote: >>>> On Mon, 11 Nov 2024 12:25:35 +0000 >>>> Ryan Roberts wrote: >>>> >>>>> Hi Petr, >>>>> >>>>> On 11/11/2024 12:14, Petr Tesarik wrote: >>>>>> Hi Ryan, >>>>>> >>>>>> On Thu, 17 Oct 2024 13:32:43 +0100 >>>>>> Ryan Roberts wrote: >>>>> [...] >>>>>> Third, a few micro-benchmarks saw a significant regression. >>>>>> >>>>>> Most notably, getenv and getenvT2 tests from libMicro were 18% and 20% >>>>>> slower with variable page size. I don't know why, but I'm looking into >>>>>> it. The system() library call was also about 18% slower, but that might >>>>>> be related. >>>>> >>>>> OK, ouch. I think there are some things we can try to optimize the >>>>> implementation further. But I'll wait for your analysis before digging myself. >>>> >>>> This turned out to be a false positive. The way this microbenchmark was >>>> invoked did not get enough samples, so it was mostly dependent on >>>> whether caches were hot or cold, and the timing on this specific system >>>> with the specific sequence of bencnmarks in the suite happens to favour >>>> my baseline kernel. >>>> >>>> After increasing the batch count, I'm getting pretty much the same >>>> performance for 6.11 vanilla and patched kernels: >>>> >>>> prc thr usecs/call samples errors cnt/samp >>>> getenv (baseline) 1 1 0.14975 99 0 100000 >>>> getenv (patched) 1 1 0.14981 92 0 100000 >>> >>> Oh that's good news! Does this account for all 3 of the above tests (getenv, >>> getenvT2 and system())? >> >> It does for getenvT2 (a variant of the test with 2 threads), but not >> for system. Thanks for asking, I forgot about that one. >> >> I'm getting substantial difference there (+29% on average over 100 runs): >> >> prc thr usecs/call samples errors cnt/samp command >> system (baseline) 1 1 6937.18016 102 0 100 A=$$ >> system (patched) 1 1 8959.48032 102 0 100 A=$$ >> >> So, yeah, this should in fact be my priority #1. > > Further testing reveals the workload is bimodal, that is to say the > distribution of results has two peaks. The first peak around 3.2 ms > covers 30% runs, the second peak around 15.7 ms covers 11%. Two per > cent are faster than the fast peak, 5% are slower than slow peak, the > rest is distributed almost evenly between them. FWIW, One source of bimodality I've seen on Ampere systems with 2 NUMA nodes is placement of the kernel image vs placement of the running thread. If they are remote from eachother, you'll see a slowdown. I've hacked this source away in the past by effectively using only a single NUMA node (with the help of 'maxcpus' and 'mem' kernel cmdline options). > > 100 samples were not sufficient to see this distribution, and it was > mere bad luck that only the patched kernel originally reported bad > results. I can now see bad results even with the unpatched kernel. > > In short, I don't think there is a difference in system() performance. > > I will still have a look at dup() and VMA performance, but so far it > all looks good to me. Good job! ;-) Thanks for digging into all this! > > I will also try running a more complete set of benchmarks during next > week. That's SUSE Hack Week, and I want to make a PoC for the MM > changes I proposed at LPC24, so I won't need this Ampere system for > interactive use. > > Petr T