From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DCCFC3DA7F for ; Mon, 5 Aug 2024 03:05:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7E5AF6B007B; Sun, 4 Aug 2024 23:05:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7961E6B0082; Sun, 4 Aug 2024 23:05:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65D066B0085; Sun, 4 Aug 2024 23:05:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 48EED6B007B for ; Sun, 4 Aug 2024 23:05:57 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id AE7D841E3A for ; Mon, 5 Aug 2024 03:05:56 +0000 (UTC) X-FDA: 82416702312.26.4BDFB78 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.18]) by imf05.hostedemail.com (Postfix) with ESMTP id 07CD4100003 for ; Mon, 5 Aug 2024 03:05:53 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=AypT3BlY; spf=pass (imf05.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.18 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722827124; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9jJKPud1fpQRKzHtPjcPPPRVTlWvZ78tnqhVM/VteCY=; b=R7Hwo3Q2NpI9UNRTh5SO8xOz5E4BZoA/TNpF3Gl+S0UXsrnLZsvTTcmzU+Qrcp7EFiePxW 9d4+YCGWO3fkvspk4yGQ2RkJSdyicOeKvmud34iODkMsEP0obJHIETt91P+djR+u50dWCg IbeGK/5GeXYP4JPvGrOnmkQ9TAqFZ4E= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=AypT3BlY; spf=pass (imf05.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.18 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722827124; a=rsa-sha256; cv=none; b=ohCf0JUvNoh/21cn2LGJ3kQkanQyr+3hvwiu8K9QfnM19/8ZdJw/uGNBDQU8pGmdVAMBtV gFN85/mBGJyCNPP+9X98m69tZZnmRQ+cIOCI8icswWHabJuWxgRU1HTt1ymCSiL3f5y5Ez UrfEEF2Nw/S4Zcpiv+jQdxwDyweNF0M= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1722827154; x=1754363154; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version:content-transfer-encoding; bh=UikW0DjSQ4NBdWwFF3UrHQtgkLZXjkywX8235WRdWjk=; b=AypT3BlY+5U9slwkk+L3jGz0krKE5Dgn1Lt6zgm4K6QEFmnEAJlUPKM/ D7cDdzaK3Qi6lPZHHgSXnq/uHXABaoGKuV/3uW2UZt1qTTE9F4yQoQFuU ondUc2+EWofpqaiqTakgAhSEjhZsopz3vRgnXA2N7YicTYw/uzSs0W3Xo f+jA5KEwA3mslUwy9GQV2yIMCFZwCpnBYoOgpOJIhx/GXZz4lkCcnc9rC dYYctMNbC2D6Mc6f7cNtOQ95AnZo8CY9Yrva4A3d0B/BWLQKDh6aArDoR QzKegWnxNNp9Jx8YBoK6+SCYOenZ5IvUaU9gGjrIVWB5yDMh5sadN2/1c w==; X-CSE-ConnectionGUID: nea3R8HcTQKkdTugxQze7g== X-CSE-MsgGUID: pDwdUKFeRRCxgPO4t8sdCA== X-IronPort-AV: E=McAfee;i="6700,10204,11154"; a="20909085" X-IronPort-AV: E=Sophos;i="6.09,263,1716274800"; d="scan'208";a="20909085" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa110.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Aug 2024 20:05:52 -0700 X-CSE-ConnectionGUID: VDGtLQZLR1anKKDEvma93w== X-CSE-MsgGUID: jfJf61asRv6JC3Id1dwsbQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,263,1716274800"; d="scan'208";a="55957038" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Aug 2024 20:05:50 -0700 From: "Huang, Ying" To: Yafang Shao Cc: akpm@linux-foundation.org, mgorman@techsingularity.net, linux-mm@kvack.org, Matthew Wilcox , David Rientjes Subject: Re: [PATCH v3 3/3] mm/page_alloc: Introduce a new sysctl knob vm.pcp_batch_scale_max In-Reply-To: (Yafang Shao's message of "Mon, 5 Aug 2024 09:58:33 +0800") References: <20240804080107.21094-1-laoar.shao@gmail.com> <20240804080107.21094-4-laoar.shao@gmail.com> <87r0b3g35e.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Mon, 05 Aug 2024 11:02:17 +0800 Message-ID: <87mslrfz9i.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 07CD4100003 X-Stat-Signature: i4znpg5wk1y7uig6cksxkiwyhuh4iuas X-HE-Tag: 1722827153-588936 X-HE-Meta: U2FsdGVkX1/PBLIsi54rwEtDaj28ohifH4JeJFwSVInIG4XmXlC2xwn2DLRTp8BEXcJ9Zzd422PJfZzYY0XWq6RP59rBL/WcsLub8j092jXpTv3J/jJx3daaxYCqjZ1OlAkWI3U1e0U3dl5WDuS7HdvCedKy9Rq0cu1VW+NepkIKDmwlNA+XTXabq9ZfSvGECx/upYTlhsqBpjAl49w+jhy9zsLPSLz5ZjFTKrZtImbTZg12lETUAE6xKMGUVKc/FGyYZjbs9vS0CshMYEw1A9E856CvbDipRnnHAW5xLqUtxQmLjJ82o1f/wFCfzDfwMjQkIDY1uDfGvF9eqr2KZ9w3NI6Au3zf7+dHq3KJLp4ZrlJBwWmfZG2C5PdU5VLeNcRUNd5zfyZOjiB28pugea1z5QXGg26J4c5uXWshndj74uhaCT3DSiHRxrypgKYIfH5MPGePXIbJ5O1z8CMj9td6ihMt3qGAoCPxKr7qxIfokuMM6WA7PFLW/UhfdHuhyTGJjrOZgaVGpfH6tFqJD8z71bmyzzbOQ25/LDCZu9Ickl5tB71ARowedRDdBQ+sVQz3rWyGRUELSxrNjsd6sE/xycvKGdbXI8NqoUAfuT/bMWtpQxO2CgC8NsHEDa8yhl7VNs5/qt3gQBfXJXFs740P2UdXcVNcZeKjMfSPYRO3bdJWJnqSQfVk2sSAeoI8gL4z6HTscPK3RmN+UfuVx8w2x7N558GPK1ufulxUlnYfV5zWKK/48dVzbfs7IZP1sbtP85K9ZTx/Nn66Ai7sHxLzgRLOYqFTHUFH03fDcX/DBGTXxa8eYaSxvX6U247lfyzvy1UbGligdaAmpqYl69ldVPlhamlf0nkiPr2TFyaiXhq6FRIhdalhV1GUL+vq9KFLk+06TTqVeJuRr8sRZQin5uCOlM4UyhVANeloi1TIkElS/KcB+RR5wjTpepO/e6IJ/yhRcL5ij+zM57e dLqEqUAY zTNFraTP2ViALDQKhisiXKd0tx2Cpit79n1MyBvcxxLnfV2w15dJtRgdv+DYt30X7xlWt8CmmLHI/MWlPq7hbRrJYQm79PbZ2/qF2NvKb8qdqt4C6h1JD4/2NY6xuTLrbz38BcDp7czHRcczJrOK/4cmJDXdcLf66LwPDDAPbEoPH6ByW3WozlnGSKTN2C64Afe6uxd6WUl8C9k4bPCLkC1tFENYdAFnUpj/3/vsKm1siFOpx3cIbhGJ2dMX6EIpc9d48h6bUlikTt9awuJ7jN6O97J5yszJFAbbYTn1AM8SCtYJ61Af26OBgQw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Yafang Shao writes: > On Mon, Aug 5, 2024 at 9:41=E2=80=AFAM Huang, Ying = wrote: >> >> Yafang Shao writes: >> >> [snip] >> >> > >> > Why introduce a systl knob? >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D >> > >> > From the above data, it's clear that different CPU types have varying >> > allocation latencies concerning zone->lock contention. Typically, peop= le >> > don't release individual kernel packages for each type of x86_64 CPU. >> > >> > Furthermore, for latency-insensitive applications, we can keep the def= ault >> > setting for better throughput. >> >> Do you have any data to prove that the default setting is better for >> throughput? If so, that will be a strong support for your patch. > > No, I don't. The primary reason we can't change the default value from > 5 to 0 across our fleet of servers is that you initially set it to 5. > The sysadmins believe you had a strong reason for setting it to 5 by > default; otherwise, it would be considered careless for the upstream > kernel. I also believe you must have had a solid justification for > setting the default value to 5; otherwise, why would you have > submitted your patches? In commit 52166607ecc9 ("mm: restrict the pcp batch scale factor to avoid too long latency"), I tried my best to run test on the machines available with a micro-benchmark (will-it-scale/page_fault1) which exercises kernel page allocator heavily. From the data in commit, larger CONFIG_PCP_BATCH_SCALE_MAX helps throughput a little, but not much. The 99% alloc/free latency can be kept within about 100us with CONFIG_PCP_BATCH_SCALE_MAX =3D=3D 5. So, we chose 5 as default value. But, we can always improve the default value with more data, on more types of machines and with more types of benchmarks, etc. Your data suggest smaller default value because you have data to show that larger default value has the latency spike issue (as large as tens ms) for some practical workloads. Which weren't tested previously. In contrast, we don't have strong data to show the throughput advantages of larger CONFIG_PCP_BATCH_SCALE_MAX value. So, I suggest to use a smaller default value for CONFIG_PCP_BATCH_SCALE_MAX. But, we may need more test to check the data for 1, 2, 3, and 4, in addtion to 0 and 5 to determine the best choice. >> >> > In our production environment, we set this >> > value to 0 for applications running on Kubernetes servers while keepin= g it >> > at the default value of 5 for other applications like big data. It's n= ot >> > common to release individual kernel packages for each application. >> > >> >> [snip] >> -- Best Regards, Huang, Ying