From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33C6EC3DA41 for ; Thu, 11 Jul 2024 09:52:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ABE356B008C; Thu, 11 Jul 2024 05:52:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A6E686B0093; Thu, 11 Jul 2024 05:52:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 935E96B0095; Thu, 11 Jul 2024 05:52:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 765906B008C for ; Thu, 11 Jul 2024 05:52:17 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 27891A04BC for ; Thu, 11 Jul 2024 09:52:17 +0000 (UTC) X-FDA: 82327006314.02.5B0F57E Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) by imf05.hostedemail.com (Postfix) with ESMTP id 6052110000E for ; Thu, 11 Jul 2024 09:52:15 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TNF24cYm; spf=pass (imf05.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.219.48 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720691491; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+SGOETrlErZ1QTSgR312Mn5cjSigFdcWUjBY/vTt6Gk=; b=4lc7yE9oIFZLPanGry/HO/euCjCv7Ck8sYxf+8IT1ecfRWiXzJCwfWuteoYZ1yK+AYPQju A+vaufTNW0WTb37Pzg3HA4Nn1SzP+3T5Sqryu81Yiv7GV20OqU65gMgKn3dCzrXWeE3QyH 0SMNfVBGmbyLjIzcqe908iNuvbHlXH4= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TNF24cYm; spf=pass (imf05.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.219.48 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720691491; a=rsa-sha256; cv=none; b=QUApuLpMNd95oIxrxDH8K/x8nvSAh/qlJxydJINg4DyY9KyO34HMa9K9GNR49jmgjqnVPb Hgl3tGmcy5ia28uL94Qe7Yknl2co3lNR5/iCRrLqsrw0ZVgNEpmLScmMXet0QrsSR4IG6G FGOU76+o1jMI6KLykt5esjFCWq0YUjU= Received: by mail-qv1-f48.google.com with SMTP id 6a1803df08f44-6b5f46191b5so3873116d6.3 for ; Thu, 11 Jul 2024 02:52:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720691534; x=1721296334; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=+SGOETrlErZ1QTSgR312Mn5cjSigFdcWUjBY/vTt6Gk=; b=TNF24cYmWvXE7U5kCynWwAjr4g1ZQJYPjFSOMfxVV9Z9QUYerWzCEFkJLmU1o4JQJs Uj29Wylo4hSg5zg9sL1ab7h8EQEiyPvIgk9eajxYCAT2w9zZvXFZU8cg3VB4nQEbB8u0 cgPT2Aa8CJpT44+nMLzoiL3pgE8NYIsB+Ojb7Du4CcSHgSaEhgDzsAWfu+Spsyt8GrF9 2phqANBjv0YTBxCWKsEL45AIqhvX6i+BAl0xOq3eGaP7cud5IR0bFE4y7HBWFZ0DLvXB dOC7G7WJEb+o/NQinBUzfj99mM2jXQQwOKCkFemOWXNuCrejp6+JlkfNfFjN+SEuu4GT VYww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720691534; x=1721296334; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+SGOETrlErZ1QTSgR312Mn5cjSigFdcWUjBY/vTt6Gk=; b=uhUWEJWdhJOKW3gUhGX4FAcxYfMN50OEUE97vOSwZO+7b3QnlmCNYVVwkkFQo6tN+R KnGp5WcnzkSvVE6VXdYQ77OTB6VGL8za7m0D9iGBqKxI+xE/iU3jU+1e+mxQsbqtQdBA a58DrQmNHnOfcXV33o7ntqMxVrXYC8te8putcFQvLeOuioQcVqxU1kosJouYdQXas1Av V5JbxYyt9qAZyA0mVIkuTEOBtokGKJiqqugnBCSLc7hjs4q1vkThntwVQ6IVAjXUl3kZ dgnIACQ+LtTWGBngB0Lpr+l4e9ldT1uQnitUoxb740Nnl/4WUe8tVv1HGQDwy5AuTTa6 tP8A== X-Forwarded-Encrypted: i=1; AJvYcCVu3f3xzQqQ/RXeRHAY422MFxny0mZqLpiZavVxfU4Lkk+jKVmqAfp5Tise/KWrMkSAICdDw+IlAIXVLmW7Cv+Lz2g= X-Gm-Message-State: AOJu0YyjIZWJNpRd8/ZwxM7WBN/+J4+77x4/H/zTxb4j6P5HAPYtCKo/ CDrbDz5teQGYJof758aDET4jes5P92/i3YuMCpv0a0m6NTAwF53paEcGUBLeAwYkGyAH022rmcK LfWgakDoscbHe6AmLQCmmNGMXtnk= X-Google-Smtp-Source: AGHT+IHNptpEfuRB1Kp9p5AUkVQOUzcUjMeypm0bsZLjZUzOAkCI5elx+q36HcyUTwP8whbHgnb5gnOZA5gc9Cq4elA= X-Received: by 2002:ad4:574c:0:b0:6b5:3f58:bb7d with SMTP id 6a1803df08f44-6b61bc80b22mr110634236d6.5.1720691534505; Thu, 11 Jul 2024 02:52:14 -0700 (PDT) MIME-Version: 1.0 References: <20240707094956.94654-1-laoar.shao@gmail.com> <20240707094956.94654-4-laoar.shao@gmail.com> <878qyaarm6.fsf@yhuang6-desk2.ccr.corp.intel.com> <87o774a0pv.fsf@yhuang6-desk2.ccr.corp.intel.com> <87frsg9waa.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <87frsg9waa.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Yafang Shao Date: Thu, 11 Jul 2024 17:51:38 +0800 Message-ID: Subject: Re: [PATCH 3/3] mm/page_alloc: Introduce a new sysctl knob vm.pcp_batch_scale_max To: "Huang, Ying" Cc: akpm@linux-foundation.org, mgorman@techsingularity.net, linux-mm@kvack.org, Matthew Wilcox , David Rientjes Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 6052110000E X-Stat-Signature: hggy8nwawsgpwooxm5xsqybngd8mq5io X-Rspam-User: X-HE-Tag: 1720691535-312461 X-HE-Meta: U2FsdGVkX18sUKXWnMrgyto+YL7MnjwMjmLy3E30LJQuqk8MrRYQ3gjD9glLNgnJfx8mA52rNia+2znBxty0G5bHpPHqGXlxdgkfJ9zFKdC2HCBWND/Hyg53wyge61b5vphGKJDE2EjNKbZIumyhkgjzQh+LXcAR/db31YSL9wa/BkFT722lINV7JYR95PpN4Yv3zUdoIU49HcQl0JeipMr3IrJqLZMfjbglK5IXO0JfqxUttF5rl+dDLagqhJP/WwKC/tfKyG/0VyYih90vfnM9T9541yQ5Cv+MHzdP+SkzjRxHKEDRhW4uaEUZf/WWE7+NWGrstTBI/A4fVjFGs128OkbvMwuqxT4JGtJ+e2iwjo6Z1fca+bXtPxzpvk7xplNxOQuAnOsKqCwOQyKESncoJjs+4YmuDuicEt437IvvejYyIcCdaMM8N0l/nB/vjIeuZUoiV2hv17WfcGAOWQUXFw4njYgVr76Wzj7OY6VdDytnn8Nm5QsnrM0nrIyGhQE8TmOsFEE8a93Ed81OPw3/bV/FNx9mQGHZFlr0BqI9Gyf6Lr393sNuxqGsz2gAKWswKDZi820G6fhW1kQkAvaFEuOdvj/McaVYtMgxmmS6fiuUAiVG0phLD/UpOpAyI/chUq5mCLy+fYV8i25zhLN4KYFxA8nt9zbxFFE+oiYHNNnRUHjBKRADNDTi7QZn/7AZ/uM9+wYBSUp/eMp51vMUyXzQRO3UFoJBZz7qlLRgtaf6gU/qeg52TLuRC3AqjmNdt7bhHvSxzU4mA8atE0R6YlZnN81mdQzHvXqMFcr/5etTHac/RWbfeXBuuOJ51o+mhLN72uPAQC5P8ey6bXBXi2F5klV5rFeHzeut6ye9mJ1v/fnRU2k6y5dZGYs+pPNjraC5cxHyD36rmb7qQbUZvL0PUXkb4i/Xo8Of0kLudBYZ2JNMd6WkoXXcgSKAb7ZQntC2bIKV7qUmr5v +oK689Jp pRB+EhKoDWyEAJvyMltT/pb7vczeUfwocqYMFs7Qb04+7dUgwCYaT/XQ+gzgL08MlzytF9XpKCQVkgZkcPnNZixk5/c7qYfhKowtFZT4aluw0DcxKo0Z17+g7NL3SGQwJn3Vk9rLgf/kj4YU519zKxqp0zpkQtFx6Jiw1OfgTUbuMMst7HTXvDmDSFCjxmdPQEmuTaXm2EQLvH8vNDOzUylkrtBYKc/GhvqPMvhlYQ28qehdun+BpRSsKHA5cGhY/plvRJuciMPJAX9uEWFxzLDOH5f7JTA6WgPZx4UrIGi0PpasU9GNOl8QvA1hrn1+w4fOvQVbNgS5aOvYwRvbkuqlaWInFg/2A579KR0AK04RwbG8fi7Ano+P0wTL6KhzCchNqDx8I8PrTB0kp7h83RTPbuD+iGvlZC41x X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jul 11, 2024 at 4:20=E2=80=AFPM Huang, Ying = wrote: > > Yafang Shao writes: > > > On Thu, Jul 11, 2024 at 2:44=E2=80=AFPM Huang, Ying wrote: > >> > >> Yafang Shao writes: > >> > >> > On Wed, Jul 10, 2024 at 10:51=E2=80=AFAM Huang, Ying wrote: > >> >> > >> >> Yafang Shao writes: > >> >> > >> >> > The configuration parameter PCP_BATCH_SCALE_MAX poses challenges = for > >> >> > quickly experimenting with specific workloads in a production env= ironment, > >> >> > particularly when monitoring latency spikes caused by contention = on the > >> >> > zone->lock. To address this, a new sysctl parameter vm.pcp_batch_= scale_max > >> >> > is introduced as a more practical alternative. > >> >> > >> >> In general, I'm neutral to the change. I can understand that kerne= l > >> >> configuration isn't as flexible as sysctl knob. But, sysctl knob i= s ABI > >> >> too. > >> >> > >> >> > To ultimately mitigate the zone->lock contention issue, several s= uggestions > >> >> > have been proposed. One approach involves dividing large zones in= to multi > >> >> > smaller zones, as suggested by Matthew[0], while another entails = splitting > >> >> > the zone->lock using a mechanism similar to memory arenas and shi= fting away > >> >> > from relying solely on zone_id to identify the range of free list= s a > >> >> > particular page belongs to[1]. However, implementing these soluti= ons is > >> >> > likely to necessitate a more extended development effort. > >> >> > >> >> Per my understanding, the change will hurt instead of improve zone-= >lock > >> >> contention. Instead, it will reduce page allocation/freeing latenc= y. > >> > > >> > I'm quite perplexed by your recent comment. You introduced a > >> > configuration that has proven to be difficult to use, and you have > >> > been resistant to suggestions for modifying it to a more user-friend= ly > >> > and practical tuning approach. May I inquire about the rationale > >> > behind introducing this configuration in the beginning? > >> > >> Sorry, I don't understand your words. Do you need me to explain what = is > >> "neutral"? > > > > No, thanks. > > After consulting with ChatGPT, I received a clear and comprehensive > > explanation of what "neutral" means, providing me with a better > > understanding of the concept. > > > > So, can you explain why you introduced it as a config in the beginning = ? > > I think that I have explained it in the commit log of commit > 52166607ecc9 ("mm: restrict the pcp batch scale factor to avoid too long > latency"). Which introduces the config. What specifically are your expectations for how users should utilize this config in real production workload? > > Sysctl knob is ABI, which needs to be maintained forever. Can you > explain why you need it? Why cannot you use a fixed value after initial > experiments. Given the extensive scale of our production environment, with hundreds of thousands of servers, it begs the question: how do you propose we efficiently manage the various workloads that remain unaffected by the sysctl change implemented on just a few thousand servers? Is it feasible to expect us to recompile and release a new kernel for every instance where the default value falls short? Surely, there must be more practical and efficient approaches we can explore together to ensure optimal performance across all workloads. When making improvements or modifications, kindly ensure that they are not solely confined to a test or lab environment. It's vital to also consider the needs and requirements of our actual users, along with the diverse workloads they encounter in their daily operations. -- Regards Yafang