From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A987EB64DC for ; Mon, 17 Jul 2023 13:50:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9CCD98D0002; Mon, 17 Jul 2023 09:50:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 97D3D6B0074; Mon, 17 Jul 2023 09:50:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 844A38D0002; Mon, 17 Jul 2023 09:50:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 75D7E6B0072 for ; Mon, 17 Jul 2023 09:50:24 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3E758B0A24 for ; Mon, 17 Jul 2023 13:50:24 +0000 (UTC) X-FDA: 81021238368.29.53642D7 Received: from outbound-smtp28.blacknight.com (outbound-smtp28.blacknight.com [81.17.249.11]) by imf14.hostedemail.com (Postfix) with ESMTP id 42871100009 for ; Mon, 17 Jul 2023 13:50:20 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf14.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.11 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689601821; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T1flYcIcKkoL6Q2cNqLqqCQC7gfFf+ToSrC/tA6Jplk=; b=269EIyIWOsT0UPoqIsUqeT0pOjj42MGnr8INd4gYGELfxHzNVUTJSm/AB5FC0ejAHfS2HY Q8Q35apoCQlQqDQXESfS0TFn3grNdB4Lzv/g0KfsVs5mnrJH75MbFuHZqt5KtSrK4fsssG tbKKkr7JsAiQWOpJNn5LIHCGqgLPMic= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf14.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.11 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689601821; a=rsa-sha256; cv=none; b=KfgA7tIKW8NEZzVAXvNtAXYc+rXH1HuMYVTOZB/xn/yF6KuFTf1lY7oLj1jS2Lcb61FVo6 W5j1oNdd3SY8bS1AsPaot9LC8eqxYF6d9wpYHYS6TCUVeYjmwV3BGVjcNFfAxl4pDA1XFG WkdiTlPy37al/xZQR4AK3rpPMUigbZM= Received: from mail.blacknight.com (pemlinmail01.blacknight.ie [81.17.254.10]) by outbound-smtp28.blacknight.com (Postfix) with ESMTPS id 28E51CCB6E for ; Mon, 17 Jul 2023 14:50:19 +0100 (IST) Received: (qmail 650 invoked from network); 17 Jul 2023 13:50:19 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[84.203.20.191]) by 81.17.254.9 with ESMTPSA (AES256-SHA encrypted, authenticated); 17 Jul 2023 13:50:18 -0000 Date: Mon, 17 Jul 2023 14:50:17 +0100 From: Mel Gorman To: "Huang, Ying" Cc: Michal Hocko , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Arjan Van De Ven , Andrew Morton , Vlastimil Babka , David Hildenbrand , Johannes Weiner , Dave Hansen , Pavel Tatashin , Matthew Wilcox Subject: Re: [RFC 2/2] mm: alloc/free depth based PCP high auto-tuning Message-ID: <20230717135017.7ro76lsaninbazvf@techsingularity.net> References: <20230710065325.290366-1-ying.huang@intel.com> <20230710065325.290366-3-ying.huang@intel.com> <20230712090526.thk2l7sbdcdsllfi@techsingularity.net> <871qhcdwa1.fsf@yhuang6-desk2.ccr.corp.intel.com> <20230714140710.5xbesq6xguhcbyvi@techsingularity.net> <87pm4qdhk4.fsf@yhuang6-desk2.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: <87pm4qdhk4.fsf@yhuang6-desk2.ccr.corp.intel.com> X-Rspamd-Queue-Id: 42871100009 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: sm84ukhe8gfpbr68r4uae5ma3ex3anp8 X-HE-Tag: 1689601820-327540 X-HE-Meta: U2FsdGVkX19P/K8cRZe6Z0jQmvKkB/kvpEveW/ekpUMx6TzXNc8GCSjgg5ILk5jbmOHHyu/EvVIwqGtrlHGsenMlf1CxsRa2PijWr0kmMIZ7sdNqtOclp6nG1hGGCHGStaLEZ0aARCDb8Mm/Z/9GGLmRLCkLmI4OIi9gDdDrfVINdcJqNF3dpA645LGyt/Ahw4I40qib7eQd1mGuA5Qi2Wyr5/GQYkya9QQQatva+OBHSoWWFzAylHbxa7CoDbV17eygieQwUO6W2P3NUjakaoayptq5TXCrfrSWj32pGc6GXrLpRoehvszaN25Dx1B32LEGkwQcd9G0IeJvTeoutDpvQxMrgScR11WRjjyKx+krsuOapO91mAeiYEl2XjFlI63naprSQILxUuK2RngT42XYGZP0jgWP8DrqaCm3StrJY5q9Wqh+hXzCo0Y7qBcseUPf5nh0IdpqFXuQC3Wx/cU9LLqGoTFB4uifgAS7YcbqTJtQBnFhbzwhTk+mxluSh4UN5XTWqrQvtMgKRdE2zbaf927TC/Z2Bpepynm2s99SxCZ47p2ZXOXszg9WWHGdWIriAd+9AtoqBosforIXsEE+NWshvjgW0BRxVVQERlBDDqPsPq0R9Z0jmJ2ysT4pVZpi1NU48MXgZZZw7ZEuUGZdRwkL5t2j2cX6JBzpIWpqRqTMIC7TG083aYm6rKqV5isCXbbKfYzpz9lNqDeZgs03w6OVmJfK3txNUPByUjEFx6UOiwCqXXAJCNjSam135SM+9cXhrzM1V0RVajL9NNbvuHW7S1FQ/AIIzdvAUf4b4RXZL3DnYzPalhQQCcxK4WyDLzqcj83azCbvgrMzUrqCm5RJQkYvlmuu3Is+5CJw3F5Zi0SKpZext5d71B3lHnL0juRL9m1hTR1UEIg65TGi/I1HZrkLhirr9/E18CYOJdWOhZIykjT7KsJCPGb8dyLo/kv7YqiQch7zPSO hdPdFsM0 hWZWfSfopfhNtYOFgRzJDx8+prHyDnDdcPfI6+95JtElAgBp+ITfAlGL901eP1c7FLHZWqSTSW3eJ/kbNVqXN/OBPh8v/YOFAHtkWj87k41dUCfDiyGxd4nt2dvujdM/Y+d6UGCTR6S9hRZsrtoyGEas+Lg5GAb3kJuk2Aj+4T6sswYgrnAXbKXdCjTPnDew0zeu5uxma79qFhVpgXxUG422A9P1YeVm0ki+k X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jul 17, 2023 at 05:16:11PM +0800, Huang, Ying wrote: > Mel Gorman writes: > > > Batch should have a much lower maximum than high because it's a deferred cost > > that gets assigned to an arbitrary task. The worst case is where a process > > that is a light user of the allocator incurs the full cost of a refill/drain. > > > > Again, intuitively this may be PID Control problem for the "Mix" case > > to estimate the size of high required to minimise drains/allocs as each > > drain/alloc is potentially a lock contention. The catchall for corner > > cases would be to decay high from vmstat context based on pcp->expires. The > > decay would prevent the "high" being pinned at an artifically high value > > without any zone lock contention for prolonged periods of time and also > > mitigate worst-case due to state being per-cpu. The downside is that "high" > > would also oscillate for a continuous steady allocation pattern as the PID > > control might pick an ideal value suitable for a long period of time with > > the "decay" disrupting that ideal value. > > Maybe we can track the minimal value of pcp->count. If it's small > enough recently, we can avoid to decay pcp->high. Because the pages in > PCP are used for allocations instead of idle. Implement as a separate patch. I suspect this type of heuristic will be very benchmark specific and the complexity may not be worth it in the general case. > > Another question is as follows. > > For example, on CPU A, a large number of pages are freed, and we > maximize batch and high. So, a large number of pages are put in PCP. > Then, the possible situations may be, > > a) a large number of pages are allocated on CPU A after some time > b) a large number of pages are allocated on another CPU B > > For a), we want the pages are kept in PCP of CPU A as long as possible. > For b), we want the pages are kept in PCP of CPU A as short as possible. > I think that we need to balance between them. What is the reasonable > time to keep pages in PCP without many allocations? > This would be a case where you're relying on vmstat to drain the PCP after a period of time as it is a corner case. You cannot reasonably detect the pattern on two separate per-cpu lists without either inspecting remote CPU state or maintaining global state. Either would incur cache miss penalties that probably cost more than the heuristic saves. -- Mel Gorman SUSE Labs