From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90D10CDB474 for ; Mon, 23 Oct 2023 09:26:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB6B56B00B9; Mon, 23 Oct 2023 05:26:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A672C6B00BA; Mon, 23 Oct 2023 05:26:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 97D446B00BB; Mon, 23 Oct 2023 05:26:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 84AF86B00B9 for ; Mon, 23 Oct 2023 05:26:55 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4318AA0A1D for ; Mon, 23 Oct 2023 09:26:55 +0000 (UTC) X-FDA: 81376196790.25.C438E44 Received: from outbound-smtp28.blacknight.com (outbound-smtp28.blacknight.com [81.17.249.11]) by imf12.hostedemail.com (Postfix) with ESMTP id 3CB3B4001F for ; Mon, 23 Oct 2023 09:26:51 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf12.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.11 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698053213; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SaCtDCwZshQ+K4pA6TVD8e0/ADE1IDaZ7OdH3iWGsHg=; b=Eqqvz5Dz9zeJEvPtFCA2x5tBnLyEkMTohF0its84JGHFx9U56OwJ+LV59/e1nt9xjUswqB RmUtULkkuwSOiYaZS9rokm8qeJYPNTnwGHW6qwvOFaOeWMdQ0l0TJPJL+kSxbOnVF7qV0c xgErzeGFsr9LrWUQlBiMwUUnAJYAz3M= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf12.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.11 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698053213; a=rsa-sha256; cv=none; b=Ki0Yg9dVGHDmFBp5Nmmp9aSe4eDQNXqX+msRmt7ApseRoZgISscnROGCWAY8DMyB92UFGG JGwxOf0LeL+fALb1QN/D07kb9ZaWm5S7MG2XlFkAb+O4WBAfyYNdihsyRuaKzfhjXwC1d8 yJCb+grKoVYgf0e+BpfdMLa+UjNjoVQ= Received: from mail.blacknight.com (pemlinmail02.blacknight.ie [81.17.254.11]) by outbound-smtp28.blacknight.com (Postfix) with ESMTPS id 48477CCFC1 for ; Mon, 23 Oct 2023 10:26:50 +0100 (IST) Received: (qmail 30348 invoked from network); 23 Oct 2023 09:26:50 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[84.203.199.31]) by 81.17.254.9 with ESMTPSA (AES256-SHA encrypted, authenticated); 23 Oct 2023 09:26:49 -0000 Date: Mon, 23 Oct 2023 10:26:47 +0100 From: Mel Gorman To: "Huang, Ying" Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Arjan Van De Ven , Vlastimil Babka , David Hildenbrand , Johannes Weiner , Dave Hansen , Michal Hocko , Pavel Tatashin , Matthew Wilcox , Christoph Lameter Subject: Re: [PATCH -V3 8/9] mm, pcp: decrease PCP high if free pages < high watermark Message-ID: <20231023092647.dxu45uq4lncw5asy@techsingularity.net> References: <20231016053002.756205-1-ying.huang@intel.com> <20231016053002.756205-9-ying.huang@intel.com> <20231019123310.pirier6sk6iaqr3n@techsingularity.net> <87o7guezrm.fsf@yhuang6-desk2.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: <87o7guezrm.fsf@yhuang6-desk2.ccr.corp.intel.com> X-Rspam-User: X-Stat-Signature: wmcipqhof76k8x3odf34iyfosq4uo9os X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 3CB3B4001F X-HE-Tag: 1698053211-437265 X-HE-Meta: U2FsdGVkX19K5mVXcWX9/wFdT+Ncb8MY2iCUYc2mCByHcGG7CqLfHLknci7HoUZ31kphy1TV7KU/r6mIVbHU0ndLxutWG5zofT+A9NaIo7Ux+HOQ3HoNHt/Dj9Hr+7dU7iUulF37BCLIdMGlxJYMD3KMea2evb1K9VtpQCFg+NQvDMDh4GjRglrJYLd8++lbTpKclY9Qi5v9r/dD8KkKSX2mAY+AsVIkfZ93oZJJEK6QMoiucfdtdX94Ykz7HkZD6uancFD51j9+7PL4DusmQOQjFf/dkDjUj56TgxJlrbLw1RyFaIAOysK+5hFs1z927sJPpLwGMpqshpD5G2lIqnohJHRaG9GUhTQMkoLMdvgLiT9oDpuRUWmi25FgoOCbDN4QSQtrqreU8H/9TuNBOQ6ZRN1lTl/+ECNVOkdIvsJUJNVmIlAog3wcsdQTSBeuP/BpLBrVYMyV0ef9Uhd7yOZNgOBi+D9VHkXQRTmtLBWNNz5V+HwVi0rRW6T0pZBj0AMFViJuYKTqt9U7rl0N4xkdod6oaphs1YjAX0zR+Qs0krs9IqnBw+G1JdVhMLT0nX18l5OhrGrqgS0dOy4SmMDMGg/YSRL9vbpPNcUBGyln1GbFBwdVt/JkPOXW6dAPVywYmdvh57ObZustC3Yq8weA8WVHeyrs7J1prOl81xnxPe15gShHM6uKLiU1tMalin+xk0Hw0NHvG1AByDxy2nJerSCkSDmagHcDXPxDLrlr7Ao1HXvsJrTA55c7/VEeatY3YkfAH8GXXrKkWGjj+uvKZ+NIz6hqNNyDWW9Y5z5Ad6qjgf7TLM0TgO8xrefOy53BE9CFjQD8mRprddtsuIweXj9HgRGzXuk273/h7OaWO6dQuPKXRNxM3ZUBNDJ8fDJGXa8V1kVuzPr7Es/+ZBPnRtat4DqsqO18Zul0osYN8ouuKc88uYcDV3rhYzi2e4eaV51rSmXJQ7O5qBW IAeULUkT 0tgPzLxG6eanawjNmIYgASHUetwYPIVai9oZYj0Df0U6x8tbn0VwqGNZfH41Oxb/AIJyQuTES/VoByKxFzawCgQF4+buUaZxN9E1g3BAEcShGCqLSVJmUV1RE4553ki4jaduh1fZAoCNRVV31NMP7UkuTW4LI1Se1fyM4BgEgKVHYe7Trl+lLWI2HS6S3pHozuGm3oVvfRNY0/TIeGxKDOwTmptFDyMYViPxf+d6C7qxEqyEeXL0wSfVseJFf6xECiYRSz2HO/5CgMXwnfISJAMsNGHVH0GisvcSL/o3kTECdL8FuoH95WLv6YM5flutXe/aUITY9MPJmwD1AtNcdQuDWZfQ1JMe35IDQ8qENKyrzySKh35nD2za2rp7ienjOAv7k2sSBg1GL7auFwaYaacScwclwvU1HRL0dBrm/DsItfh2+ha49tqw2gi0a8Qu7iP8i4pdivtgYwBSujMrZOR87SDkGZGJcZz6468jaRkjvw76MHLfWexmtFDIcwY7KgdidXJwrFYHBhbx6lgwTb4U6N+xjHivK1qnr4oA+jo3gZq6jn49tXYRxcdZuOJifWWkr X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Oct 20, 2023 at 11:30:53AM +0800, Huang, Ying wrote: > Mel Gorman writes: > > > On Mon, Oct 16, 2023 at 01:30:01PM +0800, Huang Ying wrote: > >> One target of PCP is to minimize pages in PCP if the system free pages > >> is too few. To reach that target, when page reclaiming is active for > >> the zone (ZONE_RECLAIM_ACTIVE), we will stop increasing PCP high in > >> allocating path, decrease PCP high and free some pages in freeing > >> path. But this may be too late because the background page reclaiming > >> may introduce latency for some workloads. So, in this patch, during > >> page allocation we will detect whether the number of free pages of the > >> zone is below high watermark. If so, we will stop increasing PCP high > >> in allocating path, decrease PCP high and free some pages in freeing > >> path. With this, we can reduce the possibility of the premature > >> background page reclaiming caused by too large PCP. > >> > >> The high watermark checking is done in allocating path to reduce the > >> overhead in hotter freeing path. > >> > >> Signed-off-by: "Huang, Ying" > >> Cc: Andrew Morton > >> Cc: Mel Gorman > >> Cc: Vlastimil Babka > >> Cc: David Hildenbrand > >> Cc: Johannes Weiner > >> Cc: Dave Hansen > >> Cc: Michal Hocko > >> Cc: Pavel Tatashin > >> Cc: Matthew Wilcox > >> Cc: Christoph Lameter > >> --- > >> include/linux/mmzone.h | 1 + > >> mm/page_alloc.c | 33 +++++++++++++++++++++++++++++++-- > >> 2 files changed, 32 insertions(+), 2 deletions(-) > >> > >> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h > >> index ec3f7daedcc7..c88770381aaf 100644 > >> --- a/include/linux/mmzone.h > >> +++ b/include/linux/mmzone.h > >> @@ -1018,6 +1018,7 @@ enum zone_flags { > >> * Cleared when kswapd is woken. > >> */ > >> ZONE_RECLAIM_ACTIVE, /* kswapd may be scanning the zone. */ > >> + ZONE_BELOW_HIGH, /* zone is below high watermark. */ > >> }; > >> > >> static inline unsigned long zone_managed_pages(struct zone *zone) > >> diff --git a/mm/page_alloc.c b/mm/page_alloc.c > >> index 8382ad2cdfd4..253fc7d0498e 100644 > >> --- a/mm/page_alloc.c > >> +++ b/mm/page_alloc.c > >> @@ -2407,7 +2407,13 @@ static int nr_pcp_high(struct per_cpu_pages *pcp, struct zone *zone, > >> return min(batch << 2, pcp->high); > >> } > >> > >> - if (pcp->count >= high && high_min != high_max) { > >> + if (high_min == high_max) > >> + return high; > >> + > >> + if (test_bit(ZONE_BELOW_HIGH, &zone->flags)) { > >> + pcp->high = max(high - (batch << pcp->free_factor), high_min); > >> + high = max(pcp->count, high_min); > >> + } else if (pcp->count >= high) { > >> int need_high = (batch << pcp->free_factor) + batch; > >> > >> /* pcp->high should be large enough to hold batch freed pages */ > >> @@ -2457,6 +2463,10 @@ static void free_unref_page_commit(struct zone *zone, struct per_cpu_pages *pcp, > >> if (pcp->count >= high) { > >> free_pcppages_bulk(zone, nr_pcp_free(pcp, batch, high, free_high), > >> pcp, pindex); > >> + if (test_bit(ZONE_BELOW_HIGH, &zone->flags) && > >> + zone_watermark_ok(zone, 0, high_wmark_pages(zone), > >> + ZONE_MOVABLE, 0)) > >> + clear_bit(ZONE_BELOW_HIGH, &zone->flags); > >> } > >> } > >> > > > > This is a relatively fast path and freeing pages should not need to check > > watermarks. > > Another stuff that mitigate the overhead is that the watermarks checking > only occurs when we free pages from PCP to buddy. That is, in most > cases, every 63 page freeing. > True > > While the overhead is mitigated because it applies only when > > the watermark is below high, that is also potentially an unbounded condition > > if a workload is sized precisely enough. Why not clear this bit when kswapd > > is going to sleep after reclaiming enough pages in a zone? > > IIUC, if the number of free pages is kept larger than the low watermark, > then kswapd will have no opportunity to be waken up even if the number > of free pages was ever smaller than the high watermark. > Also true and I did think of that later. I guess it's ok, the chances are that the series overall offsets any micro-costs like this so I'm happy. If, for some reason, this overhead is noticable (doubtful), then it can be revisted. Thanks. -- Mel Gorman SUSE Labs