From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C7C6C001DC for ; Mon, 24 Jul 2023 01:11:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C3C216B0071; Sun, 23 Jul 2023 21:11:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BED0B6B0074; Sun, 23 Jul 2023 21:11:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB47A6B0075; Sun, 23 Jul 2023 21:11:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9C28F6B0071 for ; Sun, 23 Jul 2023 21:11:20 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 5BB881C96EA for ; Mon, 24 Jul 2023 01:11:20 +0000 (UTC) X-FDA: 81044727120.24.DBB88E3 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf16.hostedemail.com (Postfix) with ESMTP id C82D5180005 for ; Mon, 24 Jul 2023 01:11:17 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JyrbsA7C; spf=pass (imf16.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690161078; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=emTPugFu5GJT74frbV1JZ2ddNLPUjzTgQ5Y39GZUPXY=; b=VrN1vfLtJfRyPlAqOJA85iqskTlOFYc9cFANHUf07cSHGqajuT53R4Rk7BY7B2esdXLbmB ofMaPyexBktpMn5+qk9R7iHwHfOPvl3x8/ccBHOv/eERh9iA4GnnLd/y6be/aFP9O61k6r /Zi1BXuTCJ0465PMcLGzq0Yy6vRIGJg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690161078; a=rsa-sha256; cv=none; b=F3kfeC/YxaaKw3tAcc87iPh+hHwB8M/YU6SLomHTxnD5V11dXZO5QS+CvT6BWALI9zKCT9 9VbegiPIzi2kpaBZEqzlUTXb3ltJFbToooxr0MQrs4HaIOZVdu3/6OXURkXzHEXdKBNJfU X3jgzBeU+ucCvFk1v0GWm+ND5bKK1qY= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JyrbsA7C; spf=pass (imf16.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690161077; x=1721697077; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=2bkwBVVjsPHIqYj7TayR1Uu4788OLpgRpr080tPdSn0=; b=JyrbsA7C06iNFVtsNy3W+UVtYUeRZcApKwYctEpU9FCG5+VQENUmcX95 blgOlLk4AqPM0SBagJARpnh7RmUbn/r7mMakF3jygymHmyiGoW7FM5CqG wNUZaMWEnp85GEnp0Nhbct0BqmxkF51rJMSyWbgS+CuRw0SrvqqMMznZu VZMhfBkBPA5N0LJYER83c5SX7CEGvKZHrv+uTt+xQIEWEnGAJW9mUayMg bpDoGjuVcsZbw7SyIDtm0y78oODoJiDA6iedscjwWGLkMuTUTQand+L5/ lMj2XK/t4Kz0TfnsZFxxa5vM5yrtqlcQiW1YE9bKmZB87f7pRjQjoz3yB Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10780"; a="433562624" X-IronPort-AV: E=Sophos;i="6.01,228,1684825200"; d="scan'208";a="433562624" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jul 2023 18:11:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10780"; a="719473880" X-IronPort-AV: E=Sophos;i="6.01,228,1684825200"; d="scan'208";a="719473880" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jul 2023 18:11:12 -0700 From: "Huang, Ying" To: Mel Gorman Cc: Michal Hocko , , , Arjan Van De Ven , Andrew Morton , Vlastimil Babka , David Hildenbrand , Johannes Weiner , Dave Hansen , Pavel Tatashin , Matthew Wilcox Subject: Re: [RFC 2/2] mm: alloc/free depth based PCP high auto-tuning References: <20230712090526.thk2l7sbdcdsllfi@techsingularity.net> <871qhcdwa1.fsf@yhuang6-desk2.ccr.corp.intel.com> <20230714140710.5xbesq6xguhcbyvi@techsingularity.net> <87pm4qdhk4.fsf@yhuang6-desk2.ccr.corp.intel.com> <20230717135017.7ro76lsaninbazvf@techsingularity.net> <87lefeca2z.fsf@yhuang6-desk2.ccr.corp.intel.com> <20230718123428.jcy4avtjg3rhuh7i@techsingularity.net> <87mszsbfx7.fsf@yhuang6-desk2.ccr.corp.intel.com> <20230719090518.67g7hascnfcly6hk@techsingularity.net> <87fs5h7mfo.fsf@yhuang6-desk2.ccr.corp.intel.com> <20230721092119.5nzpru7ttfudqzbg@techsingularity.net> Date: Mon, 24 Jul 2023 09:09:31 +0800 In-Reply-To: <20230721092119.5nzpru7ttfudqzbg@techsingularity.net> (Mel Gorman's message of "Fri, 21 Jul 2023 10:21:19 +0100") Message-ID: <87bkg26rp0.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Queue-Id: C82D5180005 X-Rspam-User: X-Stat-Signature: nhjpd9899i94jdt9k7py5ej3m6giuwrn X-Rspamd-Server: rspam03 X-HE-Tag: 1690161077-863688 X-HE-Meta: U2FsdGVkX1/wnAKgs0bwGnpQnaWKZ4ojwC1pr08qfKYXgcS8qo7rhcBXNN1HHmf5sKhlVLv9Wc7J70cnokWSGKLQPHa3nRGG9sCLSk6wC55Xif1DwyJkmyQ6FFTFAuBy2FqW63+EOLjKir6Q2lpsyJFmFUo6ShUBQrcKWRrxuMOb9pLV2r3DB6S6OKf2l8XVc+7bEqTWl2gLsUdzzRk23NMgJznM9PeyfzwV1CO7loNsKbEQNvw496uJzgNWz8caqZxqnNpmtG50lkEa/hzSu6qfiAltwyCutvsr7av+O/7d1otV9JVMicyjy6STysmdMMNRCFAvLk+pxFMJFkMmb6J8saLLTCYJhmKn93Q/AUZzAr9mP1/3fltt5euluZGkGkiW4dUAqtrWNxajJrnOu72ROj1Tg3+MkXMqWhzmJSLvXZXJFabEZaGq9hvg0T5EumX/oaWnjZkKo4Ne+Ae3mpPhd0cIxIqQ3MFmjNSl5M07ExyC514PKAwaCa2jlJU7D+WGt0EPtEWcOazmIhM+As1vYcr2RGL+JRJX/5C4+CawbwppyEIp43TdLu0kFbixrQE4yyUZfNMXm73RqJXJOuRXtjUA3tKD4xMNGxiJe58bnGFi831v6+K/KXwFpirS1T0wGUQeIwGj1sCgEj1/o30UdiOuHmekMtguADkbI6boJ+ilwuL31SSFEJpfnXd6enh9cOiq6ReNK39IBkfaYJtD/9RUZEe5qiaHidPXgPdfFS5rcQiuTAJrqvu8eiUjzYN4RogyQAhAiCgiM6VBTa+j/5wsI8YsAm7DAaLNQqSwErVK3q8SREflLc4i1WA2h2nlZXAjJ0PG/kluJcrvwhAspKn6CiyUVeLO312nXdE13RQdwk/vo7P10d/rhp/9mHP3l5R1abDX/e/m1mC/odKVeXL1Otk2axWNyc9hMoi4pEvyzqVBPQrtPt/6JLzY5CxcV0CCvCHKZw73yUY vjVwvjbX NXRE+nE4F0Uf1BYPNj5UOXDs0VEphOlhAUOEbR8CW5RJL6rl0F9m/RFMLXK3h+6bAgcwXBzBnWCIdbpVvb0U91SoYAjtOj6jyhuowNr/5fAezyODLulohvUNpMtEw1d4NRsltza3iVdlmgBBv4AdKAOmskFfrNrVZE/f4M4kT7r802V5sMETduiT+61lsBmkqOqyuPiFmqTPb3JJuF16qz9T8WA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Mel Gorman writes: > On Fri, Jul 21, 2023 at 03:28:43PM +0800, Huang, Ying wrote: >> Mel Gorman writes: >> >> > On Wed, Jul 19, 2023 at 01:59:00PM +0800, Huang, Ying wrote: >> >> > The big remaaining corner case to watch out for is where the sum >> >> > of the boosted pcp->high exceeds the low watermark. If that should ever >> >> > happen then potentially a premature OOM happens because the watermarks >> >> > are fine so no reclaim is active but no pages are available. It may even >> >> > be the case that the sum of pcp->high should not exceed *min* as that >> >> > corner case means that processes may prematurely enter direct reclaim >> >> > (not as bad as OOM but still bad). >> >> >> >> Sorry, I don't understand this. When pages are moved from buddy to PCP, >> >> zone NR_FREE_PAGES will be decreased in rmqueue_bulk(). That is, pages >> >> in PCP will be counted as used instead of free. And, in >> >> zone_watermark_ok*() and zone_watermark_fast(), zone NR_FREE_PAGES is >> >> used to check watermark. So, if my understanding were correct, if the >> >> number of pages in PCP is larger than low/min watermark, we can still >> >> trigger reclaim. Whether is my understanding correct? >> >> >> > >> > You're right, I didn't check the timing of the accounting and all that >> > occurred to me was "the timing of when watermarks trigger kswapd or >> > direct reclaim may change as a result of PCP adaptive resizing". Even >> > though I got the timing wrong, the shape of the problem just changes. >> > I suspect that excessively large PCP high relative to the watermarks may >> > mean that reclaim happens prematurely if too many pages are pinned by PCP >> > pages as the zone free pages approaches the watermark. >> >> Yes. I think so too. In addition to reclaim, falling back to remote >> NUMA node may happen prematurely too. >> > > Yes, with the added bonus that this is relatively easy to detect from > the NUMA miss stats. I say "relative" because in a lot of cases, it'll be > difficult to distinguish from the noise. Hence, it's better to be explicit in > the change log that the potential problem is known and has been considered. > That way, if bisect points the finger at adaptive resizing, there will be > some notes on how to investigate the bug. Sure. Will do that. >> > While disabling the adaptive resizing during reclaim will limit the >> > worst of the problem, it may still be the case that kswapd is woken >> > early simply because there are enough CPUs pinning pages in PCP >> > lists. Similarly, depending on the size of pcp->high and the gap >> > between the watermarks, it's possible for direct reclaim to happen >> > prematurely. I could still be wrong because I'm not thinking the >> > problem through fully, examining the code or thinking about the >> > implementation. It's simply worth keeping in mind the impact elevated >> > PCP high values has on the timing of watermarks failing. If it's >> > complex enough, it may be necessary to have a separate patch dealing >> > with the impact of elevated pcp->high on watermarks. >> >> Sure. I will keep this in mind. We may need to check zone watermark >> when tuning pcp->high and free some pages from PCP before falling back >> to other node or reclaiming. >> > > That would certainly be one option, a cap on adaptive resizing as memory > gets lower. It's not perfect but ideally the worst-case behaviour would be > that PCP adaptive sizing returns to existing behaviour when memory usage > is persistently high and near watermarks within a zone. -- Best Regards, Huang, Ying