From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0132CC54E41 for ; Wed, 28 Feb 2024 22:36:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 58C5C6B009B; Wed, 28 Feb 2024 17:36:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 53C3D6B009C; Wed, 28 Feb 2024 17:36:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3DC706B009D; Wed, 28 Feb 2024 17:36:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2E02D6B009B for ; Wed, 28 Feb 2024 17:36:10 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id EC5F5411DA for ; Wed, 28 Feb 2024 22:36:09 +0000 (UTC) X-FDA: 81842672058.09.3837B9F Received: from mail-oi1-f169.google.com (mail-oi1-f169.google.com [209.85.167.169]) by imf27.hostedemail.com (Postfix) with ESMTP id E6DEE40010 for ; Wed, 28 Feb 2024 22:36:07 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=qbO+dg1j; spf=pass (imf27.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.167.169 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709159768; a=rsa-sha256; cv=none; b=lz+clu5J/0GTNBwKlU9VSGupibEoUjE9cuanaL2OeSReF294rX6uVV3LKVOkHr5COfxWp8 w8IuVubN967PaSDOlOYlbACmHc5eGzDAWp754kak4gGrrcxMCTs77tA3vXySJPoi9NcgAW 0XUj0USCSpONvbWN/GCyOHYJsKwI8Ho= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=qbO+dg1j; spf=pass (imf27.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.167.169 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709159768; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=syMwEHkL47bP4Zl9OiTuPEHWa+rdfzjECj4WhEn1mRk=; b=3ObUy/JbP4xdoiMa0B2uuFISXpM/32KTT+qLTM7NAjqVG2SFMh3nVbgHUWVJ9C39ynvZtf XhB5Dm+41TygK92pk3wcHs17kBJjCJafVVPN3hAyAwcyiAkSGuzTQGKGdTCnXeUGgzpDuG Q3ihmwwP99afLg3WizWjN8jkBXkrt/I= Received: by mail-oi1-f169.google.com with SMTP id 5614622812f47-3c1a172e46bso145896b6e.3 for ; Wed, 28 Feb 2024 14:36:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1709159767; x=1709764567; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=syMwEHkL47bP4Zl9OiTuPEHWa+rdfzjECj4WhEn1mRk=; b=qbO+dg1jTm8qS/i0qrBy9JhO9caG/tsVCW4dXfxVQbVWYCpBlkYtCRcvvqHW13rZ7F 4cmQTpZWpwuO3kJRGGOMTMFEJT3CYN6Niw49GblYcWfPyNMMHN2cPVnmQv+yFZ7BUfmj Qbt0NShcPByry8GPgqU7+62ADDtQE3lsY7CVQ7dvrE+c0RIq8TQcv8vJgKOjd8FDlDbT Z6OsTgxTRtYYFRyB6LTySnwPSNJ9x9Hn59G/9vf84Lbgsjgm6T3SeWSuZpwJ03UPU2NE FrSkuQdPniDM4CJ4XpjG3mObReRZQawXgFCjuDMbQO0hddXQSVtKTmJ2LZZ+eQZSqDRn XTtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709159767; x=1709764567; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=syMwEHkL47bP4Zl9OiTuPEHWa+rdfzjECj4WhEn1mRk=; b=V0CCRBMZRLluxIrLp02f0V4kz4p6nYUFVZg16LKhPmysZkbufXCSfFrrdF3R6Wo0dN rMpYJvrfQFPh8CKymR9BSawp4z2eJBWhoT7vYrlojBX1+qyR3AWxmkfz4UQQlS6MWlJK GzPb8egFxQzFXje1AMX3ARp/4MkswFkC/FfSpbZPDmILPfTUveAQJffWUu0FIx90l0jk CVvomMvfBwV6lm8omYexrek0oGnjs2K52TGib0L4G/HYi+vBoow9kMz/fj3HmwoWDh4u W89YfxAjEL1ZSAh9/eGs40E0AS63D45ILZ1IztgvSbmMyiywBa7apTEt41uSgBjrY99g AXtg== X-Forwarded-Encrypted: i=1; AJvYcCWBVINN5qSK6FsAf+QaSWc18JD/QJSfAYr7vi8j+/ME0cgE2jjJhSznk22dzOUfp7xi4Hv5jurH4y0huT24JLyv+W4= X-Gm-Message-State: AOJu0YxPZK8XAwme6TSO8Sm9K3Hw/JBVlK8d42mmKk5mUHQBLz6iscQW lbxtPWQ6quoFfqIZb6kwlzqdqBp4Tvl3LTcIz9Ipfw4i7tOtNXp6BUqxJuyJ0iw= X-Google-Smtp-Source: AGHT+IExUcM840ET+bEdCk7f8mXQR796jT2WCTIOfn/XY++JShOyjlDzdn+e2qLuPH8qPePixihg8w== X-Received: by 2002:aca:1c02:0:b0:3c1:c1ae:462b with SMTP id c2-20020aca1c02000000b003c1c1ae462bmr378528oic.19.1709159767002; Wed, 28 Feb 2024 14:36:07 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id q14-20020a0cfa0e000000b0068f920768a5sm35813qvn.140.2024.02.28.14.36.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Feb 2024 14:36:06 -0800 (PST) Date: Wed, 28 Feb 2024 17:36:01 -0500 From: Johannes Weiner To: Michal Hocko Cc: Byungchul Park , akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, yuzhao@google.com, ying.huang@intel.com, Mel Gorman , Vlastimil Babka Subject: Re: [PATCH v3] mm, vmscan: do not turn on cache_trim_mode if it doesn't work Message-ID: <20240228223601.GA53666@cmpxchg.org> References: <20240223054407.14829-1-byungchul@sk.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: E6DEE40010 X-Stat-Signature: hjqfzq3fpmaw1w63fmquotfwkmb33irn X-Rspam-User: X-HE-Tag: 1709159767-760981 X-HE-Meta: U2FsdGVkX198zTyAi/1J0cHUa9jEG5zXRZ4KeQ6V3qiaPtdXbhGFgLXcPdOZ4m/UkVR8+oaO1BNLiYkIEVi3Va2x4R61gzZmnokjky2cB6E0NuTUq7ak3mNXxXgqTeki6YqjZiXsofFNkrzLde3+3ik+cMXH1QPRl5QkKuR19Geemdw73NWpaiwY66BkO45REFuG7RdXdO49VcHpc3vfkPyrDMJ3HTxfHpGPHpKXAE50Xvxeq83xcO1FTpDg2wibuNtr4XZlbNxvBp5qT138gNRLLEuD9buq9sbOp9Ov5H6PNNfUoEHAA5gseqc4y81RTTt1/nbCZ0Kh0+A+OBjg3rZGQ5OFZoEnqkrEW5pxCCUnMgw7aXVe7ODHnIbOyb0uBhuGcwpqlK1W6bFHVNvMUazEso4nUKlVx9+uX+LIrx67O9FdYUOXMWAD/T24yvVK+CfnZpf7OOeWj6cGb1WGC3QsMQMpa1E8WNblqVwZkklIfC20OAw5dA4sh0zCx59DeELhQOTsecL5tLP3JTfEOe6IeKsrJ9mF3LeCd9DN1jlerrRQb5vG6nJe6/MijIIzvaAsvNixyNlMsTDWCDiFtgfpenH+/JRIWk9NTGwQtkIpiOPbx09P9F7KHY8zHUmU6qekDLZNFf8+jM6siRBwbZyFH5kpl0d2AsFAiB4NdBHuEnSEI9FGAd1iZiZigi5dx4x+lGQi7YdEy52PIdxXFrAChczCP/A6od9WxT7yXsBfgptWtDnhZF8hk0A+IIKuqVJ/lUvOZ7tzFf3ALH8L8x67uhJT8QtkN6PFLLh/HistZBq/BIXHlDcJFnlEcYtoscnP9FhvzCR3MQ2Po6A0AGK8Bz2NsBC3KSz+d7iutLbtE1GJlIUfica8KSLZ6TZyXcGhqky274NKBM6zLtDCgWGj0zfGIvvShRBssOH0HSoVg2jM+vSLnLo2EdDNaxWx2xDz6NUYnBxUUykZy10 94voDHQS fLo6zHxHmBYRGHv3CQa/oAVWBmIXCyFg+I9WME+egvGFxcnT86K+qsJtF5yDsDo4TbyyJNLENre22qrJ57uwoXi3JUNySfpeyJY4fxPb19fCNQpm3HTzThLOEUpUJR7GM0c7wyb1CKr3e/L0rmiCa2e8ohFSOKJthsWop5p1cliEywdLh/t/7qYwxo+lBMK9ehmjK5qJpy6mKHWzFm3cUI6hxk0nTdhJsKjOox/FvBhOrI63eKhcQqkayBzEhZYja9oJ41QwiDjs73YKkyfmEpaPzWS91rmYjz76jX9jsPY96DSLXYzG9r29gKg8vCZMI1HoOgXV94Po88xziDwhsmEnmp7EDjsZm3T9dmxq9Vj35A6ExA5EaBCv2V8rA1lwhbD7oE6zzXcCKqyW5yrSZlpSDSh16cRR4O6GrwmK6M5SEvR81ftRz7VTbBg2/qZtm0nYnkdfzAXZWoSkkuZzUbu0z5NAoTz65jIJ0KCqljHJ+bFjHEeMqik6UicP2zhZZvHboBMEKfnw56vU6m/uXe5ZrtRFRwzJAfTO1PnVTAwe9jp9mLyMa6Nr7wQ+3gnuHYRQh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Feb 26, 2024 at 02:06:30PM +0100, Michal Hocko wrote: > [CC Mel, Vlastimil and Johannes for awareness] > > On Fri 23-02-24 14:44:07, Byungchul Park wrote: > > Changes from v2: > > 1. Change the condition to stop cache_trim_mode. > > > > From - Stop it if it's at high scan priorities, 0 or 1. > > To - Stop it if it's at high scan priorities, 0 or 1, and > > the mode didn't work in the previous turn. > > > > (feedbacked by Huang Ying) > > > > 2. Change the test result in the commit message after testing > > with the new logic. > > > > Changes from v1: > > 1. Add a comment describing why this change is necessary in code > > and rewrite the commit message with how to reproduce and what > > the result is using vmstat. (feedbacked by Andrew Morton and > > Yu Zhao) > > 2. Change the condition to avoid cache_trim_mode from > > 'sc->priority != 1' to 'sc->priority > 1' to reflect cases > > where the priority goes to zero all the way. (feedbacked by > > Yu Zhao) > > > > --->8--- > > >From 05846e34bf02ac9b3e254324dc2d7afd97a025d9 Mon Sep 17 00:00:00 2001 > > From: Byungchul Park > > Date: Fri, 23 Feb 2024 13:47:16 +0900 > > Subject: [PATCH v3] mm, vmscan: do not turn on cache_trim_mode if it doesn't work > > > > With cache_trim_mode on, reclaim logic doesn't bother reclaiming anon > > pages. However, it should be more careful to turn on the mode because > > it's going to prevent anon pages from being reclaimed even if there are > > a huge number of anon pages that are cold and should be reclaimed. Even > > worse, that leads kswapd_failures to reach MAX_RECLAIM_RETRIES and > > stopping kswapd from functioning until direct reclaim eventually works > > to resume kswapd. > > > > So do not turn on cache_trim_mode if the mode doesn't work, especially > > while the sytem is struggling against reclaim. > > > > The problematic behavior can be reproduced by: > > > > CONFIG_NUMA_BALANCING enabled > > sysctl_numa_balancing_mode set to NUMA_BALANCING_MEMORY_TIERING > > numa node0 (8GB local memory, 16 CPUs) > > numa node1 (8GB slow tier memory, no CPUs) > > > > Sequence: > > > > 1) echo 3 > /proc/sys/vm/drop_caches > > 2) To emulate the system with full of cold memory in local DRAM, run > > the following dummy program and never touch the region: > > > > mmap(0, 8 * 1024 * 1024 * 1024, PROT_READ | PROT_WRITE, > > MAP_ANONYMOUS | MAP_PRIVATE | MAP_POPULATE, -1, 0); > > > > 3) Run any memory intensive work e.g. XSBench. > > 4) Check if numa balancing is working e.i. promotion/demotion. > > 5) Iterate 1) ~ 4) until numa balancing stops. > > > > With this, you could see that promotion/demotion are not working because > > kswapd has stopped due to ->kswapd_failures >= MAX_RECLAIM_RETRIES. > > > > Interesting vmstat delta's differences between before and after are like: > > > > +-----------------------+-------------------------------+ > > | interesting vmstat | before | after | > > +-----------------------+-------------------------------+ > > | nr_inactive_anon | 321935 | 1636737 | > > | nr_active_anon | 1780700 | 465913 | > > | nr_inactive_file | 30425 | 35711 | > > | nr_active_file | 14961 | 8698 | > > | pgpromote_success | 356 | 1267785 | > > | pgpromote_candidate | 21953245 | 1745631 | > > | pgactivate | 1844523 | 3309867 | > > | pgdeactivate | 50634 | 1545041 | > > | pgfault | 31100294 | 6411036 | > > | pgdemote_kswapd | 30856 | 2267467 | > > | pgscan_kswapd | 1861981 | 7729231 | > > | pgscan_anon | 1822930 | 7667544 | > > | pgscan_file | 39051 | 61687 | > > | pgsteal_anon | 386 | 2227217 | > > | pgsteal_file | 30470 | 40250 | > > | pageoutrun | 30 | 457 | > > | numa_hint_faults | 27418279 | 2752289 | > > | numa_pages_migrated | 356 | 1267785 | > > +-----------------------+-------------------------------+ > > > > Signed-off-by: Byungchul Park > > --- > > mm/vmscan.c | 24 +++++++++++++++++++----- > > 1 file changed, 19 insertions(+), 5 deletions(-) > > > > diff --git a/mm/vmscan.c b/mm/vmscan.c > > index bba207f41b14..f7312d831fed 100644 > > --- a/mm/vmscan.c > > +++ b/mm/vmscan.c > > @@ -127,6 +127,9 @@ struct scan_control { > > /* One of the zones is ready for compaction */ > > unsigned int compaction_ready:1; > > > > + /* If the last try was reclaimable */ > > + unsigned int reclaimable:1; > > + > > /* There is easily reclaimable cold cache in the current node */ > > unsigned int cache_trim_mode:1; > > > > @@ -2266,9 +2269,14 @@ static void prepare_scan_control(pg_data_t *pgdat, struct scan_control *sc) > > * If we have plenty of inactive file pages that aren't > > * thrashing, try to reclaim those first before touching > > * anonymous pages. > > + * > > + * It doesn't make sense to keep cache_trim_mode on if the mode > > + * is not working while struggling against reclaim. So do not > > + * turn it on if so. Note the highest priority of kswapd is 1. > > */ > > file = lruvec_page_state(target_lruvec, NR_INACTIVE_FILE); > > - if (file >> sc->priority && !(sc->may_deactivate & DEACTIVATE_FILE)) > > + if (file >> sc->priority && !(sc->may_deactivate & DEACTIVATE_FILE) && > > + !(sc->cache_trim_mode && !sc->reclaimable && sc->priority <= 1)) > > sc->cache_trim_mode = 1; > > else > > sc->cache_trim_mode = 0; The overall goal makes sense to me. file >> priority is just a heuristic that there are enough potential candidate pages, not a guarantee that any forward progress will happen. So it makes sense to retry without before failing. The way you wrote this conditional kind of hurts my head, though. Please don't write negations of complex terms like this. It expands to this: !sc->cache_trim_mode || sc->reclaimable || sc->priority > 1 which I'm not sure makes sense. Surely it should be something like !sc->cache_trim_mode && sc->reclaimable && sc->priority > 1 instead? Also if (!sc->cache_trim_mode) sc->cache_trim_mode = 1 else sc->cache_trim_mode = 0 will toggle on every loop. So if direct reclaim runs through a zonelist, it'll cache trim every other numa node...? > > @@ -5862,7 +5870,6 @@ static void shrink_node(pg_data_t *pgdat, struct scan_control *sc) > > { > > unsigned long nr_reclaimed, nr_scanned, nr_node_reclaimed; > > struct lruvec *target_lruvec; > > - bool reclaimable = false; > > > > if (lru_gen_enabled() && root_reclaim(sc)) { > > lru_gen_shrink_node(pgdat, sc); > > @@ -5877,6 +5884,14 @@ static void shrink_node(pg_data_t *pgdat, struct scan_control *sc) > > nr_reclaimed = sc->nr_reclaimed; > > nr_scanned = sc->nr_scanned; > > > > + /* > > + * Reset to the default values at the start. > > + */ > > + if (sc->priority == DEF_PRIORITY) { > > + sc->reclaimable = 1; > > + sc->cache_trim_mode = 0; > > + } > > + > > prepare_scan_control(pgdat, sc); > > > > shrink_node_memcgs(pgdat, sc); > > @@ -5890,8 +5905,7 @@ static void shrink_node(pg_data_t *pgdat, struct scan_control *sc) > > vmpressure(sc->gfp_mask, sc->target_mem_cgroup, true, > > sc->nr_scanned - nr_scanned, nr_node_reclaimed); > > > > - if (nr_node_reclaimed) > > - reclaimable = true; > > + sc->reclaimable = !!nr_node_reclaimed; The scope of this doesn't quite make sense. If direct reclaim scans multiple nodes, reclaim failure on the first node would disable cache trim mode on the second node, which is totally unrelated. I think it needs separate paths for direct reclaim and kswapd. For direct reclaim, the retry should be before these similar retry catches in do_try_to_free_pages(), after all zones have been considered: /* * We make inactive:active ratio decisions based on the node's * composition of memory, but a restrictive reclaim_idx or a * memory.low cgroup setting can exempt large amounts of * memory from reclaim. Neither of which are very common, so * instead of doing costly eligibility calculations of the * entire cgroup subtree up front, we assume the estimates are * good, and retry with forcible deactivation if that fails. */ if (sc->skipped_deactivate) { sc->priority = initial_priority; sc->force_deactivate = 1; sc->skipped_deactivate = 0; goto retry; } /* Untapped cgroup reserves? Don't OOM, retry. */ if (sc->memcg_low_skipped) { sc->priority = initial_priority; sc->force_deactivate = 0; sc->memcg_low_reclaim = 1; sc->memcg_low_skipped = 0; goto retry; } and for kswapd it looks like it should be in balance_pgdat(), after the priority loop, before increasing kswapd_failures. Instead of sc->reclaimable, which is very broad, it would be better to call it sc->may_cache_trim_mode. This is in line with a bunch of other such mechanisms in scan_control.