From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FA47C35FE1 for ; Sun, 15 Sep 2024 01:00:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72B8C6B007B; Sat, 14 Sep 2024 21:00:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B40B6B0082; Sat, 14 Sep 2024 21:00:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 506326B0083; Sat, 14 Sep 2024 21:00:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2E3866B007B for ; Sat, 14 Sep 2024 21:00:46 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 964B61A109E for ; Sun, 15 Sep 2024 01:00:45 +0000 (UTC) X-FDA: 82565167650.11.6C99A2B Received: from m16.mail.163.com (m16.mail.163.com [117.135.210.4]) by imf21.hostedemail.com (Postfix) with ESMTP id E62181C0002 for ; Sun, 15 Sep 2024 01:00:41 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=163.com header.s=s110527 header.b=bxKp0Af3; dmarc=pass (policy=none) header.from=163.com; spf=pass (imf21.hostedemail.com: domain of a929244872@163.com designates 117.135.210.4 as permitted sender) smtp.mailfrom=a929244872@163.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726361955; a=rsa-sha256; cv=none; b=hVGTDqH9hb1L1IjN7YBRv/X9aEFeUZYzO9jS6YqMqiMeX99PWyvV9vh1uG5M3JBtWMeBmr 1JF0aOj2Czi5hGRY/1VV1rvOw2Egdhnp143vg0pmsJQU5jItTDQSGz3QraqWjYgvzfZY8T PE/4nwGDgp2Irjn328VR+SpabaKmIFw= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=163.com header.s=s110527 header.b=bxKp0Af3; dmarc=pass (policy=none) header.from=163.com; spf=pass (imf21.hostedemail.com: domain of a929244872@163.com designates 117.135.210.4 as permitted sender) smtp.mailfrom=a929244872@163.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726361955; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mn7qb12XU3SlbuhQAOiseJ+pHaTAZ+pcvZrKp8F6Tic=; b=lW4enGEAt730dhAgIvA0K7CFaxz4NVALfTopc+N1JCS7AUVu2WKDciEEx7M0UpfHSqtglb giQGFuFR9Ou6Oja1XdkEoFTBJTWIJ9q6N+jgJtp6uSESxs1vVnKPs9L+xXUgPRm3OKZrWw QJPxLaZTQEwk24T6++OaLRNp516qIhY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:Subject:Date:Message-ID:MIME-Version: Content-Type; bh=mn7qb12XU3SlbuhQAOiseJ+pHaTAZ+pcvZrKp8F6Tic=; b=bxKp0Af3aMZLMR+hIHF+2XzFaOxlwuct2OX8/EDKNo0LSRROoNsM/bgTLGC2bk 1RgWi4+Kk3OoLmOONCalbibP6bakvtv4lf6+M1IV33KSAHPaMvsGDC9MQgSESBaL X7MbKOMtB58E7Iu173gF0xeYLTPmR5NL2i2T4iSvH2Giw= Received: from DESKTOPG6SHR7J (unknown [112.10.227.147]) by gzsmtp3 (Coremail) with SMTP id sigvCgDnsZUtMeZmX2OfAQ--.36272S2; Sun, 15 Sep 2024 08:58:22 +0800 (CST) From: "wang wei" To: "'Barry Song'" <21cnbao@gmail.com>, , Cc: , , , , , , , , , , , , , , "'Barry Song'" References: <20240914063746.46290-1-21cnbao@gmail.com> In-Reply-To: <20240914063746.46290-1-21cnbao@gmail.com> Subject: RE: [PATCH RFC] mm: mglru: provide a separate list for lazyfree anon folios Date: Sun, 15 Sep 2024 08:58:27 +0800 Message-ID: <014f01db070a$61976220$24c62660$@163.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 16.0 Content-Language: zh-cn Thread-Index: AQJy7sFNPkNNwxQPmRpO6KjBLXfo2bEoNjVQ X-CM-TRANSID:sigvCgDnsZUtMeZmX2OfAQ--.36272S2 X-Coremail-Antispam: 1Uf129KBjvJXoW3tr47WrWktFykCw4xGrWDCFg_yoWkGw4rpF Z8GwnrArZ5Jr47Grs5Jr4vkF1SkrWrGryUtFyxW342kF1aqaykKa45Kw1UtFWrCr18ZFyS va4UGF9rW3WjqrUanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07j72-5UUUUU= X-Originating-IP: [112.10.227.147] X-CM-SenderInfo: jdzsmjiuuylji6rwjhhfrp/1tbiLhBbpmV4Lt7vRQAAsE X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: E62181C0002 X-Stat-Signature: rbuo1tfbetc86phh75389qwu8ab3nztc X-Rspam-User: X-HE-Tag: 1726362041-33251 X-HE-Meta: U2FsdGVkX19ky9AapXuxSnatTGYXWljN/OxaZQKLS0qL5Ms+UIwgZyqzVsOuVdXJ4LSIfCl8q8dC9Am3vRPeMe5GnGOpvSFOnmGNFhLH3sHkg5lmth3eOyP0zPlESxafzYEG+yDTq7RNQazJ4rjeuPzvRDTllPq9aex0OMfzLC4F1fEWX/dUrrpLmWxJyHKDoFXmbIQcuso2af1J/KXjnRZadN537FGyH+UdxSlzUhDqy23k6NNhQKZY1KqAzLQW2McyaU247FD8yeowwqQqOk92wbWk4PzcFgCsYZlxGE7Ed9VPsc8u7/xCmixTVFTpqsUlDPrwjWO3ChTmqa6HrJiW1b6npmClXd5lehXPddCjuySmK9wRocSHw9f4Si3//oB86E63eq3AmVPNZm3Uo2qxq17VQKMxv3PfJsNKdhAB1A3aX0khgO7oV8dxPwWYZdYEQ9ipT1JEVYqR8Av83eizanacLQpKSUZdBKOca2NBNpPN7wEWHxxJLolELa5Y7lfwFJBApCO41Klm6Y4dWkPSzTkg7ONgo638DdiLFhNHAqNQP90KVh+uxda6QCXvmDNGCLDWOCaVu4X96VQBuhOaQZo9rXSmmhp3sMT62K4K6dEi0c9sAzar0QvVWT/N8vl5wvT4Zu1k04XazCc43+icziBw7JNMc4f8a/mg/r50EiN9/OZTt7Qp/eXNuOdIJYdng7abOZ+7DEzh7oN1/WTd6xScHsxxt6tjbjtPrHcMi55NI4PUV6gBL+z/GLeOEBVwxbfpGh9F19BQu/58aftRJLhfOB3MyP3jDucAxhiFG8eDeRUcIxlIzdPXUG/CBhwNb2J3A49gSSzT658AmRvvF355afPAXqwKSAwCbOk9Y4Fof2t9+QFWDj86x4ka6D2YVJcR+oHjdrl+9KzORWmA71MqvK+nA2/7nSxbuSvepLjiiT+K1FwcJ/SfuPxTIyw4eztNZEYI5Y/Zl2D ycfCDki9 sfgBMYEOdEvgjVuOmp3TTG7oEhQ4Dng52B8d84+DrLLRYOx2xl3cp+MP8C27uLKPpSF10UG7PA8fbV0aPcqKGZGYnWnsK7mIShwQTiur6szy5uOKaPufIcZuIFznerhgGNOkpgu4lKak4QgNMa6utCWFvHVbdlnO47fUjsuxIoQ4q+Tqocsmt9YmFcADKELKlXbQPP2pi8/YF42/1bjzq3e+DKS9ya3sGTodedxWF+dpqH+9sRYqdJdNtvYRquULzHQfFGLk1KYvMzAj63IMdN5h1z0YmYkeNlwKNif/R7l1tXqiCo5ICJlWp2+q7O/wHSQCaXmVCd4UzQIc3VmgBo8A9sIG83TNDP6CwqwZxQwYWhD8ibO8cMkW7Y/u0fURpFOEsukdGekEjY4q4sSXrBxnTVY1E8vkmj43GDKzlMJntthc62GA6JFhn5cyPBpLpy2pOr5HVvc2icLD8UBeAB9ET2K9Hfy8Ij4pgW+EMQVTWsP0bmKPKD78G3cFt25tV0hkLKpAa5ccfXLOcfc3NtLvBnBgZA8n0Tr6KJkiLOTXtfzvI/tfdGx32jZOfRf/LQ8Ncjqva9SL+r63YaoA8Rtm/WKqw9ergFjQdO8GAQ6EfB5o2rA8bMrXiUp7T2IvFV6Jbiwk8B3cMefmI+5YEHvlCcNLphmXAuX2nYiayfl1U/IDZw8+ctWWQ+EAkXrQ36mX1b3PIGu+XQh0HChsBksqsQAyT3+M7uB/OMVF0PD6PGXhee1a0bamMz+06npFf9pcQZ69gn1wOKBtqGEZp/gb4RFlAuZIaB5r+4xBqLP4rpog= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: > -----Original Message----- > From: linux-kernel+bounces-329184-a929244872=163.com@vger.kernel.org > On > Behalf Of Barry Song > Sent: Saturday, September 14, 2024 2:38 PM > To: akpm@linux-foundation.org; linux-mm@kvack.org > Cc: mhocko@suse.com; fengbaopeng@honor.com; gaoxu2@honor.com; > hailong.liu@oppo.com; kaleshsingh@google.com; linux- > kernel@vger.kernel.org; lokeshgidra@google.com; ngeoffray@google.com; > shli@fb.com; surenb@google.com; yipengxiang@honor.com; > david@redhat.com; yuzhao@google.com; minchan@kernel.org; Barry Song > > Subject: [PATCH RFC] mm: mglru: provide a separate list for lazyfree anon > folios > > From: Barry Song > > This follows up on the discussion regarding Gaoxu's work[1]. It's unclear if > there's still interest in implementing a separate LRU list for lazyfree folios, but I > decided to explore it out of curiosity. > > According to Lokesh, MADV_FREE'd anon folios are expected to be released > earlier than file folios. One option, as implemented by Gao Xu, is to place > lazyfree anon folios at the tail of the file's `min_seq` generation. However, this > approach results in lazyfree folios being released in a LIFO manner, which > conflicts with LRU behavior, as noted by Michal. > > To address this, this patch proposes maintaining a separate list for lazyfree > anon folios while keeping them classified under the "file" LRU type to minimize > code changes. These lazyfree anon folios will still be counted as file folios and > share the same generation with regular files. In the eviction path, the lazyfree > list will be prioritized for scanning before the actual file LRU list. > > [1] https://lore.kernel.org/linux- > mm/f29f64e29c08427b95e3df30a5770056@honor.com/ > > Signed-off-by: Barry Song > --- > include/linux/mm_inline.h | 5 +- > include/linux/mmzone.h | 2 +- > mm/vmscan.c | 97 +++++++++++++++++++++++---------------- > 3 files changed, 61 insertions(+), 43 deletions(-) > > diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index > f4fe593c1400..118d70ed3120 100644 > --- a/include/linux/mm_inline.h > +++ b/include/linux/mm_inline.h > @@ -225,6 +225,7 @@ static inline bool lru_gen_add_folio(struct lruvec > *lruvec, struct folio *folio, > int gen = folio_lru_gen(folio); > int type = folio_is_file_lru(folio); > int zone = folio_zonenum(folio); > + int lazyfree = type ? folio_test_anon(folio) : 0; > struct lru_gen_folio *lrugen = &lruvec->lrugen; > > VM_WARN_ON_ONCE_FOLIO(gen != -1, folio); @@ -262,9 +263,9 > @@ static inline bool lru_gen_add_folio(struct lruvec *lruvec, struct folio > *folio, > lru_gen_update_size(lruvec, folio, -1, gen); > /* for folio_rotate_reclaimable() */ > if (reclaiming) > - list_add_tail(&folio->lru, &lrugen->folios[gen][type][zone]); > + list_add_tail(&folio->lru, &lrugen->folios[gen][type + > +lazyfree][zone]); > else > - list_add(&folio->lru, &lrugen->folios[gen][type][zone]); > + list_add(&folio->lru, &lrugen->folios[gen][type + > lazyfree][zone]); > > return true; > } > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index > 17506e4a2835..5d2331778528 100644 > --- a/include/linux/mmzone.h > +++ b/include/linux/mmzone.h > @@ -434,7 +434,7 @@ struct lru_gen_folio { > /* the birth time of each generation in jiffies */ > unsigned long timestamps[MAX_NR_GENS]; > /* the multi-gen LRU lists, lazily sorted on eviction */ > - struct list_head > folios[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES]; > + struct list_head folios[MAX_NR_GENS][ANON_AND_FILE + > 1][MAX_NR_ZONES]; This also divides lazy free filio into MAX_NR_ZONES generations. The gen of a lazy free filio depends on the gen in the anno list before it is marked as lazy free. Whether it will happen that lazy free filios are released in an order that is not consistent with the order of the mark? > /* the multi-gen LRU sizes, eventually consistent */ > long nr_pages[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES]; > /* the exponential moving average of refaulted */ diff --git > a/mm/vmscan.c b/mm/vmscan.c index 96abf4a52382..9dc665dc6ba9 > 100644 > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -3725,21 +3725,25 @@ static bool inc_min_seq(struct lruvec *lruvec, int > type, bool can_swap) > > /* prevent cold/hot inversion if force_scan is true */ > for (zone = 0; zone < MAX_NR_ZONES; zone++) { > - struct list_head *head = &lrugen- > >folios[old_gen][type][zone]; > + int list_num = type ? 2 : 1; > + struct list_head *head; > > - while (!list_empty(head)) { > - struct folio *folio = lru_to_folio(head); > + for (int i = list_num - 1; i >= 0; i--) { > + head = &lrugen->folios[old_gen][type + i][zone]; > + while (!list_empty(head)) { > + struct folio *folio = lru_to_folio(head); > > - > VM_WARN_ON_ONCE_FOLIO(folio_test_unevictable(folio), folio); > - VM_WARN_ON_ONCE_FOLIO(folio_test_active(folio), > folio); > - > VM_WARN_ON_ONCE_FOLIO(folio_is_file_lru(folio) != type, folio); > - > VM_WARN_ON_ONCE_FOLIO(folio_zonenum(folio) != zone, folio); > + > VM_WARN_ON_ONCE_FOLIO(folio_test_unevictable(folio), folio); > + > VM_WARN_ON_ONCE_FOLIO(folio_test_active(folio), folio); > + > VM_WARN_ON_ONCE_FOLIO(folio_is_file_lru(folio) != type, folio); > + > VM_WARN_ON_ONCE_FOLIO(folio_zonenum(folio) != zone, folio); > > - new_gen = folio_inc_gen(lruvec, folio, false); > - list_move_tail(&folio->lru, &lrugen- > >folios[new_gen][type][zone]); > + new_gen = folio_inc_gen(lruvec, folio, false); > + list_move_tail(&folio->lru, &lrugen- > >folios[new_gen][type + > +i][zone]); > > - if (!--remaining) > - return false; > + if (!--remaining) > + return false; > + } > } > } > done: > @@ -4291,6 +4295,7 @@ static bool sort_folio(struct lruvec *lruvec, struct > folio *folio, struct scan_c > int refs = folio_lru_refs(folio); > int tier = lru_tier_from_refs(refs); > struct lru_gen_folio *lrugen = &lruvec->lrugen; > + int lazyfree = type ? folio_test_anon(folio) : 0; > > VM_WARN_ON_ONCE_FOLIO(gen >= MAX_NR_GENS, folio); > > @@ -4306,7 +4311,7 @@ static bool sort_folio(struct lruvec *lruvec, struct > folio *folio, struct scan_c > > /* promoted */ > if (gen != lru_gen_from_seq(lrugen->min_seq[type])) { > - list_move(&folio->lru, &lrugen->folios[gen][type][zone]); > + list_move(&folio->lru, &lrugen->folios[gen][type + > lazyfree][zone]); > return true; > } > > @@ -4315,7 +4320,7 @@ static bool sort_folio(struct lruvec *lruvec, struct > folio *folio, struct scan_c > int hist = lru_hist_from_seq(lrugen->min_seq[type]); > > gen = folio_inc_gen(lruvec, folio, false); > - list_move_tail(&folio->lru, &lrugen->folios[gen][type][zone]); > + list_move_tail(&folio->lru, &lrugen->folios[gen][type + > +lazyfree][zone]); > > WRITE_ONCE(lrugen->protected[hist][type][tier - 1], > lrugen->protected[hist][type][tier - 1] + delta); @@ - > 4325,7 +4330,7 @@ static bool sort_folio(struct lruvec *lruvec, struct folio > *folio, struct scan_c > /* ineligible */ > if (!folio_test_lru(folio) || zone > sc->reclaim_idx) { > gen = folio_inc_gen(lruvec, folio, false); > - list_move_tail(&folio->lru, &lrugen->folios[gen][type][zone]); > + list_move_tail(&folio->lru, &lrugen->folios[gen][type + > +lazyfree][zone]); > return true; > } > > @@ -4333,7 +4338,7 @@ static bool sort_folio(struct lruvec *lruvec, struct > folio *folio, struct scan_c > if (folio_test_locked(folio) || folio_test_writeback(folio) || > (type == LRU_GEN_FILE && folio_test_dirty(folio))) { > gen = folio_inc_gen(lruvec, folio, true); > - list_move(&folio->lru, &lrugen->folios[gen][type][zone]); > + list_move(&folio->lru, &lrugen->folios[gen][type + > lazyfree][zone]); > return true; > } > > @@ -4377,7 +4382,7 @@ static bool isolate_folio(struct lruvec *lruvec, struct > folio *folio, struct sca static int scan_folios(struct lruvec *lruvec, struct > scan_control *sc, > int type, int tier, struct list_head *list) { > - int i; > + int i, j; > int gen; > enum vm_event_item item; > int sorted = 0; > @@ -4399,33 +4404,38 @@ static int scan_folios(struct lruvec *lruvec, struct > scan_control *sc, > LIST_HEAD(moved); > int skipped_zone = 0; > int zone = (sc->reclaim_idx + i) % MAX_NR_ZONES; > - struct list_head *head = &lrugen->folios[gen][type][zone]; > - > - while (!list_empty(head)) { > - struct folio *folio = lru_to_folio(head); > - int delta = folio_nr_pages(folio); > - > - > VM_WARN_ON_ONCE_FOLIO(folio_test_unevictable(folio), folio); > - VM_WARN_ON_ONCE_FOLIO(folio_test_active(folio), > folio); > - > VM_WARN_ON_ONCE_FOLIO(folio_is_file_lru(folio) != type, folio); > - > VM_WARN_ON_ONCE_FOLIO(folio_zonenum(folio) != zone, folio); > - > - scanned += delta; > + int list_num = type ? 2 : 1; > + struct list_head *head; In addition, scan_folios will also age lazy free list. Is this necessary? > + > + for (j = list_num - 1; j >= 0; j--) { > + head = &lrugen->folios[gen][type + j][zone]; > + while (!list_empty(head)) { > + struct folio *folio = lru_to_folio(head); > + int delta = folio_nr_pages(folio); > + > + > VM_WARN_ON_ONCE_FOLIO(folio_test_unevictable(folio), folio); > + > VM_WARN_ON_ONCE_FOLIO(folio_test_active(folio), folio); > + > VM_WARN_ON_ONCE_FOLIO(folio_is_file_lru(folio) != type, folio); > + > VM_WARN_ON_ONCE_FOLIO(folio_zonenum(folio) != zone, folio); > + > + scanned += delta; > + > + if (sort_folio(lruvec, folio, sc, tier)) > + sorted += delta; > + else if (isolate_folio(lruvec, folio, sc)) { > + list_add(&folio->lru, list); > + isolated += delta; > + } else { > + list_move(&folio->lru, &moved); > + skipped_zone += delta; > + } > > - if (sort_folio(lruvec, folio, sc, tier)) > - sorted += delta; > - else if (isolate_folio(lruvec, folio, sc)) { > - list_add(&folio->lru, list); > - isolated += delta; > - } else { > - list_move(&folio->lru, &moved); > - skipped_zone += delta; > + if (!--remaining || max(isolated, > skipped_zone) >= MIN_LRU_BATCH) > + goto isolate_done; > } > - > - if (!--remaining || max(isolated, skipped_zone) >= > MIN_LRU_BATCH) > - break; > } > > +isolate_done: > if (skipped_zone) { > list_splice(&moved, head); > __count_zid_vm_events(PGSCAN_SKIP, zone, > skipped_zone); @@ -5586,8 +5596,15 @@ void lru_gen_init_lruvec(struct > lruvec *lruvec) > for (i = 0; i <= MIN_NR_GENS + 1; i++) > lrugen->timestamps[i] = jiffies; > > - for_each_gen_type_zone(gen, type, zone) > + for_each_gen_type_zone(gen, type, zone) { > INIT_LIST_HEAD(&lrugen->folios[gen][type][zone]); > + /* > + * lazyfree anon folios have a separate list while using > + * file as type > + */ > + if (type) > + INIT_LIST_HEAD(&lrugen->folios[gen][type + > 1][zone]); > + } > > if (mm_state) > mm_state->seq = MIN_NR_GENS; > -- > 2.39.3 (Apple Git-146)