From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4EB06EB64DD for ; Wed, 2 Aug 2023 02:08:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 59A20280117; Tue, 1 Aug 2023 22:08:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 54A1C280112; Tue, 1 Aug 2023 22:08:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41132280117; Tue, 1 Aug 2023 22:08:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 32794280112 for ; Tue, 1 Aug 2023 22:08:18 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 046D2120D55 for ; Wed, 2 Aug 2023 02:08:17 +0000 (UTC) X-FDA: 81077529876.13.33B7984 Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by imf16.hostedemail.com (Postfix) with ESMTP id 3F4E9180017 for ; Wed, 2 Aug 2023 02:08:15 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=6MnyYdm0; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.169 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690942095; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Aja2sp26dbGeO9esoM3QzIDVku+dq0XwVDIc+IsolCQ=; b=Ue/eR7B9bIhvbGe7wLuNnFVaRahs81b9iv0QBkzM+LPHhZjihOHiQOGIWeV2NJYkVLzWa6 ucLF7cEwbcjvWWmnAHPFl+RJ6Q9bcEqvDg7AY26LFoYq+ZKRiOTRgVO4nBGbxMCIzYSxL7 p2s2XowdLehLq5G1xfA8mX4MgPFgaog= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=6MnyYdm0; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.169 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690942095; a=rsa-sha256; cv=none; b=Ppd/IeFHXiww5KhxOb6Iv+7DeR3q3+YtAvyWC+svj/nnmwdotmQBaS6mnRqGcRp8kfe8CA TeXj9Un2TYQXpLBpuRTs5iGpg0Lijxhk3XjfRCdRRdh6ltG2v95GLzygTYPgRbv/3rjVZc tJHRDcwXFqrzG+CIssNinbds3ud4QAw= Received: by mail-qt1-f169.google.com with SMTP id d75a77b69052e-4036bd4fff1so158071cf.0 for ; Tue, 01 Aug 2023 19:08:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690942094; x=1691546894; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Aja2sp26dbGeO9esoM3QzIDVku+dq0XwVDIc+IsolCQ=; b=6MnyYdm06DH0WFOhS8+wUQPsLSbrpKWd3npWMcNWWyzAoz/kWWU4HkWHEZkS61E5pe 9+jM2Wx2GDwjmZYAXMQ3oB/Ejtai2i/RIC/5EhBihdDTYwCEGEc0i91VlXsTHUJNwSLg HEWLCMrTrjQPcHYz/1kyIkGqMQcNEkWIjapol0tMtvd6wLBG/kZFFiygKH+0VY7jQciM llbYmlAKWBPiPtT85B8qB78/xEDiFTc4jhn4UorM7ZYY2b8Kp0Hsfr/3asBIIHRoIz4G 9GRBAmP5iodnS7ZBJgfcQQApGF89b1H00DiaC3rJYn/6IOMhoWejdsbrBkUAlYw2vfXb 6EXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690942094; x=1691546894; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Aja2sp26dbGeO9esoM3QzIDVku+dq0XwVDIc+IsolCQ=; b=P+466pudyoPUm1+PdA+jI8uqxiuCYYV2r03KuE6K6aoyTM5TgHu9zHJFBKZI6bvFnU Qt7PeH3+pq9TQ/eHIwZ5EtTwrnah+5M9E6TqVLjhCYK/pUweN2+t+nFDsTC3f/uyGN9p GwXjV30wyCp4Npm1LS91PcvZ279ozmufVEaxe7Qz0mM06TL9MZ0NLlCJ7eU/hytrpN+B i2y8Vb69JseiDb/lASBvpp7y4FhHa+rJalSCDBqQErDrDGCpDqe1uGALZdb+V8p85rov 1fabGvkwSfnKeeqqEJJDRaU9IgumzZpVGIb79RXrHKQ4DCavz5GIfEeGI1qS082c6Avb wvwA== X-Gm-Message-State: ABy/qLaIp2WEqSWkfxnwACOQ1EjBJnynIMNVNft6QzFCbNy8x4kTXQma OpD99iQG6csNsuVSqleRZ4E4aat/YFZj98kg6XvJ5Q== X-Google-Smtp-Source: APBJJlFI0LpzT3Dxb29zrFcaiBceXtmAw2GDelbar44DfZF8uuQLNGEI78+/S1i3xVkHS0a1TO1gl0VExunoRFjXvvE= X-Received: by 2002:ac8:584b:0:b0:40f:dc70:fde2 with SMTP id h11-20020ac8584b000000b0040fdc70fde2mr129571qth.26.1690942094306; Tue, 01 Aug 2023 19:08:14 -0700 (PDT) MIME-Version: 1.0 References: <20230802001938.3913174-1-kaleshsingh@google.com> In-Reply-To: <20230802001938.3913174-1-kaleshsingh@google.com> From: Yu Zhao Date: Tue, 1 Aug 2023 20:07:38 -0600 Message-ID: Subject: Re: [PATCH 1/3] mm-unstable: Multi-gen LRU: Fix per-zone reclaim To: Kalesh Singh Cc: akpm@linux-foundation.org, surenb@google.com, android-mm@google.com, kernel-team@android.com, Charan Teja Kalla , Lecopzer Chen , Matthias Brugger , AngeloGioacchino Del Regno , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 3F4E9180017 X-Stat-Signature: ac649aoufhe15gfg6ymw6xes5t75sk9y X-HE-Tag: 1690942095-450609 X-HE-Meta: U2FsdGVkX1/q0Wsx63hr1GYnn4SBFZXjb1aWg0DQnJZLc01mz2Ebp8YQq8bXFzcS8WwjEfARWFeNvOIcPe2rA1DsckdeT8fRR5ksaS0EAsMAzCw4k/A0ZVS9LdtgYSCqJi4rq5SoivF9BXIW4chZ4UT8BZGfc4Gq5JPEUYbuRRM6jvm77uX835TMuLWhUHMBLFG4Eb7zLe/h6rNURfC4BRbF8Pw8PeilJYUvL5BlM1u5jrHD6lkiCIEWhGxTSb5WnMhkVo+4oXeCP53rh/keGTFS1clyo3lVyqaPMedAyoJC8lK5Sc036DPtXq2qr+GBu7rbq6Saire9rzcT4CVv+NaPLyxbS3XqkXZfJPlRzmb1JU3udI0ESLVNRt3+L5E6tDRw5K23hQMcFhP+j5RTDh4lw1l33BEXD7S61WihoRTh3IZsYyCqk8eJQsAGB8N0Q7YvUyaWtBB6+o2kAFBFQBzvBPtAyB+iDm+yx4KxgS6TJWehfm/y9rgyIhqwm5GoJMU7e/eYk00rjAxfLenQ0wopsK5ghrqw134h/IgI9a0OmNLw9xfIYn0wylAowu3I8h3XeL3FfPEGWR8Y/sQC0VlWAEc8bGFK5f/VONSZQqevku3HdrpreibwUSJfncrVMmV5xcmEURguh9EnY0n7trm85516uLq7xSIRcS6aGLoQ1ilrOGv+id0Yiki8M+g/HBlqxxAkXZj6fzj3tvH2pe1ISw1uheJhFtdZjSSidg8IAw21byUgAwBgffhHAQkTDLjOy7YMtRwkYtBOo8Zzk28MkKwUwBKPkQQs7SeAfKuk8e9CTCyCTvRAGSFLGSgjWCN7gvmM07g/2eDB/XcuHbj9sg/pGGEOk0V1qKtDvD84a4SXNJUBB0WRvKzIxewGwF0cM/PMNawE/RmnlN9nI7a/WzJ71Fnhx3U3C/NMQLLQ8P2x9a7zm3c1i29C03o5zrCmfqAGpc9/IH+OPk9 7F5c61pO YPilnb3byWrvm1Dw2N+B2gl+jMrbrZ5INAkCItcC7H04+a+fKjFCZnJZgJlmQA4gdj05Jd9OHbLKRtjGtcVZ8tSebv+azlJPtBz6NqGP0Jk+x2UQs7SBQ7J4cWeQD8NPSUGQk4BHubYlE9kwYgysOtmTt2aieZdpTW6T7PGiSDRoK0L1whNWFtTzc7f/waabt42wYsZOfrYnb8GqfB8jz/Zd3y0bHHdzSXhXbGI54wz8QTR2F4pe1znaFvo+dcuU2c3oWKOy82Zie9vP/GVZrAa2POAMQo532xYkMreZPBG2/rA3oZHoDS6Y1kQMlTMmseqHpl+zWWzG9XhtJLq4Ncgr+O7qYhu2mdoEMYmVDUTXbG6OqHxxSF8G51iiyhJi0jh2ipH2ffVDI9+IasM3GQk5Gr1gVU5psAc0js2qwyYxSDf/5QYf7S00UT3tt4QF6kgGbD4U+v9khStE5+ZXCIRObrf1TyKg4rL9gHPNp2g0ZRz8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.223077, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Aug 1, 2023 at 6:19=E2=80=AFPM Kalesh Singh wrote: > > MGLRU has a LRU list for each zone for each type (anon/file) in each > generation: > > long nr_pages[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES]; > > The min_seq (oldest generation) can progress independently for each > type but the max_seq (youngest generation) is shared for both anon and > file. This is to maintain a common frame of reference. > > In order for eviction to advance the min_seq of a type, all the per-zone > lists in the oldest generation of that type must be empty. > > The eviction logic only considers pages from eligible zones for > eviction or promotion. > > scan_folios() { > ... > for (zone =3D sc->reclaim_idx; zone >=3D 0; zone--) { > ... > sort_folio(); // Promote > ... > isolate_folio(); // Evict > } > ... > } > > Consider the system has the movable zone configured and default 4 > generations. The current state of the system is as shown below > (only illustrating one type for simplicity): > > Type: ANON > > Zone DMA32 Normal Movable Device > > Gen 0 0 0 4GB 0 > > Gen 1 0 1GB 1MB 0 > > Gen 2 1MB 4GB 1MB 0 > > Gen 3 1MB 1MB 1MB 0 > > Now consider there is a GFP_KERNEL allocation request (eligible zone > index <=3D Normal), evict_folios() will return without doing any work > since there are no pages to scan in the eligible zones of the oldest > generation. Reclaim won't make progress until triggered from a ZONE_MOVAB= LE > allocation request; which may not happen soon if there is a lot of free > memory in the movable zone. This can lead to OOM kills, although there > is 1GB pages in the Normal zone of Gen 1 that we have not yet tried to > reclaim. > > This issue is not seen in the conventional active/inactive LRU since > there are no per-zone lists. > > If there are no (not enough) folios to scan in the eligible zones, move > folios from ineligible zone (zone_index > reclaim_index) to the next > generation. This allows for the progression of min_seq and reclaiming > from the next generation (Gen 1). > > Qualcomm, Mediatek and raspberrypi [1] discovered this issue independentl= y. > > [1] https://github.com/raspberrypi/linux/issues/5395 > > Cc: Yu Zhao > Cc: Andrew Morton > Reported-by: Charan Teja Kalla > Reported-by: Lecopzer Chen > Signed-off-by: Kalesh Singh LGTM. But I think we need the Fixes tag and Cc stable.