From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF29BC4167B for ; Wed, 13 Dec 2023 07:59:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6EC776B00A6; Wed, 13 Dec 2023 02:59:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 69CCA6B0479; Wed, 13 Dec 2023 02:59:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53C666B047A; Wed, 13 Dec 2023 02:59:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 441B76B00A6 for ; Wed, 13 Dec 2023 02:59:55 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 10C7040123 for ; Wed, 13 Dec 2023 07:59:55 +0000 (UTC) X-FDA: 81561046350.17.0A3F9AF Received: from mail-qt1-f173.google.com (mail-qt1-f173.google.com [209.85.160.173]) by imf13.hostedemail.com (Postfix) with ESMTP id 5479220011 for ; Wed, 13 Dec 2023 07:59:53 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KHpZjtU1; spf=pass (imf13.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.173 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702454393; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7SCRsk8v2NeRoEP69IlZxtD0S6CzE5b/aKxi3o7os3s=; b=18u/hYoVMwwWiql0ID4fAiVr77m1tzaZ9Q8Eo954luK5zAQTtY0NXz37+q6qS+DLbRtFOq +DOalXehss6jeIXi4mbqiJtMi8AWn7Gx0Az8Tbsl3VxwIpNrSMgwfrlMF5DYUk7c68Egmx aHPNBbOpeKr6rFILqeh9eeI3ln0ath8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702454393; a=rsa-sha256; cv=none; b=4vtqTzuyLR4yRY7Up5LUdHsVbADkxD2OzvHcVy3bRoRzNDHQWir5LylKitbzIikMp22/0E Mbk33rwNvorp22tVtrIBJh13COQRvUXSV0uV94uqVUYaNEHhb60lf4hbxRmPLjbLU/1hRz P11gOMi4DcTKbJL3lphn7hZM+tvSaWw= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KHpZjtU1; spf=pass (imf13.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.173 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-qt1-f173.google.com with SMTP id d75a77b69052e-425c1d7d72eso169811cf.1 for ; Tue, 12 Dec 2023 23:59:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702454392; x=1703059192; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=7SCRsk8v2NeRoEP69IlZxtD0S6CzE5b/aKxi3o7os3s=; b=KHpZjtU1fh0oYtVqSZr/D8jryUBHOnpgp/b5mLwpfX6TTDQIRYDf422PHluJGpPc2o duSasNrtfCilgo4qH21vLlMDD7wilXwKfslw1qVI86tycfbBxLRcr/6ekbTWBxmddKYC In3+vIDjO6IWCwHWpd12DCtiqGOC835MPT1WnQZHK3DqQ+K47E4dAiPMqBl6bEOzr9R2 UqM4j9Nv2eRxORJobHQfMdfv0+Bp3kD2XVNeAXZ02LxvwuxuQhJgmF0e/TXrLz92+rmT aQz08vW58WtF6r0HllJUMVjT640RIoCDJWD25DcsDycpCsCZ1jKPjjJo3xmIxANJYtuE qIpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702454392; x=1703059192; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7SCRsk8v2NeRoEP69IlZxtD0S6CzE5b/aKxi3o7os3s=; b=Qn8/C4EgbZX9dBb/fszn+Lrf5WAGxgHbKGLt7W6McEIRAMh9SJ91EuW3iA5myg5tNb fy+M8Zve6kZpb6hyyrBjX/KvulzpIXo2gip6zfUisuexxz9wUvfJCU2kzZiappyyLK+4 VuIKNvrzDveyVS3UmAjWnaPJcXPQmmoL5oslX7OOiBGhT7bTM03+QVgHeGaM5PI7SPnv DwAaZurW9NJ+BfqsrfIpOMG/dNJmeqVBtwf52EHavjJP9JHfw9mAqGGIuCakD8NrrkuW tv8x3vze4DR9qMHzPq54hpx+EPuUUfFwQw/RkIIcrcO2KEUTUizteBuoKZpH2WmtnF3I Ob/w== X-Gm-Message-State: AOJu0YxSQBr5jw8Dpgj+RPI4jkkaxfhD6dCSyc4l/dBkjRFxFVqX3UzJ hTh+JcLOQD1WQFR2AeNDSYowsVFlLrnI5/egYp48jg== X-Google-Smtp-Source: AGHT+IEpUfZumpmFkAwQhMzxID9arjyLxuQJmf93yVKOo9TY8C7lbVucG1fgltdFUHZLYcEy1F6UJMTjytiuxR7ltBE= X-Received: by 2002:a05:622a:190c:b0:423:b6a1:2088 with SMTP id w12-20020a05622a190c00b00423b6a12088mr1307562qtc.5.1702454392208; Tue, 12 Dec 2023 23:59:52 -0800 (PST) MIME-Version: 1.0 References: <20231208061407.2125867-1-yuzhao@google.com> In-Reply-To: From: Yu Zhao Date: Wed, 13 Dec 2023 00:59:14 -0700 Message-ID: Subject: Re: [PATCH mm-unstable v1 1/4] mm/mglru: fix underprotected page cache To: Kairui Song Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Charan Teja Kalla , Kalesh Singh , stable@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: mm4o9oqc8kygnkgr75yhesqmz7wc3sgn X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 5479220011 X-Rspam-User: X-HE-Tag: 1702454393-981876 X-HE-Meta: U2FsdGVkX19pESeENyPMmgw2GFq9EQKJHAkbqqopix5MucbAP2VHPieo72p1N1XymCG0K3C2CLxDLwuFmwB1RZQ6jRaqc5lYsBvFFXTcR6JE61eVQIJl+Tw2lhNiqIP7Sv1ONvpIpfqk+L/XQ3Bgy5HHZb8s5HNr4l0IT+nOueELabFm1REol+BRX/Y+od1OjjA5kMFFLzxeaX4AN4T0qNa/0jP26yFP+0mNGQ39KkP/Lf11R6EE7iB6npYa/QXA7ltk2etZ4VsfJJnogJCmTTX/eo7RVfZSq6o14AMGsLZB3kNhjPvoAtJL1zn6giUiEPjxxPhoZTLzCgsEjWvh2b2PL9A/QlvII389UVLA/czjd6FZdl64fECz7eCcK1LCfhAhiTERoMQTws/F2wATZgfh2cdXZDdrGT8Kef7ddLzQMGPFZK2vnfee4Hv8Hng9mAiQ4e5mS4vSry3L+wdtEWnudTT/20Kul6aBpW/EXpNt56XOKF89wpZnGFrZDMB5Dcey+KWvVPqVd9AXpguia94NNW0OVnRY20ZesM8Br5LksDFyMWrbJDyq5iW89+MRhgbs4avUopwjzVyD5gHdT/YtZVh78PbVEMipRMBHyUepPubmtnrzusz/2XdvDHgoWxqtrLTz6z5EWwobA4uxUg7NHCDbvAKLWQYEqML4zSMUiyKWW2qQiD9vGXnDw5lqKGrDYc75a7tdB0JEHH4PEFEHL9dvMld8EYK/8N7OvZMrsgs0R/BJZF3YH5UJFd0qvj3246h0y4ujc0TXaJmoTm4osrGjNuIp62EVwLJ5FXYqOK4H0eqli5iqzO7AyYiEVphCTtjxL7kYUjLqiDUJxshlikzea2QT8q7n7VhnXWv+rD5bhbCD65V5RfV3F7Zpf59p+gkcvXm4orM4VIKUieJNoJZnogwCesQ2ws5VbEYLSnFSknctNmmdPvYo8g+ssnEatfFRW4uNE8dV1Id vd4Q/Rov EYyO/bJ3RS3/SJbsAHSGej9DKKXjP68oFr7hLLxNbk/CeCV8wEHBu/BkiiSXSf1gq8PlXoKbbgN2WJGwK/WYOZr+lIS9+zAS0psA3VWeQYxO758naRow6MKdruZ/IUUvViZti3uvLGpllVLyH7lLh1c4sG19az2xL6p6aYWQx3Y6n9LYeP3S8pj1MLN4RrWeOXIQz/UGQKF++FnNGOsmk+icQoBLypgqigO5EVgvf03i+IuAD4EgBxVK9EudPqAsZUdpjGvMB2R4w7F5E5b1lZY2z22Kftg0dx1TaO+N4J8dHEhupR526ByajUVoWmO7dfYt1q3syXvLF4ak= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Dec 12, 2023 at 8:03=E2=80=AFPM Kairui Song wrot= e: > > Kairui Song =E4=BA=8E2023=E5=B9=B412=E6=9C=8812=E6=97= =A5=E5=91=A8=E4=BA=8C 14:52=E5=86=99=E9=81=93=EF=BC=9A > > > > Yu Zhao =E4=BA=8E2023=E5=B9=B412=E6=9C=8812=E6=97= =A5=E5=91=A8=E4=BA=8C 06:07=E5=86=99=E9=81=93=EF=BC=9A > > > > > > On Fri, Dec 8, 2023 at 1:24=E2=80=AFAM Kairui Song = wrote: > > > > > > > > Yu Zhao =E4=BA=8E2023=E5=B9=B412=E6=9C=888=E6= =97=A5=E5=91=A8=E4=BA=94 14:14=E5=86=99=E9=81=93=EF=BC=9A > > > > > > > > > > Unmapped folios accessed through file descriptors can be > > > > > underprotected. Those folios are added to the oldest generation b= ased > > > > > on: > > > > > 1. The fact that they are less costly to reclaim (no need to walk= the > > > > > rmap and flush the TLB) and have less impact on performance (d= on't > > > > > cause major PFs and can be non-blocking if needed again). > > > > > 2. The observation that they are likely to be single-use. E.g., f= or > > > > > client use cases like Android, its apps parse configuration fi= les > > > > > and store the data in heap (anon); for server use cases like M= ySQL, > > > > > it reads from InnoDB files and holds the cached data for table= s in > > > > > buffer pools (anon). > > > > > > > > > > However, the oldest generation can be very short lived, and if so= , it > > > > > doesn't provide the PID controller with enough time to respond to= a > > > > > surge of refaults. (Note that the PID controller uses weighted > > > > > refaults and those from evicted generations only take a half of t= he > > > > > whole weight.) In other words, for a short lived generation, the > > > > > moving average smooths out the spike quickly. > > > > > > > > > > To fix the problem: > > > > > 1. For folios that are already on LRU, if they can be beyond the > > > > > tracking range of tiers, i.e., five accesses through file > > > > > descriptors, move them to the second oldest generation to give= them > > > > > more time to age. (Note that tiers are used by the PID control= ler > > > > > to statistically determine whether folios accessed multiple ti= mes > > > > > through file descriptors are worth protecting.) > > > > > 2. When adding unmapped folios to LRU, adjust the placement of th= em so > > > > > that they are not too close to the tail. The effect of this is > > > > > similar to the above. > > > > > > > > > > On Android, launching 55 apps sequentially: > > > > > Before After Change > > > > > workingset_refault_anon 25641024 25598972 0% > > > > > workingset_refault_file 115016834 106178438 -8% > > > > > > > > Hi Yu, > > > > > > > > Thanks you for your amazing works on MGLRU. > > > > > > > > I believe this is the similar issue I was trying to resolve previou= sly: > > > > https://lwn.net/Articles/945266/ > > > > The idea is to use refault distance to decide if the page should be > > > > place in oldest generation or some other gen, which per my test, > > > > worked very well, and we have been using refault distance for MGLRU= in > > > > multiple workloads. > > > > > > > > There are a few issues left in my previous RFC series, like anon pa= ges > > > > in MGLRU shouldn't be considered, I wanted to collect feedback or t= est > > > > cases, but unfortunately it seems didn't get too much attention > > > > upstream. > > > > > > > > I think both this patch and my previous series are for solving the > > > > file pages underpertected issue, and I did a quick test using this > > > > series, for mongodb test, refault distance seems still a better > > > > solution (I'm not saying these two optimization are mutually exclus= ive > > > > though, just they do have some conflicts in implementation and solv= ing > > > > similar problem): > > > > > > > > Previous result: > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > Execution Results after 905 seconds > > > > ------------------------------------------------------------------ > > > > Executed Time (=C2=B5s) Rate > > > > STOCK_LEVEL 2542 27121571486.2 0.09 txn/s > > > > ------------------------------------------------------------------ > > > > TOTAL 2542 27121571486.2 0.09 txn/s > > > > > > > > This patch: > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > Execution Results after 900 seconds > > > > ------------------------------------------------------------------ > > > > Executed Time (=C2=B5s) Rate > > > > STOCK_LEVEL 1594 27061522574.4 0.06 txn/s > > > > ------------------------------------------------------------------ > > > > TOTAL 1594 27061522574.4 0.06 txn/s > > > > > > > > Unpatched version is always around ~500. > > > > > > Thanks for the test results! > > > > > > > I think there are a few points here: > > > > - Refault distance make use of page shadow so it can better > > > > distinguish evicted pages of different access pattern (re-access > > > > distance). > > > > - Throttled refault distance can help hold part of workingset when > > > > memory is too small to hold the whole workingset. > > > > > > > > So maybe part of this patch and the bits of previous series can be > > > > combined to work better on this issue, how do you think? > > > > > > I'll try to find some time this week to look at your RFC. It'd be a > > Hi Yu, > > I'm working on V4 of the RFC now, which just update some comments, and > skip anon page re-activation in refault path for mglru which was not > very helpful, only some tiny adjustment. > And I found it easier to test with fio, using following test script: > > #!/bin/bash > swapoff -a > > modprobe brd rd_nr=3D1 rd_size=3D16777216 > mkfs.ext4 /dev/ram0 > mount /dev/ram0 /mnt > > mkdir -p /sys/fs/cgroup/benchmark > cd /sys/fs/cgroup/benchmark > > echo 4G > memory.max > echo $$ > cgroup.procs > echo 3 > /proc/sys/vm/drop_caches > > fio -name=3Dmglru --numjobs=3D12 --directory=3D/mnt --size=3D1024m \ > --buffered=3D1 --ioengine=3Dio_uring --iodepth=3D128 \ > --iodepth_batch_submit=3D32 --iodepth_batch_complete=3D32 \ > --rw=3Drandread --random_distribution=3Dzipf:0.5 --norandommap = \ > --time_based --ramp_time=3D5m --runtime=3D5m --group_reporting > > zipf:0.5 is used here to simulate a cached read with slight bias > towards certain pages. > Unpatched 6.7-rc4: > Run status group 0 (all jobs): > READ: bw=3D6548MiB/s (6866MB/s), 6548MiB/s-6548MiB/s > (6866MB/s-6866MB/s), io=3D1918GiB (2060GB), run=3D300001-300001msec > > Patched with RFC v4: > Run status group 0 (all jobs): > READ: bw=3D7270MiB/s (7623MB/s), 7270MiB/s-7270MiB/s > (7623MB/s-7623MB/s), io=3D2130GiB (2287GB), run=3D300001-300001msec > > Patched with this series: > Run status group 0 (all jobs): > READ: bw=3D7098MiB/s (7442MB/s), 7098MiB/s-7098MiB/s > (7442MB/s-7442MB/s), io=3D2079GiB (2233GB), run=3D300002-300002msec > > MGLRU off: > Run status group 0 (all jobs): > READ: bw=3D6525MiB/s (6842MB/s), 6525MiB/s-6525MiB/s > (6842MB/s-6842MB/s), io=3D1912GiB (2052GB), run=3D300002-300002msec > > - If I change zipf:0.5 to random: > Unpatched 6.7-rc4: > Patched with this series: > Run status group 0 (all jobs): > READ: bw=3D5975MiB/s (6265MB/s), 5975MiB/s-5975MiB/s > (6265MB/s-6265MB/s), io=3D1750GiB (1879GB), run=3D300002-300002msec > > Patched with RFC v4: > Run status group 0 (all jobs): > READ: bw=3D5987MiB/s (6278MB/s), 5987MiB/s-5987MiB/s > (6278MB/s-6278MB/s), io=3D1754GiB (1883GB), run=3D300001-300001msec > > Patched with this series: > Run status group 0 (all jobs): > READ: bw=3D5839MiB/s (6123MB/s), 5839MiB/s-5839MiB/s > (6123MB/s-6123MB/s), io=3D1711GiB (1837GB), run=3D300001-300001msec > > MGLRU off: > Run status group 0 (all jobs): > READ: bw=3D5689MiB/s (5965MB/s), 5689MiB/s-5689MiB/s > (5965MB/s-5965MB/s), io=3D1667GiB (1790GB), run=3D300003-300003msec > > fio uses ramdisk so LRU accuracy will have smaller impact. The Mongodb > test I provided before uses a SATA SSD so it will have a much higher > impact. I'll provides a script to setup the test case and run it, it's > more complex to setup than fio since involving setting up multiple > replicas and auth and hundreds of GB of test fixtures, I'm currently > occupied by some other tasks but will try best to send them out as > soon as possible. Thanks! Apparently your RFC did show better IOPS with both access patterns, which was a surprise to me because it had higher refaults and usually higher refautls result in worse performance. So I'm still trying to figure out why it turned out the opposite. My current guess is that: 1. It had a very small but stable inactive LRU list, which was able to fit into the L3 cache entirely. 2. It counted few folios as workingset and therefore incurred less overhead from CONFIG_PSI and/or CONFIG_TASK_DELAY_ACCT. Did you save workingset_refault_file when you ran the test? If so, can you check the difference between this series and your RFC?