From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46017C3DA6E for ; Sun, 17 Dec 2023 18:32:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2F1E96B006E; Sun, 17 Dec 2023 13:32:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2A2936B0074; Sun, 17 Dec 2023 13:32:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 142696B0075; Sun, 17 Dec 2023 13:32:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 033626B0074 for ; Sun, 17 Dec 2023 13:32:22 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id BBF171207B4 for ; Sun, 17 Dec 2023 18:32:21 +0000 (UTC) X-FDA: 81577155282.21.951ADDF Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by imf18.hostedemail.com (Postfix) with ESMTP id CDBCF1C001C for ; Sun, 17 Dec 2023 18:32:19 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=3s4yQ3e+; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.48 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702837939; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xi8WxnO/5ju1HSV+e8T/TEEABVPzUu+B2D1s+/TqHas=; b=GmHD3uF9B7z4Xhl1qxC6KpibKFV+yeNwphwjoyKzncX63lS9TP5Zt+WmFKrSXbPoQKJ61v ZOT0kEra+falPmrmVGUfNQiFzogdJUR+ssF9eF2K7HXqhRFeZe5witl+g2ednLQ+G7BxRB DVInF1agBiUSTXnUMOOGtueV7zCMAE0= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=3s4yQ3e+; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.48 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702837939; a=rsa-sha256; cv=none; b=XVM9XQSWK7VV2XknqI1E4sWNLLG3ZbNoG1USvwylERcO1GhlUYlPdBCQckeQYWP4j0s3Sx kX3l8l1/rs3XkbcF5kDEuHAQk/76nm5kx2XCbv9DqX+GAxuCDdbPoAJVmvBC8lCRa1vf9Z KtJQxd3I6nBg1D8k2wqGmZOejMgyFTw= Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-40c3963f9fcso40305e9.1 for ; Sun, 17 Dec 2023 10:32:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702837938; x=1703442738; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=xi8WxnO/5ju1HSV+e8T/TEEABVPzUu+B2D1s+/TqHas=; b=3s4yQ3e+lyEEbFVjkaKVMR+qv75kwaVQEeaCi6p4esP6yYTxRrWFQEYEL6erN6BQIs pm6xiEu2lEbo58VamI+xsRioK/D9jGkx1aRCeFrj/d0EcjGaprJH70q5cJpNwFifjraX /KWkysElDNJJbTiOvkadiwCeMBIXSMgr3R4FeC+s5AM098p1I0+/N6T4HujfrVcaAx/Y 0gU6Riv833AsJKxX2ayz21wCk7gbJ5fzS9oAh0xY4yclYTjvYEHL4oOyrkz9YBQ8D+R5 Tyha9AigIraSh2v66DP3/5o1ity7isfZMjLxUyaKAQvLmSy//iYxTUYb4Kl+LSlzRFg3 7CJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702837938; x=1703442738; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xi8WxnO/5ju1HSV+e8T/TEEABVPzUu+B2D1s+/TqHas=; b=eaP4yycGja7kwaEuLJjTHAwhJ3LNZyDgrj55MaZDr5+tlUrvytBhr30Nz+JRwiIe0l ajZWjnKmNb9ouoO9Ms/YCtjbiM7NnxGJmMMkFqGd06M5MfT7LrgFOjOz2g/ADZFsvIMQ 3jxpTOFtK7eoH5vxXCl275DoM7Ytuzt6+mvIFj8/WO68Dqrgqg77OUKlWqH9mlfykboL GOFSZyh1gxl/7M8aqijYA7XKdZlVq3srCI8PS9mJrK+iLTmsJtDPbe5N9M7Suh75P2yh mp0a5+lpJ31RwxyVWIVMbKrUNU94vnmC42o4GZkzsT4WR2zL1dvGyj+YBU4xrFg3oLKu JxCQ== X-Gm-Message-State: AOJu0YxYi2UlCcZ57dDzp4ao6FLDeW7LB1wVzxe5bU0tCXCawSSyF4Ot 9kcRYUH0t9hDHVgyGLlowFuBOOuzB6TMQXaX4b1Yn08kauGg X-Google-Smtp-Source: AGHT+IGdXOu9eyPcTs+cJBufM93L37weDLAN8z1jWXvoCUM6NrDSrdmQ2RCVAhUxS6+gFnQdcTdQA5rb3u3Bvdsop+0= X-Received: by 2002:a05:600c:4655:b0:40d:1482:896a with SMTP id n21-20020a05600c465500b0040d1482896amr96929wmo.1.1702837937966; Sun, 17 Dec 2023 10:32:17 -0800 (PST) MIME-Version: 1.0 References: <20231208061407.2125867-1-yuzhao@google.com> In-Reply-To: From: Yu Zhao Date: Sun, 17 Dec 2023 11:31:39 -0700 Message-ID: Subject: Re: [PATCH mm-unstable v1 1/4] mm/mglru: fix underprotected page cache To: Kairui Song Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Charan Teja Kalla , Kalesh Singh , stable@vger.kernel.org, Matthew Wilcox Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: CDBCF1C001C X-Stat-Signature: c1195ff1ui6ogyghipwndbxitwuspe4h X-Rspam-User: X-HE-Tag: 1702837939-505909 X-HE-Meta: U2FsdGVkX1+q3ihgIAlXsmgquZQ2h5UkZmVZ7R7ZHXNfO24GaMezMzG0voWgXEgi3sk83KnBxdOKuU1qnWA5ZVPYaNLhUrd7XQu7q1joyd8O+r+7rcSrQbx/8eAPnSWKeeTkLqnY6GLS69X4OYgV6lFh5rSLj13j3Dktikq8hasPKHrYpdM9lHb2RX9bodUlXX2cnrGtprN/F2/JcBh9Bv9Qy9PsYHQ/r62ox5yi5h3w6o8r10PEgZ4nLUa7V9U9jFWqc3BR7LKTFc6zQbSM0y/Hqfvm6xnY8oH/tTWGlKvN7VHOxtVZQqv0PVjV5WHDAz/+G39VuObo0BXuw/jl1Qvbxm7vAsUcZhIxZDjOTxhZJu26n+riAJYb/48bVmC421hI2TYei+DvIuJFx91/7gczmfsreMcjVlpdIwGLfLKmvc5+hcJweN5svHCUbG2wi42jiSuR7W0od3Cb4wnBI+7ZK/o3yZERicdxnIV5sXucw59zb/NTS7pcC3T8x8zmlPdlcTtuqpWKmMnW7XTW6Tr5RIyQM7L8WuDM+3/nLmqKlN6jjOaE+wKgBhmHZaSft/O/x+x6WUZxSNUxfzGXcsWwVH81X9VQEOwagWanrKIU45jC2y00J4v6GZYBV9bYm6CH8mPltHTes71Ub7bK9NPjcnY84YNeYHX3RaZlxV1M30ENj/eCQtO1mIDSFWkvhfc6NVWRuAVhT0TORTfb2A64a16AerOLfIsM6U1jU+HAYctkcfnYKbvSZ13Y2WBxcjbQo374oBFnQ0ik8oiCtyns+L2EyhoMOymTv2weAcEdSAq36xDjbqQTuCGejbTnYYsNewnOG048+0krgXZGmeS7oTdgIIlbR7NX+9naNBznNHumyu2OSd0Pem7y4mz6qcjth7pz3f6Kjd/+33R20IWyd163oqZg/S+vk9GJGG0iMkAUZ4l4KvVTD/31Xz/e3oOXgZd9yao/P716Lte 8qIobR+u wjtQ20LfiAfcRq6l4IgqlYk7tXzAsp895rdnEK4Z5WxvOV/Zri39uadz5pivkHzuas959tiK7hjGWFA0fAP3zwFhXEA8BxryWOMHZBMuOD+QwvERRObNx/1o8PsnKtJ7CUVIJLEMgqg3zVGrmVNIW8RijKN7MTsuSh5EB/n4jBYacN0Recv+8YkYGTz1RJG/ox/EqxT69tjHHQKV4XOvda5UPp1Fmdkd48Kl+E6CmQE6MjGHgPWQl8iBh2RONrZbUWlUgWoTSyOx87SRwwe9pt+LUHkNzOV6zCJ9GyFXHCSc3WNNqXbiJfIy+yfvkDxjwVAWL0YSF47bjjFU+FMvacDSsFxM7tqmjDIFd/PvrAvDM/emgAH2K2ef6lj//BNaaAf+CHTIK0/DXMv7M/8eyCxxUPmC4xYnN4Zi0QUjEBspKatPk6baAVoefELqo40s6i2cg+Et0aTOHSl4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Dec 14, 2023 at 4:51=E2=80=AFPM Yu Zhao wrote: > > On Thu, Dec 14, 2023 at 11:38=E2=80=AFAM Kairui Song w= rote: > > > > Yu Zhao =E4=BA=8E2023=E5=B9=B412=E6=9C=8814=E6=97= =A5=E5=91=A8=E5=9B=9B 11:09=E5=86=99=E9=81=93=EF=BC=9A > > > On Wed, Dec 13, 2023 at 12:59:14AM -0700, Yu Zhao wrote: > > > > On Tue, Dec 12, 2023 at 8:03=E2=80=AFPM Kairui Song wrote: > > > > > > > > > > Kairui Song =E4=BA=8E2023=E5=B9=B412=E6=9C=881= 2=E6=97=A5=E5=91=A8=E4=BA=8C 14:52=E5=86=99=E9=81=93=EF=BC=9A > > > > > > > > > > > > Yu Zhao =E4=BA=8E2023=E5=B9=B412=E6=9C=8812= =E6=97=A5=E5=91=A8=E4=BA=8C 06:07=E5=86=99=E9=81=93=EF=BC=9A > > > > > > > > > > > > > > On Fri, Dec 8, 2023 at 1:24=E2=80=AFAM Kairui Song wrote: > > > > > > > > > > > > > > > > Yu Zhao =E4=BA=8E2023=E5=B9=B412=E6=9C= =888=E6=97=A5=E5=91=A8=E4=BA=94 14:14=E5=86=99=E9=81=93=EF=BC=9A > > > > > > > > > > > > > > > > > > Unmapped folios accessed through file descriptors can be > > > > > > > > > underprotected. Those folios are added to the oldest gene= ration based > > > > > > > > > on: > > > > > > > > > 1. The fact that they are less costly to reclaim (no need= to walk the > > > > > > > > > rmap and flush the TLB) and have less impact on perfor= mance (don't > > > > > > > > > cause major PFs and can be non-blocking if needed agai= n). > > > > > > > > > 2. The observation that they are likely to be single-use.= E.g., for > > > > > > > > > client use cases like Android, its apps parse configur= ation files > > > > > > > > > and store the data in heap (anon); for server use case= s like MySQL, > > > > > > > > > it reads from InnoDB files and holds the cached data f= or tables in > > > > > > > > > buffer pools (anon). > > > > > > > > > > > > > > > > > > However, the oldest generation can be very short lived, a= nd if so, it > > > > > > > > > doesn't provide the PID controller with enough time to re= spond to a > > > > > > > > > surge of refaults. (Note that the PID controller uses wei= ghted > > > > > > > > > refaults and those from evicted generations only take a h= alf of the > > > > > > > > > whole weight.) In other words, for a short lived generati= on, the > > > > > > > > > moving average smooths out the spike quickly. > > > > > > > > > > > > > > > > > > To fix the problem: > > > > > > > > > 1. For folios that are already on LRU, if they can be bey= ond the > > > > > > > > > tracking range of tiers, i.e., five accesses through f= ile > > > > > > > > > descriptors, move them to the second oldest generation= to give them > > > > > > > > > more time to age. (Note that tiers are used by the PID= controller > > > > > > > > > to statistically determine whether folios accessed mul= tiple times > > > > > > > > > through file descriptors are worth protecting.) > > > > > > > > > 2. When adding unmapped folios to LRU, adjust the placeme= nt of them so > > > > > > > > > that they are not too close to the tail. The effect of= this is > > > > > > > > > similar to the above. > > > > > > > > > > > > > > > > > > On Android, launching 55 apps sequentially: > > > > > > > > > Before After Change > > > > > > > > > workingset_refault_anon 25641024 25598972 0% > > > > > > > > > workingset_refault_file 115016834 106178438 -8% > > > > > > > > > > > > > > > > Hi Yu, > > > > > > > > > > > > > > > > Thanks you for your amazing works on MGLRU. > > > > > > > > > > > > > > > > I believe this is the similar issue I was trying to resolve= previously: > > > > > > > > https://lwn.net/Articles/945266/ > > > > > > > > The idea is to use refault distance to decide if the page s= hould be > > > > > > > > place in oldest generation or some other gen, which per my = test, > > > > > > > > worked very well, and we have been using refault distance f= or MGLRU in > > > > > > > > multiple workloads. > > > > > > > > > > > > > > > > There are a few issues left in my previous RFC series, like= anon pages > > > > > > > > in MGLRU shouldn't be considered, I wanted to collect feedb= ack or test > > > > > > > > cases, but unfortunately it seems didn't get too much atten= tion > > > > > > > > upstream. > > > > > > > > > > > > > > > > I think both this patch and my previous series are for solv= ing the > > > > > > > > file pages underpertected issue, and I did a quick test usi= ng this > > > > > > > > series, for mongodb test, refault distance seems still a be= tter > > > > > > > > solution (I'm not saying these two optimization are mutuall= y exclusive > > > > > > > > though, just they do have some conflicts in implementation = and solving > > > > > > > > similar problem): > > > > > > > > > > > > > > > > Previous result: > > > > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > > > Execution Results after 905 seconds > > > > > > > > -----------------------------------------------------------= ------- > > > > > > > > Executed Time (=C2=B5s) Rate > > > > > > > > STOCK_LEVEL 2542 27121571486.2 0.09 txn/= s > > > > > > > > -----------------------------------------------------------= ------- > > > > > > > > TOTAL 2542 27121571486.2 0.09 txn/= s > > > > > > > > > > > > > > > > This patch: > > > > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > > > Execution Results after 900 seconds > > > > > > > > -----------------------------------------------------------= ------- > > > > > > > > Executed Time (=C2=B5s) Rate > > > > > > > > STOCK_LEVEL 1594 27061522574.4 0.06 txn/= s > > > > > > > > -----------------------------------------------------------= ------- > > > > > > > > TOTAL 1594 27061522574.4 0.06 txn/= s > > > > > > > > > > > > > > > > Unpatched version is always around ~500. > > > > > > > > > > > > > > Thanks for the test results! > > > > > > > > > > > > > > > I think there are a few points here: > > > > > > > > - Refault distance make use of page shadow so it can better > > > > > > > > distinguish evicted pages of different access pattern (re-a= ccess > > > > > > > > distance). > > > > > > > > - Throttled refault distance can help hold part of workings= et when > > > > > > > > memory is too small to hold the whole workingset. > > > > > > > > > > > > > > > > So maybe part of this patch and the bits of previous series= can be > > > > > > > > combined to work better on this issue, how do you think? > > > > > > > > > > > > > > I'll try to find some time this week to look at your RFC. It'= d be a > > > > > > > > > > Hi Yu, > > > > > > > > > > I'm working on V4 of the RFC now, which just update some comments= , and > > > > > skip anon page re-activation in refault path for mglru which was = not > > > > > very helpful, only some tiny adjustment. > > > > > And I found it easier to test with fio, using following test scri= pt: > > > > > > > > > > #!/bin/bash > > > > > swapoff -a > > > > > > > > > > modprobe brd rd_nr=3D1 rd_size=3D16777216 > > > > > mkfs.ext4 /dev/ram0 > > > > > mount /dev/ram0 /mnt > > > > > > > > > > mkdir -p /sys/fs/cgroup/benchmark > > > > > cd /sys/fs/cgroup/benchmark > > > > > > > > > > echo 4G > memory.max > > > > > echo $$ > cgroup.procs > > > > > echo 3 > /proc/sys/vm/drop_caches > > > > > > > > > > fio -name=3Dmglru --numjobs=3D12 --directory=3D/mnt --size=3D1024= m \ > > > > > --buffered=3D1 --ioengine=3Dio_uring --iodepth=3D128 \ > > > > > --iodepth_batch_submit=3D32 --iodepth_batch_complete=3D= 32 \ > > > > > --rw=3Drandread --random_distribution=3Dzipf:0.5 --nora= ndommap \ > > > > > --time_based --ramp_time=3D5m --runtime=3D5m --group_re= porting > > > > > > > > > > zipf:0.5 is used here to simulate a cached read with slight bias > > > > > towards certain pages. > > > > > Unpatched 6.7-rc4: > > > > > Run status group 0 (all jobs): > > > > > READ: bw=3D6548MiB/s (6866MB/s), 6548MiB/s-6548MiB/s > > > > > (6866MB/s-6866MB/s), io=3D1918GiB (2060GB), run=3D300001-300001ms= ec > > > > > > > > > > Patched with RFC v4: > > > > > Run status group 0 (all jobs): > > > > > READ: bw=3D7270MiB/s (7623MB/s), 7270MiB/s-7270MiB/s > > > > > (7623MB/s-7623MB/s), io=3D2130GiB (2287GB), run=3D300001-300001ms= ec > > > > > > > > > > Patched with this series: > > > > > Run status group 0 (all jobs): > > > > > READ: bw=3D7098MiB/s (7442MB/s), 7098MiB/s-7098MiB/s > > > > > (7442MB/s-7442MB/s), io=3D2079GiB (2233GB), run=3D300002-300002ms= ec > > > > > > > > > > MGLRU off: > > > > > Run status group 0 (all jobs): > > > > > READ: bw=3D6525MiB/s (6842MB/s), 6525MiB/s-6525MiB/s > > > > > (6842MB/s-6842MB/s), io=3D1912GiB (2052GB), run=3D300002-300002ms= ec > > > > > > > > > > - If I change zipf:0.5 to random: > > > > > Unpatched 6.7-rc4: > > > > > Patched with this series: > > > > > Run status group 0 (all jobs): > > > > > READ: bw=3D5975MiB/s (6265MB/s), 5975MiB/s-5975MiB/s > > > > > (6265MB/s-6265MB/s), io=3D1750GiB (1879GB), run=3D300002-300002ms= ec > > > > > > > > > > Patched with RFC v4: > > > > > Run status group 0 (all jobs): > > > > > READ: bw=3D5987MiB/s (6278MB/s), 5987MiB/s-5987MiB/s > > > > > (6278MB/s-6278MB/s), io=3D1754GiB (1883GB), run=3D300001-300001ms= ec > > > > > > > > > > Patched with this series: > > > > > Run status group 0 (all jobs): > > > > > READ: bw=3D5839MiB/s (6123MB/s), 5839MiB/s-5839MiB/s > > > > > (6123MB/s-6123MB/s), io=3D1711GiB (1837GB), run=3D300001-300001ms= ec > > > > > > > > > > MGLRU off: > > > > > Run status group 0 (all jobs): > > > > > READ: bw=3D5689MiB/s (5965MB/s), 5689MiB/s-5689MiB/s > > > > > (5965MB/s-5965MB/s), io=3D1667GiB (1790GB), run=3D300003-300003ms= ec > > > > > > > > > > fio uses ramdisk so LRU accuracy will have smaller impact. The Mo= ngodb > > > > > test I provided before uses a SATA SSD so it will have a much hig= her > > > > > impact. I'll provides a script to setup the test case and run it,= it's > > > > > more complex to setup than fio since involving setting up multipl= e > > > > > replicas and auth and hundreds of GB of test fixtures, I'm curren= tly > > > > > occupied by some other tasks but will try best to send them out a= s > > > > > soon as possible. > > > > > > > > Thanks! Apparently your RFC did show better IOPS with both access > > > > patterns, which was a surprise to me because it had higher refaults > > > > and usually higher refautls result in worse performance. > > > > > > > > So I'm still trying to figure out why it turned out the opposite. M= y > > > > current guess is that: > > > > 1. It had a very small but stable inactive LRU list, which was able= to > > > > fit into the L3 cache entirely. > > > > 2. It counted few folios as workingset and therefore incurred less > > > > overhead from CONFIG_PSI and/or CONFIG_TASK_DELAY_ACCT. > > > > > > > > Did you save workingset_refault_file when you ran the test? If so, = can > > > > you check the difference between this series and your RFC? > > > > > > > > > It seems I was right about #1 above. After I scaled your test up by 2= 0x, > > > I saw my series performed ~5% faster with zipf and ~9% faster with ra= ndom > > > accesses. > > > > Hi Yu, > > > > Thank you so much for testing and sharing this result. > > > > I'm not sure about #1, the ramdisk size, access data, are far larger > > than L3 (16M on my CPU) even in down scaled test, and both random/zipf > > shows similar result. > > It's the LRU list not pages. IOW, the kernel data structure, not the > content in LRU pages. Does it make sense? FYI. Willy just reminded me that he explained it a lot better than I did: https://lore.kernel.org/linux-mm/ZTc7SHQ4RbPkD3eZ@casper.infradead.org= / > > > IOW, I made rd_size from 16GB to 320GB, memory.max from 4GB to 80GB, > > > --numjobs from 12 to 60 and --size from 1GB to 4GB. > > Would you be able to try a larger configuration like above instead?