From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B810AC3DA4A for ; Sat, 3 Aug 2024 17:09:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC46F6B0088; Sat, 3 Aug 2024 13:09:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C74516B008A; Sat, 3 Aug 2024 13:09:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B144C6B0092; Sat, 3 Aug 2024 13:09:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 920146B0088 for ; Sat, 3 Aug 2024 13:09:40 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 232D014175F for ; Sat, 3 Aug 2024 17:09:40 +0000 (UTC) X-FDA: 82411570920.19.6B15BDB Received: from mail-qt1-f181.google.com (mail-qt1-f181.google.com [209.85.160.181]) by imf20.hostedemail.com (Postfix) with ESMTP id 4AA1C1C0006 for ; Sat, 3 Aug 2024 17:09:38 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="d/S1C9yJ"; spf=pass (imf20.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.181 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722704971; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uJmgT6oTpLrEgwNYBG6iDU6/PwiyyeFSzdFO7EuJwL4=; b=CtSurIEeHZd5Of3PZOhgPwh1+9kIZWG4E7HZ15+h47BTKMwbBEKq+zE1Dt9Auia+ZwP1Iz TllF43e2jd1l3cN8MY1Io8Qs22iZOrl4eRrJXXoXH4UVDdkIGSG66yVIK3cUVkJMcpi96S GgcI0x9AtzxI7WZzD8qTwWscOMeX1L0= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="d/S1C9yJ"; spf=pass (imf20.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.181 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722704971; a=rsa-sha256; cv=none; b=n1ggF2yhKS2sFXXhsbT2YUzRSvdZ1BvUyHSYRQoZO8lGef8GK/A+kGWYwDb0rlM0Ne/Yf9 Sey6uwuaC5K3DL0Ec08mBBny6kU6cuFqK5dbq3IQUa2G5gt/DYVDBcDjM7k0+FjWRFinAD 51KX+3oc2Rt1oSBIj9hONpQ8EMSBv/4= Received: by mail-qt1-f181.google.com with SMTP id d75a77b69052e-4518d9fa2f4so459571cf.0 for ; Sat, 03 Aug 2024 10:09:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1722704977; x=1723309777; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=uJmgT6oTpLrEgwNYBG6iDU6/PwiyyeFSzdFO7EuJwL4=; b=d/S1C9yJvY3m7VNtjjCh/gQQDapvNzfbj3KODNdY+dYt6MSEViMDcVQ8cu3HB9K6rw PiaVIlWBCuhbGQh7nk568ygUdPk9RJMkGQjyX5mG4cwI6KeSfSv6L3CPFgrXR4b/7edk V6C+cbuGICsrnsuboVA9OqLLwNTwiMxhT8xyv2XmyPM6Ai1M/M0vEvTY3BQgcDVyvwN7 gMnND6g7dj9bowqYNpwn50jHBZzcYrx4dYfAIQaOZVWsrEaNw9s9aWERLUJ9roaGeSH0 k8/lVI2bDcJcxS+CmbpAF9/Rwh3aoS1Ks4uYPzTClhM4OT0JBb6e976WR0Np1HemNNTu ZNgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722704977; x=1723309777; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uJmgT6oTpLrEgwNYBG6iDU6/PwiyyeFSzdFO7EuJwL4=; b=Y5bibo81maeS05aWHwF6fIjhCxS4VxCMVIN3v44mkfnBE8ZrL5Fg/QT4+yPt+o4PWB 1igay/DAJCB3aHKWqgHTbIcyoQYH4TOXKtOYTkAvASQEZTYY1qsJ4hdlap1saTdcoO/X kCjmL2WxoCiElSgtlo0UP6Z8C2HyWV+FhTVyT+Ltee/IKBOd1pDyvDf+Ceyh/WRrK77P PnFbg1jibajDsWVf/7pC1N/xawqL9kK1CKN+c1j8YMbSAVjx0SRo6HoXs0G0aJAsV+Zm Xn3mc42fhD9Bz/EosTEIiKTovuKX0nZKg4biH9JaQ2YsbuiRVXn4b2CgMu99m4+LkTVP Ct5A== X-Forwarded-Encrypted: i=1; AJvYcCVc9sLWH3L4CIMHJuXZQUmgzRkYeFpzCZb1GCEnsDIBzzXPIl0OLI5e1ZvRzhsOJCa4K0kkvU/+Fj7zcmeFn+1CBiI= X-Gm-Message-State: AOJu0YyDrr3E4bDRklJJ6/wiY9S9Pj/udOXsVXi8kEgPQCfON5YNkw4X nJ4cIskb/Cw7FOqRiMze7hOidwD9Me/NME6QLWAxBK0YrwyOFNXiTjn7cAx798q++t7mGkwmiLg OYAFIdlEAYNhxnizH0OYHD4y9Dmtkzts+WMbe X-Google-Smtp-Source: AGHT+IF65CauaimP3NlXguEo+6ib8VGPl94a1Rq/hA9I/V2MtuTtcR15tRDCJOV1vVIreODcBjNKh0YrXFu5xHR3u2U= X-Received: by 2002:a05:622a:591:b0:44f:ea7a:2119 with SMTP id d75a77b69052e-4519ae21848mr1632181cf.18.1722704977148; Sat, 03 Aug 2024 10:09:37 -0700 (PDT) MIME-Version: 1.0 References: <1719038884-1903-1-git-send-email-yangge1116@126.com> <0f9f7a2e-23c3-43fe-b5c1-dab3a7b31c2d@126.com> <00a27e2b-0fc2-4980-bc4e-b383f15d3ad9@126.com> In-Reply-To: <00a27e2b-0fc2-4980-bc4e-b383f15d3ad9@126.com> From: Yu Zhao Date: Sat, 3 Aug 2024 11:08:59 -0600 Message-ID: Subject: Re: [PATCH V2] mm/gup: Clear the LRU flag of a page before adding to LRU batch To: Ge Yang Cc: Chris Li , Andrew Morton , linux-mm , LKML , stable@vger.kernel.org, Barry Song <21cnbao@gmail.com>, David Hildenbrand , baolin.wang@linux.alibaba.com, liuzixing@hygon.cn, Hugh Dickins Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Stat-Signature: 1y5ytgchhpbs9oifatfkbz3x6ixrgdh3 X-Rspamd-Queue-Id: 4AA1C1C0006 X-Rspamd-Server: rspam11 X-HE-Tag: 1722704978-267056 X-HE-Meta: U2FsdGVkX18dwYcA7lAtj/Q2315TCID/4yR13L27GX2tp+xVaZRvXWnReRnpXkcKRD6mTg+Bh71qJ9eAsD4Lg8sDLiHlqq9GFztq7tPpC5iWHcfYcORG9ea5cBjOzdCnFdcZ5onMfPNw9ZEBsV9wndw/aiosQc+VWgiYuSMexP3CJUWDVo7olSBV3rfKFvI9kQEOTInHxiTeRYLyEk/HrlqIzaFhX6V2R4mjKi7k+Iapka37hCQyO9zEhOFTsGDit1jqQqnEP4wpso5+VVv21YsQuwOhsh402cvCRL06TA23TOlx4QEQJdoMqiN1AalBRbnCTupsOVDyaZRnA91h/PnwDkxPo0wH7tFDLqE3wy5JOJO5Z3JRuj3prtewRdNCqhG1uy+HECADy7KyHWYWXKoh/rMbwfX9wxwNtNrlqQ4plGwDuFQjhKxfQdB2WaO/IFB4ledqnQBZPOZd98ffaal2QIra5VByj7y45y2IueVlomzb44TghpLHhs/E1VAiCTZOvbDtWv/ffwsojVWZmswbjtUpxkzvcf2GMSSKAlkIyM4rSCLnmYRYZNtzAqzXZk64N5lW/NAhcDH1zwphrtSFIoHKlkjeeE0Sq2jDBd92qhBXRdZKT4phJqWWCts7r3ocRqJiWPKyUwT6w4zAcfXYA5v94A3W1mKKELxIICXM843g/BE8rQW774Q7tnVhXnkl4Izl4lQhKH8ZgA3OFcBfdAVwnyS9Rshcg/NyEmNQCXi65fsP9k8qaJxt9yDJSrew4Tpz5kpzG1kuSSz8Pye2l55M3t1qvFOryNyBAevpi4n4KkNiRQq6y50U+QQAG8QRVfaHweeszsSeZ5ZoWqSwfzglkWzTNoAM1RO02fwh968ayRrRep/oo62BxhY2VAKtYsGCdZXumnBXt73AunQjxLhbh0pQATkRJOktublbGOleRsfPC+4x9NXk9HzSHuHqYkDH+RLNx7IR5/D ouqD9vy7 zWH2B8503pjn82fF4gpzsfm+MuRGiHLUR+XC9Zqk0jS2LHIIFKJX8RQy6+Sd+mM21PvoeW/tWJWXYw6YJuQ395+hTP9he2zhuwb+CmMWJZI/acW89e0+Wq/u1P2T/vUt9Gxr/homSQnbHgkavvLWf02EEl9lRSkSf7dk0jk1XMafGaM+dQ7HnKaETSCf+SL6dqmi9F29s0JQTQCLb26lXbVPd+M/ROq8sm2s5XzXTyUxYLu/ML9UOsYG9aUuiWg0cfPe2ZUaaXKU1qMRfcVpPTTUS+DYTAPPDU2AcoF/PmWtykH5IWQkVvki/nQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000103, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, Aug 3, 2024 at 2:31=E2=80=AFAM Ge Yang wrote: > > > > =E5=9C=A8 2024/8/3 4:18, Chris Li =E5=86=99=E9=81=93: > > On Thu, Aug 1, 2024 at 6:56=E2=80=AFPM Ge Yang wro= te: > >> > >> > >> > >>>> I can't reproduce this problem, using tmpfs to compile linux. > >>>> Seems you limit the memory size used to compile linux, which leads t= o > >>>> OOM. May I ask why the memory size is limited to 481280kB? Do I also > >>>> need to limit the memory size to 481280kB to test? > >>> > >>> Yes, you need to limit the cgroup memory size to force the swap > >>> action. I am using memory.max =3D 470M. > >>> > >>> I believe other values e.g. 800M can trigger it as well. The reason t= o > >>> limit the memory to cause the swap action. > >>> The goal is to intentionally overwhelm the memory load and let the > >>> swap system do its job. The 470M is chosen to cause a lot of swap > >>> action but not too high to cause OOM kills in normal kernels. > >>> In another word, high enough swap pressure but not too high to bust > >>> into OOM kill. e.g. I verify that, with your patch reverted, the > >>> mm-stable kernel can sustain this level of swap pressure (470M) > >>> without OOM kill. > >>> > >>> I borrowed the 470M magic value from Hugh and verified it works with > >>> my test system. Huge has a similar swab test up which is more > >>> complicated than mine. It is the inspiration of my swap stress test > >>> setup. > >>> > >>> FYI, I am using "make -j32" on a machine with 12 cores (24 > >>> hyperthreading). My typical swap usage is about 3-5G. I set my > >>> swapfile size to about 20G. > >>> I am using zram or ssd as the swap backend. Hope that helps you > >>> reproduce the problem. > >>> > >> Hi Chris, > >> > >> I try to construct the experiment according to your suggestions above. > > > > Hi Ge, > > > > Sorry to hear that you were not able to reproduce it. > > > >> High swap pressure can be triggered, but OOM can't be reproduced. The > >> specific steps are as follows: > >> root@ubuntu-server-2204:/home/yangge# cp workspace/linux/ /dev/shm/ -r= f > > > > I use a slightly different way to setup the tmpfs: > > > > Here is section of my script: > > > > if ! [ -d $tmpdir ]; then > > sudo mkdir -p $tmpdir > > sudo mount -t tmpfs -o size=3D100% nodev $tmpdir > > fi > > > > sudo mkdir -p $cgroup > > sudo sh -c "echo $mem > $cgroup/memory.max" || echo setup > > memory.max error > > sudo sh -c "echo 1 > $cgroup/memory.oom.group" || echo setup > > oom.group error > > > > Per run: > > > > # $workdir is under $tmpdir > > sudo rm -rf $workdir > > mkdir -p $workdir > > cd $workdir > > echo "Extracting linux tree" > > XZ_OPT=3D'-T0 -9 =E2=80=93memory=3D75%' tar xJf $linux_src || = die "xz > > extract failed" > > > > sudo sh -c "echo $BASHPID > $cgroup/cgroup.procs" > > echo "Cleaning linux tree, setup defconfig" > > cd $workdir/linux > > make -j$NR_TASK clean > > make defconfig > /dev/null > > echo Kernel compile run $i > > /usr/bin/time -a -o $log make --silent -j$NR_TASK || die "mak= e failed" > > > > > Thanks. > > >> root@ubuntu-server-2204:/home/yangge# sync > >> root@ubuntu-server-2204:/home/yangge# echo 3 > /proc/sys/vm/drop_cache= s > >> root@ubuntu-server-2204:/home/yangge# cd /sys/fs/cgroup/ > >> root@ubuntu-server-2204:/sys/fs/cgroup/# mkdir kernel-build > >> root@ubuntu-server-2204:/sys/fs/cgroup/# cd kernel-build > >> root@ubuntu-server-2204:/sys/fs/cgroup/kernel-build# echo 470M > memor= y.max > >> root@ubuntu-server-2204:/sys/fs/cgroup/kernel-build# echo $$ > cgroup.= procs > >> root@ubuntu-server-2204:/sys/fs/cgroup/kernel-build# cd /dev/shm/linux= / > >> root@ubuntu-server-2204:/dev/shm/linux# make clean && make -j24 > > > > I am using make -j 32. > > > > Your step should work. > > > > Did you enable MGLRU in your .config file? Mine did. I attached my > > config file here. > > > > The above test didn't enable MGLRU. > > When MGLRU is enabled, I can reproduce OOM very soon. The cause of > triggering OOM is being analyzed. I think this is one of the potential side effects -- Huge mentioned earlier about isolate_lru_folios(): https://lore.kernel.org/linux-mm/503f0df7-91e8-07c1-c4a6-124cad9e65e7@googl= e.com/ Try this: diff --git a/mm/vmscan.c b/mm/vmscan.c index cfa839284b92..778bf5b7ef97 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4320,7 +4320,7 @@ static bool sort_folio(struct lruvec *lruvec, struct folio *folio, struct scan_c } /* ineligible */ - if (zone > sc->reclaim_idx || skip_cma(folio, sc)) { + if (!folio_test_lru(folio) || zone > sc->reclaim_idx || skip_cma(folio, sc)) { gen =3D folio_inc_gen(lruvec, folio, false); list_move_tail(&folio->lru, &lrugen->folios[gen][type][zone= ]); return true; > >> Please help to see which step does not meet your requirements. > > > > How many cores does your server have? I assume your RAM should be > > plenty on that server. > > > > My server has 64 cores (128 hyperthreading) and 160G of RAM.