From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A87AC4167D for ; Mon, 11 Dec 2023 22:07:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 037EA6B0246; Mon, 11 Dec 2023 17:07:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EDBC16B0248; Mon, 11 Dec 2023 17:07:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D57256B0249; Mon, 11 Dec 2023 17:07:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BFEBD6B0246 for ; Mon, 11 Dec 2023 17:07:29 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 97EC8C07A5 for ; Mon, 11 Dec 2023 22:07:29 +0000 (UTC) X-FDA: 81555924618.24.8B0058A Received: from mail-qt1-f173.google.com (mail-qt1-f173.google.com [209.85.160.173]) by imf10.hostedemail.com (Postfix) with ESMTP id D1C57C0012 for ; Mon, 11 Dec 2023 22:07:26 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="xK5zC/7J"; spf=pass (imf10.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.173 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702332446; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rEQXjkg8ZdEhUtl2x53QcNzk2aEcDFh+4Uvu9gMAPOM=; b=3HXfnekPk2HVPp+YW4m5zcUbOzllgwqR+lm5U36Lt8ZgcSH11Abltq1QMRpTStufEeosYQ X4qLtd2jNy9H3yX1vw8W76+DcpLNX+WpOBcN7iWg2hVtOPDzgjX6IAXAdeGYNVYvSZpckl 7qqdfjNINLs1zgqCfLfPUIhrzCQBhUc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702332446; a=rsa-sha256; cv=none; b=qbS50lgCdZsLSmNWL/5sqZGDp/9tYgWZVevhPkZliWW/vyQdTyT+VSVtdxAxXQ0UmuIBpK QqMaWD9tL0gjLb1XG6NgJwaTlajJjG9ZIGeUIaBspJWPg1nZSVKCK3v6iFGcUsHy+up7c2 pWWkzEnZ9a8i4znJZpPKTIEXqHxrQIo= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="xK5zC/7J"; spf=pass (imf10.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.173 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-qt1-f173.google.com with SMTP id d75a77b69052e-425c1d7d72eso36191cf.1 for ; Mon, 11 Dec 2023 14:07:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702332446; x=1702937246; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=rEQXjkg8ZdEhUtl2x53QcNzk2aEcDFh+4Uvu9gMAPOM=; b=xK5zC/7JKYz64B5HifWGp5uQoXSFmdlBrBu45A2q1jP24RlRy6pu7b2zWJtxy+JFbU QViDi5+2vybwK90W/+AFEqlRVPVJIirPndH6gxRZ10J781+INsIiSlF9uBuNSTa1wOAQ UcFtDqjTzBXeLan23T/OU+StqFtY3qtDDtl63ewC5VRIXwjIpMK7F/MHnkKquOoQj8+S c3TBEYoj2c2uruzw2hJ7YZi6CAcoRoKOnsBazhvk2nMEcd88IGM2IkO/q7+JaulsLvfQ bFDPGKdjS6dbRsrBp1LNQRwOR9+BCapyLIlp+l8IyVaarN7KJsp1tloD6lo7YkaIbeXj V8SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702332446; x=1702937246; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rEQXjkg8ZdEhUtl2x53QcNzk2aEcDFh+4Uvu9gMAPOM=; b=WaUPT4aGxGO3+8I/zAME73f2PRYRbzSbRW9Hg4URFB4gylA94dyJSZyEefXD/4gE00 t3ckjvpr9iAD0pVYM6yy0KD8fsXOawPAr10ZaALkCGIyN4iJlXT65DQrNyN1j5PqFK0h lde6E7kF1Mib5RWEUdJBBfexMDkVG+PdxDZxnPJi67XFOsKg1zkmEFBXK2wJ454ECFJN 0NkHToWtwEiiKxHi2UTeUOMtkIEeMNL0EVj6BzQI4YePwANcm99npkjRTZrQTcB9WOGl YCvf9PiNLwhf7kBWTsmN67sb1m56eMmqaK2ZHP89JALc9T7DRsTJBmwYlujnFn4clcao vBXg== X-Gm-Message-State: AOJu0Yz+gKT3ddIKZbdBr3sTiTKBGjvJKBUmO8YKw4ZiADPh7zFNgYJz XKmebGr1Vl8YW4ZzuQA37gLbHbF2+R6ZkKpO785Ehw== X-Google-Smtp-Source: AGHT+IH+/dyc/hZNX51k+t0PKYhPnetIx3LhhZb7gJ9YizNg8qLu/nCEqnbEKRgxA7byr+po3RGzCwYOrfHmvYMDq5M= X-Received: by 2002:ac8:5992:0:b0:421:c331:7df4 with SMTP id e18-20020ac85992000000b00421c3317df4mr827815qte.18.1702332445538; Mon, 11 Dec 2023 14:07:25 -0800 (PST) MIME-Version: 1.0 References: <20231208061407.2125867-1-yuzhao@google.com> In-Reply-To: From: Yu Zhao Date: Mon, 11 Dec 2023 15:06:48 -0700 Message-ID: Subject: Re: [PATCH mm-unstable v1 1/4] mm/mglru: fix underprotected page cache To: Kairui Song Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Charan Teja Kalla , Kalesh Singh , stable@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: D1C57C0012 X-Rspam-User: X-Stat-Signature: r6dsdx4z4xz3k6orkna7fuyyppue8tnt X-Rspamd-Server: rspam03 X-HE-Tag: 1702332446-421457 X-HE-Meta: U2FsdGVkX1+Ao4K8Lgmct8YP/HQzJ3nljO6jfzpuBeOCXXK6ApiDj9iltJH7zooGFpIp8/hZjKIRNWmOi2YRlRsK7JG1qzxOKqbeA2hCWwuY+C0gh8Z8ot3aU4WoJGPDxhA8eMrKk7tGzNlVNYRsJD4XxY9z0r8mocWVlypzlwcY1vU+PFUfscNyk99foXRlUoX/SeJbIV68wPxFfTNTZ+MYiyy3XJRx1FlIQwOLx8v6IjbGG6f+LzqTGrvUE7xT0Ywhmq4CHqCO74RtPHRr9bZkYAqonM49JZ6zvq9XA9qzM0rWH9GCUabwaKPknlHET79Rj6ogYngV6aWs/mfjU5z1t78GR/D1gT09PIjgBNPwBmXD7vk0iPIeZw9VaMXJS++UL+w30kUbdxILD5dPA8qINs/+eqSs9swVLpvbamLCpVw5rE/clW4ay9saWON32yp8cnpemTHDybCLaEFEVRfayl8ysyhQomAXKa7dL2M5INOKrmnadyJjpLTbzAj31N0fRHUmd2YOKCT9xeaUFeO3ZNFtsz1Sv1oaN8Bofzo0Oi/r1RX8IkM+R4m0OGQr5NsYStx2uOzWY91tQRKRQ2ANMZQQP9fcgquek0WtDUahxzpJBLIXwIi0JUTgW/vD4EB/mN8V8DMUperPd3eXt2UnhCCkl/qWBNAWlgbqjk9yU6sgviu4RIaIHZbB3nj11kHuirqCdRcOm+idfaIu32NyPN8zkeAhaAbEb2m8xvR7jXHvUgmkWsyRF3dbqqP39pCVJTPZoJiFnSnkaA1Yxs7bCyYJ8fHl5ubYk2pP57wccTTjVFC74cN38BAcGTe4nyshqwumS3TDOOztW1YzYveKwv6YymM20NZiguMhsInXkVcOzOxMkVcWgNLOSO3YECOe1GPCU63Mo+kvOSCXwJv1ezP6XRkL5xBp0+gBh3Zjv5B1ncHbfxy7n5ZXbO+Y07FQLUfIm3J4tSL18rg 4Z8zZln0 a/CxPWZJmi40vSl1vNPkiGpvj+7lQnpP0OtuwhunY6sWi8PVHmWWqBc7enQhya+kkCJYFT6ZCgWsKOMEdBTccSN2gVV7ZljP+GBqNTbpRK8urY66bwixMiguW7ru0I47Ga5ctsWeg9T2rdq92ByOZeesAgsKPCVAl86sdoVv7BVJlO8VLnV7g0ThX81GQ8RbZdOUgrkFO/HO5XVdp21O4/Lj7CmHY47vjztCWoQJmIqXJ7gBA+uNH/dsMiepfB9TkIGLXL8rhM6CDufDi/yy63igKGUfUTXUh2DsjxhzICYbB3QRdaQI81Ki4xGsOgIup/xpYafWuJyNVNF0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Dec 8, 2023 at 1:24=E2=80=AFAM Kairui Song wrote= : > > Yu Zhao =E4=BA=8E2023=E5=B9=B412=E6=9C=888=E6=97=A5= =E5=91=A8=E4=BA=94 14:14=E5=86=99=E9=81=93=EF=BC=9A > > > > Unmapped folios accessed through file descriptors can be > > underprotected. Those folios are added to the oldest generation based > > on: > > 1. The fact that they are less costly to reclaim (no need to walk the > > rmap and flush the TLB) and have less impact on performance (don't > > cause major PFs and can be non-blocking if needed again). > > 2. The observation that they are likely to be single-use. E.g., for > > client use cases like Android, its apps parse configuration files > > and store the data in heap (anon); for server use cases like MySQL, > > it reads from InnoDB files and holds the cached data for tables in > > buffer pools (anon). > > > > However, the oldest generation can be very short lived, and if so, it > > doesn't provide the PID controller with enough time to respond to a > > surge of refaults. (Note that the PID controller uses weighted > > refaults and those from evicted generations only take a half of the > > whole weight.) In other words, for a short lived generation, the > > moving average smooths out the spike quickly. > > > > To fix the problem: > > 1. For folios that are already on LRU, if they can be beyond the > > tracking range of tiers, i.e., five accesses through file > > descriptors, move them to the second oldest generation to give them > > more time to age. (Note that tiers are used by the PID controller > > to statistically determine whether folios accessed multiple times > > through file descriptors are worth protecting.) > > 2. When adding unmapped folios to LRU, adjust the placement of them so > > that they are not too close to the tail. The effect of this is > > similar to the above. > > > > On Android, launching 55 apps sequentially: > > Before After Change > > workingset_refault_anon 25641024 25598972 0% > > workingset_refault_file 115016834 106178438 -8% > > Hi Yu, > > Thanks you for your amazing works on MGLRU. > > I believe this is the similar issue I was trying to resolve previously: > https://lwn.net/Articles/945266/ > The idea is to use refault distance to decide if the page should be > place in oldest generation or some other gen, which per my test, > worked very well, and we have been using refault distance for MGLRU in > multiple workloads. > > There are a few issues left in my previous RFC series, like anon pages > in MGLRU shouldn't be considered, I wanted to collect feedback or test > cases, but unfortunately it seems didn't get too much attention > upstream. > > I think both this patch and my previous series are for solving the > file pages underpertected issue, and I did a quick test using this > series, for mongodb test, refault distance seems still a better > solution (I'm not saying these two optimization are mutually exclusive > though, just they do have some conflicts in implementation and solving > similar problem): > > Previous result: > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Execution Results after 905 seconds > ------------------------------------------------------------------ > Executed Time (=C2=B5s) Rate > STOCK_LEVEL 2542 27121571486.2 0.09 txn/s > ------------------------------------------------------------------ > TOTAL 2542 27121571486.2 0.09 txn/s > > This patch: > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Execution Results after 900 seconds > ------------------------------------------------------------------ > Executed Time (=C2=B5s) Rate > STOCK_LEVEL 1594 27061522574.4 0.06 txn/s > ------------------------------------------------------------------ > TOTAL 1594 27061522574.4 0.06 txn/s > > Unpatched version is always around ~500. Thanks for the test results! > I think there are a few points here: > - Refault distance make use of page shadow so it can better > distinguish evicted pages of different access pattern (re-access > distance). > - Throttled refault distance can help hold part of workingset when > memory is too small to hold the whole workingset. > > So maybe part of this patch and the bits of previous series can be > combined to work better on this issue, how do you think? I'll try to find some time this week to look at your RFC. It'd be a lot easier for me if you could share 1. your latest tree, preferably based on the mainline, and 2. your VM image containing the above test.