From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7129CC4167B for ; Sat, 2 Dec 2023 04:44:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F0D976B04F3; Fri, 1 Dec 2023 23:44:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EBD116B04F4; Fri, 1 Dec 2023 23:44:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D84626B04F5; Fri, 1 Dec 2023 23:44:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C849C6B04F3 for ; Fri, 1 Dec 2023 23:44:27 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A1792A0939 for ; Sat, 2 Dec 2023 04:44:27 +0000 (UTC) X-FDA: 81520636974.03.7E706F2 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf30.hostedemail.com (Postfix) with ESMTP id C98268000E for ; Sat, 2 Dec 2023 04:44:25 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=cdJDNDrQ; spf=pass (imf30.hostedemail.com: domain of bagasdotme@gmail.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=bagasdotme@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701492265; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mkjJumzQC9OLzeUlKf2cdrps8W5WGvu5yds3ov88Mec=; b=5sRK84y4BfXEThNTT/+IhKDBBq8Zzz22J64OCLPkzCO+7Dfs8zwHuE7BhPvD705o6uR+x/ N5S6+e4ejhv1prGItGLZoOnlk4yO+Yf0xhPekVob7iQYeNDfLmRsMz0jT9AVBxMuP0EAPd aLJ1IPqfuAdu4Ygiy2fsWfxtueYAIf8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701492265; a=rsa-sha256; cv=none; b=t/UO0UdOMMnPMESTlN4R+hA2t6WU93xP+Ym9O0M7uo9ZEMTKvWrkrQKBasNR5oZWTjp0gW SMiAyoVQxTxg9o/stPTDftJVpVLvrCypM14HAtoqytlbaieY9cEw8DywgtVAFIIKkbUIrF DU0twYmcvxHdOrUI8A45vVKQNWfd+R0= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=cdJDNDrQ; spf=pass (imf30.hostedemail.com: domain of bagasdotme@gmail.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=bagasdotme@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1d06d42a58aso4638655ad.0 for ; Fri, 01 Dec 2023 20:44:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701492264; x=1702097064; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=mkjJumzQC9OLzeUlKf2cdrps8W5WGvu5yds3ov88Mec=; b=cdJDNDrQyQ7z2QSjWiLVz3LIFNM0Ixv4+Ir2dOcTtIPCAk9j7hlpwumMQUsXaJEhg7 skDY6EnNIb3Ak/5Jjr2BGiyvI/tl4AX97/aTuoP1Sly55L+dh0iXyAGrr/HVQXOIAIXJ KyHaycsi8/sYLbLXptiZcj6qNYGm4V9yU1Iem5BN0dz7rxC8Hai9Idxjm1pAcSVePcsd 4gXvwYOW/yZXLG7IWmHv+XtgM/a7qLUpi4q8CnZUdgrn20/84c+Ji2bVNKFkqjWJWrJ0 twhU38sVfDc7E7VnaZJnsULIVl7JPx3NHvZE6oT9NxP9LwNb/8vfzAAinqKlI+cweSoT SaHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701492264; x=1702097064; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=mkjJumzQC9OLzeUlKf2cdrps8W5WGvu5yds3ov88Mec=; b=eIOMIKvZ9wvBmIYtbECfkYoU7sFktfGltnWImRFt19TJtmYfHfN915AiQAkM4SbQgz Er1wU8LElwhnpzeNzXXJQ3qS7jl59eqTt2D8KZgOxlMe8t75pI7/4NHWSAp88ZXw3lcT mXDeJ9URljG+FHTYaT4ak702JqC7zW/UYuQlAHEJ/4dm42Ku5k1hUmlNkK+S5Al3akYv BB30PraaS8jcBrsb7G5pnVy/2lvlZcqf7kxS0SO5KcFB5lb69OUs1A80/CD/2HIsrhDQ fdiWZvZ1a3Ol2O8jP95s1c2ekFkEhy8AHJoxzojouDQyNMWWDUTi1aGRaGNut9mqcnxq ijgg== X-Gm-Message-State: AOJu0Yyn76djZePBviWBnT+tIZDkEK+wM+cKjGbdaPijpjiORzRRM2/K C7a7MwYig9+k9c08dYIfkCk= X-Google-Smtp-Source: AGHT+IG+AyARMGDcYbSUtUR0Ml9fa0udGX+8XMSUKKs+Zr4+J4uBha8QS1idDiF0qZFBJrJwksUIvw== X-Received: by 2002:a17:902:dacb:b0:1d0:68a:4a46 with SMTP id q11-20020a170902dacb00b001d0068a4a46mr798452plx.3.1701492264465; Fri, 01 Dec 2023 20:44:24 -0800 (PST) Received: from archie.me ([103.131.18.64]) by smtp.gmail.com with ESMTPSA id p14-20020a1709028a8e00b001cf6453b237sm4173552plo.236.2023.12.01.20.44.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Dec 2023 20:44:23 -0800 (PST) Received: by archie.me (Postfix, from userid 1000) id B89B9110DB381; Sat, 2 Dec 2023 11:44:18 +0700 (WIB) Date: Sat, 2 Dec 2023 11:44:18 +0700 From: Bagas Sanjaya To: Nhat Pham , akpm@linux-foundation.org Cc: hannes@cmpxchg.org, cerasuolodomenico@gmail.com, yosryahmed@google.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, chrisl@kernel.org, linux-mm@kvack.org, kernel-team@meta.com, Linux Kernel Mailing List , Linux CGroups , Linux Documentation , Linux Kernel Selftests , shuah@kernel.org Subject: Re: [PATCH v7 0/6] workload-specific and memory pressure-driven zswap writeback Message-ID: References: <20231127234600.2971029-1-nphamcs@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="L3dnvOdQcggt5SPe" Content-Disposition: inline In-Reply-To: <20231127234600.2971029-1-nphamcs@gmail.com> X-Stat-Signature: fumjf5f8f5b717xik5tw8b97znchd519 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: C98268000E X-Rspam-User: X-HE-Tag: 1701492265-246845 X-HE-Meta: U2FsdGVkX1/b2XGoZMK43semQdgz79He5FAwAITDgaysFBkxMK0CDC8lfCeahqXhHtkQRsZynI+ZeEOc/IGalkHoJ2df+KWEqpZz5P+mAo2NkVtrKOPWjgZ5krD9uYIC9GC/RJR0DT2hcL9S/3hALfWWJuAONj5vP1hiPptDLXvbfft5x8qNtkA1NFBnP6Nuh1smGajvG/qDFJ/BK3929u6YoyKctTON4i+bm2FBs7vtBwdHFDfetjFdNZrExueHgtnESAv2FD+Vc3H+K8O0yqGpqfFK5EEYjy78aOPvO1b7OSGSs+YEYaam7uV6ByNt3ajUQY47eTJqkoJ2oIcXec6s3qj0o7PzOvZ7/cck239RO0fjGg2BEtMYbXiCDZouxlr7gMmjCqTnhHSx5gqfNSfCckdXiz5wRFFtxM1YolvT3fZUHyVbLIsUp7JkxrE5Cfufx0X1r/aOWHXJd08l22TBZSaQghougw5BqFu9D03vgSy8JAdzLuij4KQ7aiwQerTpyVLrLCnvabH/OsFYTi04Bcd9V8k1tbgjOsNZEpwJWp6TpSxpSY+r8ycQhmxanBjVZXaoSYD0ygF3AfWVVVHj3Eva2NpjZuY/Q1FBAvbnzUHqDFjXWiH3HjnKYcJ6hJTlVkQ5lMsjcxkOcrhj6WCY3K03KShaAjAqKEX7Nw1Ba79T9EyCbAPHSiE41srWH9vfg7rEtIFek6ddu1jFjPTUJcgCnrhlkN20cPSzqrW+m6qzhGj/wqrj5EWbAJw/L/fN5Wb/NS1HX1Yr/O85dR3+vo2S9gK8zpbn+0fuGdp13+DVIVEepq4I1PcG5yxae830hrSEMcXHThxn6FDPO3xID0FzCTazWyg9Eao3T3b+GJ51eqeB2tBZkDdzxgTbiYpbqi0zY40138LgVpMnLWaCsfsso/L/qRGyAlNowlG5B/VMrr2eqw+DEwiYD7v50nPkPBX4GAgguQce85k cqZSGdCh 3jpI5je72IzfH0v35+rUKFwi6Y073vcbd9ydibzA8rxd0fce4+bsa6KLC9B9cauSBrzM4/WwzS8CG2zcp+IA3WlYTqsYGFDqem8CGMfwd+XVKm+rno/WnCwmIfWv5G+H6mwfVORMX5s6UQIYSUXlrF/CgF9r2Hs8zlyg369PAPjOIyVJteHfBEnlIDwRTSTriyZNcldQGHqv+Xo7H1NPs/QnyvWYhk5IJGCUWeIgrft2DlzUnus9delBS6a5lTWLFVHvBPpZKjCNoaxEzuFCxOLF8CZ20u80j0ifuafDw8/AF8V8qEMg3FYHSqiekkewmOFVPVOcgstUUI3EbYDJQNDu3EPQkLP4C4g94rS+nUcvxlEijho0s2C6OuI4jx8+d7npwS6ZPIMOATla25ZYWmJv93KwwfZlUaaJEcNGb6wYybTI4mYRkbDspnzI5wm9o2XzaC7iDpeuc+if3PPncxkFdFxdWaCVQt73mRqN7OSRLcH0yfwgdWuuDNhRtbkKBG0NWJaEdjTgcduXBuPdC/E7USAYcmhrhDzwUlTQzvZvLZ9hzqRWcNDtf4V30wdhwi0OpE4XOGogVjVwArmhunTYTUI00ayY86FBXXCNrDZ5SGzDl7R2oI09QczVZOjUqTGs7r4KwxroWrME= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: --L3dnvOdQcggt5SPe Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Nov 27, 2023 at 03:45:54PM -0800, Nhat Pham wrote: > Changelog: > v7: > * Added the mem_cgroup_iter_online() function to the API for the new > behavior (suggested by Andrew Morton) (patch 2) > * Fixed a missing list_lru_del -> list_lru_del_obj (patch 1) > v6: > * Rebase on top of latest mm-unstable. > * Fix/improve the in-code documentation of the new list_lru > manipulation functions (patch 1) > v5: > * Replace reference getting with an rcu_read_lock() section for > zswap lru modifications (suggested by Yosry) > * Add a new prep patch that allows mem_cgroup_iter() to return > online cgroup. > * Add a callback that updates pool->next_shrink when the cgroup is > offlined (suggested by Yosry Ahmed, Johannes Weiner) > v4: > * Rename list_lru_add to list_lru_add_obj and __list_lru_add to > list_lru_add (patch 1) (suggested by Johannes Weiner and > Yosry Ahmed) > * Some cleanups on the memcg aware LRU patch (patch 2) > (suggested by Yosry Ahmed) > * Use event interface for the new per-cgroup writeback counters. > (patch 3) (suggested by Yosry Ahmed) > * Abstract zswap's lruvec states and handling into=20 > zswap_lruvec_state (patch 5) (suggested by Yosry Ahmed) > v3: > * Add a patch to export per-cgroup zswap writeback counters > * Add a patch to update zswap's kselftest > * Separate the new list_lru functions into its own prep patch > * Do not start from the top of the hierarchy when encounter a memcg > that is not online for the global limit zswap writeback (patch 2) > (suggested by Yosry Ahmed) > * Do not remove the swap entry from list_lru in > __read_swapcache_async() (patch 2) (suggested by Yosry Ahmed) > * Removed a redundant zswap pool getting (patch 2) > (reported by Ryan Roberts) > * Use atomic for the nr_zswap_protected (instead of lruvec's lock) > (patch 5) (suggested by Yosry Ahmed) > * Remove the per-cgroup zswap shrinker knob (patch 5) > (suggested by Yosry Ahmed) > v2: > * Fix loongarch compiler errors > * Use pool stats instead of memcg stats when !CONFIG_MEMCG_KEM >=20 > There are currently several issues with zswap writeback: >=20 > 1. There is only a single global LRU for zswap, making it impossible to > perform worload-specific shrinking - an memcg under memory pressure > cannot determine which pages in the pool it owns, and often ends up > writing pages from other memcgs. This issue has been previously > observed in practice and mitigated by simply disabling > memcg-initiated shrinking: >=20 > https://lore.kernel.org/all/20230530232435.3097106-1-nphamcs@gmail.com= /T/#u >=20 > But this solution leaves a lot to be desired, as we still do not > have an avenue for an memcg to free up its own memory locked up in > the zswap pool. >=20 > 2. We only shrink the zswap pool when the user-defined limit is hit. > This means that if we set the limit too high, cold data that are > unlikely to be used again will reside in the pool, wasting precious > memory. It is hard to predict how much zswap space will be needed > ahead of time, as this depends on the workload (specifically, on > factors such as memory access patterns and compressibility of the > memory pages). >=20 > This patch series solves these issues by separating the global zswap > LRU into per-memcg and per-NUMA LRUs, and performs workload-specific > (i.e memcg- and NUMA-aware) zswap writeback under memory pressure. The > new shrinker does not have any parameter that must be tuned by the > user, and can be opted in or out on a per-memcg basis. >=20 > As a proof of concept, we ran the following synthetic benchmark: > build the linux kernel in a memory-limited cgroup, and allocate some > cold data in tmpfs to see if the shrinker could write them out and > improved the overall performance. Depending on the amount of cold data > generated, we observe from 14% to 35% reduction in kernel CPU time used > in the kernel builds. >=20 > Domenico Cerasuolo (3): > zswap: make shrinking memcg-aware > mm: memcg: add per-memcg zswap writeback stat > selftests: cgroup: update per-memcg zswap writeback selftest >=20 > Nhat Pham (3): > list_lru: allows explicit memcg and NUMA node selection > memcontrol: add a new function to traverse online-only memcg hierarchy > zswap: shrinks zswap pool based on memory pressure >=20 > Documentation/admin-guide/mm/zswap.rst | 7 + > drivers/android/binder_alloc.c | 7 +- > fs/dcache.c | 8 +- > fs/gfs2/quota.c | 6 +- > fs/inode.c | 4 +- > fs/nfs/nfs42xattr.c | 8 +- > fs/nfsd/filecache.c | 4 +- > fs/xfs/xfs_buf.c | 6 +- > fs/xfs/xfs_dquot.c | 2 +- > fs/xfs/xfs_qm.c | 2 +- > include/linux/list_lru.h | 54 ++- > include/linux/memcontrol.h | 18 + > include/linux/mmzone.h | 2 + > include/linux/vm_event_item.h | 1 + > include/linux/zswap.h | 27 +- > mm/list_lru.c | 48 ++- > mm/memcontrol.c | 32 +- > mm/mmzone.c | 1 + > mm/swap.h | 3 +- > mm/swap_state.c | 26 +- > mm/vmstat.c | 1 + > mm/workingset.c | 4 +- > mm/zswap.c | 426 +++++++++++++++++--- > tools/testing/selftests/cgroup/test_zswap.c | 74 ++-- > 24 files changed, 641 insertions(+), 130 deletions(-) >=20 >=20 > base-commit: 5cdba94229e58a39ca389ad99763af29e6b0c5a5 No regressions when booting kernel with series applied. Tested-by: Bagas Sanjaya --=20 An old man doll... just what I always wanted! - Clara --L3dnvOdQcggt5SPe Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYKAB0WIQSSYQ6Cy7oyFNCHrUH2uYlJVVFOowUCZWq2HwAKCRD2uYlJVVFO o+mhAP98o1RpDS79BK+Q11P7wQTxU2MYmimbjb5iykn5K7OINwEAx7hxtVV/yTcR X+TAaHXp3ua/dAycrhD7Qx+QUiyCRgw= =qSwI -----END PGP SIGNATURE----- --L3dnvOdQcggt5SPe--