From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87726C35274 for ; Mon, 18 Dec 2023 14:44:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 25D7E6B0087; Mon, 18 Dec 2023 09:44:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 20C876B0088; Mon, 18 Dec 2023 09:44:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D4056B0089; Mon, 18 Dec 2023 09:44:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id ED0026B0087 for ; Mon, 18 Dec 2023 09:44:40 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B4AD11C1053 for ; Mon, 18 Dec 2023 14:44:40 +0000 (UTC) X-FDA: 81580210320.25.5AAD82E Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by imf24.hostedemail.com (Postfix) with ESMTP id 7E551180003 for ; Mon, 18 Dec 2023 14:44:38 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=rjW6DaJq; spf=pass (imf24.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.128.54 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702910678; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WE6lIWDf/kPieQyyeHfgiHrbETC606qdiK73vUfwMAY=; b=ilX7AGQvSF0fT1YTwH3QwCVuHouzm61vWXVrJoeTKKwV4XU3S46JBUfnvopTSHrKejpdlI Ua4zwSrryLk01l7M7dFbZlGebAySjuHb2ZTlvPAJsWftI8JdeMRnPyj9OjZCcPbdu71jVa BuQuuYPwnN8jN/JmfuCciCQG5bBAgVk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702910678; a=rsa-sha256; cv=none; b=FiHZpAGIgt+cuHbPHrJpdA+m5VeIexYrkc4kWkoLObHgPlNsmquF65tHAuqwSo1x8+Gnn+ XSjeIb/gX9ZdZTeH7N2u6sPGN3Lqj8LNpitu57Bc/mbzwXW6omb/QMOBT3733hIspwelag aeRyTiGtiOVkh+t+mQgJ++XPw1Evef4= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=rjW6DaJq; spf=pass (imf24.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.128.54 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-40c38e292c8so16452245e9.0 for ; Mon, 18 Dec 2023 06:44:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1702910677; x=1703515477; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=WE6lIWDf/kPieQyyeHfgiHrbETC606qdiK73vUfwMAY=; b=rjW6DaJqy1rt4IJtkTyhrnPRDRTo8v86QHFFcbdTx5uIfAWOKYS08J+Fz63ljuE9tI q/G3hv7DL55BUF/ctgCsE07XRw045l6rNoPQY3SJKd6JS06CSDdto2mueuK+ZHDAPUjI Qd2ZJsfjkCx6Bckxh7UP5YTTCFBYCBCewi1dhtLdIoED9qTsJQgFYLjsZ1CQZJDbCt+v rnhlEs2bR852mdJDxr1NkMz/4+1xXYISwBEALm6xd9IBElFupAtYUIFtT2s3mbe8fQa6 iFPgSxrO/IWc1CtHNDWoRDVKI4syVX3gk/SusezHTSIbybHQlKXp0HB/X2mtDCx995vw aKfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702910677; x=1703515477; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WE6lIWDf/kPieQyyeHfgiHrbETC606qdiK73vUfwMAY=; b=oEWrITVzGqjUmrFyXbYmrp6u4YuuKwUxKetGDqnmhxJA6mqwfeLMq2QYohINKuiavP YizHmAnIWB2ulCd4KZjKOYJJPhOGx2YGeBw8tcBL78fm1RPs0mMUvMMJnENgpDYfIxkm HTOafeOkKTuWCk9WOcvgju3HOtpU0i7c/wKLz38Dr82j3vI0gdDB8qJSVVKppbKgroF0 H+proHbUMNLfMX0dCRKvJEPHdghKU69ZA6HX6O1E/A3nrmiIgrFo/sNn+6RG7TUXX3dA ccMWlCJBu6ITIKyHAdFgSug5Ls8mPp7fDveHJtmLGEGj+VhO1buT/eacYSnjFE0Xa8X9 riww== X-Gm-Message-State: AOJu0YyHha9mADTsqZ8d4MnFshPz9foHPnD6JspjW44oMkoDxDMsvdeY XjvPAtxyLGnO7u+nNdw0nJKsrA== X-Google-Smtp-Source: AGHT+IFCF0xM4rxtEjkGeNdb1pqxvZqkgbp76i4kF8Ih7/vn1FDsVKwbLgmryjhQY8w6tSGBJegf/w== X-Received: by 2002:a7b:cd8e:0:b0:40c:269c:518f with SMTP id y14-20020a7bcd8e000000b0040c269c518fmr8241906wmj.115.1702910676582; Mon, 18 Dec 2023 06:44:36 -0800 (PST) Received: from localhost ([2620:10d:c091:400::5:44d1]) by smtp.gmail.com with ESMTPSA id z14-20020a05600c0a0e00b0040b3867a297sm43245191wmp.36.2023.12.18.06.44.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Dec 2023 06:44:36 -0800 (PST) Date: Mon, 18 Dec 2023 15:44:31 +0100 From: Johannes Weiner To: Yosry Ahmed Cc: Nhat Pham , akpm@linux-foundation.org, tj@kernel.org, lizefan.x@bytedance.com, cerasuolodomenico@gmail.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, hughd@google.com, corbet@lwn.net, konrad.wilk@oracle.com, senozhatsky@chromium.org, rppt@kernel.org, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, david@ixit.cz, chrisl@kernel.org Subject: Re: [PATCH v6] zswap: memcontrol: implement zswap writeback disabling Message-ID: <20231218144431.GB19167@cmpxchg.org> References: <20231207192406.3809579-1-nphamcs@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: 7E551180003 X-Rspam-User: X-Stat-Signature: jfcsirmbik1gxsfuuxyi8dyg4seu45d8 X-Rspamd-Server: rspam03 X-HE-Tag: 1702910678-894958 X-HE-Meta: U2FsdGVkX1++tfSirWDrJwOzDB1kbkao+I/l7yuB4XtOLK+p74Bo/EnE6txjILnryk+yUG177ZzA32QtTCP86ohFoqVrOYqDlNeyadq+tryRHoCUwzMFq7OHiCUW4P7wP8hcjxPljmfXHOqp1q6ksWhIJxr8uWWqUCwlwEZevfGpexNoF/HrfyStCopxFs4NO+8k16qN6JQLDrscv4BEdme6iRpCR2YcirM7/0eXpNkak/KoXXkwIEWk4B/gxiNR5MHTgwtzWx6M+p6evnfms8LWsnZToFhaNMKnciC55jTbyIi4QdIc9yC4VZ4Bmu2/8mjxyFfyyZru5cHO71SSI4JEHbW+A/QX1Juh2f/1WFDV9TtMl1HIGpA9fHkHFtAqxX80Z4nb+nhgNYPL5KMMzb+Nydky86hz+Yg9suhNxAl0dMr9T3dP9HMd9YjB+XDSZsgQL54H438NbkL3BMskhOOd4c6oi6QvmY7+CZo+VYMf3Bcy1a4F/3RyLNAXK/1g9o0bowR7v1dG9mWsfLhuavRRFsid2u38KnG13ZMeNgI0DJQzDzgrwCQ2JzxL0Qcvyz8P0sothfKY+JVmA5a9uEmPvijEYkcPoDkPvc2VnruummQ4pRJYMjyFF16LgK6hUYqYT68+rOB352bvT4YA3OOk0SiqTUMNb5+xpBBfLnDZsjUo8JrgWpSenR1dv4enes+27NzYzyt5CB364rVPVomv9qYydTIk3o8TUPCeLQZO0geLMDq+OxaOOY4IQqBJFjvFemiSfaaJD8ZyDP5stIDVrUBYbq9rFikcQFlu2/uHV8JItPckYrt69qAMb6NZCYe8tIMZBRkPR76utz7atBMjLiymtOC+y+VtnxGryaIDIsQnl+XIRWFKYnitGfRoa0GZ4Y4eN5c2bHycoIs3CghqvdZf4yi9MUUWzQeEea8/pXJHUWU74MIiqndTfSmT4LGDN+FG8ImE69H5rOm uUSN6r+N JwnAB6NYiMl+DsA2IRWrNqpvNPwta1pIM6S4SrvRmx5yKqC+IZ9o6Qz6ViFsuKC/8uNC+4gx+QRnrDCFFArkhUhjy2kBQKfybG5MlLGGn9ATq4fwKqiSNIW0kOfA18Y3/qKGXG6fua/iT9bhfFjkN2I2i6tvPFxhkzyz2bXn0gXJem9ZTNAMQmRg2PuUMSWEbnraReuyr2An/zy7EPXGoxR5uLp7zxQGphfsGNjGMduasKtL9Hk5E0BH9liGgS8H8QY5XHFsDDnp3Izmou+4GK4pNmK7HjFnoYD9YeOAevSDWvlGjY+7q1dkcFC6tjKQX/fo3Xiu2ycZhwRlz8vSMlF5kadhhCd5UUrQGV+CmwEJj/esAYg2SGpWwWxoSsY1f9pfuRXAiOUTL1yN08KahxMFIWPuWEWdX+gg+COMT57enT4qjDjw2X6zQxnNe/FK1WsQLFGARbhJap7l/FbMVZ1MNx8Pst3aZGEe3IW0NJ3CN27ThVgtLo1xMTabzh0Fm00mKn+ZCnKAqPrl/ChJ1JIyXFuQvjBG2EkCKyFVIK139LqG/xyJW4XhGnF774w4VyWSQcsL+YzGQQdK01H2V+lTCMAeFzwgM/fgATiaA8wmBmj08df707TpHDuL6vd9HnLlU4dRjET6lDSyocwLPtVNNKvg912CA2bgR+fdpv4CShBSLhOG2WicZNw56sbFIgnYL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Dec 15, 2023 at 01:21:57PM -0800, Yosry Ahmed wrote: > On Thu, Dec 7, 2023 at 11:24 AM Nhat Pham wrote: > > > > During our experiment with zswap, we sometimes observe swap IOs due to > > occasional zswap store failures and writebacks-to-swap. These swapping > > IOs prevent many users who cannot tolerate swapping from adopting zswap > > to save memory and improve performance where possible. > > > > This patch adds the option to disable this behavior entirely: do not > > writeback to backing swapping device when a zswap store attempt fail, > > and do not write pages in the zswap pool back to the backing swap > > device (both when the pool is full, and when the new zswap shrinker is > > called). > > > > This new behavior can be opted-in/out on a per-cgroup basis via a new > > cgroup file. By default, writebacks to swap device is enabled, which is > > the previous behavior. Initially, writeback is enabled for the root > > cgroup, and a newly created cgroup will inherit the current setting of > > its parent. > > > > Note that this is subtly different from setting memory.swap.max to 0, as > > it still allows for pages to be stored in the zswap pool (which itself > > consumes swap space in its current form). > > > > This patch should be applied on top of the zswap shrinker series: > > > > https://lore.kernel.org/linux-mm/20231130194023.4102148-1-nphamcs@gmail.com/ > > > > as it also disables the zswap shrinker, a major source of zswap > > writebacks. > > > > Suggested-by: Johannes Weiner > > Signed-off-by: Nhat Pham > > Reviewed-by: Yosry Ahmed > > Taking a step back from all the memory.swap.tiers vs. > memory.zswap.writeback discussions, I think there may be a more > fundamental problem here. If the zswap store failure is recurrent, > pages can keep going back to the LRUs and then sent back to zswap > eventually, only to be rejected again. For example, this can if zswap > is above the acceptance threshold, but could be even worse if it's the > allocator rejecting the page due to not compressing well enough. In > the latter case, the page can keep going back and forth between zswap > and LRUs indefinitely. > > You probably did not run into this as you're using zsmalloc, but it > can happen with zbud AFAICT. Even with zsmalloc, a less problematic > version can happen if zswap is above its acceptance threshold. > > This can cause thrashing and ineffective reclaim. We have an internal > implementation where we mark incompressible pages and put them on the > unevictable LRU when we don't have a backing swapfile (i.e. ghost > swapfiles), and something similar may work if writeback is disabled. > We need to scan such incompressible pages periodically though to > remove them from the unevictable LRU if they have been dirited. I'm not sure this is an actual problem. When pages get rejected, they rotate to the furthest point from the reclaimer - the head of the active list. We only get to them again after we scanned everything else. If all that's left on the LRU is unzswappable, then you'd assume that remainder isn't very large, and thus not a significant part of overall scan work. Because if it is, then there is a serious problem with the zswap configuration. There might be possible optimizations to determine how permanent a rejection is, but I'm not sure the effort is called for just yet. Rejections are already failure cases that screw up the LRU ordering, and healthy setups shouldn't have a lot of those. I don't think this patch adds any sort of new complications to this picture.