From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07B39C4167B for ; Fri, 15 Dec 2023 21:22:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A57D6B00FB; Fri, 15 Dec 2023 16:22:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 956196B0101; Fri, 15 Dec 2023 16:22:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F6186B0105; Fri, 15 Dec 2023 16:22:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6E2176B00FB for ; Fri, 15 Dec 2023 16:22:40 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2FC1E141006 for ; Fri, 15 Dec 2023 21:22:40 +0000 (UTC) X-FDA: 81570326880.02.B6EA14F Received: from mail-ed1-f46.google.com (mail-ed1-f46.google.com [209.85.208.46]) by imf12.hostedemail.com (Postfix) with ESMTP id 6A0834001B for ; Fri, 15 Dec 2023 21:22:37 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="pSF/jDdJ"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.46 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702675357; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fXf+McqFtrDiVJug5PNpwx3kDJ7o/xVXss/36ywlgjE=; b=q0CL95BsF25wBcptiAl+rZWUFEm9NJb0aj73OTJf+aF0NKkIYov245T8jqSBrnLFj4ULMt /eEGUG2pXNSftYan451TZi4h8qa8Y4+pPYNbHUR5HrYYeUgwYSEA+FbPLLNtU+kwPX6GRA HpuXzT0CdY6pYjjinPa/JC/ImqkVBYo= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="pSF/jDdJ"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.46 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702675357; a=rsa-sha256; cv=none; b=FH1ca1loLhjKQRpePPQBeKdFFjKYTWNygVtO/t4uF6zlRSTQX20j8FnxfvfAh7axAXvGBJ fCktimk04b+OBgXiKL4w7MtcArfkWvZT91G4hcko/eJzsoY6DXqzAFN6mZYb9pgFKC1X5J AyNkP8F6F7rTX4hMCVfc7hf9LEoMiq4= Received: by mail-ed1-f46.google.com with SMTP id 4fb4d7f45d1cf-552231d9c1dso1566740a12.0 for ; Fri, 15 Dec 2023 13:22:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702675356; x=1703280156; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=fXf+McqFtrDiVJug5PNpwx3kDJ7o/xVXss/36ywlgjE=; b=pSF/jDdJ5fGcwysPAHQt08fImG3thG7wqElhD7FKVuBwrIS32PaVslJxOypvcHh/T0 yArCwA8xLgJ+t8r8b+MTO0xZMr7XmiYcNHvHNHM1jqPIvGaGEVjA0bPusC8NHUkYuav0 hkCk5DVH+ZZHLeUMLWE+diLlPaC3qLcvK+SMMHzdFsnUBW7Zw8LzFmS6pdeaEXK94dYb h+/OmF4FsZzpx/dooaRLG5Ujk3kkS2x+7z/9EmhzLbR8y8UKiGBZaIlqctvp/mJq0TuG JvegmT1/NC0AWQlJldbr5WbezSZq61OFB3P2MNSzR5AJpWLEuWOOMTvG5898444pHjBu Wh7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702675356; x=1703280156; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fXf+McqFtrDiVJug5PNpwx3kDJ7o/xVXss/36ywlgjE=; b=tZBoTULF5BZSdnC6bZ1Y0g1odF4IgwCunmkGWRQgFzg20bI0Z/nLAf+PPjwhNObvJR PVZVfH7H06SCvXZhZutQTwKRRjvtdCWzmzmly3PQFrDdHWHhyuZT+Wk99ZYshH/j4gra aomZff9qKidw4xIFmPe7W0s4/C8PVv4aQY+nJeuEKO6jDS+pxU6Z4ntz14C2oFHS/JWD WXPDl0/G3yghlIuZNE+8lLUX2y3dkh+aVnbpRU8jTqJmmNcnE5xWu7uRN1YcaNwcedei hiMqujZosVGevyN4GebYG9Vz0O0iP7yk1YQZls0KCqjaJ8iB//IyXBdfnPxqGvblIsej u34A== X-Gm-Message-State: AOJu0YxIVPHVjyGwdF8G4m5+wsaXmssvl9E11D/kvjbniW8C7fO2bgwV cglBg9k5I24jZO2q8CdsRuIfG6YiCXLtnquQDS6DlQ== X-Google-Smtp-Source: AGHT+IHlKzE+tjnDj59p47h256/B9DFQX7PwwwbdzBZeZgmu28yrZn9vMa4l2i82Y/r5ncHDvDrcNjOZ5KCY7NicpR4= X-Received: by 2002:a17:906:100d:b0:9fe:3bb6:99fe with SMTP id 13-20020a170906100d00b009fe3bb699femr7354917ejm.2.1702675355578; Fri, 15 Dec 2023 13:22:35 -0800 (PST) MIME-Version: 1.0 References: <20231207192406.3809579-1-nphamcs@gmail.com> In-Reply-To: <20231207192406.3809579-1-nphamcs@gmail.com> From: Yosry Ahmed Date: Fri, 15 Dec 2023 13:21:57 -0800 Message-ID: Subject: Re: [PATCH v6] zswap: memcontrol: implement zswap writeback disabling To: Nhat Pham Cc: akpm@linux-foundation.org, tj@kernel.org, lizefan.x@bytedance.com, hannes@cmpxchg.org, cerasuolodomenico@gmail.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, hughd@google.com, corbet@lwn.net, konrad.wilk@oracle.com, senozhatsky@chromium.org, rppt@kernel.org, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, david@ixit.cz, chrisl@kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6A0834001B X-Stat-Signature: r3n6pfzdgwn4uajgawp5r53j81op35ry X-HE-Tag: 1702675357-425766 X-HE-Meta: U2FsdGVkX18qt0PKxD2WOUEg3otCuw11TucFVJYYssm8pCJp6jI/wINHyCzW6BiM7nYS4/1arJVhZ0eWqU2zW/WnQxbxmyoqxQg1Ul/GyP7Z41h5icVVK2/RTxJnjVcq+192W+AAXya1/nhVgzpRmnwj6kcRgJVNxf5Kt7ob6lbT1jbb6sVFJAnMNOAqAXz+vnRfgXymn8CpXQkGznY1r/rbVOYBt05bewiObq+SatmWCdrNoxHWcZGZ3cPw4tprO/bS5T6/qno1EiFZmIYFDZMgvV+hzORQ9M1x9tEVmBp00JCDr7Rbt5Wt53D/FyS5x8WBc9TdUIvD8MKBIc6o2YzjYifCTFD/yzRCFZUxOXCRHea3RzO5PNwqogtQt+7qP3eCeLs6p7HZmloB/hRIsoLNLtLQ+uYu1C2Sv9K3qf8QL6RMA+ZP1Z99ITD7DBtNR+kPqMcq2AD+48KvHZLcPrQmwG079UDq8Lz/CTiriAyaKDv9kLuJlle9jbWAooXnqJYdmz5wVr9c99l4v80uQJjafE2PIfpxI/K6Js5+BV5conmQTTRGNZFqyo3Pb9zL7iPawNmniinHQuwHpV8+EsE7yYmz1kCY8Hwtufaplx5dJfIXvldVDaHDss7fTuBN1lvJ3Qdx9LG7tOu8714v6EgRAVuTWSboxk5Yel8GMWuYoMN/MAN6DDWIu4vXWVQPYiWsDMWLUVh8vy+U9T+wEYGMUBJYr3MMA0wShfjJ7r0MVH5PEpXLsyFVCIFJLMgktdJUOuWzablPrtrfyuaOhyHG9NBBlR+L3IwiB45YFWe/wn1y7MZOjdLKczjSkJS+N00ms1uWv+TASzB58q55OOt/DTAa1wPU5R9fZiNP/EWdk50CwxPEJ8w6sVfdUTEvbyhzXa2YXIIvjHYG+ETm6oahnWLHXUnpEfQdjGeWYf7xL9+ksOxqcnrGVlSRhqHI2KR2ZzPzsEzJVkI5+P7 qZ5brSak 2iDAjCkp7M2SmRAZ6CP19dkgZjZh7sU5eU7SE7m/uvzNguiE85ip/7zGee9evg5GGuzkvjtE/WjQ0BneqEBzAwLLCAtsWMrkRl4eC+tgzR3WbCb9c8i0LjOZNznuDEAw8ObPAYhrbrp99mAFr0DuuEbTpHszv7C61bVHbHp5tHcTXqsEYAdObCTo+yDbYKWRLKlg8oX3OTIz2CYmrcVXppDg+3Op7u6m7M85oTO22ezGHaDk7ubicmCf2FW/tadF0Pl9P5Un+N+fGzqwMJsjbAg5ntLWnjvw7FXuWwZVWS7HFXXWknmYIzQhNsCwIuDWhC4dm1iUkgM44OJtmTUahORb7FAgpbgOpv2Pka3UIlw1W+l0BO0BNvjUsCqPVTnE6FDoVIjy+Zc4T62q7bTh4I0Ca4TxNqCPMYwto X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Dec 7, 2023 at 11:24=E2=80=AFAM Nhat Pham wrote= : > > During our experiment with zswap, we sometimes observe swap IOs due to > occasional zswap store failures and writebacks-to-swap. These swapping > IOs prevent many users who cannot tolerate swapping from adopting zswap > to save memory and improve performance where possible. > > This patch adds the option to disable this behavior entirely: do not > writeback to backing swapping device when a zswap store attempt fail, > and do not write pages in the zswap pool back to the backing swap > device (both when the pool is full, and when the new zswap shrinker is > called). > > This new behavior can be opted-in/out on a per-cgroup basis via a new > cgroup file. By default, writebacks to swap device is enabled, which is > the previous behavior. Initially, writeback is enabled for the root > cgroup, and a newly created cgroup will inherit the current setting of > its parent. > > Note that this is subtly different from setting memory.swap.max to 0, as > it still allows for pages to be stored in the zswap pool (which itself > consumes swap space in its current form). > > This patch should be applied on top of the zswap shrinker series: > > https://lore.kernel.org/linux-mm/20231130194023.4102148-1-nphamcs@gmail.c= om/ > > as it also disables the zswap shrinker, a major source of zswap > writebacks. > > Suggested-by: Johannes Weiner > Signed-off-by: Nhat Pham > Reviewed-by: Yosry Ahmed Taking a step back from all the memory.swap.tiers vs. memory.zswap.writeback discussions, I think there may be a more fundamental problem here. If the zswap store failure is recurrent, pages can keep going back to the LRUs and then sent back to zswap eventually, only to be rejected again. For example, this can if zswap is above the acceptance threshold, but could be even worse if it's the allocator rejecting the page due to not compressing well enough. In the latter case, the page can keep going back and forth between zswap and LRUs indefinitely. You probably did not run into this as you're using zsmalloc, but it can happen with zbud AFAICT. Even with zsmalloc, a less problematic version can happen if zswap is above its acceptance threshold. This can cause thrashing and ineffective reclaim. We have an internal implementation where we mark incompressible pages and put them on the unevictable LRU when we don't have a backing swapfile (i.e. ghost swapfiles), and something similar may work if writeback is disabled. We need to scan such incompressible pages periodically though to remove them from the unevictable LRU if they have been dirited.