From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5CE6C47258 for ; Wed, 17 Jan 2024 19:31:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 215B46B0088; Wed, 17 Jan 2024 14:31:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C5AA6B0089; Wed, 17 Jan 2024 14:31:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 08D9E6B008A; Wed, 17 Jan 2024 14:31:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id ED9076B0088 for ; Wed, 17 Jan 2024 14:31:31 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C159F140431 for ; Wed, 17 Jan 2024 19:31:31 +0000 (UTC) X-FDA: 81689797182.29.254137A Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) by imf13.hostedemail.com (Postfix) with ESMTP id E6CD020025 for ; Wed, 17 Jan 2024 19:31:29 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=C9XY1HQN; spf=pass (imf13.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.53 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705519890; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OUMHX6S+0IO5CvL0DocoythkWeWqWs33LQSD4/f8L0Y=; b=E+6NYeFThaM/vaKdGO8mPB+U0kZxQbD2pm0bWmGEU+eIdsUfPE0Z8XuYHx9u/Wi64zKDU4 4kNbz0ejSF02J6MBogWwXt/mrNww5zAfRxT3j2MITC6i0JvIB4HQ2C32ofagruFTlIRuCl SgmPjTnpJRFYrl0kHylyhQlL3MQXOg4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705519890; a=rsa-sha256; cv=none; b=qxRjnEVCDoyO1EJ/fwBh7slgp7esTO9yL1EIGtdlLwc4ajceg1IhWhNcA7yN8U2xFf1X5G WQjV8MLefft+UAEv8PwAGTjKL4O4RqKKp+iHyESiA6Ohk1aJTD+IArSQ5WyIpUtwUCBiJr 9nqM3cVPmXmZopCbHbqqdp9LkfVIobE= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=C9XY1HQN; spf=pass (imf13.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.53 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-ed1-f53.google.com with SMTP id 4fb4d7f45d1cf-559edcee495so702342a12.3 for ; Wed, 17 Jan 2024 11:31:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1705519888; x=1706124688; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=OUMHX6S+0IO5CvL0DocoythkWeWqWs33LQSD4/f8L0Y=; b=C9XY1HQN9Q/gHO82hJbJmuktlawWrRKM/dILAJpCZtDfiz2uimCGmuqfopeHxuz6Ys GaxAMLgwgkyzAOdWIOM3XH7QXRUYg0qKZV8EsMvy5VrGRG4CLW23YLAGUO/FwlyCHrNM oN/wE+IJzMcuZ5Rcz7WiwbBG9uHMSQhZXWYyK6g8C7Wzij/m9rUiw2lKcZtBpIeScXlC a+A6TtJkolkR10dnvslTKiSkdJN0f/eFM6UMl2rE2OrkDpm2KDqgataUXLI50Kgn9nP2 dGFVyDvIOusfN1o2THepVGtHi6qb3oNP0u1D0cvlA8Jw9SzDisfr64SPgPxhwvn5gglA KInQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705519888; x=1706124688; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OUMHX6S+0IO5CvL0DocoythkWeWqWs33LQSD4/f8L0Y=; b=eTZ3C2aaqu+QcrnnQIaxdWEQMkRXbODHfZX4U6LjH8sQOoxKdFwvL5pQ2dMV+LfeO6 9J/VD/43alCufNdZLB7xStnhMkK+QlWrlxLkdmyXIcCpovSlhGUVK/A1cVRxjP8UfUG2 8Jj6SHXyfvchomv7VHv+lE6ADboSIpivfEzZXFiEZal2qV7ql+OYLnsyQa66a/J3BCdc nPJnTZjdvUmeB83eBMBUSX5EhM0smEL27tWe/6fVsyfSWMaLgvPWrLyhs19AKu6RzMOr uWvfzo825JQ3EVHU5dkXXkm8hSsMip/kvGysDcu6FTsFZrbjBG8TIGeYc/quoOEb15Re d3VA== X-Gm-Message-State: AOJu0Yx/vyYtO9jcWeQxxMGrh7JT0f5UGJx+N5YlO+GH5WJFvtabTOE4 TSe4HzsRmcFWVKxL6YUlJgHwKRQ3npuqw//gCC9f5VsY2kDl X-Google-Smtp-Source: AGHT+IESU8PNUGdO93lxbT3HckP0KLGKpX9LFWXeWklpI+hjGR0SlguYycREh9vBYKKO/sV4r8TYeRv9itfv8v8Dluo= X-Received: by 2002:a17:906:2695:b0:a23:1e0d:565b with SMTP id t21-20020a170906269500b00a231e0d565bmr4826659ejc.1.1705519888090; Wed, 17 Jan 2024 11:31:28 -0800 (PST) MIME-Version: 1.0 References: <20240116133145.12454-1-debug.penguin32@gmail.com> In-Reply-To: From: Yosry Ahmed Date: Wed, 17 Jan 2024 11:30:50 -0800 Message-ID: Subject: Re: [PATCH] mm/zswap: Improve with alloc_workqueue() call To: Nhat Pham Cc: Ronald Monthero , sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, akpm@linux-foundation.org, chrisl@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Johannes Weiner Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: xubme7dpsxohyjnqb7b95efw1od8w5ed X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: E6CD020025 X-Rspam-User: X-HE-Tag: 1705519889-671085 X-HE-Meta: U2FsdGVkX19EHLo3uf3SbQ2CTMbT2/GCDR46PVgwScVYkkedtYuZ1NHj4qQu7jOTIzvjUKt4BN6hAhLk4SNBLHqz+rVZzhqFGP2BfaiTRSR6K5+IhGr0kcNBTVrMb77fc3CHzQ2JIgbHak0yXMPCs7uvTm+0Dxyr4PCeJxP5kGhTirEKeGjhs4e5sSOTX3IXZusa/lxhwV9F/UecgxBp2PEMYjzwPDbdtUYdhhl+xQh1DyWtRwZeXS6k68Z5i7g9rNGcQMLmxfgk5R/cM8rbJHICDe4psYNVIGJz2Y2MDgSDS/iNvHzScq5hL9sN+fxEeRsnMPDkOE0CsPeYmRJ9t1Ziv1PaaNfIyg909i8EuH8PNifGvXXUFyO0+N8zOTp55OgEBzaMDZnM2OJ/bJ1c61cygI+ti/sLJpZqNVXf+vlVIo8pznSBMJByZucC88G2MW8tKmsKdHSA7RAlOfdRKs07XIW5hYYQ0K1BIMlQuR8ZaJ7MfmV+5TezwAfRrjMWJUroCtVXZyjYAOvYgiuwRAK2fUSzSFf5ZMD6lXDA0Rg3tdF2NQM4OVW7aqKgC9B/encBNNPqSLLfwUAZeMNSPDZvY+1YsREE2N6PSE1DcU3cDHHCmwhpt/gUiHaG1F1yn50SUGvD4Y+viyr0njycs9xa3YwkPkIIh/UbguYe3htlLDfpa0nLZhywpHf2iRJjr+TQYCzSrRoUpc/wM+RnvJiun1pD3IDQtimdmyyyBhL2xeznJNioA8grsKYHU4+zPNIo7doiFmUvNTXjPP8g/odXanxGGIzTQfoU+ZM/2LL2TzHa+2LaZsyElreQ3eyCtveS1/PaxYz8HB+jlZbS+/+Y2AgO7PDb+jJ2/0IJT5bN7IZbWeKYW2ZpPwtzyXCUp1q6LZ4ga9g0X24qdHQFKTz9j1wETZclNRu3D/WYA99awARYj2Gn9M2fPZXUmkpPcIlt5cJ8kD/0kv1M9tG s0B9g4u9 1nyHGfFTLscLUiCUef8Wqcsm4N87JV4cSIf19+ljigdci6tbUyMNn54FxnIWfU7sIECOXWJ7LsnGtvhVltrEUg05HYr2c+sQqlCb1Re7FRiDamHn/TKDSoG8ViviSSxffOJ53f2aA26RuSuT0xqCC8tIfRBrDqB6axi9Pb9AIvOhouDfusd8ozcxcyl2Sg7cQiOPhd+y2iLPRyiQNeQqohx/g6nR68dQl9ElUZ8eGLVs60E8cOfHN8tcN1zYNNJrfoDPgeHDC2Gi7f5K6YZagtdV+89TSBm4cVmcFQ27pcjRcbcDVuQ7T1e6Q7FSl7AHjWgheI6Mmsna0KNjxpu6YWS8C+o8WLKrTs48f18w11cxwL9jViEo98JMLXw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jan 17, 2024 at 11:14=E2=80=AFAM Nhat Pham wrot= e: > > On Tue, Jan 16, 2024 at 5:32=E2=80=AFAM Ronald Monthero > wrote: > > + Johannes and Yosry > > > > > The core-api create_workqueue is deprecated, this patch replaces > > the create_workqueue with alloc_workqueue. The previous > > implementation workqueue of zswap was a bounded workqueue, this > > patch uses alloc_workqueue() to create an unbounded workqueue. > > The WQ_UNBOUND attribute is desirable making the workqueue > > not localized to a specific cpu so that the scheduler is free > > to exercise improvisations in any demanding scenarios for > > offloading cpu time slices for workqueues. > > nit: extra space between paragraph would be nice. > > > For example if any other workqueues of the same primary cpu > > had to be served which are WQ_HIGHPRI and WQ_CPU_INTENSIVE. > > Also Unbound workqueue happens to be more efficient > > in a system during memory pressure scenarios in comparison > > to a bounded workqueue. > > > > shrink_wq =3D alloc_workqueue("zswap-shrink", > > WQ_UNBOUND|WQ_MEM_RECLAIM, 1); > > > > Overall the change suggested in this patch should be > > seamless and does not alter the existing behavior, > > other than the improvisation to be an unbounded workqueue. > > > > Signed-off-by: Ronald Monthero > > --- > > mm/zswap.c | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/mm/zswap.c b/mm/zswap.c > > index 74411dfdad92..64dbe3e944a2 100644 > > --- a/mm/zswap.c > > +++ b/mm/zswap.c > > @@ -1620,7 +1620,8 @@ static int zswap_setup(void) > > zswap_enabled =3D false; > > } > > > > - shrink_wq =3D create_workqueue("zswap-shrink"); > > + shrink_wq =3D alloc_workqueue("zswap-shrink", > > + WQ_UNBOUND|WQ_MEM_RECLAIM, 1); > > Have you benchmarked this to check if there is any regression, just to > be safe? With an unbounded workqueue, you're gaining scheduling > flexibility at the cost of cache locality. My intuition is that it > doesn't matter too much here, but you should probably double check by > stress testing - run some workload with a relatively small zswap pool > limit (i.e heavy global writeback), and see if there is any difference > in performance. I also think this shouldn't make a large difference. The global shrinking work is already expensive, and I imagine that it exhausts the caches anyway by iterating memcgs. A performance smoketest would be reassuring for sure, but I believe it won't make a difference. Keep in mind that even with WQ_UNBOUND, we prefer the local CPU (see wq_select_unbound_cpu()), so it will take more than global writeback to observe a difference. The local CPU must not be in wq_unbound_cpumask, or CONFIG_DEBUG_WQ_FORCE_RR_CPU should be on. > > > if (!shrink_wq) > > goto fallback_fail; > > > > -- > > 2.34.1 > > > > On a different note, I wonder if it would help to perform synchronous > reclaim here instead. With our current design, the zswap store failure > (due to global limit hit) would leave the incoming page going to swap > instead, creating an LRU inversion. Not sure if that's ideal. The global shrink path keeps reclaiming until zswap can accept again (by default, that means reclaiming 10% of the total limit). I think this is too expensive to be done synchronously.