From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AB43C2BB3F for ; Mon, 20 Nov 2023 18:53:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A48376B0476; Mon, 20 Nov 2023 13:53:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9F8006B0477; Mon, 20 Nov 2023 13:53:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 911356B0478; Mon, 20 Nov 2023 13:53:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 819196B0476 for ; Mon, 20 Nov 2023 13:53:09 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 61080801F0 for ; Mon, 20 Nov 2023 18:53:09 +0000 (UTC) X-FDA: 81479230098.26.0E0FDB9 Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by imf18.hostedemail.com (Postfix) with ESMTP id 8EE2A1C0014 for ; Mon, 20 Nov 2023 18:53:07 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CdxhD8PC; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700506387; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9CBhC1m0OhsuzwJIA2sP5RLtq30X0JjG4f15SHsDupg=; b=I18UKgrdCLg8+7/tDJmTFvNxorS/xNUBuXAT2qyzeKhNQnT9d+2xPDFM0YHT3CcbjCQi3X PUdwTW7AEcr52s55vKPYoZYj939cVpLiqHj6ZoVs0RsbVUx+ayItbWTmQnL9dNY478DgJL xKTYet2kvmTC1rYq2XSwxCFUmKjTxqo= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CdxhD8PC; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700506387; a=rsa-sha256; cv=none; b=MC5OK6xd2iGVcqInsfe/ZQSH9fEYfZqaMpfqOBia6JT51/yd9DIQPmsUEaSj9BlONVjXOa MoLRtQ9tS+ng3JGi3DE6TpxACtatBDoN4tEwSX9iVHymIyDJNWJlVuDPzk+PTaizRrjAbL EHRqdyPTgs3bg/FX8vZPK8zddc7s6oY= Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-9d2e7726d5bso633872566b.0 for ; Mon, 20 Nov 2023 10:53:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1700506386; x=1701111186; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=9CBhC1m0OhsuzwJIA2sP5RLtq30X0JjG4f15SHsDupg=; b=CdxhD8PCTKs/3kZ1zJze25RBqrW/4JoqR9/xTfwE2MhJtWVSbMZLitr9/LO6JQwtoo vWEBofTRKE9G6VLtIsJcAJcbCu71BNhgu6w+yBTsRF+DR1us61UlfTQ1w4j+bzOsdzhX MIoOtFrHPjbJC7lfopx+6yZt//kMr8ro1654kP7WKiGfWuR63K3QfYHh5NrCEQf6YmGk zBwalQzyCjr5W0YoLxIMA0eL/9Ftmjn3buwL39yIUqG29jeXKeeTP5B4eUgYeo7Ea13Y zWyDjvWFBUcMCKeR3+oic20h2Wa4LkOKK+9anMLycyCOBGTj+2eoHucS0Tqyctd65OxK PK3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700506386; x=1701111186; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9CBhC1m0OhsuzwJIA2sP5RLtq30X0JjG4f15SHsDupg=; b=v2h+S2zADFWxomGbzjhc/uBQWK1tUmDIrBeYBCyIfinWVbWOfoy0ApfpnLv6mZEkvw 8rt6bPPajs1lLIcYFiU89aw4+Lc1fPW0qO4KF23D2FsD6ijnySQ+73nSvvDyEjwiX5LW +Fg8HPNDpI0dN/c60rQvBUaQmy++5ZlX2cYSDNsBodAbMKf0EE4fW5X2ZADmocqESZaZ GzTvTshkgIOz5bGt6Ei6GkqkSntQ0dfdGtAM4b8X3NdslPtxOWgiy/6CR8FNlW2/dmKr Ku/MYn3tW0qAk9jp9VBsNrJQuAvnom1rFl0taMvcJ6l/ZfhDVAZWDwudZeRcfVbdBe1Q eieQ== X-Gm-Message-State: AOJu0YzSId2QeB9ECTXoZzYKNcrYb38Ap6Fn/Tx5S5p0U9y7ZF1kf/1O EPiXbJT/RaqZm82zkmBUt7hXrO+WDGywxYthNP/bCA== X-Google-Smtp-Source: AGHT+IFgxqrtHRBfQhtaT65udIR9IUe1v9BKLhGOQFwasQm69jVp7/yuqde7t4blz4WxygyCw6vdNKG6xTKPO86+yM4= X-Received: by 2002:a17:906:52c1:b0:9dd:d85e:b23f with SMTP id w1-20020a17090652c100b009ddd85eb23fmr5763192ejn.67.1700506385805; Mon, 20 Nov 2023 10:53:05 -0800 (PST) MIME-Version: 1.0 References: <20231113130601.3350915-1-hezhongkun.hzk@bytedance.com> <8734x1cdtr.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <8734x1cdtr.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Yosry Ahmed Date: Mon, 20 Nov 2023 10:52:25 -0800 Message-ID: Subject: Re: [PATCH] mm:zswap: fix zswap entry reclamation failure in two scenarios To: "Huang, Ying" Cc: Chris Li , Zhongkun He , Andrew Morton , Johannes Weiner , Nhat Pham , Seth Jennings , Dan Streetman , Vitaly Wool , linux-mm , LKML Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 8EE2A1C0014 X-Stat-Signature: h8jmfesdkzmic9ugxwck9ozifu9tnrwe X-HE-Tag: 1700506387-127424 X-HE-Meta: U2FsdGVkX18A+9SxF9W5KXqenr1eB2zbkrHF0G5hmVl683LLLTGD+t81soY9IdNpDQl7GLgSUJOzU0GyWzMg0LO6lwnnWndaEjuUeahQmXRRnw8YdSJxYfAlXusDkhJyp/QUPuTZS4NiDUulqAmGKbSwUTXNp9TW2By8esbiIPvVTNrHp1HYvwutoK5vBKfICjHA35nK2m6a9F910MEErzBBR1eP57gDjxDGEr8CbIyc9jvvDL+4v6/KX8qf24L/DVph16fk9bWiJnOg1ZD6TorHT6Sgo8JKj7PqT9ovE1qShRDLNSMCVSczKvHhzdFlcevTV+H6MpHBWbv3zetq7aMz1ucJNFdHMzYJ9YFE7xFzH4Sx2bZNwChQW8NQtJjY3Q2cdR+XHlbE8StiKyQhHRM76bSx0XkgmgvkRyUnHK9NlAKfsSwYTiZ+ecRBObPtfXPVcgzxEtFqHtblwWpVCCOW6E8iK8iAV3tOXCVgWTAxlyn/W5MghEv5+2qc6UDqo2czUzLo6LkYgZ7jRELOiJEsB9OH15yQS6JRprxjyUfnErxAImKu2L9Gpu29CcwAWT8u7Wo7nx7dhFjKMjeh2eQQ9eVUmOj2u6hX+WUiX9bLWhC/nyFDa2uY6wwizTqOrwL76Qs0zMxx7H66JmO3pHF6LWAI0Gz4L3p8cnky/KZ5mY4jSGZ/zZQUP5DLf7lMMorxZNHfNVDLoYZrCF7NA9BkFiYwRNvwkrrxcFF0H6N+qF+2+/nWO294f6E/GtR0Iq+iujMxE9911M7x7Ef7GBt0ysM6UJMqjiDbCsiYhZfttTGgauAdYDm/emYnHqvCyLJNpMTniF6M2Uea3jwU2XeFrqNIGl9Lvi9KXEJhrUpcYMOuGHrkUgpNsIKoRVbvcd7/MEr8krGtambrn8C7O81VBBuSSvnZHM1v6TJqariwHGyWxlEsfYX7gLNsa5bypKaOSsYAwbealcEWWOh K2r1bPMC ohgycTlbmDjE7QvKlH8svPKXIOcP8q00X9g96NyZxezF7jJqU8oRafys/QXQELTBjMh3fWXg1LYlAUrxPaHaGju5KT3kOCmm5ghP/xUgWjYjv8TN1JW1w4pZ2FcqUzJlUQ591iOSAz06Xdsk4992CgO4nigW2u1/ybNMugpXsE8QT6jdDMZPI8IUCw+65baAjTqI5/WWjYqjDtenf1ChkjGVBlca/jJHgznvpga4vz6mIvM/ghwSbCE6gr4gLXwz3E5xzHSyLV1Y7HjdNcQWehIbuDA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sun, Nov 19, 2023 at 7:20=E2=80=AFPM Huang, Ying = wrote: > > Chris Li writes: > > > On Thu, Nov 16, 2023 at 12:19=E2=80=AFPM Yosry Ahmed wrote: > >> > >> Not bypassing the swap slot cache, just make the callbacks to > >> invalidate the zswap entry, do memg uncharging, etc when the slot is > >> no longer used and is entering the swap slot cache (i.e. when > >> free_swap_slot() is called), instead of when draining the swap slot > >> cache (i.e. when swap_range_free() is called). For all parts of MM > >> outside of swap, the swap entry is freed when free_swap_slot() is > >> called. We don't free it immediately because of caching, but this > >> should be transparent to other parts of MM (e.g. zswap, memcg, etc). > > > > That will cancel the batching effect on the swap slot free, making the > > common case for swapping faults take longer to complete, righ? > > If I recall correctly, the uncharge is the expensive part of the swap > > slot free operation. > > I just want to figure out what we are trading off against. This is not > > one side wins all situations. > > Per my understanding, we don't batch memcg uncharging in > swap_entry_free() now. Although it's possible and may improve > performance. Yes. It actually causes a long tail in swapin fault latency as Chris discovered in our prod. I am wondering if doing the memcg uncharging outside the slots cache will actually amortize the cost instead. Regardless of memcg charging, which is more complicated, I think we should at least move the call to zswap_invalidate() before the slots cache. I would prefer that we move everything non-swapfile specific outside the slots cache layer (zswap_invalidate(), arch_swap_invalidate_page(), clear_shadow_from_swap_cache(), mem_cgroup_uncharge_swap(), ..). However, if some of those are controversial, we can move some of them for now. When draining free swap slots from the cache, swap_range_free() is called with nr_entries =3D=3D 1 anyway, so I can't see how any batching is going on. If anything it should help amortize the cost. > > -- > Best Regards, > Huang, Ying