From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 082A7CD11DF for ; Thu, 28 Mar 2024 18:46:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F51F6B0087; Thu, 28 Mar 2024 14:46:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 47DFF6B0088; Thu, 28 Mar 2024 14:46:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F77A6B0089; Thu, 28 Mar 2024 14:46:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0C9DB6B0087 for ; Thu, 28 Mar 2024 14:46:08 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 9AE481C0397 for ; Thu, 28 Mar 2024 18:46:07 +0000 (UTC) X-FDA: 81947327574.25.8FC846B Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by imf11.hostedemail.com (Postfix) with ESMTP id DB61740010 for ; Thu, 28 Mar 2024 18:46:05 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=qoBYOZvB; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.52 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711651566; a=rsa-sha256; cv=none; b=6iHs8OnhaGb8I+yxn2E9Pl+6OaZjuQL5DqA1UbYEyOJvHg1JPYZObU0RAFeASQo/01qPwK SwkUGnUyQiPzHEw4pymQtfLByBvWcATPOwIxcto0ZbHwIVhl54WY1q3wSdKpoaTmJ/yAR6 5sFyXLNv/zuwsitMSOZYh9WxW2ITGt0= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=qoBYOZvB; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.52 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711651566; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Id1/BhFDfw+YMKndBIzO44ul9b3NoasO2NXD5GD17qU=; b=LT+DieNjxes7uQOXhQnhfv1ALDnwEt/Vp4sU3JI3WroIPAA6zK6GsdMswHGCN5gbi8x4jI QM6V519Kv9LWRfc+2xD9yWkdr6QlRvb9xp/fWXXAWC41nNw4fxXQF0jA+wJxpwhNVOs56T t4p6SS9CpV1/yByS+okVwQaH4vRwX6Q= Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a45f257b81fso151310566b.0 for ; Thu, 28 Mar 2024 11:46:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1711651564; x=1712256364; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Id1/BhFDfw+YMKndBIzO44ul9b3NoasO2NXD5GD17qU=; b=qoBYOZvBJ6h4g9T9r0+ycEgYzzZ9B4R+vGFh0ndZ+5tk128YHZhybKTZVSi/vgrtKC idDj7wPJbsRB+n2ZhLOKWLLsUwywsJSSh8G7C48rP7dfJeTLja46+Wmf0HkwZxIhpGmE /z9ws10SBSCMY+zZpJyn58jQeBGjgiRRXiwGAlz6gF0+H85BaJ0AwpvMwgUFA3RoENXO NyqwN9tIjCFkaKGwVT/E02k7YBG4OREV+OQ6CmA1r/Be2xW2VxvygvfGXWrUm0d56Gwe 1W6fCmiJabZh34LE11yGmWo8prXsJ/X8jBdVw1RMWsmC73ohpuOkjkI7eH4ayPWACeof xomw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711651564; x=1712256364; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Id1/BhFDfw+YMKndBIzO44ul9b3NoasO2NXD5GD17qU=; b=FMPlmXUDiVtaYzsr2CsOFp54lRSnZXmuaQ8btdLBRTuXxPnSm4IuVUXfO4ALfiDE6A wVyLyemOBkJ9MlsHLjLuQJiAqT/KFeonO//mt7oTuooT+wf+sc8kJLnMH5zM/JevJYMi Tx5TtS9LyzeleNO8QgXwvUu+2vy4qkbwM5ZyI7v3+Sm1v9DcuiT18S1xy4f93DjecGFi 2yml8Kk3qi07WKxKuNodJ2xwKaryz5YAGhLlwFYhtk5nF67nxtaUuR5rxGVH0axdBkBT Al2MwIV3+Sxyo0Y2c/dVhWokjNI+85Uhwust2KkIyTFfWCMH1h3ZGhhPXESQFeZEUMWx I5HQ== X-Forwarded-Encrypted: i=1; AJvYcCXpDdZCVzUGtOCoVPcd6a/UbtEkJOwhzzO/QpGYU5kuMyXiDCvPZJxu24k77UFTPzJa0GCIBKLMeozi/inIUd9EYlQ= X-Gm-Message-State: AOJu0YzHywxqG8ClpTYbdqg5Hppimia0Os9nSu13PkVGPpWPAm6fmJzI p9Voc3vmvGcOvUV2q9jTC7AuuQN3bxR9Hy8Tb2UKhHoFNd6bSNF7J85yy6+QWB3gcdwY8RlhKeW VPm+o/jwDyRLDaJqrcVkWEr3ltC/Oe1dq8DCa X-Google-Smtp-Source: AGHT+IHryiPRPxQDrTg55XLq+V7QJeRsJ+yEGzh1/0zvgIomnS33RrLd0QVMtT+9vBzw9M4q56R1LAYW9+pLKYElaVM= X-Received: by 2002:a17:907:7216:b0:a4e:14e6:2084 with SMTP id dr22-20020a170907721600b00a4e14e62084mr93370ejc.51.1711651564051; Thu, 28 Mar 2024 11:46:04 -0700 (PDT) MIME-Version: 1.0 References: <20240325235018.2028408-1-yosryahmed@google.com> <20240325235018.2028408-8-yosryahmed@google.com> <098bfa48-75d5-45b5-b81d-a2a84b394352@linux.dev> In-Reply-To: <098bfa48-75d5-45b5-b81d-a2a84b394352@linux.dev> From: Yosry Ahmed Date: Thu, 28 Mar 2024 11:45:26 -0700 Message-ID: Subject: Re: [RFC PATCH 7/9] mm: zswap: store zero-filled pages without a zswap_entry To: Chengming Zhou Cc: Andrew Morton , Johannes Weiner , Nhat Pham , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: DB61740010 X-Stat-Signature: i7nkbfrmynbo757kxmoy3bg3gsxbsspy X-HE-Tag: 1711651565-748203 X-HE-Meta: U2FsdGVkX1+d+zJRyBWNCFVoWYCerKFLnweSH0ABGKBrvdETymspl0FRwhm6iYdGlEOruM9WXNaf1oVlvA5YbYrGBzUljc5/78/T+dyDf2iFUdWNJrT3xYwKAf0Tt153R/OSSextPvyRHHcCLBKUvbZGmnR7quTQ0Fm3wJS5gegCbX4yqiohWnhPkucv6ExUkSZCnZ060Ig8sEwwWrXzBAh4hjILgYcDtpXZVyqNCJaZATcfAzeh6wiAnRhZR3tz1BRwYKYOxDuLDOaSEQ3PV1a2mP+6r/7nuGCuuzHwtjxuMYRvEyINe9+OzG/+HNzB7sHi8T/gOsPBmwiURml3Og82IyQumCr4k0iO6zOsFBWygcSpTUMx2EaSdX3kE99qQA120mackJhEoq1flPQ4KemtYCR1uD1/0FPN9r/RhWqAqbcvjKuXTr3I5ujUsok7Vp0xBUu+4C6ZCIQterAukh9ySIogdiB68tIEUxuCh8WMbrJjUooxcykzOYa0zdQ9hekWzCLNQFUrqyhPPf5ZxCKsMRyKmP1hH+5IpEZ7dhYeRvjn0mPod9WczNJ71jjQm2vvePIQRxWjPWskzJe90Xp2mO2YxJXqqfXfD641yV499jXaHz4pAfk8Ekz9ZWHYz1tnBTtgaY4lj8nN94hIvR4+LlAIXo60E8KXyId7pgd+19szl3PgKa49hKw8Do3gQspQbvElBP1OhN3S28QPGmeXxPo5ozBmgmyvMaTeizJA7dJc2/sjJLgX0zM8ntLhdIqjGFaWkx4qGOT3zvf5H/+9HeUVzmk/pyRmARABz39TN7aaI8RFP0+8mCu/4BZ5UIVWufkLtb9jHNYqLHChZIBB/4k2+QYaEFPSfF+hfTHh5lAL3nvY/UXhCpXXG3TbT3FzZunse/+Ka9eR0+fghEfIVI31WfAqTYlnW3YyK8LTsU1ziDhjlqbJXdMV99+HhBDfgny6pRqb14dcOVW fGRibRua ffgntcUmiR49OKsUePBrlYDblS02G4/+IZDBw9Hq3nPTvXa0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Mar 28, 2024 at 1:12=E2=80=AFAM Chengming Zhou wrote: > > On 2024/3/26 07:50, Yosry Ahmed wrote: > > After the rbtree to xarray conversion, and dropping zswap_entry.refcoun= t > > and zswap_entry.value, the only members of zswap_entry utilized by > > zero-filled pages are zswap_entry.length (always 0) and > > zswap_entry.objcg. Store the objcg pointer directly in the xarray as a > > tagged pointer and avoid allocating a zswap_entry completely for > > zero-filled pages. > > > > This simplifies the code as we no longer need to special case > > zero-length cases. We are also able to further separate the zero-filled > > pages handling logic and completely isolate them within store/load > > helpers. Handling tagged xarray pointers is handled in these two > > helpers, as well as the newly introduced helper for freeing tree > > elements, zswap_tree_free_element(). > > > > There is also a small performance improvement observed over 50 runs of > > kernel build test (kernbench) comparing the mean build time on a skylak= e > > machine when building the kernel in a cgroup v1 container with a 3G > > limit. This is on top of the improvement from dropping support for > > non-zero same-filled pages: > > > > base patched % diff > > real 69.915 69.757 -0.229% > > user 2956.147 2955.244 -0.031% > > sys 2594.718 2575.747 -0.731% > > > > This probably comes from avoiding the zswap_entry allocation and > > cleanup/freeing for zero-filled pages. Note that the percentage of > > zero-filled pages during this test was only around 1.5% on average. > > Practical workloads could have a larger proportion of such pages (e.g. > > Johannes observed around 10% [1]), so the performance improvement shoul= d > > be larger. > > > > This change also saves a small amount of memory due to less allocated > > zswap_entry's. In the kernel build test above, we save around 2M of > > slab usage when we swap out 3G to zswap. > > > > [1]https://lore.kernel.org/linux-mm/20240320210716.GH294822@cmpxchg.org= / > > > > Signed-off-by: Yosry Ahmed > > The code looks good, just one comment below. > > Reviewed-by: Chengming Zhou Thanks! > > > --- > > mm/zswap.c | 137 ++++++++++++++++++++++++++++++----------------------- > > 1 file changed, 78 insertions(+), 59 deletions(-) > > > > diff --git a/mm/zswap.c b/mm/zswap.c > > index 413d9242cf500..efc323bab2f22 100644 > > --- a/mm/zswap.c > > +++ b/mm/zswap.c > > @@ -183,12 +183,11 @@ static struct shrinker *zswap_shrinker; > > * struct zswap_entry > > * > [..] > > > > @@ -1531,26 +1552,27 @@ bool zswap_load(struct folio *folio) > > struct page *page =3D &folio->page; > > struct xarray *tree =3D swap_zswap_tree(swp); > > struct zswap_entry *entry; > > + struct obj_cgroup *objcg; > > + void *elem; > > > > VM_WARN_ON_ONCE(!folio_test_locked(folio)); > > > > - entry =3D xa_erase(tree, offset); > > - if (!entry) > > + elem =3D xa_erase(tree, offset); > > + if (!elem) > > return false; > > > > - if (entry->length) > > + if (!zswap_load_zero_filled(elem, page, &objcg)) { > > + entry =3D elem; > > nit: entry seems no use anymore. I left it here on purpose to avoid casting elem in the next two lines, it is just more aesthetic. > > > + objcg =3D entry->objcg; > > zswap_decompress(entry, page); > > - else > > - clear_highpage(page); > > + } > > > > count_vm_event(ZSWPIN); > > - if (entry->objcg) > > - count_objcg_event(entry->objcg, ZSWPIN); > > - > > - zswap_entry_free(entry); > > + if (objcg) > > + count_objcg_event(objcg, ZSWPIN); > > > > + zswap_tree_free_element(elem); > > folio_mark_dirty(folio); > > - > > return true; > > } > [..]