From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CF4DC27C4F for ; Thu, 13 Jun 2024 21:22:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 222426B0095; Thu, 13 Jun 2024 17:22:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D2B76B0096; Thu, 13 Jun 2024 17:22:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C1946B0098; Thu, 13 Jun 2024 17:22:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DFF8B6B0095 for ; Thu, 13 Jun 2024 17:22:29 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 54A90140385 for ; Thu, 13 Jun 2024 21:22:28 +0000 (UTC) X-FDA: 82227139176.21.5FDA3B8 Received: from mail-lf1-f51.google.com (mail-lf1-f51.google.com [209.85.167.51]) by imf29.hostedemail.com (Postfix) with ESMTP id 64C5F120013 for ; Thu, 13 Jun 2024 21:22:26 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="Va5r/1us"; spf=pass (imf29.hostedemail.com: domain of yosryahmed@google.com designates 209.85.167.51 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718313745; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DvpeA/1HRnfLZyI1RmxZmnl2ghDAFMHZ/pZktbhRNJ8=; b=w33IwbCJkIWVC0mJoA+kUn6/VN7ebPkE2/+yE/wZ7imkSxxtMKzusBy5x1Qigory1kQb2E H/3RT2H1bAFOH1t5dZQFrwUT4IhW0zXCBMva5iC1xL9EUZa6Xf2syNa6n05OecG/ae/aat TqH7VgIA4JrRBzLWXxJX1YsUWUaAWqc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718313745; a=rsa-sha256; cv=none; b=YeyCA8+7YYM59Omc06f4Eiqc8iNPAQxVqInNQmS0NWGrxlGrvdY0w/GiEuzBL9nzzYb0tw fIXIbXcp/tkzWWiuop9mAlq4xm0bylChZs70EAJa+ZvbM531xif9lM+qDcxiWDMjq7zgka g67a2QzFnSQlrEyARCABq/VJxfrwc3o= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="Va5r/1us"; spf=pass (imf29.hostedemail.com: domain of yosryahmed@google.com designates 209.85.167.51 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-lf1-f51.google.com with SMTP id 2adb3069b0e04-52c89d6b4adso1470042e87.3 for ; Thu, 13 Jun 2024 14:22:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1718313745; x=1718918545; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=DvpeA/1HRnfLZyI1RmxZmnl2ghDAFMHZ/pZktbhRNJ8=; b=Va5r/1us9YwPolsfpCUyerIsuqxGNM0t62589PLwPjTyN1PSlmLkpbcsORyZ/FfxpK 6qchA8cDhnd35avZhTqXXA8DeBI7IclyZyNYFeP1y2jyHK+zzUukTrF9nxdHc7+rxdnm rqRtMLl63D3mCw6K90DM5g9p2aqbH/rJcrIMa3kTP6oIu5y9ojXlHphFpBPeKojv+alc sioEuG+nI//QEbftWMb/LLDFF4XYfcRNdGw49ZDf3sApYAXVf4Wah+jdxc9u5Dy4ubza RPQ27n1iQHG5LL4NTP6+5mSq1e3oPV/SZziz4bjp+hL8XZAcEvgTVRb0+ywAAc4LwYzW uSZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718313745; x=1718918545; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DvpeA/1HRnfLZyI1RmxZmnl2ghDAFMHZ/pZktbhRNJ8=; b=aAl2gxj2Woxp1PPlOfMQEMwdktfk4vndRqkYwvg7V5hz5zuSzv8B6mXYTNJ24UMyCW dzc6AjwESj8A9V7j/zqmQ0mEvXy8rqnJk2mr3zthMxTTVhD726Nuq89+kOlmmkvR8psn 80xOnznfL33rNgES3MEaqLGzkj852bnDZWE0IfcK5qHly/tDCvoK+XapvY2qBSYHOL9V 99rX3i9zFif/+Rrhf7z4ijMUxy9hfkvvZD2u1AlYbp/QeTbG3tr25eVK/b+Xt+08nJXF dDCR1lf01D//OFTej0zpdJqdaCXIBvFs5rOPKQiiSpvBWTc1fbxHQkvL+XvmmLfwPyXB SLqA== X-Forwarded-Encrypted: i=1; AJvYcCUR7D3QaMOLYHI6wkLPPq2vcAOUYESS5PyBX+ZJnPXec0ZI96RFnhrlkUYvBWYRWyxc85iefCz/MHilG+7jSAKZrL4= X-Gm-Message-State: AOJu0Yw03Ej0tHmYO6ZQ+Hz9Bv5vcpctSthVAhmkrKy8WKeraasJaX5r UPMHWrI5eczWrMl20/np3Lq9N9ROvJHO+37mHQek7LjrRzVySagU6ZEwn8Hknr+gr1Iocrpyo4P DP4UrAkgbAihQGdLeKiNvXIW9lDlnFdNz3Hy4 X-Google-Smtp-Source: AGHT+IFMJjg0s0ZFkpT9FjDot5/rGhh6fqId5ocyCohFuRj4gm6D2a1YSR0zkLByFCM9X6pUHqADSOmQqcapzFUROdM= X-Received: by 2002:ac2:4247:0:b0:52b:c14f:4f84 with SMTP id 2adb3069b0e04-52ca6e658ecmr524796e87.21.1718313744315; Thu, 13 Jun 2024 14:22:24 -0700 (PDT) MIME-Version: 1.0 References: <20240610121820.328876-1-usamaarif642@gmail.com> In-Reply-To: <20240610121820.328876-1-usamaarif642@gmail.com> From: Yosry Ahmed Date: Thu, 13 Jun 2024 14:21:45 -0700 Message-ID: Subject: Re: [PATCH v3 0/2] mm: store zero pages to be swapped out in a bitmap To: Usama Arif Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, david@redhat.com, ying.huang@intel.com, hughd@google.com, willy@infradead.org, nphamcs@gmail.com, chengming.zhou@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 64C5F120013 X-Stat-Signature: aso9kff3kmdriz3rsu1z7aodfy9sfo33 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1718313746-997533 X-HE-Meta: U2FsdGVkX18Xxpe0JSP7oGZRMJnrqMwoaCvHEkLK16UZoNnoBNM8aom5eVs8xLMK9KSApWvxZwT1WnQ4hvMA7ykBWWBUTGKUhcWcBF4oT8W7bpyeunBPpWUeK+qzBsrhmirIp+uKRnPsr+mTH2eQOpIGYzH9+qMSxGv1zH6UG9++A0XsCCrxykg/81jwnqXncHZRsqOxQQoFg4rPEJK3M89uW4wWbuFTY96MbWdsZmlCwAxCXIMyasAEdEn3UHuNLovqEFKUm97NkPsvdk9aV2soCc8kRhBL9znOMwdc+FLdJqalOY2ckju2NqwIENP8abOobYmN1mtrTsBprmPtEYnrn7Jo5HLNU8HCv5++TIRAnJiCb7yuUpuVwqJYT6IJz8n09mQCwnWDtsbFSg9d6thO0gTR7jYPM2CyYQqhjffJ5CPy5Sqsao8lkhCyU0CfbNIX3Va8BQttuqgnE8gVQ4IisF6loSPc/EK2MSIgbigMDhOUdjFTiU0OS/UqYog3iz2OTLZlOaQl2OBpuOJ/ivNes+NcZx7xpybCCDK7mGlBWiFA5mnb5aRL4L3kDSavgZsLqc/HTmg0MXvR8dxZSqxAlQtgmVIbGNhixPvRcCnNLyKsVtbm19lru0xsDRdDBqLmIpKKL1GxN39X1QzkFiuhkdhrKJT/ZlZyxpdaICM41FcQVylF3TZ759WYiuwqdzvvRUqBkq27rYsFfNHY0oZvX8x8kkKFA+3sKOmo/Hpc1D49pcSCIz42O1pH3hmEM1m+Gr6WIFfRBkp3gARy6SXxg6aY2/rGswZKPm92HjKMXgKbb8+NykFgJge3NQFeZ+dWvsqXi9Zm6oe0VIdXfecWiwQmyAmMlWqB2LfM1HyuQg4iDGunMk0ydcJ9IdHVgSkXfrBROKQ2mcCmffUoQ/7t95U2Zhs81CP6/yVx2rzYZP/Ul2JVP0XTw+d6M7m4nBLY3ZgShZ5r+UKm/bT b3TNZEdh FhBp5vXmOCt7EOpQpRFYmt9KIBp7sIPihs4EJ5EuYv7yOAqkiD3yAhuNfIPbz4uEli7YRQ5bwCnjrjtCfdpGyfE0nP7XZcQixMJa5jneM5xQqwLHNZOwewFM7NVo16n7WFsEEU+Psld3k+7MdppGsszbLJJc+UT5/ItyJNYZlZJmP0B63lwBtPLonJNKdkqctBEO/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000018, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Jun 10, 2024 at 5:18=E2=80=AFAM Usama Arif = wrote: > > Going back to the v1 implementation of the patchseries. The main reason > is that a correct version of v2 implementation requires another rmap > walk in shrink_folio_list to change the ptes from swap entry to zero page= s to > work (i.e. more CPU used) [1], is more complex to implement compared to v= 1 > and is harder to verify correctness compared to v1, where everything is > handled by swap. > > --- > As shown in the patchseries that introduced the zswap same-filled > optimization [2], 10-20% of the pages stored in zswap are same-filled. > This is also observed across Meta's server fleet. > By using VM counters in swap_writepage (not included in this > patchseries) it was found that less than 1% of the same-filled > pages to be swapped out are non-zero pages. > > For conventional swap setup (without zswap), rather than reading/writing > these pages to flash resulting in increased I/O and flash wear, a bitmap > can be used to mark these pages as zero at write time, and the pages can > be filled at read time if the bit corresponding to the page is set. > > When using zswap with swap, this also means that a zswap_entry does not > need to be allocated for zero filled pages resulting in memory savings > which would offset the memory used for the bitmap. > > A similar attempt was made earlier in [3] where zswap would only track > zero-filled pages instead of same-filled. > This patchseries adds zero-filled pages optimization to swap > (hence it can be used even if zswap is disabled) and removes the > same-filled code from zswap (as only 1% of the same-filled pages are > non-zero), simplifying code. > > This patchseries is based on mm-unstable. Aside from saving swap/zswap space and simplifying the zswap code (thanks for that!), did you observe any performance benefits from not having to go into zswap code for zero-filled pages? In [3], I observed ~1.5% improvement in kernbench just by optimizing zswap's handling of zero-filled pages, and that benchmark only produced around 1.5% zero-filled pages. I imagine avoiding the zswap code entirely, and for workloads that have 10-20% zero-filled pages, the performance improvement should be more pronounced. When zswap is not being used and all swap activity translates to IO, I imagine the benefits will be much more significant. I am curious if you have any numbers with or without zswap :)