From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9438EB64D8 for ; Fri, 16 Jun 2023 08:05:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 428286B0081; Fri, 16 Jun 2023 04:05:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D7EC6B0082; Fri, 16 Jun 2023 04:05:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 251638E0002; Fri, 16 Jun 2023 04:05:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1181C6B0081 for ; Fri, 16 Jun 2023 04:05:02 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D163C40B61 for ; Fri, 16 Jun 2023 08:05:01 +0000 (UTC) X-FDA: 80907875202.15.E9B7545 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by imf26.hostedemail.com (Postfix) with ESMTP id EB50A140011 for ; Fri, 16 Jun 2023 08:04:59 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=rfzoXUjK; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.52 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686902700; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YXX6whrEOn2eQbzXPk3jhKKZQgf895lWSSyWx6pgZ/Q=; b=mVSZcCq4QGy0DJlOyj//ncw8XFwTERbi0UpcxVZBR7SSfKm+9KpuKsowglATj9sDyxD2Iy YXjLwgok9YwUqQi7Z7m5IPP4fLSeJrQJ5lEUEDLoNwgjcegWoRcdysLe3xnFfoL2/NadpD TENs3EkcmpSQC3hDRYvI2I4D7ZVLQ6M= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=rfzoXUjK; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.52 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686902700; a=rsa-sha256; cv=none; b=ZEoNNXfS+586NBpDM5kSyECaOMMQlm0u63FXyC68t0bBfxeXZKsLMWi7XocS5sjvT/gkyF KZzJF3UHtMv9ffT+M6mHyzhpOPn/q1WK+8VcKdiZR+osla0x1fv+X24kTEx1To3rI4DoV/ wr5A5YpSST10vnQ0FMgEi+10h6PKRXw= Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-982acf0a4d2so49896466b.3 for ; Fri, 16 Jun 2023 01:04:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686902698; x=1689494698; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=YXX6whrEOn2eQbzXPk3jhKKZQgf895lWSSyWx6pgZ/Q=; b=rfzoXUjKWJC8c0uQjpT5e2w67xqgQ7njy4eKPdn233HlreIfZxLqjcenWdO+XxoI/l 1Ee14lfpEEGlUNkflK5dfrGnNaD1eBKu/tm2IIpEoc0jrELVV5BCGtPAeTRsJsNx4V5x 4HNTisQ6lIYdZvJMkFkIkzz8A0CIVl7UhfKkis+by025pcmsT/Rh6qMeuU4npZnCnQqA ukY8zD3fqaA14g0EE/ZmmTehxbk+sR/i0RPBUwZGgUG6n2gA42fVGWAQCk0pezH8j6OU DbPYtVBut+/M2qHLl6Wj4l3K3FPC3PyhXk7ObRLb39/e8EZKoYFCYC14mqoyrDHTDTeL +rBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686902698; x=1689494698; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YXX6whrEOn2eQbzXPk3jhKKZQgf895lWSSyWx6pgZ/Q=; b=b414W/beeF3m1p6zb+Y1l9ry5wU8wgWBw5+QBbdIW4MN2tGjW4UkjJMycr1mkLbhkM iMtEmNkx5ebUWmapgoqsOZlr7bLFzQjFQ6l6Z9XcVceinicDLgtZW1T9QoqN6YqHfmu1 SG5rywelTI3FIW6+87xMxMhCffpNDumdzOeGDYKG78c/SQwpEaEAQ3RqFtQb0ap08qwz bvfy3TAiQ4wBWhG5kUd3FG7CMyizv5Gflg3p2GFiUXyC4P7uOKkQr2QDOJW43vxnG2GO MVmRIslzmVqiIwS6Vk1O1vqdnJfa+PjwMFVYFJ3TvD5GeTown5/k/KGSZNfB8OHImJo1 PSdg== X-Gm-Message-State: AC+VfDzjgcBJyyvgeBUq1VbjH8OXednNQLE6i9qptz/5g+rAAY8zvYRg 5ao5a63lN6pZLAqI/F56U2BAVqNpCY1yFjYoJqL1u5UcF789J6iGGtg= X-Google-Smtp-Source: ACHHUZ61O0r9MfXSpz4gJIWkwrbq1lgMgPTPILCLWpXVKQYSPgM6Ed4rssqmEJDJ7x8tsVmz/iVjC1DCVxDsEv/jKQ4= X-Received: by 2002:a17:907:7f13:b0:97b:e0bb:8c2f with SMTP id qf19-20020a1709077f1300b0097be0bb8c2fmr976886ejc.4.1686902698171; Fri, 16 Jun 2023 01:04:58 -0700 (PDT) MIME-Version: 1.0 References: <20230615034830.1361853-1-hezhongkun.hzk@bytedance.com> <576b7ba6-4dcd-48c9-3917-4e2a25aaa823@redhat.com> In-Reply-To: <576b7ba6-4dcd-48c9-3917-4e2a25aaa823@redhat.com> From: Yosry Ahmed Date: Fri, 16 Jun 2023 01:04:21 -0700 Message-ID: Subject: Re: [External] Re: [RFC PATCH 1/3] zram: charge the compressed RAM to the page's memcgroup To: David Hildenbrand Cc: =?UTF-8?B?6LS65Lit5Z2k?= , Yu Zhao , minchan@kernel.org, senozhatsky@chromium.org, mhocko@suse.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrea Arcangeli , Fabian Deutsch Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: EB50A140011 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: io6nazniqe98radbej68itxy1ijz8sc8 X-HE-Tag: 1686902699-775218 X-HE-Meta: U2FsdGVkX1+iIZS1wQbv2+S93SQ4liSO+vjW9RF6xj1NcSvZ+FgU32jtfwyc/AJWAXQwImpptMhXCSiDimnYBVNiYtPrVFOq1Zv5/sU96pRfmKo0v3xPmjMc4e2IcYoNrn0ZbIKGX3FEjN6z/nGS/bGW/NODKSMk64AeorcMzPXwZr/m7LL/fkZOoTxgA0T8P+wKRbqAXddGYEjy6GoAOd1e7EtIvghgLA9o8wCUKgBop4i+SBPgw9l95wv3dJ/AN8Le9dsEh3xvzyeCiTGBwZwOO4kdFRvza5d6/8spEnVai+g+hH32fT1MIbhWuedm1fE0E/k2fXGCU3MEV3z1IdL9CxNVyVYUQU9HdFJf0WiAUkESYwU3mk/yRRU4SoTyRbNCJSbQvjgwyuGox50eqskzginuBEYT4NwJAqrGaLvd3zm8+NYhgyZhTzDrjuhJb9DnGJZomZbMBK3e/i7cDVOek/dUX2qy/955fCdWxEPeb3hlzmUpZrXc3TtqVo96mz8wfCY+teJ95Fn0J0GR9d60d8qiDHJalx8cdrBGK828PlH+Up5ak9kPGuZevBgcMNkEQlfXsOoZiUvln5P/vYiyKKwZjpBdOMXb4ZBr40ZFI+V64x+zgE0EgTZLJ8LFPX8zo4MUzDkSwuZhqOR3UMKrTWNPR0nVqWrDqtDNSjLSjGKAvQNifpLjZ1e3sr5C8xsq9ZsmO56LbQ3+ZJimRsVSrR/hnv3781OEhgzJCcinTzLoWjs07u3QRY2/EF7J6ZWgB9C6typJzQa3kFT/Qgkzbn0ZzJnQVuA5jx2bkZf7tp5CNFF++6NakyeCb7ZSuKEbqNbeL8oMLrHOvsyZlcE/lQ7Byf4nlDrrczG9iqscIJaX4L4+tQcDSJGG646tvvOfcS9u+0Z1JknjsDOTLtBpTXvAe96PybYppzDG0KHe5SZpuKUlfZ6xj5ueIAIQEgDvLTYNsWWQcLfy/x1 n57Hy0X4 u+2PhklWMlVeyLAXmulcnFXBgKc0JR5QWp27tKNeSNVesEQXgIlcQLS7dn+jaIubL8r3Vks8rIJm+IizCkaGJi9xyZEC62+U5/3BDYrhNrs5JmaOsTq+jBL69iEX+JhiN3wYlHJOed5OZZSZN8nrt5/h9BqRdh3uKkjAwKlb5m6qQvgUSV89MiSPPG/BJXgvBioNeTuuA970zsXxZBUlwjoHM46fomsqPJkZej6vq8jCheJD3RlPcs+vFGkbe+syeSi9U+LgIYsnjpLS1mmopk4twk6IYZp7L/HIei3GrmDybloG80s+uScdsog== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Jun 16, 2023 at 12:57=E2=80=AFAM David Hildenbrand wrote: > > On 16.06.23 09:37, Yosry Ahmed wrote: > > On Thu, Jun 15, 2023 at 9:41=E2=80=AFPM =E8=B4=BA=E4=B8=AD=E5=9D=A4 wrote: > >> > >>> Thanks Fabian for tagging me. > >>> > >>> I am not familiar with #1, so I will speak to #2. Zhongkun, There are > >>> a few parts that I do not understand -- hopefully you can help me out > >>> here: > >>> > >>> (1) If I understand correctly in this patch we set the active memcg > >>> trying to charge any pages allocated in a zspage to the current memcg= , > >>> yet that zspage will contain multiple compressed object slots, not > >>> just the one used by this memcg. Aren't we overcharging the memcg? > >>> Basically the first memcg that happens to allocate the zspage will pa= y > >>> for all the objects in this zspage, even after it stops using the > >>> zspage completely? > >> > >> It will not overcharge. As you said below, we are not using > >> __GFP_ACCOUNT and charging the compressed slots to the memcgs. > >> > >>> > >>> (2) Patch 3 seems to be charging the compressed slots to the memcgs, > >>> yet this patch is trying to charge the entire zspage. Aren't we doubl= e > >>> charging the zspage? I am guessing this isn't happening because (as > >>> Michal pointed out) we are not using __GFP_ACCOUNT here anyway, so > >>> this patch may be NOP, and the actual charging is coming from patch 3 > >>> only. > >> > >> YES=EF=BC=8C the actual charging is coming from patch 3. This patch ju= st > >> delivers the BIO page's memcg to the current task which is not the > >> consumer. > >> > >>> > >>> (3) Zswap recently implemented per-memcg charging of compressed > >>> objects in a much simpler way. If your main interest is #2 (which is > >>> what I understand from the commit log), it seems like zswap might be > >>> providing this already? Why can't you use zswap? Is it the fact that > >>> zswap requires a backing swapfile? > >> > >> Thanks for your reply and review. Yes, the zswap requires a backing > >> swapfile. The I/O path is very complex, sometimes it will throttle the > >> whole system if some resources are short , so we hope to use zram. > > > > Is the only problem with zswap for you the requirement of a backing swa= pfile? > > > > If yes, I am in the early stages of developing a solution to make > > zswap work without a backing swapfile. This was discussed in LSF/MM > > [1]. Would this make zswap usable in for your use case? > > Out of curiosity, are there any other known pros/cons when using > zswap-without-swap instead of zram? > > I know that zram requires sizing (size of the virtual block device) and > consumes metadata, zswap doesn't. We don't use zram in our data centers so I am not an expert about zram, but off the top of my head there are a few more advantages to zswap: (1) Better memcg support (which this series is attempting to address in zram, although in a much more complicated way). (2) We internally have incompressible memory handling on top of zswap, which is something that we would like to upstream when zswap-without-swap is supported. Basically if a page does not compress well enough to save memory we reject it from zswap and make it unevictable (if there is no backing swapfile). The existence of zswap in the MM layer helps with this. Since zram is a block device from the MM perspective, it's more difficult to do something like this. Incompressible pages just sit in zram AFAICT. (3) Writeback support. If you're running out of memory to store compressed pages you can add a swapfile in runtime and zswap will start writing to it freeing up space to compress more pages. This wouldn't be possible in the same way in zram. Zram supports writing to a backing device but in a more manual way (userspace has to write to an interface to tell zram to write some pages). The disadvantage is that zswap doesn't currently support being used without a swapfile, but once this support exists, I am not sure what other disadvantages zswap would have compared to zram. > > -- > Cheers, > > David / dhildenb >