From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82378C4332F for ; Wed, 1 Nov 2023 01:41:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D418B6B02A0; Tue, 31 Oct 2023 21:41:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF1A26B02A2; Tue, 31 Oct 2023 21:41:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C08F86B02A3; Tue, 31 Oct 2023 21:41:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id AE25B6B02A0 for ; Tue, 31 Oct 2023 21:41:47 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8DC2A1CB96A for ; Wed, 1 Nov 2023 01:41:47 +0000 (UTC) X-FDA: 81407683854.28.9CC2055 Received: from mail-io1-f52.google.com (mail-io1-f52.google.com [209.85.166.52]) by imf08.hostedemail.com (Postfix) with ESMTP id DD5F6160002 for ; Wed, 1 Nov 2023 01:41:45 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gs9v6rZU; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.166.52 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698802905; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WBro9oe1zZ628ykQDCAf+JGt6uQlYsotXXea9Ml/Ecg=; b=aswtn2kEWaTjdbVMKLHY1PH7ta2yvMEK40ZrxYRSsiV2uf0ZvQVUCt9AoCd41d5EoD5+0d +r5/NULIsld+KnARhN4R6IYqS2AoKZDK3C75IkGXYfW9mmvn3zSurfIyIunYUY32OufGNR kT7LHF0jC8/54pupqxprwkwacC2CmyM= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gs9v6rZU; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.166.52 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698802905; a=rsa-sha256; cv=none; b=tvRqCczn5/31ug0YWb4GHvhTSMLOyimQ+q56OmPvouuv4Zj6lB/QngRQBpjM4QzW3D98iN sSKvtkXJuhj4CorhgSPOfERJGUy9vLaNCVgf5YcMyjuhAQxyPtuBTpN1CALFK+Tn/JPXow RSIz29YTKFuD20pkV5TD3q9l5cLx4AI= Received: by mail-io1-f52.google.com with SMTP id ca18e2360f4ac-7a67b9cd230so226364839f.1 for ; Tue, 31 Oct 2023 18:41:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698802905; x=1699407705; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=WBro9oe1zZ628ykQDCAf+JGt6uQlYsotXXea9Ml/Ecg=; b=gs9v6rZUDYyyCZaFCPcxjOuauedCzfTOD1QhiQmt2Bhlp08JYGj8gMdQz/WlENdi+d 91qYtpDzPZT3zjFWcYIjz1lnFwZPxr1ee9Ek2Bi33AbQgoCPovv/BAwmXvNQ9hMLzKn1 RpI+fAtL1BdFzxU6BYSUj5vgBq8h/k7J4E3b98Z9YNo/7C2uU1xu6J3Glnf3ZKNO5PkB vPHpbL8VarVlX7vKa1jFIworbktf86kgyxBwyj6QKhYWAwyhMr7KExRnTaRZa90Jsg5m 0/GD2fPoN5gfn/SOY/Ggow+SdI6ELEC0JN6nruV0thvk/EAPxu4kZu0OlAB08e3kJ6ZJ WG4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698802905; x=1699407705; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WBro9oe1zZ628ykQDCAf+JGt6uQlYsotXXea9Ml/Ecg=; b=GLIdpxr/9NnpYwbsA+Bs/7lzJHZGCTNKcIyLEfMseW37wXemKzlT39hvzZNCpgmhK6 dX1IQGDknOxkXnamu3eZ+xYcd7fagGxrG+tRiWPAcg4LcV7QGjj9Jvuu9p0vrC8sC9eF lEvGIhbCRHh5K1H0gyd+ROopGtzWAwouASgh5cxzoQUdxq5BlNggLnksYGI4M1Fab5kX G33zqZr0heS5T40iIFK/6T1wRsiWbzWFFs0PIzDVqqJnMAlD6uVVboHldR8JjVKAmMLm betMBEp6thhWoFo3EHTn/MeABx7QtbIxpZ9BHlPV0AiLnCMESBAeO06yAu/3hLmNoQD2 0mag== X-Gm-Message-State: AOJu0YwfRlHuADGwllE8trroUX2o0sep/Q2RaU0jnGsuc288KhEX7M3X niKQE6HsNjm/FZVpQrFOIGbkV+KlwXEkPD61o98= X-Google-Smtp-Source: AGHT+IFcLCrOGmA9ztpEwfKMHuMNJuvYheXneWeTkMPZswyZG+bUfW17ZWq1pqW3GmAZbWmjH6EFqFHzSB+zb1p1iVg= X-Received: by 2002:a05:6602:3c8:b0:7a9:b1c9:4380 with SMTP id g8-20020a05660203c800b007a9b1c94380mr18950530iov.1.1698802904942; Tue, 31 Oct 2023 18:41:44 -0700 (PDT) MIME-Version: 1.0 References: <20231024203302.1920362-3-nphamcs@gmail.com> <20231101012614.186996-1-nphamcs@gmail.com> In-Reply-To: From: Nhat Pham Date: Tue, 31 Oct 2023 18:41:33 -0700 Message-ID: Subject: Re: [PATCH v4 2/5] zswap: make shrinking memcg-aware To: Yosry Ahmed Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, cerasuolodomenico@gmail.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, chrisl@kernel.org, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: DD5F6160002 X-Stat-Signature: aresfgxi6ha5s7mibaz89citc4zbbss8 X-Rspam-User: X-HE-Tag: 1698802905-29946 X-HE-Meta: U2FsdGVkX18vYH0Bl4B0v8iHFFY9mzMyLB9fUqlH8Xyune8M+Ucqf/FC+EsjopXsUumHNjXUQZRUBpPN7GhDzcx2Yz4fEkM7QgvCbwofJv5HxDlC1POOb8lnHeBY7wLVSEuNXN/2Mwvl31wHjhzQExWjmP5nuCFvb4JOA/s1oJ7/OqfpM1JlBLXFzBu8q1qkwfQ3zRbfy+N+31B4V/JHNLTet+fyOaNc0xjaXuAInqICgJP+8TvmKlYPRYq9MSFepGAkZ1mjPpGDtorcrRK+RuW6l4YIdpURQ3vvhX8KJ0/otc4gIjrJ5M2rEExszDpw4LrJAz0q6Rz4fjEsMBYv4SVnXp1RiKZcZ2unOml58p0xHsB8HPqOXEVhuk2Ln/idTetmbAaE+ypw/Sq8Vbam3mMWOFNimYwV1Y538xS+Kb/omN7XpD2gzTzfJVaXK2ROjFEOYZ7yZPU3miMSHciZhoI7GbzBcKS2dvzLOCR0KUsfJjjQsZA7i8cML0scpq2whef3JjQ9Lvv9pjRJzwLmLmr/NZWfHB4k9RyJ6QnivpR/jIOrniPbMzdVB+8utQPIxaCWlH6uREGuyij+fHf5n0peGmTM61ocuAfch3AKKjkvyaaehbnavWAz2/9Qo1AN/CmhApjHBQkOtYdOGLjJLwKnKchCs5cbZSAKyUyr15koO3eCElrXOchmxBJEpb8uBxqZOfRiJSGLkDmIBRjuydpl1TGn1T3QwTdlhh2z4zq+3ElMdHYYDEmZL8ufvvTRGZXGItuR0egxtFt2VQAWeAc5iTmO0nkJXj5hgv7sQkgXFYtTzs4+1Y6Y7fxQS4pKJqAiOkOyV2E9CjnQ9elW5hROKAttEqYC4cg8N0+An+1g274tmcbTNWPUWKAC1CMg3LRSe5GKmx2LTkWlezInWxqnq6ON/BxNHR+52UkOrx/4cUaIlLj+mD23u+nehTZ4ZbFnG6Kz5tC3c2sBYyC MCWU5Gpr vBXVKE1xMx4IWb66EZ2/Lw+OAfTRJDiyKMOXR4KQER4mI7Bzi6YcCAN6cZ6gAU8OMXUbk6o8UdxGS7jjawe2e52dB9/V8XRAesejhWf20VRgQlY7V5rv1fQtmqxu6NhzBTBrFjx1GfYLi7ZCMNeh9AawADMsXx4gKj2AA4Me6ruoyFfYJTK5mKk//odXdvgu/L6vX3ELdMSZGJFHpEVA9O43IlK234LsP2VMgFL1Uhia+3Lu97eprHHphWa+fg+NsRdR0OgW9jNtBbNbXOX4cP15nZtb+QQr3iNxJQq/AmS9wLrcDyTt/QTQsF78asX90pKIoFvQ1lEx8OB/QHEgtp3ClaYzSi39To/czslDwNWgqiNt3ttvuygMx+0mvR5WM3PerBnrQefCwzfk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.010152, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Oct 31, 2023 at 6:33=E2=80=AFPM Yosry Ahmed = wrote: > > On Tue, Oct 31, 2023 at 6:26=E2=80=AFPM Nhat Pham wro= te: > > > > cc-ing Johannes, Roman, Shakeel, Muchun since you all know much more > > about memory controller + list_lru reparenting logic than me. > > > > There seems to be a race between memcg offlining and zswap=E2=80=99s > > cgroup-aware LRU implementation: > > > > CPU0 CPU1 > > zswap_lru_add() mem_cgroup_css_offline() > > get_mem_cgroup_from_objcg() > > memcg_offline_kmem() > > memcg_reparent_objcgs() > > memcg_reparent_list_lrus() > > memcg_reparent_list_lru() > > memcg_reparent_list_lru= _node() > > list_lru_add() > > memcg_list_lru_free() > > > > > > Essentially: on CPU0, zswap gets the memcg from the entry's objcg > > (before the objcgs are reparented). Then it performs list_lru_add() > > after the list_lru entries reparenting (memcg_reparent_list_lru_node()) > > step. If the list_lru of the memcg being offlined has not been freed > > (i.e before the memcg_list_lru_free() call), then the list_lru_add() > > call would succeed - but the list will be freed soon after. The new > > zswap entry as a result will not be subjected to future reclaim > > attempt. IOW, this list_lru_add() call is effectively swallowed. And > > worse, there might be a crash when we invalidate the zswap_entry in the > > future (which will perform a list_lru removal). > > > > Within get_mem_cgroup_from_objcg(), none of the following seem > > sufficient to prevent this race: > > > > 1. Perform the objcg-to-memcg lookup inside a rcu_read_lock() > > section. > > 2. Checking if the memcg is freed yet (with css_tryget()) (what > > we're currently doing in this patch series). > > 3. Checking if the memcg is still online (with css_tryget_online()) > > The memcg can still be offlined down the line. > > > > > > I've discussed this privately with Johannes, and it seems like the > > cleanest solution here is to move the reparenting logic down to release > > stage. That way, when get_mem_cgroup_from_objcg() returns, > > zswap_lru_add() is given an memcg that is reparenting-safe (until we > > drop the obtained reference). > > The objcgs hold refs to the memcg, which are dropped during > reparenting. How can we do reparenting in the release stage, which > IIUC happens after all refs are dropped? Oh I meant just the list_lru reparenting. The list_lru themselves don't hold any ref I believe, right? Then it's safe to perform this at the release stage. (also, I think I might have messed up the encoding for the email above. Let me know if people cannot view it, and I'll resend it :( )