From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31033E8FDC1 for ; Wed, 4 Oct 2023 00:02:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9F63F8D00A3; Tue, 3 Oct 2023 20:02:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9A60D8D0003; Tue, 3 Oct 2023 20:02:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 846C98D00A3; Tue, 3 Oct 2023 20:02:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 74F868D0003 for ; Tue, 3 Oct 2023 20:02:41 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E36E81204C5 for ; Wed, 4 Oct 2023 00:02:40 +0000 (UTC) X-FDA: 81305827680.22.851F273 Received: from mail-io1-f50.google.com (mail-io1-f50.google.com [209.85.166.50]) by imf15.hostedemail.com (Postfix) with ESMTP id E24A5A0019 for ; Wed, 4 Oct 2023 00:02:38 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=eT9pK0L2; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf15.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.166.50 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696377758; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=i+o9h8Nzc+9aYQKkbD61EGaTMw7bMaNO2j+MzlluneE=; b=RY3xLWP9YB+7p4D1+N0foNAehSRZfDc09VyW7ckKW2QEzK6Vm1rNchg9PM5OaHQghxRY2U PxqVQ4192HwmcUMZuBt13vr+tGS2I95+WuHD54pqil0whZDJSkd/MA0Bg19j0p+TvmMzoz 43wvPFkXynP8NZCG2IVyy3XL9/Bt3Ow= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=eT9pK0L2; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf15.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.166.50 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696377758; a=rsa-sha256; cv=none; b=f+m1eVyblaQs3GELIn6nsSqJZ2o2cEQsJfsm7Q0l2OftBTYABRyh1g3iwWxB8JmROMdC6V HLl9AE1yL0kx3GvJZ9P39F/MLrwsnnFsrAi6WUag5M9X1hLUzUOEg3On9M4ujVE+JelyNn Px72NOzU+Vz+SvfNLJ2IKQhMV7P9btA= Received: by mail-io1-f50.google.com with SMTP id ca18e2360f4ac-7a2a3fb0713so58177739f.1 for ; Tue, 03 Oct 2023 17:02:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696377758; x=1696982558; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=i+o9h8Nzc+9aYQKkbD61EGaTMw7bMaNO2j+MzlluneE=; b=eT9pK0L2SHLAZn4R+XrtytdMBCEt3PW0up6FWJRriDm1DNOxCoWacGQ3vg7/jQ3kQe OzhKkVoi+OAS047FjW8p7vjC7ztyuloVbss7uPJOe9NP/cPNNaVNW2ueszZt2N4wpbhr tOEIfPp7he8V+Opc/i2jfnvzC34pzkbbk6UxQVB/0vGEe9/OeWfajHkEaTZ8oVeV8EfH GKa+H5srksz7cSH9hx+GaQyzBgPHbsxjz+1/zlAQ5h0oiqhLr7XuYFg/dCQedfHBO8kQ edamWppUjlaQ9lLkt8j008rfIUwzAVzBXSDhD6JRQEo3kBC6X8DN1iYGAuVlCXRwGdrO AhYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696377758; x=1696982558; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i+o9h8Nzc+9aYQKkbD61EGaTMw7bMaNO2j+MzlluneE=; b=TJhCG5s99FNGBCj4Ca9qPuRvyB9KqrZNdxleTY0uzUFKAYRkkzpVyB9tnhzzuRhtgD M8TTWumM7UBCW9XCVWafbr27+rnHgQkC7XXj2LF3LVR4w64fNX8Ah+eBANdHPT9jRF18 NWSYMXazFHqq2sZoC2In7f0iYiXnQg3OilAzQqc+dvnnkei1JNYK+y1EjdJ0cPA23LH5 Z6YCblSze1/MHGxEcd3/fzD2jQXem/Pl74ua5uvF30Mto12jv/3EoGx47n+2EbQl/EOr lAUM2l3KcZOPAc+9wg6erGAYgK0uLMdHsoVV67ksZgAOucM1XgXWLbGPgSm0GN9Ktpdn MfCw== X-Gm-Message-State: AOJu0Yx3aELTSv1RLgtoTqA4lNUYVb0nf9J9DE5eLjSYEq4Nvl82ep3J YTwjnDLL5AeY+rXMr673D4X2ogI1XdrhfF54ySY= X-Google-Smtp-Source: AGHT+IEwIM9/xlWr0clvgcTRvKO0iWCf/2pYa5PPpj4WehHR2oXTWpXlcqux/Fdf+BTYuo92qGRtAy39M7RVe909bjM= X-Received: by 2002:a5d:8713:0:b0:792:9a3e:2dd with SMTP id u19-20020a5d8713000000b007929a3e02ddmr1080957iom.3.1696377757928; Tue, 03 Oct 2023 17:02:37 -0700 (PDT) MIME-Version: 1.0 References: <20231003171329.GB314430@monkey> <20231003231422.4046187-1-nphamcs@gmail.com> In-Reply-To: <20231003231422.4046187-1-nphamcs@gmail.com> From: Nhat Pham Date: Tue, 3 Oct 2023 17:02:24 -0700 Message-ID: Subject: Re: [PATCH] memcontrol: only transfer the memcg data for migration To: akpm@linux-foundation.org Cc: riel@surriel.com, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, tj@kernel.org, lizefan.x@bytedance.com, shuah@kernel.org, mike.kravetz@oracle.com, yosryahmed@google.com, fvdl@google.com, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: E24A5A0019 X-Stat-Signature: kfkzj8xsq8841pmf461fszy69f166p5s X-HE-Tag: 1696377758-660095 X-HE-Meta: U2FsdGVkX1+Sl6ZEHTOVKo0Ux7ZYPMSDgWJcmYw9rjdOvpFh9/Z67BplDhdvGkHuRlUqHq/g1m4zjr9AWxZqjmv756cFNK3lLukqAu9aFjLqu58+IW0IXXoBqp/jHaqCVK1LdHoE998TQVWFegQE39GKnhJySlwsETXJjO8wE5fklS6OyCkZvAzGf9BTjvZy26vIcWgF6rxK35g6uLQYXVHo3fHxog4P4SASaWM3Ia64xClEujh6RucmDkOD8zHwncCFh5M/1INt5h+waRrCff5RRIGdvlHgugWPN9NaTnNtYZf0602Lp5Tsoz07T0TCN8dZpYxw9rZYF/1mzOw/Lw0R8SBVEmzed2kB9TGgCFPRmRPYW7YX/jQQxSzZJe38PZrzHQGTVvZUyQ2aAkkNreHH8mT++EHFOMVU0uyv3x6OLxLqmeND2N1p7EMmmgC/hXKzy05uIUOBGF7tRY7vla2/kZ3lSmzq6yTrwnO0VUNcIhdtLFxyvHAIALWQalmpN/Ms1rIrw58+HtPMV9CaYhN4MyBAn4bEM/JIHePYJEdWa68cWexDpL0PIu5vEg5CyF++Bo/mmM0wy/1kyDHcEdM2oiKd6Wslrx7OleKK3U8BBXKzd0DdSRtr0zZ0Pv5hMgBQl0BQpLvxrkMfwvf0YXA/qSITslh9AA0eUsSNmcPcUDWt9UkPfEBKZsMdVmApxmeRp0Rj1FnhWoTi1r6Yjjhaqt4zpl0JBKKynSybVgmCZCkj2GPUp/5rxhiKYhIaeX4QT0y0ypMcUFknP0xjAF94nuE/laZtOf/dWvyVhh9NjIiuT1vF5JUOFBgvcCK4BxOy4s3/puZtXNpRt7h46Eb8L/RAi+1ZD8sKAHqACL+w+wi9i2pxS7OOaOAJEyKP0gEi01EGayIiPq+JgpQdRtxX2WhC3K96dMGkEXmKHwb8nsTDVf30/wxp6BI8a3IbAQEWkcs5JxexATFGttg wGS9tnUR 8hvbCGTOTP6Yd4BIOnsl+7CTcgGvS6yx8p5WRi1oQv1tr9/8eDVcygX2DoF6PATNPKAZKpfHaGYMOHm84nuLjpvx08ROe185Ue14drJgykmwLGz2dV9mON2DNaunCT7AA4OeuAxH+wx+sU4rO+/I/eFsuch0Ll/CJQour1gWzP77S/oYOhmRq21A2/SBVkXhMurHzOdJfVab384GtDJyo7G+nVgHZURJlXOKtux6gGIfmx4v0m9OezKJBcjm1r8o9KmWMk30aHHQJZBbIsg0aDMqh9JllnfvLnXrUnLfHBpSboTWcImyewrRLM8/phSZgIuzh+9aIeWoL040/2+j5I9Vb6nmzlIXI2ccvXxVoAzc71ExCB60opJgRGYXWXK2joNVXYKq3NmfubiaSYjfkMBUPUdOpBY9mc7gQBpBGNGn1Gl5HsTkGgpGbSD4OE45eXvNKZ7Uw+1Ke/hVFzFZRvpYcIe6HXA50aBU0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Oct 3, 2023 at 4:14=E2=80=AFPM Nhat Pham wrote: > > For most migration use cases, only transfer the memcg data from the old > folio to the new folio, and clear the old folio's memcg data. No > charging and uncharging will be done. These use cases include the new > hugetlb memcg accounting behavior (which was not previously handled). > > This shaves off some work on the migration path, and avoids the > temporary double charging of a folio during its migration. > > The only exception is replace_page_cache_folio(), which will use the old > mem_cgroup_migrate() (now renamed to mem_cgroup_replace_folio). In that > context, the isolation of the old page isn't quite as thorough as with > migration, so we cannot use our new implementation directly. > > This patch is the result of the following discussion on the new hugetlb > memcg accounting behavior: > > https://lore.kernel.org/lkml/20231003171329.GB314430@monkey/ For Andrew: This fixes the following patch (which has not been merged to stable): https://lore.kernel.org/lkml/20231003001828.2554080-3-nphamcs@gmail.com/ and should ideally be squashed to it. My apologies for not making it clear in the changelog. Let me know if you'd like to see a new version instead of this fixlet. > > Reported-by: Mike Kravetz > Closes: https://lore.kernel.org/lkml/20231003171329.GB314430@monkey/ > Suggested-by: Johannes Weiner > Signed-off-by: Nhat Pham > --- > include/linux/memcontrol.h | 7 ++++++ > mm/filemap.c | 2 +- > mm/memcontrol.c | 45 +++++++++++++++++++++++++++++++++++--- > mm/migrate.c | 3 +-- > 4 files changed, 51 insertions(+), 6 deletions(-) > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > index a827e2129790..e3eaa123256b 100644 > --- a/include/linux/memcontrol.h > +++ b/include/linux/memcontrol.h > @@ -711,6 +711,8 @@ static inline void mem_cgroup_uncharge_list(struct li= st_head *page_list) > > void mem_cgroup_cancel_charge(struct mem_cgroup *memcg, unsigned int nr_= pages); > > +void mem_cgroup_replace_folio(struct folio *old, struct folio *new); > + > void mem_cgroup_migrate(struct folio *old, struct folio *new); > > /** > @@ -1294,6 +1296,11 @@ static inline void mem_cgroup_cancel_charge(struct= mem_cgroup *memcg, > { > } > > +static inline void mem_cgroup_replace_folio(struct folio *old, > + struct folio *new) > +{ > +} > + > static inline void mem_cgroup_migrate(struct folio *old, struct folio *n= ew) > { > } > diff --git a/mm/filemap.c b/mm/filemap.c > index 9481ffaf24e6..673745219c82 100644 > --- a/mm/filemap.c > +++ b/mm/filemap.c > @@ -819,7 +819,7 @@ void replace_page_cache_folio(struct folio *old, stru= ct folio *new) > new->mapping =3D mapping; > new->index =3D offset; > > - mem_cgroup_migrate(old, new); > + mem_cgroup_replace_folio(old, new); > > xas_lock_irq(&xas); > xas_store(&xas, new); > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index 6660684f6f97..cbaa26605b3d 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -7316,16 +7316,17 @@ void __mem_cgroup_uncharge_list(struct list_head = *page_list) > } > > /** > - * mem_cgroup_migrate - Charge a folio's replacement. > + * mem_cgroup_replace_folio - Charge a folio's replacement. > * @old: Currently circulating folio. > * @new: Replacement folio. > * > * Charge @new as a replacement folio for @old. @old will > - * be uncharged upon free. > + * be uncharged upon free. This is only used by the page cache > + * (in replace_page_cache_folio()). > * > * Both folios must be locked, @new->mapping must be set up. > */ > -void mem_cgroup_migrate(struct folio *old, struct folio *new) > +void mem_cgroup_replace_folio(struct folio *old, struct folio *new) > { > struct mem_cgroup *memcg; > long nr_pages =3D folio_nr_pages(new); > @@ -7364,6 +7365,44 @@ void mem_cgroup_migrate(struct folio *old, struct = folio *new) > local_irq_restore(flags); > } > > +/** > + * mem_cgroup_migrate - Transfer the memcg data from the old to the new = folio. > + * @old: Currently circulating folio. > + * @new: Replacement folio. > + * > + * Transfer the memcg data from the old folio to the new folio for migra= tion. > + * The old folio's data info will be cleared. Note that the memory count= ers > + * will remain unchanged throughout the process. > + * > + * Both folios must be locked, @new->mapping must be set up. > + */ > +void mem_cgroup_migrate(struct folio *old, struct folio *new) > +{ > + struct mem_cgroup *memcg; > + > + VM_BUG_ON_FOLIO(!folio_test_locked(old), old); > + VM_BUG_ON_FOLIO(!folio_test_locked(new), new); > + VM_BUG_ON_FOLIO(folio_test_anon(old) !=3D folio_test_anon(new), n= ew); > + VM_BUG_ON_FOLIO(folio_nr_pages(old) !=3D folio_nr_pages(new), new= ); > + > + if (mem_cgroup_disabled()) > + return; > + > + memcg =3D folio_memcg(old); > + /* > + * Note that it is normal to see !memcg for a hugetlb folio. > + * It could have been allocated when memory_hugetlb_accounting wa= s not > + * selected, for e.g. > + */ > + VM_WARN_ON_ONCE_FOLIO(!memcg, old); > + if (!memcg) > + return; > + > + /* Transfer the charge and the css ref */ > + commit_charge(new, memcg); > + old->memcg_data =3D 0; > +} > + > DEFINE_STATIC_KEY_FALSE(memcg_sockets_enabled_key); > EXPORT_SYMBOL(memcg_sockets_enabled_key); > > diff --git a/mm/migrate.c b/mm/migrate.c > index 7d1804c4a5d9..6034c7ed1d65 100644 > --- a/mm/migrate.c > +++ b/mm/migrate.c > @@ -633,8 +633,7 @@ void folio_migrate_flags(struct folio *newfolio, stru= ct folio *folio) > > folio_copy_owner(newfolio, folio); > > - if (!folio_test_hugetlb(folio)) > - mem_cgroup_migrate(folio, newfolio); > + mem_cgroup_migrate(folio, newfolio); > } > EXPORT_SYMBOL(folio_migrate_flags); > > -- > 2.34.1 >