From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7BECC54798 for ; Tue, 27 Feb 2024 16:39:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5EBF4940013; Tue, 27 Feb 2024 11:39:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 59B6194000E; Tue, 27 Feb 2024 11:39:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 43C63940013; Tue, 27 Feb 2024 11:39:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3235694000E for ; Tue, 27 Feb 2024 11:39:11 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 035B1120D60 for ; Tue, 27 Feb 2024 16:39:10 +0000 (UTC) X-FDA: 81838143702.09.FE344EB Received: from mail-yw1-f169.google.com (mail-yw1-f169.google.com [209.85.128.169]) by imf26.hostedemail.com (Postfix) with ESMTP id 63E76140021 for ; Tue, 27 Feb 2024 16:39:09 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CC31ehcI; spf=pass (imf26.hostedemail.com: domain of surenb@google.com designates 209.85.128.169 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709051949; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=z/ds6xBAyNMUyUudVMgCcZQOOdZHJrbf6LFZuMELU1U=; b=2tqSRExaP0YlYvZCeQRZEmz8k407PblBsDwryr86MpIxUDVxD+R3rrylM521H1/jigGU6x /jJVR2VsdHLOK+RO3iGlgwdb+B8gLgU8IVxHkxTPX6M0dtz8iXGnyZJf9M1Me/Y7xIEAwM DyPfmGHMfkEq6Z6aFbXgmMpFBCgibUo= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CC31ehcI; spf=pass (imf26.hostedemail.com: domain of surenb@google.com designates 209.85.128.169 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709051949; a=rsa-sha256; cv=none; b=w+Td+vcwD2hnoSSBWmov/2xO1trzcskjiMYixAYvBC9h4Bfo5UHX9W8JpIdEYi1H1hLpgO H01wufcKcDUECy2kzBQC64QltrnYA4XpZGoif3PH18ecVNaH0vGDvhrG57ANlj4dal+TP9 VmLXlH1G/xNj8gmePK4PHF8zyYjgmCI= Received: by mail-yw1-f169.google.com with SMTP id 00721157ae682-60922e16f6fso12242637b3.3 for ; Tue, 27 Feb 2024 08:39:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709051948; x=1709656748; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=z/ds6xBAyNMUyUudVMgCcZQOOdZHJrbf6LFZuMELU1U=; b=CC31ehcIotR9EogY/fKLFwkWEGsr45hihHkxbaD5eGBHeNQX2OJCs12UU0DOJ1RTYW jX2zlXoyzBog7vZJsfLyKxIIzRYlXek+t4uuWEhPwk1KNb8LDBdQ6o3biuTKt6d+AyQU BAHWm5QWXyvBfUWevOC8iw0uPNTcmvmpz2lSkSzI3EZ2Ir5IsH35ISrUdO5NsZHFBuht UJNoO75PjXzbqRdAB+d5nGgxLsw8zfKDKgtDTTIS2j9TR+gdw/KfqMqn//z5RZg+5qxy a6XiEeZ7ovFqI+sphQoizWFXR5dmM/tce2nELsHA7t6qhGFeX0U7lBYnls+bhCt+/0RM ffOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709051948; x=1709656748; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=z/ds6xBAyNMUyUudVMgCcZQOOdZHJrbf6LFZuMELU1U=; b=ftuWh+UTblYnE5RT/yrpSkNfcVRYKNl577EAGvYGcpRsKdMVN8CNsNR5oySSROmkrL ofpQ3od9QN/ghEY9ubh2vZN4HPqqACyvXEj3DiETwS64nRLsL6ROJGTWSwXneXitlJbm 8YkrNml8P12idEW42FJEdJXCffW+R91deU1SB2W7jB8CcZNQ0ySjc6Uwp+KdGkcwUYJF ggewzfh9lGJYuoYs8Jyuw2T1hi/GoQaAxAEYmt1zySQpuQWzbaFrv8kzkvioSTlj+HsJ Rj2C2WORR8OnYzeTFCgyJvYsLqJL1NHxYpMvTF63IVFwgeeW230Ai8FAIzIt/+AHYo41 GuCg== X-Forwarded-Encrypted: i=1; AJvYcCX8/Yu/B8kq5mQQ8VjgaU/piL6dOj/Hi9kSPNF5OIglOV8UylW3SXkCoznYoR/hsWAyeUcf1PHv2rpMwl7KNU7lkhg= X-Gm-Message-State: AOJu0YzjV0FIM5eqfMnhCZimOTGrlAFdlyREE7M/EQaDKK+U31FzrAoH EsoXkm3S+Lal82VUc+5nB7hfLGl092TO0gdQv0d7TcqIpTv/vYohUdxa528YJ2bcdkSGlD3wtMp ODvLlyNC8cUrq/mL9jQ/RcLH3dAqeeWxeYDn0 X-Google-Smtp-Source: AGHT+IGYsfj1hpFG2qZ+lv3Tc2gaVahH2B+pghzlhO4uZB2pAxOqLV3b7HUJfJpPshrzze8w4TdCoPFLjSNMBWpeWIM= X-Received: by 2002:a0d:cc52:0:b0:609:2c38:4dd2 with SMTP id o79-20020a0dcc52000000b006092c384dd2mr1712145ywd.42.1709051948102; Tue, 27 Feb 2024 08:39:08 -0800 (PST) MIME-Version: 1.0 References: <20240221194052.927623-1-surenb@google.com> <20240221194052.927623-20-surenb@google.com> <2daf5f5a-401a-4ef7-8193-6dca4c064ea0@suse.cz> In-Reply-To: <2daf5f5a-401a-4ef7-8193-6dca4c064ea0@suse.cz> From: Suren Baghdasaryan Date: Tue, 27 Feb 2024 08:38:54 -0800 Message-ID: Subject: Re: [PATCH v4 19/36] mm: create new codetag references during page splitting To: Vlastimil Babka Cc: akpm@linux-foundation.org, kent.overstreet@linux.dev, mhocko@suse.com, hannes@cmpxchg.org, roman.gushchin@linux.dev, mgorman@suse.de, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, penguin-kernel@i-love.sakura.ne.jp, corbet@lwn.net, void@manifault.com, peterz@infradead.org, juri.lelli@redhat.com, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, x86@kernel.org, peterx@redhat.com, david@redhat.com, axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, dennis@kernel.org, tj@kernel.org, muchun.song@linux.dev, rppt@kernel.org, paulmck@kernel.org, pasha.tatashin@soleen.com, yosryahmed@google.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, andreyknvl@gmail.com, keescook@chromium.org, ndesaulniers@google.com, vvvvvv@google.com, gregkh@linuxfoundation.org, ebiggers@google.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, shakeelb@google.com, songmuchun@bytedance.com, jbaron@akamai.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, kernel-team@android.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 63E76140021 X-Rspam-User: X-Stat-Signature: j9bbmnrczyrh9ute1ktnzem68i83jdu3 X-Rspamd-Server: rspam01 X-HE-Tag: 1709051949-223374 X-HE-Meta: U2FsdGVkX1/WV5nx64yc/9y9JbF6Y5X8A6OKZ8Jm2qVRePtAXHgLZ6H4lkanNyQtjE5Y4b4/pAmCZmze8MBPVcZKuQ0rqQO5LsPwiTSGfccD7/8JBQzR9A4Hiork1P7qomD822wEcdunxCKru+eBScUlHkHj1GhqSlFNdWedo0zXw+0Bd6QFlrNPx75FeHdx+XUTmtaYflXiNOT6YZlAu5XQmNwcar1ktzYWZzwnfWWtcRCSOu2YZly1q51KzF5UCzPzCxo+Y+xmkHe2spOEAr//RZlRiosvHxi+f2VG+78SVIGjKLLzgFnQZDGm9Mb2rYBoLdymRB4Ri9Bz9tHvdf5ZqDwJ85um10kNdpVMFR1gg9CwdZ5g8mSOwKH1rmHF9mZhKAtXNTU7ST6z8l4Z8hCRnIjt653f1/PHwOSbrt3aP0tkAh95ZegV46pSNjIVbr0e8TmBBa1d5bbd1VJL8NwVib1twE3kxqz+Cbv3SceqjiPwoAU4uYkHKlU/UU+KXX5SON1QQl3IRs50FRsjeoRgmd6S9tDyecviFE+k80IhWT1v+fmOLBI61mAtpL7KU5p6XFby9+fGCzgctd0D2616SEnkbto7Vbsv2sBVEs1rruuDvGJvqI5lN6TkIi5uXnK5y9p3gOFUuPcgvlHriBD3hZay2snmM9c85rQDkq9ieaDoN5SMrZLCleTdXg11YFRbyHXRU3qFmeueq5PYcnGawGTBU0hg5vd6o+n/JvhUvwJWCrHBIxzr1Fl4jcHWBig1dBqEwTfDn+IpbBSQG8jP5DFeCCX4egbWDYSe00zQAxYPuFCm9plkpnJHAapREd1iL/7fJNSap5zBp8vmrL0ElCZShQptUvIjTnjKC/xpm5Afz3pjB8lSHBgKyuL//d+e4lcE1dEEABRDNQp+e+yoUWR1CoL/voy34V5rPXkGjeKjbvKoCAnJkAN2HQiL0nEU7a373ZXruCbN3XA FnOh44FF M02f4U6DkyuN5Y+GDFupncvpFn08lew2W9EUiWLeEjHfLWMZVl2MlrP9fb6zFeny2/20d X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Feb 27, 2024 at 2:10=E2=80=AFAM Vlastimil Babka wr= ote: > > On 2/21/24 20:40, Suren Baghdasaryan wrote: > > When a high-order page is split into smaller ones, each newly split > > page should get its codetag. The original codetag is reused for these > > pages but it's recorded as 0-byte allocation because original codetag > > already accounts for the original high-order allocated page. > > This was v3 but then you refactored (for the better) so the commit log > could reflect it? Yes, technically mechnism didn't change but I should word it better. Smth like this: When a high-order page is split into smaller ones, each newly split page should get its codetag. After the split each split page will be referencing the original codetag. The codetag's "bytes" counter remains the same because the amount of allocated memory has not changed, however the "calls" counter gets increased to keep the counter correct when these individual pages get freed. > > > Signed-off-by: Suren Baghdasaryan > > I was going to R-b, but now I recalled the trickiness of > __free_pages() for non-compound pages if it loses the race to a > speculative reference. Will the codetag handling work fine there? I think so. Each non-compoud page has its individual reference to its codetag and will decrement it whenever the page is freed. IIUC the logic in __free_pages(), when it loses race to a speculative reference it will free all pages except for the first one and the first one will be freed when the last put_page() happens. If prior to this all these pages were split from one page then all of them will have their own reference which points to the same codetag. Every time one of these pages are freed that codetag's "bytes" and "calls" counters will be decremented. I think accounting will work correctly irrespective of where these pages are freed, in __free_pages() or by put_page(). > > > --- > > include/linux/pgalloc_tag.h | 30 ++++++++++++++++++++++++++++++ > > mm/huge_memory.c | 2 ++ > > mm/page_alloc.c | 2 ++ > > 3 files changed, 34 insertions(+) > > > > diff --git a/include/linux/pgalloc_tag.h b/include/linux/pgalloc_tag.h > > index b49ab955300f..9e6ad8e0e4aa 100644 > > --- a/include/linux/pgalloc_tag.h > > +++ b/include/linux/pgalloc_tag.h > > @@ -67,11 +67,41 @@ static inline void pgalloc_tag_sub(struct page *pag= e, unsigned int order) > > } > > } > > > > +static inline void pgalloc_tag_split(struct page *page, unsigned int n= r) > > +{ > > + int i; > > + struct page_ext *page_ext; > > + union codetag_ref *ref; > > + struct alloc_tag *tag; > > + > > + if (!mem_alloc_profiling_enabled()) > > + return; > > + > > + page_ext =3D page_ext_get(page); > > + if (unlikely(!page_ext)) > > + return; > > + > > + ref =3D codetag_ref_from_page_ext(page_ext); > > + if (!ref->ct) > > + goto out; > > + > > + tag =3D ct_to_alloc_tag(ref->ct); > > + page_ext =3D page_ext_next(page_ext); > > + for (i =3D 1; i < nr; i++) { > > + /* Set new reference to point to the original tag */ > > + alloc_tag_ref_set(codetag_ref_from_page_ext(page_ext), ta= g); > > + page_ext =3D page_ext_next(page_ext); > > + } > > +out: > > + page_ext_put(page_ext); > > +} > > + > > #else /* CONFIG_MEM_ALLOC_PROFILING */ > > > > static inline void pgalloc_tag_add(struct page *page, struct task_stru= ct *task, > > unsigned int order) {} > > static inline void pgalloc_tag_sub(struct page *page, unsigned int ord= er) {} > > +static inline void pgalloc_tag_split(struct page *page, unsigned int n= r) {} > > > > #endif /* CONFIG_MEM_ALLOC_PROFILING */ > > > > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > > index 94c958f7ebb5..86daae671319 100644 > > --- a/mm/huge_memory.c > > +++ b/mm/huge_memory.c > > @@ -38,6 +38,7 @@ > > #include > > #include > > #include > > +#include > > > > #include > > #include > > @@ -2899,6 +2900,7 @@ static void __split_huge_page(struct page *page, = struct list_head *list, > > /* Caller disabled irqs, so they are still disabled here */ > > > > split_page_owner(head, nr); > > + pgalloc_tag_split(head, nr); > > > > /* See comment in __split_huge_page_tail() */ > > if (PageAnon(head)) { > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > > index 58c0e8b948a4..4bc5b4720fee 100644 > > --- a/mm/page_alloc.c > > +++ b/mm/page_alloc.c > > @@ -2621,6 +2621,7 @@ void split_page(struct page *page, unsigned int o= rder) > > for (i =3D 1; i < (1 << order); i++) > > set_page_refcounted(page + i); > > split_page_owner(page, 1 << order); > > + pgalloc_tag_split(page, 1 << order); > > split_page_memcg(page, 1 << order); > > } > > EXPORT_SYMBOL_GPL(split_page); > > @@ -4806,6 +4807,7 @@ static void *make_alloc_exact(unsigned long addr,= unsigned int order, > > struct page *last =3D page + nr; > > > > split_page_owner(page, 1 << order); > > + pgalloc_tag_split(page, 1 << order); > > split_page_memcg(page, 1 << order); > > while (page < --last) > > set_page_refcounted(last);