From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CD47CD3441 for ; Tue, 19 Sep 2023 06:48:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F6CF6B04B1; Tue, 19 Sep 2023 02:48:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7A6DC6B04B2; Tue, 19 Sep 2023 02:48:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6BC596B04B3; Tue, 19 Sep 2023 02:48:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5B7746B04B1 for ; Tue, 19 Sep 2023 02:48:40 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2F71940A2E for ; Tue, 19 Sep 2023 06:48:40 +0000 (UTC) X-FDA: 81252418800.17.F9DA67C Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf16.hostedemail.com (Postfix) with ESMTP id D9B46180017 for ; Tue, 19 Sep 2023 06:48:37 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=g81Npwhj; spf=pass (imf16.hostedemail.com: domain of rppt@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695106118; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xqqBZyYojJCL2gtEa/wPUN8y3uBUQvJrGdJDWeWrwJI=; b=tg2biOwBoq1mnppk1QklmEvUQtYxJsW/GWtYy1HVGA9BwCrjwlEPwvya3QiWlOe4CwAUBm 2Nkwlt8qMzuCjvEnGyaRgvCRF6uCdkPo+qMiYqOMRCRqrR//xBdILfeqq+tfLyxSmDJzkv dewmUD9U390RBdsH05ClSDX1rFTgjeE= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=g81Npwhj; spf=pass (imf16.hostedemail.com: domain of rppt@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695106118; a=rsa-sha256; cv=none; b=zbEkHZwcNbuQVEJ1smffE77JKyZ1P+RbR30mjdri5rRkg38Ub8P0V8q5AbRxYy2R20pIOR Fqxb7o8IxcTPy8aN0w9lMftvGoPZvjk9qeox3eU6JM6yEf2UBYPVHaYEo+WCrUcIWO+H82 RjrLahd96dsVgN3a/m14gVEaYVQPplo= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 08B54CE11A1; Tue, 19 Sep 2023 06:48:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 45C45C433C8; Tue, 19 Sep 2023 06:48:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695106111; bh=v1PBG6Gu/3eqRITlKaenIy1Iaq3gKBzyqYlv0ai4zO8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=g81NpwhjgWC5UNpbUGov4u227vlRjEbZixKM1tjaZpaM/VY6T9DXSa/NEFVhg41PN vnrRJL/9n7SjreVQLTCw1zfG/HO8EMMJtOSa/z1D/CFbU114nEbsr06aQ520n/3u1j g1ERXarMOaeSRRWVrRVjkj3QcghAZT2RhMDgEx8AQqjIbY6048kFxlHKikjCWSyuLM iuv2rfpWoG3+m+AHxfjjzMFu11ynzIQlPTHtjh2xfDgQPFPLFv48dM8n6h4j5U4CaH Pokk/9WjdSFC6CHoDZa/xnP+QGd+sc3B23/ldGMkARS2g+aCBL6v5tzoijY2vKM65J rxr74ifBr+LMw== Date: Tue, 19 Sep 2023 09:47:44 +0300 From: Mike Rapoport To: Baolin Wang Cc: akpm@linux-foundation.org, will@kernel.org, aneesh.kumar@linux.ibm.com, npiggin@gmail.com, peterz@infradead.org, catalin.marinas@arm.com, chenhuacai@kernel.org, tsbogend@alpha.franken.de, dave.hansen@linux.intel.com, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, arnd@arndb.de, willy@infradead.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org Subject: Re: [PATCH] mm: add statistics for PUD level pagetable Message-ID: <20230919064744.GE3303@kernel.org> References: <876c71c03a7e69c17722a690e3225a4f7b172fb2.1695017383.git.baolin.wang@linux.alibaba.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <876c71c03a7e69c17722a690e3225a4f7b172fb2.1695017383.git.baolin.wang@linux.alibaba.com> X-Rspamd-Queue-Id: D9B46180017 X-Rspam-User: X-Stat-Signature: nasazr8gk8ed41ai8tnsacrhpyd9zh9o X-Rspamd-Server: rspam01 X-HE-Tag: 1695106117-492844 X-HE-Meta: U2FsdGVkX18bNG9etw5tt3wCXvFPCQwuLg44cp7pXHCb1dTarytkjxOD1LKftTiN04jfikZu4R/M61owOuHrRrRL0mR86WB0/GozKlvA2KiF0z2ZRcox3rnYYBhngdOWXqIiDoM/B699E7KFyrR2e1JP05RL3f360znD9PUjmxQawpfHxQaMQHopGGUhqKxIvS5E7nFYFY/fj99SuReqeUI9rf27ntZUQKiCXCisRRfDPRZWbSsI3sUjgpCjIOvxWkupVexNx1bXwPZkjBlBuqZh1Xx+pTJsFVSfrAVe77wyc67w2tFjtD3jjTPu+7TWJJE8sKi9Xn9u/2cj7Yh9VHxOdk062RIEw4f61QhVVhyaoMGc6uPODKGwKGxPFtmYMwUeZfTQMV9HpJGsmig7ISH/aybz3B5A/lQpTzlov5JMU8qZ1uyU841n0Vzu2YoFVEoNspoB8JLVPs2nO9IvXBKLvtxCkbHqHe9PosOuM/TZtwrYUoiJldDQrObVhatPQGBKhD4wxPeguqiFH/gdYAz4cX+HX4IqjIdeG37YLMkewEbPx8+XVIIsfLH+Z+2zhfBvJ9SQ1quUs/Lgd0tHMlDQfLOQbfZOJyk5qzjvvXE4hvB1JPvLwtKSX1+74RbChfsUL8OLA8E0Yx6Lb3C2lPECvEsTeVXxS4/5yWFv70znud3hyTvC3BJUU+WhQlCuHCx+aXgPyF6fB6YBfg7WCj0OUmhlwKlv34CFgCC5hQlM217q3xAE1dFtnQ9sO1ztha79JZkd44dIiTDVJ+Yk0YD9qDE5vYcXA5Dm+3fXOTw7u8TzzCFEDaVFROv2f4Uj41g6RnjSl9AvyqhU+WnXfIR5ot51fwqjQ6Z4mj23FVmQFXweRc2DMjd7m3MVnhkXGmYWFUcZ072VO/z2rAahTQIjAAjl4b3GShUMfETkA0wEF1K3qnodYrJCnA/UGMCwpS3nEsbvFlE1UnHXm9l l4nAaOvi YwWmy4K7o9yF35ZAId4Kx2UVTDzEoNTHyDEklxIv/R4N6rgeUJyxcduQGN8gTIm8ih5Qx1hxc+vX5nh3ix0Ga5mckq4mI4D5Besg+qYP8pncegJXe/J206wgclkN06Zpx92A48LdON+MvvColATDPgrwLY5eH2JmpIrxYI/AtvtPV1BBJeLDTdw1efLaXMLFLaWQh3iE5RWFmmYjZpeNmqW3VHXRFlLsZltx+BnKEqBQPjjj+21LMZ1uNMpL9uezxA7+hfCnCyphhRVPt63eldxi1R8ZiUgEzaLBsXN/bOvizT2K9o2P8lE+iorX7IPM1TwDGtqfHrK4kQfjdDC8eiUUD/w04ryouG7FtmLyyQWJAl/K+baz4526m2A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Sep 18, 2023 at 02:31:42PM +0800, Baolin Wang wrote: > Recently, we found that cross-die access to pagetable pages on ARM64 > machines can cause performance fluctuations in our business. Currently, > there are no PMU events available to track this situation on our ARM64 > machines, so an accurate pagetable accounting can help to analyze this > issue, but now the PUD level pagetable accounting is missed. > > So introducing pagetable_pud_ctor/dtor() to help to get an accurate > PUD pagetable accounting, as well as converting the architectures with > using generic PUD pagatable allocation to add corresponding PUD pagetable > accounting. Moreover this patch will also mark the PUD level pagetable > with PG_table flag, which will help to do sanity validation in unpoison_memory(). > > On my testing machine, I can see more pagetables statistics after the patch > with page-types tool: > > Before patch: > flags page-count MB symbolic-flags long-symbolic-flags > 0x0000000004000000 27326 106 __________________________g_________________ pgtable > After patch: > 0x0000000004000000 27541 107 __________________________g_________________ pgtable > > Signed-off-by: Baolin Wang Acked-by: Mike Rapoport (IBM) > --- > arch/arm64/include/asm/tlb.h | 5 ++++- > arch/loongarch/include/asm/pgalloc.h | 1 + > arch/mips/include/asm/pgalloc.h | 1 + > arch/x86/mm/pgtable.c | 3 +++ > include/asm-generic/pgalloc.h | 7 ++++++- > include/linux/mm.h | 16 ++++++++++++++++ > 6 files changed, 31 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h > index 2c29239d05c3..846c563689a8 100644 > --- a/arch/arm64/include/asm/tlb.h > +++ b/arch/arm64/include/asm/tlb.h > @@ -96,7 +96,10 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmdp, > static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pudp, > unsigned long addr) > { > - tlb_remove_ptdesc(tlb, virt_to_ptdesc(pudp)); > + struct ptdesc *ptdesc = virt_to_ptdesc(pudp); > + > + pagetable_pud_dtor(ptdesc); > + tlb_remove_ptdesc(tlb, ptdesc); > } > #endif > > diff --git a/arch/loongarch/include/asm/pgalloc.h b/arch/loongarch/include/asm/pgalloc.h > index 79470f0b4f1d..4e2d6b7ca2ee 100644 > --- a/arch/loongarch/include/asm/pgalloc.h > +++ b/arch/loongarch/include/asm/pgalloc.h > @@ -84,6 +84,7 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long address) > > if (!ptdesc) > return NULL; > + pagetable_pud_ctor(ptdesc); > pud = ptdesc_address(ptdesc); > > pud_init(pud); > diff --git a/arch/mips/include/asm/pgalloc.h b/arch/mips/include/asm/pgalloc.h > index 40e40a7eb94a..f4440edcd8fe 100644 > --- a/arch/mips/include/asm/pgalloc.h > +++ b/arch/mips/include/asm/pgalloc.h > @@ -95,6 +95,7 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long address) > > if (!ptdesc) > return NULL; > + pagetable_pud_ctor(ptdesc); > pud = ptdesc_address(ptdesc); > > pud_init(pud); > diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c > index 9deadf517f14..0cbc1b8e8e3d 100644 > --- a/arch/x86/mm/pgtable.c > +++ b/arch/x86/mm/pgtable.c > @@ -76,6 +76,9 @@ void ___pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd) > #if CONFIG_PGTABLE_LEVELS > 3 > void ___pud_free_tlb(struct mmu_gather *tlb, pud_t *pud) > { > + struct ptdesc *ptdesc = virt_to_ptdesc(pud); > + > + pagetable_pud_dtor(ptdesc); > paravirt_release_pud(__pa(pud) >> PAGE_SHIFT); > paravirt_tlb_remove_table(tlb, virt_to_page(pud)); > } > diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h > index c75d4a753849..879e5f8aa5e9 100644 > --- a/include/asm-generic/pgalloc.h > +++ b/include/asm-generic/pgalloc.h > @@ -169,6 +169,8 @@ static inline pud_t *__pud_alloc_one(struct mm_struct *mm, unsigned long addr) > ptdesc = pagetable_alloc(gfp, 0); > if (!ptdesc) > return NULL; > + > + pagetable_pud_ctor(ptdesc); > return ptdesc_address(ptdesc); > } > > @@ -190,8 +192,11 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr) > > static inline void __pud_free(struct mm_struct *mm, pud_t *pud) > { > + struct ptdesc *ptdesc = virt_to_ptdesc(pud); > + > BUG_ON((unsigned long)pud & (PAGE_SIZE-1)); > - pagetable_free(virt_to_ptdesc(pud)); > + pagetable_pud_dtor(ptdesc); > + pagetable_free(ptdesc); > } > > #ifndef __HAVE_ARCH_PUD_FREE > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 12335de50140..2232bfebb88a 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -3049,6 +3049,22 @@ static inline spinlock_t *pud_lock(struct mm_struct *mm, pud_t *pud) > return ptl; > } > > +static inline void pagetable_pud_ctor(struct ptdesc *ptdesc) > +{ > + struct folio *folio = ptdesc_folio(ptdesc); > + > + __folio_set_pgtable(folio); > + lruvec_stat_add_folio(folio, NR_PAGETABLE); > +} > + > +static inline void pagetable_pud_dtor(struct ptdesc *ptdesc) > +{ > + struct folio *folio = ptdesc_folio(ptdesc); > + > + __folio_clear_pgtable(folio); > + lruvec_stat_sub_folio(folio, NR_PAGETABLE); > +} > + > extern void __init pagecache_init(void); > extern void free_initmem(void); > > -- > 2.39.3 > > -- Sincerely yours, Mike.