From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50987C4167B for ; Fri, 15 Dec 2023 19:44:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 90DAB8D013A; Fri, 15 Dec 2023 14:44:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 895748D0121; Fri, 15 Dec 2023 14:44:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70FFA8D013A; Fri, 15 Dec 2023 14:44:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 598178D0121 for ; Fri, 15 Dec 2023 14:44:54 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2D21DC0599 for ; Fri, 15 Dec 2023 19:44:54 +0000 (UTC) X-FDA: 81570080508.10.8FD0ADA Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf14.hostedemail.com (Postfix) with ESMTP id 64E9C100007 for ; Fri, 15 Dec 2023 19:44:51 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702669491; a=rsa-sha256; cv=none; b=UeWM/GMyoLKc06g2/4GfDrmXCLZEitz6wIHw/UmFoTU73u7l8a4V4l/cQApLWC2Am7+zcY ubQvJczdHMYrEvppStuyXZqTn2Jl74tZvfBXsBnNQ+haUJpiUXtiYQPEuQp5RbL1fhrtT/ 2wTZBBDEmVZ7RLWSq77Au3LPOYgLNks= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702669491; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZBLOCWW+p47TPAsts5mlWpMOwmnCWBhV6p/5iDTjzTw=; b=EY5/Jhw0V7+a9BUuMeTjO9ess73cEHbY8U9QnkaRtXjayKKmPYER4dp1m4YanYkVFkqWaQ 5UAs4CYR8YmDXq4FlGuoj69NvUaHAF5nlqSndjZeV8rhJkv9K8AwmPEzFcatvJW2lBwiNd 13Q/RguD9jcvxL+9TOFTIfmJEqVZegU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 68D8D1042; Fri, 15 Dec 2023 11:45:35 -0800 (PST) Received: from [10.1.196.40] (e121345-lin.cambridge.arm.com [10.1.196.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 31CFD3F738; Fri, 15 Dec 2023 11:44:49 -0800 (PST) Message-ID: <1a8ef5e3-e478-492f-8f4d-05cc16945593@arm.com> Date: Fri, 15 Dec 2023 19:44:47 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] ARM64: Implement arch_report_meminfo() To: "Christoph Lameter (Ampere)" Cc: Catalin Marinas , Marc Zyngier , Will Deacon , Ryan Roberts , Mark Rutland , Vishal Moola , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org References: <79f9a0ad-80cc-1fc5-2d48-cc5ba82bd844@gentwo.org> Content-Language: en-GB From: Robin Murphy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 64E9C100007 X-Stat-Signature: xmcxf49wze3cxxt397jtcc8dwdfq15ak X-HE-Tag: 1702669491-917506 X-HE-Meta: U2FsdGVkX1+Gmy8ItVeKeTaoAxa00YwUilQzO00/FrSLi0a/lDD+9JkctkpFCyu8DP7LaskMZFq7QtGbZHK6fb6rZ3+B8VYFaYITh4S/ZZfwCqkD8ZN24lgPjGL3cj8XiVvRkAtvAyKypwnNktrTWCpHmTiYtoqRnhG7IGhzwGOYPDakra4YRTQBBI6bLxlYoppb0EuwGIqF3UFFe5JEzuiZruNjMiijyFWmLgxLHhxYb2Fno5zsps3uo3Mq4lN+MN1zN5TJwUvnMEkpLUncOb5aodhc4KVfQX8GVbhymifXFwC2OJTG4h2Oae/RWdFI6dupJyrc4Dog5MEZiw6iGFcoPOFvmXHzVKwKZRTP1R0wYgyZZKmh+jRUQbC+nHIYPeBRoCtVtVlMk4PNT/AQQmbOZK6yxCAgJYUuQv6O7nBP2zRufZNp8CMuL2WkpWhwYWsLR4imnFxnmmFHaNfhMTw3sZ5h29YytqDcbGMuUa6er+2dwR0MXOAsZ9+nkgvUNqcSlG1NN45NQCHB5yzq/z16W4AM1Op6QbCd84ovWeur0eZdjjiHLsoXB5SgOfKnDMoBSBOsZZXUy+SfnfNBZbQuV8Yyw0zfTVoFxoas7iw/t8Y/ElWyY0knKwo2gL6HyMLWv99FTZP2ZpTRaij+MGiOLdANIObS9ylenjF3jREsdi1LoC+FhIs0c6SP6yeD7Z74Vz8nV9/Or2aGHpnZkml1XrWVPoNRVVugRfjMwVFS0FjJ2SGOf1/pIhBvf3uO5xNRJU6a415tND6pjC8uY24dCGraq6WLKRldkv4E1K1d7DiJ7lkO5UJ+b/aSl4p93CTAmYr8EdnbXs62NZbica8CgQwLmOmUhOIdTIe/er+JXBBg7DPa9+VQax3pAtuwyJGm/kj0+97ODC7xcBOZ/0uXHe3fVVzgGrKkbQPAj34xv/LTWWRHFB62ljF09AAqxFDsWFcvReSmqQFcJk/ vx2YhVlc NUiEH9iTdy8iYfTGhJjiXQTIfZZjQp0SM6Hu2QhKxMdjFyM0WTEaUe5GnBhxQWKPpKR/e0ZxDLTB2kX5RCcDGd1DrNdfW6P9kFyQ4VT32ytJeNlbYL3xycWOa1pe2LTpw3VRsSPN71YspzgG5UZICR1x3Qrh2fCeWQpmLomc15L3fUAOiTMt0PPWtxufuv/9YeruWCBRWk7vGrGQ3eXROWMa8BiY5r0JUs4CrqFMXtTNt6OcMUPiu+siB4Gvrn46iEc8Ofvzvt1zY4wPX9DZJglMZ+l8FjU/pgXzs7Vgpp72H2pj54BoOVisHYhK5Bwf5b3ySjwZ0zXQAomcZ7Ss+cn7e7505ol3qY00ckWTYgc+CLXzz02icdnNPcpO/f3GsYfyUr184hTcTAwYyxtyOUu49SweuroYc4pHFTVymECNs5bEm/zZHMdh/loK7JMrZwJusuVLro7wD9RwAUYMqy2okp5gjqhKoh1jbC1bkX34obQgndwTac9eqzEPc5ZsmM2uYcTz1kGWKTuLZhxUrbk7EPGevFZqV+6NwqhHpCNVd3i1ITjwgMYAaW6hXsNZnawj6EQj8o0+RPTA+PiYTE5UZCvhcxM645gjyTAqfyJ3gnQ2JRctFcoxXNElfZcI6a7yeGpsjrwNjJys= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 14/12/2023 9:35 pm, Christoph Lameter (Ampere) wrote: > On Thu, 14 Dec 2023, Robin Murphy wrote: > >> It seems somewhat suspect that these counts only ever increase. It's >> not often that we change or remove parts of the linear map, but it >> certainly can happen. > > Well yes in the case of hotplug I guess ... Ok here is V2 There are also paths where we remove and reinstate parts of the linear map via set_memory_valid(), set_direct_map_*(), and possibly others. If we're exposing a user ABI that claims to be accounting kernel VA mappings, then I think users are within their rights to expect it to actually account kernel VA mappings, not just expose numbers whose only guaranteed significance is whether they are zero or nonzero. Looking again, am I also right in thinking that what I assumed were the non-contiguous counts here are actually total counts of *either* type of mapping at that level, and inclusive of the contiguous counts? If so, that seems a bit non-obvious - my intuitive expectation would be for the sum of all these numbers to represent the total amount of direct-mapped RAM, where either we're intersted in each distinct type of mapping and accounting them all separately, or we're simply interested in the general shape of the pagetables, and thus would account per-level and ignore the contiguous bit since we don't know whether it's actually doing anything useful anyweay. Thanks, Robin. > From cl@gentwo.org Fri Dec  8 13:11:58 2023 > From: "Christoph Lameter (Ampere)" > To: Catalin Marinas , Marc Zyngier > , Will Deacon , Ryan Roberts > , Mark Rutland , Vishal > Moola > Cc: linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org > Subject: [PATCH] ARM64: Implement arch_report_meminfo() > > V1->V2 hotplug and umapping support > > X86 has information in /proc/meminfo showing the use of large mappings > for the kernel direct map. This has now also become important for > ARM since the kernel default CONFIG_RODATA_FULL_DEFAULT_ENABLED > forces 4K PTE use for the direct map and users may not be aware > of the performance impact of the increased TLB use etc. > > The output of /proc/meminfo on ARM64 is then after this patch: > > 4K page size: > > Hugepagesize:       2048 kB > Hugetlb:               0 kB > DirectMap4k:      155912 kB > CONT DMap4k:        1176 kB > DirectMap2M:      722944 kB > CONT DMap2M:       28672 kB > DirectMap1G:   534773760 kB > > 64K page size: > > Hugepagesize:     524288 kB > Hugetlb:               0 kB > DirectMap64k:      882624 kB > CONT DMap64k:       19904 kB > DirectMap512M:    534773760 kB > CONT DMap512M:           0 kB > DirectMap4096G:           0 kB > > Signed-off-by: Christoph Lameter (Ampere) > > Index: linux/arch/arm64/mm/mmu.c > =================================================================== > --- linux.orig/arch/arm64/mm/mmu.c > +++ linux/arch/arm64/mm/mmu.c > @@ -76,6 +76,13 @@ EXPORT_SYMBOL(empty_zero_page); >  static DEFINE_SPINLOCK(swapper_pgdir_lock); >  static DEFINE_MUTEX(fixmap_lock); > > +static atomic_t nr_pte; > +static atomic_t nr_pmd; > +static atomic_t nr_pud; > +static atomic_t nr_pte_cont; > +static atomic_t nr_pmd_cont; > + > + >  void set_swapper_pgd(pgd_t *pgdp, pgd_t pgd) >  { >      pgd_t *fixmap_pgdp; > @@ -179,6 +186,7 @@ static void init_pte(pmd_t *pmdp, unsign >          pte_t old_pte = READ_ONCE(*ptep); > >          set_pte(ptep, pfn_pte(__phys_to_pfn(phys), prot)); > +        atomic_inc(&nr_pte); > >          /* >           * After the PTE entry has been populated once, we > @@ -223,8 +231,10 @@ static void alloc_init_cont_pte(pmd_t *p > >          /* use a contiguous mapping if the range is suitably aligned */ >          if ((((addr | next | phys) & ~CONT_PTE_MASK) == 0) && > -            (flags & NO_CONT_MAPPINGS) == 0) > +            (flags & NO_CONT_MAPPINGS) == 0) { >              __prot = __pgprot(pgprot_val(prot) | PTE_CONT); > +            atomic_inc(&nr_pte_cont); > +        } > >          init_pte(pmdp, addr, next, phys, __prot); > > @@ -249,6 +259,7 @@ static void init_pmd(pud_t *pudp, unsign >          if (((addr | next | phys) & ~PMD_MASK) == 0 && >              (flags & NO_BLOCK_MAPPINGS) == 0) { >              pmd_set_huge(pmdp, phys, prot); > +            atomic_inc(&nr_pmd); > >              /* >               * After the PMD entry has been populated once, we > @@ -301,8 +312,10 @@ static void alloc_init_cont_pmd(pud_t *p > >          /* use a contiguous mapping if the range is suitably aligned */ >          if ((((addr | next | phys) & ~CONT_PMD_MASK) == 0) && > -            (flags & NO_CONT_MAPPINGS) == 0) > +            (flags & NO_CONT_MAPPINGS) == 0) { >              __prot = __pgprot(pgprot_val(prot) | PTE_CONT); > +            atomic_inc(&nr_pmd_cont); > +        } > >          init_pmd(pudp, addr, next, phys, __prot, pgtable_alloc, flags); > > @@ -346,7 +359,7 @@ static void alloc_init_pud(pgd_t *pgdp, >             ((addr | next | phys) & ~PUD_MASK) == 0 && >              (flags & NO_BLOCK_MAPPINGS) == 0) { >              pud_set_huge(pudp, phys, prot); > - > +            atomic_inc(&nr_pud); >              /* >               * After the PUD entry has been populated once, we >               * only allow updates to the permission attributes. > @@ -859,6 +872,11 @@ static void unmap_hotplug_pte_range(pmd_ >              continue; > >          WARN_ON(!pte_present(pte)); > + > +        if (pte_cont(pte)) > +            atomic_dec(&nr_pte_cont); > +        atomic_dec(&nr_pte); > + >          pte_clear(&init_mm, addr, ptep); >          flush_tlb_kernel_range(addr, addr + PAGE_SIZE); >          if (free_mapped) > @@ -883,6 +901,11 @@ static void unmap_hotplug_pmd_range(pud_ > >          WARN_ON(!pmd_present(pmd)); >          if (pmd_sect(pmd)) { > + > +            if (pmd_cont(pmd)) > +                atomic_dec(&nr_pmd_cont); > +            atomic_dec(&nr_pmd); > + >              pmd_clear(pmdp); > >              /* > @@ -916,6 +939,9 @@ static void unmap_hotplug_pud_range(p4d_ > >          WARN_ON(!pud_present(pud)); >          if (pud_sect(pud)) { > + > +            atomic_dec(&nr_pud); > + >              pud_clear(pudp); > >              /* > @@ -1010,6 +1036,7 @@ static void free_empty_pte_table(pmd_t * >              return; >      } > > +    atomic_dec(&nr_pmd); >      pmd_clear(pmdp); >      __flush_tlb_kernel_pgtable(start); >      free_hotplug_pgtable_page(virt_to_page(ptep)); > @@ -1050,6 +1077,7 @@ static void free_empty_pmd_table(pud_t * >              return; >      } > > +    atomic_dec(&nr_pud); >      pud_clear(pudp); >      __flush_tlb_kernel_pgtable(start); >      free_hotplug_pgtable_page(virt_to_page(pmdp)); > @@ -1225,6 +1253,7 @@ int pmd_free_pte_page(pmd_t *pmdp, unsig >      } > >      table = pte_offset_kernel(pmdp, addr); > +    atomic_dec(&nr_pmd); >      pmd_clear(pmdp); >      __flush_tlb_kernel_pgtable(addr); >      pte_free_kernel(NULL, table); > @@ -1253,6 +1282,7 @@ int pud_free_pmd_page(pud_t *pudp, unsig >          pmd_free_pte_page(pmdp, next); >      } while (pmdp++, next += PMD_SIZE, next != end); > > +    atomic_dec(&nr_pud); >      pud_clear(pudp); >      __flush_tlb_kernel_pgtable(addr); >      pmd_free(NULL, table); > @@ -1486,3 +1516,36 @@ void ptep_modify_prot_commit(struct vm_a >  { >      set_pte_at(vma->vm_mm, addr, ptep, pte); >  } > + > + > +#ifdef CONFIG_PROC_FS > +void arch_report_meminfo(struct seq_file *m) > +{ > +    unsigned long pagesize_in_kb = PAGE_SIZE / 1024; > + > +    seq_printf(m, "DirectMap%luk:    %8lu kB\n", > +            pagesize_in_kb, > +              (unsigned long)atomic_read(&nr_pte) * pagesize_in_kb); > + > +        seq_printf(m, "CONT DMap%luk:    %8lu kB\n", > +            pagesize_in_kb, > +            (unsigned long)atomic_read(&nr_pte_cont) * pagesize_in_kb); > + > +    pagesize_in_kb = PMD_SIZE / 1024; > + > +        seq_printf(m, "DirectMap%luM:    %8lu kB\n", > +            pagesize_in_kb / 1024, > +            (unsigned long)atomic_read(&nr_pmd) * pagesize_in_kb); > + > +    seq_printf(m, "CONT DMap%luM:    %8lu kB\n", > +            pagesize_in_kb / 1024, > +            (unsigned long)atomic_read(&nr_pmd_cont) * pagesize_in_kb); > + > +    pagesize_in_kb = PUD_SIZE / 1024; > + > +    seq_printf(m, "DirectMap%luG:  %10lu kB\n", > +            pagesize_in_kb >> 20, > +            (unsigned long)atomic_read(&nr_pud) * pagesize_in_kb); > +} > +#endif > +