From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 082E5C433E1 for ; Fri, 21 Aug 2020 05:40:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 913B22075E for ; Fri, 21 Aug 2020 05:40:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 913B22075E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=csgroup.eu Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C7AEE8D0021; Fri, 21 Aug 2020 01:40:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C052F8D0005; Fri, 21 Aug 2020 01:40:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA4D48D0021; Fri, 21 Aug 2020 01:40:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0245.hostedemail.com [216.40.44.245]) by kanga.kvack.org (Postfix) with ESMTP id 8EE388D0005 for ; Fri, 21 Aug 2020 01:40:11 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 4DB3D824556B for ; Fri, 21 Aug 2020 05:40:11 +0000 (UTC) X-FDA: 77173475022.24.limit52_071218827036 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 262461A4A0 for ; Fri, 21 Aug 2020 05:40:11 +0000 (UTC) X-HE-Tag: limit52_071218827036 X-Filterd-Recvd-Size: 16223 Received: from pegase1.c-s.fr (pegase1.c-s.fr [93.17.236.30]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Fri, 21 Aug 2020 05:40:09 +0000 (UTC) Received: from localhost (mailhub1-int [192.168.12.234]) by localhost (Postfix) with ESMTP id 4BXr2H3pQFz9vCyR; Fri, 21 Aug 2020 07:40:07 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [192.168.12.234]) (amavisd-new, port 10024) with ESMTP id NBBJ9OMeuyRL; Fri, 21 Aug 2020 07:40:07 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 4BXr2H2m2nz9vCyQ; Fri, 21 Aug 2020 07:40:07 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 448BE8B87B; Fri, 21 Aug 2020 07:40:08 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id P7XZ6g2G32t3; Fri, 21 Aug 2020 07:40:08 +0200 (CEST) Received: from [192.168.4.90] (unknown [192.168.4.90]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 2D9A78B75F; Fri, 21 Aug 2020 07:40:07 +0200 (CEST) Subject: Re: [PATCH v5 5/8] mm: HUGE_VMAP arch support cleanup To: Nicholas Piggin , linux-mm@kvack.org, Andrew Morton Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Zefan Li , Jonathan Cameron , linuxppc-dev@lists.ozlabs.org References: <20200821044427.736424-1-npiggin@gmail.com> <20200821044427.736424-6-npiggin@gmail.com> From: Christophe Leroy Message-ID: <9b67b892-9482-15dc-0c1e-c5d5a93a3c91@csgroup.eu> Date: Fri, 21 Aug 2020 07:40:06 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: <20200821044427.736424-6-npiggin@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: fr X-Rspamd-Queue-Id: 262461A4A0 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Le 21/08/2020 =C3=A0 06:44, Nicholas Piggin a =C3=A9crit=C2=A0: > This changes the awkward approach where architectures provide init > functions to determine which levels they can provide large mappings for= , > to one where the arch is queried for each call. >=20 > This removes code and indirection, and allows constant-folding of dead > code for unsupported levels. I think that in order to allow constant-folding of dead code for=20 unsupported levels, you must define arch_vmap_xxx_supported() as static=20 inline in a .h If you have them in .c files, you'll get calls to tiny functions that=20 will always return false, but will still be called and dead code won't=20 be eliminated. And performance wise, that's probably not optimal either. Christophe >=20 > This also adds a prot argument to the arch query. This is unused > currently but could help with some architectures (e.g., some powerpc > processors can't map uncacheable memory with large pages). >=20 > Signed-off-by: Nicholas Piggin > --- > arch/arm64/mm/mmu.c | 12 +-- > arch/powerpc/mm/book3s64/radix_pgtable.c | 10 ++- > arch/x86/mm/ioremap.c | 12 +-- > include/linux/io.h | 9 --- > include/linux/vmalloc.h | 10 +++ > init/main.c | 1 - > mm/ioremap.c | 96 +++++++++++------------= - > 7 files changed, 73 insertions(+), 77 deletions(-) >=20 > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c > index 75df62fea1b6..bbb3ccf6a7ce 100644 > --- a/arch/arm64/mm/mmu.c > +++ b/arch/arm64/mm/mmu.c > @@ -1304,12 +1304,13 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_ph= ys, int *size, pgprot_t prot) > return dt_virt; > } > =20 > -int __init arch_ioremap_p4d_supported(void) > +#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP > +bool arch_vmap_p4d_supported(pgprot_t prot) > { > - return 0; > + return false; > } > =20 > -int __init arch_ioremap_pud_supported(void) > +bool arch_vmap_pud_supported(pgprot_t prot) > { > /* > * Only 4k granule supports level 1 block mappings. > @@ -1319,11 +1320,12 @@ int __init arch_ioremap_pud_supported(void) > !IS_ENABLED(CONFIG_PTDUMP_DEBUGFS); > } > =20 > -int __init arch_ioremap_pmd_supported(void) > +bool arch_vmap_pmd_supported(pgprot_t prot) > { > - /* See arch_ioremap_pud_supported() */ > + /* See arch_vmap_pud_supported() */ > return !IS_ENABLED(CONFIG_PTDUMP_DEBUGFS); > } > +#endif > =20 > int pud_set_huge(pud_t *pudp, phys_addr_t phys, pgprot_t prot) > { > diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm= /book3s64/radix_pgtable.c > index ae823bba29f2..7d3a620c5adf 100644 > --- a/arch/powerpc/mm/book3s64/radix_pgtable.c > +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c > @@ -1182,13 +1182,14 @@ void radix__ptep_modify_prot_commit(struct vm_a= rea_struct *vma, > set_pte_at(mm, addr, ptep, pte); > } > =20 > -int __init arch_ioremap_pud_supported(void) > +#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP > +bool arch_vmap_pud_supported(pgprot_t prot) > { > /* HPT does not cope with large pages in the vmalloc area */ > return radix_enabled(); > } > =20 > -int __init arch_ioremap_pmd_supported(void) > +bool arch_vmap_pmd_supported(pgprot_t prot) > { > return radix_enabled(); > } > @@ -1197,6 +1198,7 @@ int p4d_free_pud_page(p4d_t *p4d, unsigned long a= ddr) > { > return 0; > } > +#endif > =20 > int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot) > { > @@ -1282,7 +1284,7 @@ int pmd_free_pte_page(pmd_t *pmd, unsigned long a= ddr) > return 1; > } > =20 > -int __init arch_ioremap_p4d_supported(void) > +bool arch_vmap_p4d_supported(pgprot_t prot) > { > - return 0; > + return false; > } > diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c > index 84d85dbd1dad..5b8b495ab4ed 100644 > --- a/arch/x86/mm/ioremap.c > +++ b/arch/x86/mm/ioremap.c > @@ -481,24 +481,26 @@ void iounmap(volatile void __iomem *addr) > } > EXPORT_SYMBOL(iounmap); > =20 > -int __init arch_ioremap_p4d_supported(void) > +#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP > +bool arch_vmap_p4d_supported(pgprot_t prot) > { > - return 0; > + return false; > } > =20 > -int __init arch_ioremap_pud_supported(void) > +bool arch_vmap_pud_supported(pgprot_t prot) > { > #ifdef CONFIG_X86_64 > return boot_cpu_has(X86_FEATURE_GBPAGES); > #else > - return 0; > + return false; > #endif > } > =20 > -int __init arch_ioremap_pmd_supported(void) > +bool arch_vmap_pmd_supported(pgprot_t prot) > { > return boot_cpu_has(X86_FEATURE_PSE); > } > +#endif > =20 > /* > * Convert a physical pointer to a virtual kernel pointer for /dev/me= m > diff --git a/include/linux/io.h b/include/linux/io.h > index 8394c56babc2..f1effd4d7a3c 100644 > --- a/include/linux/io.h > +++ b/include/linux/io.h > @@ -31,15 +31,6 @@ static inline int ioremap_page_range(unsigned long a= ddr, unsigned long end, > } > #endif > =20 > -#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP > -void __init ioremap_huge_init(void); > -int arch_ioremap_p4d_supported(void); > -int arch_ioremap_pud_supported(void); > -int arch_ioremap_pmd_supported(void); > -#else > -static inline void ioremap_huge_init(void) { } > -#endif > - > /* > * Managed iomap interface > */ > diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h > index 0221f852a7e1..787d77ad7536 100644 > --- a/include/linux/vmalloc.h > +++ b/include/linux/vmalloc.h > @@ -84,6 +84,16 @@ struct vmap_area { > }; > }; > =20 > +#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP > +bool arch_vmap_p4d_supported(pgprot_t prot); > +bool arch_vmap_pud_supported(pgprot_t prot); > +bool arch_vmap_pmd_supported(pgprot_t prot); > +#else > +static inline bool arch_vmap_p4d_supported(pgprot_t prot) { return fal= se; } > +static inline bool arch_vmap_pud_supported(pgprot_t prot) { return fal= se; } > +static inline bool arch_vmap_pmd_supported(pgprot_t prot) { return fal= se; } > +#endif > + > /* > * Highlevel APIs for driver use > */ > diff --git a/init/main.c b/init/main.c > index ae78fb68d231..1c89aa127b8f 100644 > --- a/init/main.c > +++ b/init/main.c > @@ -820,7 +820,6 @@ static void __init mm_init(void) > pgtable_init(); > debug_objects_mem_init(); > vmalloc_init(); > - ioremap_huge_init(); > /* Should be run before the first non-init thread is created */ > init_espfix_bsp(); > /* Should be run after espfix64 is set up. */ > diff --git a/mm/ioremap.c b/mm/ioremap.c > index 6016ae3227ad..b0032dbadaf7 100644 > --- a/mm/ioremap.c > +++ b/mm/ioremap.c > @@ -16,49 +16,16 @@ > #include "pgalloc-track.h" > =20 > #ifdef CONFIG_HAVE_ARCH_HUGE_VMAP > -static int __read_mostly ioremap_p4d_capable; > -static int __read_mostly ioremap_pud_capable; > -static int __read_mostly ioremap_pmd_capable; > -static int __read_mostly ioremap_huge_disabled; > +static bool __ro_after_init iomap_allow_huge =3D true; > =20 > static int __init set_nohugeiomap(char *str) > { > - ioremap_huge_disabled =3D 1; > + iomap_allow_huge =3D false; > return 0; > } > early_param("nohugeiomap", set_nohugeiomap); > - > -void __init ioremap_huge_init(void) > -{ > - if (!ioremap_huge_disabled) { > - if (arch_ioremap_p4d_supported()) > - ioremap_p4d_capable =3D 1; > - if (arch_ioremap_pud_supported()) > - ioremap_pud_capable =3D 1; > - if (arch_ioremap_pmd_supported()) > - ioremap_pmd_capable =3D 1; > - } > -} > - > -static inline int ioremap_p4d_enabled(void) > -{ > - return ioremap_p4d_capable; > -} > - > -static inline int ioremap_pud_enabled(void) > -{ > - return ioremap_pud_capable; > -} > - > -static inline int ioremap_pmd_enabled(void) > -{ > - return ioremap_pmd_capable; > -} > - > -#else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ > -static inline int ioremap_p4d_enabled(void) { return 0; } > -static inline int ioremap_pud_enabled(void) { return 0; } > -static inline int ioremap_pmd_enabled(void) { return 0; } > +#else /* CONFIG_HAVE_ARCH_HUGE_VMAP */ > +static const bool iomap_allow_huge =3D false; > #endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */ > =20 > static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned lo= ng end, > @@ -81,9 +48,12 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long = addr, unsigned long end, > } > =20 > static int vmap_try_huge_pmd(pmd_t *pmd, unsigned long addr, unsigned= long end, > - phys_addr_t phys_addr, pgprot_t prot) > + phys_addr_t phys_addr, pgprot_t prot, unsigned int max_page_shift) > { > - if (!ioremap_pmd_enabled()) > + if (max_page_shift < PMD_SHIFT) > + return 0; > + > + if (!arch_vmap_pmd_supported(prot)) > return 0; > =20 > if ((end - addr) !=3D PMD_SIZE) > @@ -102,7 +72,8 @@ static int vmap_try_huge_pmd(pmd_t *pmd, unsigned lo= ng addr, unsigned long end, > } > =20 > static int vmap_pmd_range(pud_t *pud, unsigned long addr, unsigned lo= ng end, > - phys_addr_t phys_addr, pgprot_t prot, pgtbl_mod_mask *mask) > + phys_addr_t phys_addr, pgprot_t prot, unsigned int max_page_shift, > + pgtbl_mod_mask *mask) > { > pmd_t *pmd; > unsigned long next; > @@ -113,7 +84,7 @@ static int vmap_pmd_range(pud_t *pud, unsigned long = addr, unsigned long end, > do { > next =3D pmd_addr_end(addr, end); > =20 > - if (vmap_try_huge_pmd(pmd, addr, next, phys_addr, prot)) { > + if (vmap_try_huge_pmd(pmd, addr, next, phys_addr, prot, max_page_shi= ft)) { > *mask |=3D PGTBL_PMD_MODIFIED; > continue; > } > @@ -125,9 +96,12 @@ static int vmap_pmd_range(pud_t *pud, unsigned long= addr, unsigned long end, > } > =20 > static int vmap_try_huge_pud(pud_t *pud, unsigned long addr, unsigned= long end, > - phys_addr_t phys_addr, pgprot_t prot) > + phys_addr_t phys_addr, pgprot_t prot, unsigned int max_page_shift) > { > - if (!ioremap_pud_enabled()) > + if (max_page_shift < PUD_SHIFT) > + return 0; > + > + if (!arch_vmap_pud_supported(prot)) > return 0; > =20 > if ((end - addr) !=3D PUD_SIZE) > @@ -146,7 +120,8 @@ static int vmap_try_huge_pud(pud_t *pud, unsigned l= ong addr, unsigned long end, > } > =20 > static int vmap_pud_range(p4d_t *p4d, unsigned long addr, unsigned lo= ng end, > - phys_addr_t phys_addr, pgprot_t prot, pgtbl_mod_mask *mask) > + phys_addr_t phys_addr, pgprot_t prot, unsigned int max_page_shift, > + pgtbl_mod_mask *mask) > { > pud_t *pud; > unsigned long next; > @@ -157,21 +132,24 @@ static int vmap_pud_range(p4d_t *p4d, unsigned lo= ng addr, unsigned long end, > do { > next =3D pud_addr_end(addr, end); > =20 > - if (vmap_try_huge_pud(pud, addr, next, phys_addr, prot)) { > + if (vmap_try_huge_pud(pud, addr, next, phys_addr, prot, max_page_shi= ft)) { > *mask |=3D PGTBL_PUD_MODIFIED; > continue; > } > =20 > - if (vmap_pmd_range(pud, addr, next, phys_addr, prot, mask)) > + if (vmap_pmd_range(pud, addr, next, phys_addr, prot, max_page_shift,= mask)) > return -ENOMEM; > } while (pud++, phys_addr +=3D (next - addr), addr =3D next, addr !=3D= end); > return 0; > } > =20 > static int vmap_try_huge_p4d(p4d_t *p4d, unsigned long addr, unsigned= long end, > - phys_addr_t phys_addr, pgprot_t prot) > + phys_addr_t phys_addr, pgprot_t prot, unsigned int max_page_shift) > { > - if (!ioremap_p4d_enabled()) > + if (max_page_shift < P4D_SHIFT) > + return 0; > + > + if (!arch_vmap_p4d_supported(prot)) > return 0; > =20 > if ((end - addr) !=3D P4D_SIZE) > @@ -190,7 +168,8 @@ static int vmap_try_huge_p4d(p4d_t *p4d, unsigned l= ong addr, unsigned long end, > } > =20 > static int vmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned lo= ng end, > - phys_addr_t phys_addr, pgprot_t prot, pgtbl_mod_mask *mask) > + phys_addr_t phys_addr, pgprot_t prot, unsigned int max_page_shift, > + pgtbl_mod_mask *mask) > { > p4d_t *p4d; > unsigned long next; > @@ -201,18 +180,19 @@ static int vmap_p4d_range(pgd_t *pgd, unsigned lo= ng addr, unsigned long end, > do { > next =3D p4d_addr_end(addr, end); > =20 > - if (vmap_try_huge_p4d(p4d, addr, next, phys_addr, prot)) { > + if (vmap_try_huge_p4d(p4d, addr, next, phys_addr, prot, max_page_shi= ft)) { > *mask |=3D PGTBL_P4D_MODIFIED; > continue; > } > =20 > - if (vmap_pud_range(p4d, addr, next, phys_addr, prot, mask)) > + if (vmap_pud_range(p4d, addr, next, phys_addr, prot, max_page_shift,= mask)) > return -ENOMEM; > } while (p4d++, phys_addr +=3D (next - addr), addr =3D next, addr !=3D= end); > return 0; > } > =20 > -int ioremap_page_range(unsigned long addr, unsigned long end, phys_add= r_t phys_addr, pgprot_t prot) > +static int vmap_range(unsigned long addr, unsigned long end, phys_addr= _t phys_addr, pgprot_t prot, > + unsigned int max_page_shift) > { > pgd_t *pgd; > unsigned long start; > @@ -227,7 +207,7 @@ int ioremap_page_range(unsigned long addr, unsigned= long end, phys_addr_t phys_a > pgd =3D pgd_offset_k(addr); > do { > next =3D pgd_addr_end(addr, end); > - err =3D vmap_p4d_range(pgd, addr, next, phys_addr, prot, &mask); > + err =3D vmap_p4d_range(pgd, addr, next, phys_addr, prot, max_page_sh= ift, &mask); > if (err) > break; > } while (pgd++, phys_addr +=3D (next - addr), addr =3D next, addr !=3D= end); > @@ -240,6 +220,16 @@ int ioremap_page_range(unsigned long addr, unsigne= d long end, phys_addr_t phys_a > return err; > } > =20 > +int ioremap_page_range(unsigned long addr, unsigned long end, phys_add= r_t phys_addr, pgprot_t prot) > +{ > + unsigned int max_page_shift =3D PAGE_SHIFT; > + > + if (iomap_allow_huge) > + max_page_shift =3D P4D_SHIFT; > + > + return vmap_range(addr, end, phys_addr, prot, max_page_shift); > +} > + > #ifdef CONFIG_GENERIC_IOREMAP > void __iomem *ioremap_prot(phys_addr_t addr, size_t size, unsigned lo= ng prot) > { >=20