From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF9DEC6FD1C for ; Tue, 14 Mar 2023 08:41:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 870FC8E0005; Tue, 14 Mar 2023 04:41:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 820DB8E0002; Tue, 14 Mar 2023 04:41:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6E8648E0005; Tue, 14 Mar 2023 04:41:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5F6178E0002 for ; Tue, 14 Mar 2023 04:41:47 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2EF8B80F92 for ; Tue, 14 Mar 2023 08:41:47 +0000 (UTC) X-FDA: 80566860654.06.C8167CA Received: from mail-io1-f52.google.com (mail-io1-f52.google.com [209.85.166.52]) by imf13.hostedemail.com (Postfix) with ESMTP id 585512000B for ; Tue, 14 Mar 2023 08:41:45 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Fa6bULuY; spf=pass (imf13.hostedemail.com: domain of elver@google.com designates 209.85.166.52 as permitted sender) smtp.mailfrom=elver@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678783305; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WpsiRFIUZLP1ypzQar3A5S28uGiElt2gVT4WHx11HNE=; b=sGQlScjdEArK3DPEPyhIWtzc/IctRhsnvW7w+bwU+bHNZ19Kx2AH8mQ4fIez7UzoVTyhby duvoOja66SOMqcnXx4p4RBYoTyW+z2QKHQwAfd1kBmxggxyxg/wkQzHOrFmgcT4JuWjivG YjpP7Jz+4G24X2K8urAF1be3xNn/aX8= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Fa6bULuY; spf=pass (imf13.hostedemail.com: domain of elver@google.com designates 209.85.166.52 as permitted sender) smtp.mailfrom=elver@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678783305; a=rsa-sha256; cv=none; b=CScuazYMNcPzWVB/hW9/MQHw2TYsx0jtWi72AwHxFSten8WftCFinzfVLNd9iAgbJQHn/m lKcDTrs6NqVx547wuSky4hdqF40CyYJIKNHEjAuXGnvSyS3cZBZE/R5CADWacHNX03X2q7 XcvDnZnEHsPs/mwoKjDLvDwlRNnCAEk= Received: by mail-io1-f52.google.com with SMTP id v10so6090096iox.8 for ; Tue, 14 Mar 2023 01:41:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1678783304; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=WpsiRFIUZLP1ypzQar3A5S28uGiElt2gVT4WHx11HNE=; b=Fa6bULuYbJKT8d6ABCntGQ/vbMX7hFL/2cUUtjvNiyeGjg6UvoRX1gxbORwOoR9gP1 wGhRIU8Mhk0RlMx0GpLCtbf3xNPn9UeeEoaNwBGObwSJPWFF0FL4OvId6mNpN1kE3H97 IdRItFsX30UZKtnyhqG/vWy21XV8JirNHLY1NlsQObKcjuMT5t6bBWe4r8FxPjPy8klw oHbeCO6xF/sc0EQpNhRc8bZ0SfHTCJjbUdVF4ODE/ViJxvZikutSs9L6e5wM0dEpjGfq DOTe/9sj+l9BD9Fc2gLQEBuXmbIPyjeeuE8Jk2/9yKu8BmF/c5QZLTa8NclzlJUGipK5 Epqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678783304; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=WpsiRFIUZLP1ypzQar3A5S28uGiElt2gVT4WHx11HNE=; b=WXJ7vpuHzSF2Usc786EOOrws+jUoD4sDY9gpjiWdHjwYJjg6eeAKzp0+Nt6Nm2oLjx 5KilYgFFEvJ6vs6DtpXDjMmEAr2dRgPS0h7WBNwB39gOzSx/kIsY34zgiRRuQpyjlD1V Z4OGyL4Sp5ob3Mb2kttHo8wtSnhoUbsN9ULTccCJacW1Tv3E34iVY4BhjUuGPjZHqdQ6 HRdaL2HprLkhQbhGjZTo1vb1sCOf6HYKBGDl9YZdLLPb5NvMkAc9vZ32dDKQS5fuowwq +Gjeov9gshh2dVRC5qMAos0cPfHbLfF0PJzIOPZoLqRkVU2BvxrnSe/BzyXqDJaL6X13 F4vQ== X-Gm-Message-State: AO0yUKXRAFRvmSLZvAXwTfKOAC2Rh6TFgFnSlu6fkNdA9pG93bVNh3ib 4r7G0Fg8Sp7KPlb1FsWMseGruBx8TKMTgscjCZ8GZg== X-Google-Smtp-Source: AK7set/AAiidFEdhIaYteGEqmqpv6qDM2i0etiR38rSQv5WIuK/0Kr8BfR7BuIAHUEpwe8w2nUHiAMm81K+2shICBjM= X-Received: by 2002:a02:7a07:0:b0:3ec:dc1f:12d8 with SMTP id a7-20020a027a07000000b003ecdc1f12d8mr17450127jac.4.1678783304284; Tue, 14 Mar 2023 01:41:44 -0700 (PDT) MIME-Version: 1.0 References: <1678777502-6933-1-git-send-email-quic_zhenhuah@quicinc.com> In-Reply-To: <1678777502-6933-1-git-send-email-quic_zhenhuah@quicinc.com> From: Marco Elver Date: Tue, 14 Mar 2023 09:41:02 +0100 Message-ID: Subject: Re: [PATCH v8] mm,kfence: decouple kfence from page granularity mapping judgement To: Zhenhua Huang Cc: catalin.marinas@arm.com, will@kernel.org, glider@google.com, dvyukov@google.com, akpm@linux-foundation.org, robin.murphy@arm.com, mark.rutland@arm.com, jianyong.wu@arm.com, james.morse@arm.com, wangkefeng.wang@huawei.com, linux-arm-kernel@lists.infradead.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, quic_pkondeti@quicinc.com, quic_guptap@quicinc.com, quic_tingweiz@quicinc.com Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: hkepentduwrh7x3yqisegxdxxd6meo6z X-Rspamd-Queue-Id: 585512000B X-HE-Tag: 1678783305-376466 X-HE-Meta: U2FsdGVkX18fOi46meRSWmQsb1E1wYN64G5FM9z4sv83Mb9iRW/VI/IdZoEr+ghf72zpKd5504hMPMMiUANd+wMTXD07kyNaUSn0nN8IEy+kFIB6lBAWsZv7tqNsBnXUlu0PQN/n6ToMnvuqcjUf23WSDzsUrlLv+6QSaXsYATZgsccN/6K0Op54k5B7tv2tKtD7vD/Pyh/0ZIN2YHXiBDsm5Rn0w90ewB+J7hAQ5kif1x5/4NJsUihSw+SxjNH4P4oXPnvb7lijj1V2ahmLnxwHGZMQmr3LameXcqOExjAVuLaqU7YmoDNQf6IIUfI+RgmewhlWxmDiNvajZSyMxPnRtabZizTw5IoGAShg7omTkXgcCQzHj/L75tsmrstG/r2Ew/uN1EhEbSOrg1g00Y3QWnLTq7lEFJ4qZAY6kLQE7r3pjMzVMVrbdseK26NrrmoCmvUEWzod+dGV7GwzO2G/9II5lLEvz477t5+/H7SILMKcjhM7N3+2PE2XGn33TyzlWxrC4v2oi9/tN7Hv9MkCiVDbnquQtTPhdaLj6814MuHIBJmRV3EJTzELa6TPgpfBd1BbdL/OjYXr/N7U8aVdnSieaaDCWNDqtjbqon2AuAXXgBwG8Hyf1Sp/8Whbm8W9SdAieq4JR1NCampKPHMjqGqnm2v4C1XnkhVhs/vJseRtWgD/v5ffpXtdeyRoLethjmU6EzLM/fdeTqUi4sZooMwzVSITxRGwGCu4Z54ZfFZ1dYT5uPbV2ZJjCDnUdyyXDOwNR2AS5R/siovR069hYXNTKBHBc9bPGoLFIX9dzVyU6NTkx4/rfHBA24YFd3VH39BXCkPD3d6Zr98eljR/9vFnvXX7nl/sPgNUZKS0Zp29fRBhMdphV4meCI2BKiFtmGOUzptDKfaEU+IjPvU0M/Fcky2ZIv/QokgELQ6d3XwHGJqADSdhMY9FUtwaGX+jt1VWhcOQbpx/Stp px4miBnA 2hoU4ODVygmlSxhK276rN1FVPeemKI0K+PofvRxe0uD9O4WAiHo0xE4gcRvNWkloqBZejHd5ddyWdS2QjHCRyChIUzMHXrv5/kMcLUMD/HHXsXXGLEVf8GgapLx1xbFsz3JAZvTX3y9FhSuIgSuYtIXLwtI6LzQti8EDeA9DkKdJKDLgywx5vY+bml3bNpxWUtufhMo6Tc4XvooOvVttybcNDlABzIJ7YgtXgZLMS9w89rB+OvUTl3MwMP1OT7TVd7ZFQ9uA1Hf3bjIBy6ZlAPMQWMgqMG1l01e6QkHDW0BibgsW96BLP9m3meuDQ+opLyDjPEo8EFHCvYPZP2foGV31l6qwL3Xr45o6+aYA/2+un84zJAEl7VzZJdne8IMPhHinXLPgpuQYfpItWO1CmPvsAR72FsxuJKvnV60FO2J2FWXQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, 14 Mar 2023 at 08:05, Zhenhua Huang wrote: > > Kfence only needs its pool to be mapped as page granularity, if it is > inited early. Previous judgement was a bit over protected. From [1], Mark > suggested to "just map the KFENCE region a page granularity". So I > decouple it from judgement and do page granularity mapping for kfence > pool only. Need to be noticed that late init of kfence pool still requires > page granularity mapping. > > Page granularity mapping in theory cost more(2M per 1GB) memory on arm64 > platform. Like what I've tested on QEMU(emulated 1GB RAM) with > gki_defconfig, also turning off rodata protection: > Before: > [root@liebao ]# cat /proc/meminfo > MemTotal: 999484 kB > After: > [root@liebao ]# cat /proc/meminfo > MemTotal: 1001480 kB > > To implement this, also relocate the kfence pool allocation before the > linear mapping setting up, arm64_kfence_alloc_pool is to allocate phys > addr, __kfence_pool is to be set after linear mapping set up. > > LINK: [1] https://lore.kernel.org/linux-arm-kernel/Y+IsdrvDNILA59UN@FVFF77S0Q05N/ > Suggested-by: Mark Rutland > Signed-off-by: Zhenhua Huang > --- > arch/arm64/include/asm/kfence.h | 2 ++ > arch/arm64/mm/mmu.c | 44 +++++++++++++++++++++++++++++++++++++++++ > arch/arm64/mm/pageattr.c | 9 +++++++-- > include/linux/kfence.h | 8 ++++++++ > mm/kfence/core.c | 9 +++++++++ > 5 files changed, 70 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/include/asm/kfence.h b/arch/arm64/include/asm/kfence.h > index aa855c6..f1f9ca2d 100644 > --- a/arch/arm64/include/asm/kfence.h > +++ b/arch/arm64/include/asm/kfence.h > @@ -10,6 +10,8 @@ > > #include > > +extern phys_addr_t early_kfence_pool; > + > static inline bool arch_kfence_init_pool(void) { return true; } > > static inline bool kfence_protect_page(unsigned long addr, bool protect) > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c > index 6f9d889..7fbf2ed 100644 > --- a/arch/arm64/mm/mmu.c > +++ b/arch/arm64/mm/mmu.c > @@ -24,6 +24,7 @@ > #include > #include > #include > +#include > > #include > #include > @@ -38,6 +39,7 @@ > #include > #include > #include > +#include > > #define NO_BLOCK_MAPPINGS BIT(0) > #define NO_CONT_MAPPINGS BIT(1) > @@ -525,6 +527,33 @@ static int __init enable_crash_mem_map(char *arg) > } > early_param("crashkernel", enable_crash_mem_map); > > +#ifdef CONFIG_KFENCE > + > +static phys_addr_t arm64_kfence_alloc_pool(void) > +{ > + phys_addr_t kfence_pool; > + > + if (!kfence_sample_interval) > + return 0; > + > + kfence_pool = memblock_phys_alloc(KFENCE_POOL_SIZE, PAGE_SIZE); > + if (!kfence_pool) > + pr_err("failed to allocate kfence pool\n"); > + > + return kfence_pool; > +} > + > +#else > + > +static phys_addr_t arm64_kfence_alloc_pool(void) > +{ > + return 0; > +} > + > +#endif > + > +phys_addr_t early_kfence_pool; The compiler will not optimize out this global. This now exists in all arm64 kernel builds. Furthermore, there's no need for this to be phys_addr_t. Nothing outside map_mem() needs the address, so this can just be a bool. I'd recommend moving the variable under CONFIG_KFENCE, and in the asm header, just having a static inline helper function e.g. arm64_kfence_early_pool(). That helper just returns false in the !CONFIG_KFENCE case. > static void __init map_mem(pgd_t *pgdp) > { > static const u64 direct_map_end = _PAGE_END(VA_BITS_MIN); > @@ -543,6 +572,10 @@ static void __init map_mem(pgd_t *pgdp) > */ > BUILD_BUG_ON(pgd_index(direct_map_end - 1) == pgd_index(direct_map_end)); > > + early_kfence_pool = arm64_kfence_alloc_pool(); > + if (early_kfence_pool) > + memblock_mark_nomap(early_kfence_pool, KFENCE_POOL_SIZE); > + > if (can_set_direct_map()) > flags |= NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; > > @@ -608,6 +641,17 @@ static void __init map_mem(pgd_t *pgdp) > } > } > #endif > + > + /* Kfence pool needs page-level mapping */ > + if (early_kfence_pool) { > + __map_memblock(pgdp, early_kfence_pool, > + early_kfence_pool + KFENCE_POOL_SIZE, > + pgprot_tagged(PAGE_KERNEL), > + NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS); > + memblock_clear_nomap(early_kfence_pool, KFENCE_POOL_SIZE); > + /* kfence_pool really mapped now */ > + kfence_set_pool(early_kfence_pool); > + } > } > > void mark_rodata_ro(void) > diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c > index 79dd201..7ce5295 100644 > --- a/arch/arm64/mm/pageattr.c > +++ b/arch/arm64/mm/pageattr.c > @@ -7,10 +7,12 @@ > #include > #include > #include > +#include > > #include > #include > #include > +#include > > struct page_change_data { > pgprot_t set_mask; > @@ -22,12 +24,15 @@ bool rodata_full __ro_after_init = IS_ENABLED(CONFIG_RODATA_FULL_DEFAULT_ENABLED > bool can_set_direct_map(void) > { > /* > - * rodata_full, DEBUG_PAGEALLOC and KFENCE require linear map to be > + * rodata_full and DEBUG_PAGEALLOC require linear map to be > * mapped at page granularity, so that it is possible to > * protect/unprotect single pages. > + * > + * Kfence pool requires page granularity mapping also if we init it > + * late. > */ > return (rodata_enabled && rodata_full) || debug_pagealloc_enabled() || > - IS_ENABLED(CONFIG_KFENCE); > + (IS_ENABLED(CONFIG_KFENCE) && !early_kfence_pool); > } > > static int change_page_range(pte_t *ptep, unsigned long addr, void *data) > diff --git a/include/linux/kfence.h b/include/linux/kfence.h > index 726857a..570d4e3 100644 > --- a/include/linux/kfence.h > +++ b/include/linux/kfence.h > @@ -64,6 +64,12 @@ static __always_inline bool is_kfence_address(const void *addr) > void __init kfence_alloc_pool(void); > > /** > + * kfence_set_pool() - allows an arch to set the > + * KFENCE pool during early init > + */ > +void __init kfence_set_pool(phys_addr_t addr); > + > +/** > * kfence_init() - perform KFENCE initialization at boot time > * > * Requires that kfence_alloc_pool() was called before. This sets up the > @@ -222,8 +228,10 @@ bool __kfence_obj_info(struct kmem_obj_info *kpp, void *object, struct slab *sla > > #else /* CONFIG_KFENCE */ > > +#define KFENCE_POOL_SIZE 0 > static inline bool is_kfence_address(const void *addr) { return false; } > static inline void kfence_alloc_pool(void) { } > +static inline void kfence_set_pool(phys_addr_t addr) { } > static inline void kfence_init(void) { } > static inline void kfence_shutdown_cache(struct kmem_cache *s) { } > static inline void *kfence_alloc(struct kmem_cache *s, size_t size, gfp_t flags) { return NULL; } > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > index 5349c37..0765395 100644 > --- a/mm/kfence/core.c > +++ b/mm/kfence/core.c > @@ -814,12 +814,21 @@ void __init kfence_alloc_pool(void) > if (!kfence_sample_interval) > return; > > + /* if the pool has already been initialized by arch, skip the below */ > + if (__kfence_pool) > + return; > + > __kfence_pool = memblock_alloc(KFENCE_POOL_SIZE, PAGE_SIZE); > > if (!__kfence_pool) > pr_err("failed to allocate pool\n"); > } > > +void __init kfence_set_pool(phys_addr_t addr) > +{ > + __kfence_pool = phys_to_virt(addr); > +} Please move this function to the header as a static inline function, because nothing else other than arm64 needs this, having this function be here introduces a .text size increase for everyone. The function is so short that having it as a static inline function is fine, and will save a few bytes of .text. > static void kfence_init_enable(void) > { > if (!IS_ENABLED(CONFIG_KFENCE_STATIC_KEYS)) > -- > 2.7.4 >