From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FF30C6FD1C for ; Tue, 14 Mar 2023 10:32:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9F9A06B0072; Tue, 14 Mar 2023 06:32:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9AB678E0002; Tue, 14 Mar 2023 06:32:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 84BB68E0001; Tue, 14 Mar 2023 06:32:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 765C76B0072 for ; Tue, 14 Mar 2023 06:32:17 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 49F9DA01B3 for ; Tue, 14 Mar 2023 10:32:17 +0000 (UTC) X-FDA: 80567139114.22.8EFA905 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by imf18.hostedemail.com (Postfix) with ESMTP id DBD751C0005 for ; Tue, 14 Mar 2023 10:32:14 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=jp1JzcQx; spf=pass (imf18.hostedemail.com: domain of quic_zhenhuah@quicinc.com designates 205.220.168.131 as permitted sender) smtp.mailfrom=quic_zhenhuah@quicinc.com; dmarc=pass (policy=none) header.from=quicinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678789935; a=rsa-sha256; cv=none; b=LrWH4BjyYKMBjxtX00C8q4XRoG7u8/8hoIK9ChISOGJ6bjFnS0BjPwy+tlPv13m95+SipH kMWKMuxCrxGLXZ0HfcKavKLoz47Cwk9BDEwcO92lHii997s7Wd2h9ldVJyawGsshrdsFRt 6w+QqUH05ULhUozXf8FtC3sVigdqEto= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=jp1JzcQx; spf=pass (imf18.hostedemail.com: domain of quic_zhenhuah@quicinc.com designates 205.220.168.131 as permitted sender) smtp.mailfrom=quic_zhenhuah@quicinc.com; dmarc=pass (policy=none) header.from=quicinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678789935; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+0L/+ZLoHB8P/U6T9eCxaODfDhHGRx9v5KE7t9F/pb8=; b=xbTLSJ7s+B9aJcTx+B9Ms0qKMStJf7VI6PqtulczpClOK2mL33+fJciYiONJ8DSbjop+D7 pos5iU/9IfcNi0Q4iO2q6aHVRZDJfC8xQySBSfwuUPSnq1jvKpwTwD4Q8r538fgh6mdro3 gW7IcwZ7VJxJ1FIkICDmBJwo/pB7N+I= Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32EAScsD028598; Tue, 14 Mar 2023 10:31:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=qcppdkim1; bh=+0L/+ZLoHB8P/U6T9eCxaODfDhHGRx9v5KE7t9F/pb8=; b=jp1JzcQxbq9Xv9rm6u657NKV5RJKw/tTD9AxAow6DSmMgD0/Ap0BoRQ5Zq85+eidS+Fn saeipwUwzO9svOrKV//rhGeRKYBdNgpoVsZO9xa4PHg44onQIXSb2GXpZabXzd9rLRnA GNbr+mSbZ4AJsvRJ8jjgzpNKaeCcHMdkusHj23GpkE207xCLbxtBW37SVQ/VYfEAuyPn rLvyr1bAS0clkmJDYRnP5ahCSec+S0q7jf9l1ykNNggLemBFf2Sx6WNY74OkLFs/Qd0i 2ITe+PHQkj1oXjolwvOEcrXVH8e3+gciWTGXcGD4UNYMPT9l5kyF6w5COxkSH9b/AjOq Wg== Received: from nalasppmta02.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3paay39xx3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 14 Mar 2023 10:31:59 +0000 Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 32EAVwHT003805 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 14 Mar 2023 10:31:58 GMT Received: from [10.239.132.245] (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 14 Mar 2023 03:31:54 -0700 Message-ID: Date: Tue, 14 Mar 2023 18:31:52 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Subject: Re: [PATCH v8] mm,kfence: decouple kfence from page granularity mapping judgement Content-Language: en-US To: Marco Elver CC: , , , , , , , , , , , , , , , References: <1678777502-6933-1-git-send-email-quic_zhenhuah@quicinc.com> From: Zhenhua Huang In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: ysTxSfXRYWA1q52lwszIXn_iKXZzw_yc X-Proofpoint-GUID: ysTxSfXRYWA1q52lwszIXn_iKXZzw_yc X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-14_04,2023-03-14_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 mlxscore=0 mlxlogscore=999 malwarescore=0 adultscore=0 lowpriorityscore=0 bulkscore=0 phishscore=0 spamscore=0 impostorscore=0 priorityscore=1501 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303140090 X-Rspam-User: X-Rspamd-Queue-Id: DBD751C0005 X-Rspamd-Server: rspam01 X-Stat-Signature: z1k9i4izocq4niojiajd7yy4f17oj3ic X-HE-Tag: 1678789934-132856 X-HE-Meta: U2FsdGVkX1+HjHQZHir7JMGaJFjZhbEAzrNpvScFQKm1VEfdMysEwywJH+hv80PsFO8iST6uObku+xx6sbZpqNYttrKUVSe9tEMg9BMRyPnmIGhIZ61JcujCNDZPrsrm6mWBs3+6iIIQlNhx8ovDaU4TNg3/Ho0uxMwuYL6i7jiXcfmkf9DBFojLfhAXN1S1KiH8Dy8gf2AHVv1mnocn/1jpgG4iayMhFXNW4qy3zPZcKpWCxb1MeCDnLZg+/CHERT2tsfAVfjzzdFRb3CQP5yWZKMDbLwvuqKifQks0+jZp6zlA+xfGoez5p1lC3gLGcJI3F51eWz1oqsG81x9pkM5R3SrVYuHqrfLVDOWQa4UDaunaLxpRJpJJh7hxChK6nlyFNez4GHfnn5nDbwEKISE36MbffOdyLxMIzdbgP5xjFwiGR9cnmbvG/wHFbnieWBAf2OyIDY6FRKPsySn1auKmThE8lciRyjhnaujMGCFqLDV7Cf7IM323HVUywEAnp1BfBeoSBLtFb5N+XN1rZNQyLIQquHobugzyfp2jJs5+WfXnnOVu5+HulyxMC8ZaJdo5H6LkSE54HXbKdeHGrem1aDWuEYxGgjy48fECA5qCAi76+SWsCCaINv/qmgAA+3BA4jCPRQ2pUcW0G2VBUMvHXXHOcELkHzJ/tbPJTfdepQnDPdVy3qlLgpML1mQNEbn0YolApCCCm/P80LfpjrNVukDcIb2I99sKmCAIVbsZ/Gxt6dmXoyajB/YmbMZBLVv2B7Bua03S+PGVLlyf9fRvoDSmynDjXnXKfrah6sDyRLc4pBCQC/XarsmPgQYowXOB6DcZ/tTiNT3iJfpXgG9Nq/HFhGtPZZFcGVvbOPpq7073CChUzCEB1CSQDXDWU3OSWw5Sgkktx+gHrP0rrA9OMBOyoKFYyW1YdPTA8jJX4ZMT3cYYxXVtfZWJ3IFSlAdMbrJ//3BOBYkiFsn vxRcL7Lo ya1gMxkTUNzxeyiLLsakTAADjn/q3Hd14gN9QonlT/2/wiP2Ac/LyprkuJs7Mo4xW7O/pEfhNyl2W19HJ9VYikw8uHYr+j3PLZLdaZdq4TQaOEr366akvJBL43d4vILHCrgmiWI0U5HxKRGbQF6lLF64XDUJs9evlfzvyjfHCU8Avo6rmsh+iw7T/zVYMj6vrxuAi17SRU6Qg9My9cSffKzBWvRTj+JOZLhnLpGA3MDmkvo0ulv7nTg76QMwHCP7rpEl/8KO/2q7hND14vRatIND+BbFcd5ckxj9D3FxQRvX2zE+urB4CNOFmui9EcwvhEDym+F4x+M30V2iim9LuaxmuNCsS926txxeILLNc2VS8AzfZStOQJ1EHdh0cjMcSuMwUJkns1Wqyai9/QDAIxT9Mmg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2023/3/14 16:41, Marco Elver wrote: > On Tue, 14 Mar 2023 at 08:05, Zhenhua Huang wrote: >> >> Kfence only needs its pool to be mapped as page granularity, if it is >> inited early. Previous judgement was a bit over protected. From [1], Mark >> suggested to "just map the KFENCE region a page granularity". So I >> decouple it from judgement and do page granularity mapping for kfence >> pool only. Need to be noticed that late init of kfence pool still requires >> page granularity mapping. >> >> Page granularity mapping in theory cost more(2M per 1GB) memory on arm64 >> platform. Like what I've tested on QEMU(emulated 1GB RAM) with >> gki_defconfig, also turning off rodata protection: >> Before: >> [root@liebao ]# cat /proc/meminfo >> MemTotal: 999484 kB >> After: >> [root@liebao ]# cat /proc/meminfo >> MemTotal: 1001480 kB >> >> To implement this, also relocate the kfence pool allocation before the >> linear mapping setting up, arm64_kfence_alloc_pool is to allocate phys >> addr, __kfence_pool is to be set after linear mapping set up. >> >> LINK: [1] https://lore.kernel.org/linux-arm-kernel/Y+IsdrvDNILA59UN@FVFF77S0Q05N/ >> Suggested-by: Mark Rutland >> Signed-off-by: Zhenhua Huang >> --- >> arch/arm64/include/asm/kfence.h | 2 ++ >> arch/arm64/mm/mmu.c | 44 +++++++++++++++++++++++++++++++++++++++++ >> arch/arm64/mm/pageattr.c | 9 +++++++-- >> include/linux/kfence.h | 8 ++++++++ >> mm/kfence/core.c | 9 +++++++++ >> 5 files changed, 70 insertions(+), 2 deletions(-) >> >> diff --git a/arch/arm64/include/asm/kfence.h b/arch/arm64/include/asm/kfence.h >> index aa855c6..f1f9ca2d 100644 >> --- a/arch/arm64/include/asm/kfence.h >> +++ b/arch/arm64/include/asm/kfence.h >> @@ -10,6 +10,8 @@ >> >> #include >> >> +extern phys_addr_t early_kfence_pool; >> + >> static inline bool arch_kfence_init_pool(void) { return true; } >> >> static inline bool kfence_protect_page(unsigned long addr, bool protect) >> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c >> index 6f9d889..7fbf2ed 100644 >> --- a/arch/arm64/mm/mmu.c >> +++ b/arch/arm64/mm/mmu.c >> @@ -24,6 +24,7 @@ >> #include >> #include >> #include >> +#include >> >> #include >> #include >> @@ -38,6 +39,7 @@ >> #include >> #include >> #include >> +#include >> >> #define NO_BLOCK_MAPPINGS BIT(0) >> #define NO_CONT_MAPPINGS BIT(1) >> @@ -525,6 +527,33 @@ static int __init enable_crash_mem_map(char *arg) >> } >> early_param("crashkernel", enable_crash_mem_map); >> >> +#ifdef CONFIG_KFENCE >> + >> +static phys_addr_t arm64_kfence_alloc_pool(void) >> +{ >> + phys_addr_t kfence_pool; >> + >> + if (!kfence_sample_interval) >> + return 0; >> + >> + kfence_pool = memblock_phys_alloc(KFENCE_POOL_SIZE, PAGE_SIZE); >> + if (!kfence_pool) >> + pr_err("failed to allocate kfence pool\n"); >> + >> + return kfence_pool; >> +} >> + >> +#else >> + >> +static phys_addr_t arm64_kfence_alloc_pool(void) >> +{ >> + return 0; >> +} >> + >> +#endif >> + >> +phys_addr_t early_kfence_pool; > > The compiler will not optimize out this global. This now exists in all > arm64 kernel builds. > > Furthermore, there's no need for this to be phys_addr_t. Nothing > outside map_mem() needs the address, so this can just be a bool. Seems we need this early_kfence_bool to be explicit phys_addr_t as we need to mark/clear NOMAP for the region, so that it will not do linear mapping in the for loop. > > I'd recommend moving the variable under CONFIG_KFENCE, and in the asm > header, just having a static inline helper function e.g. > arm64_kfence_early_pool(). That helper just returns false in the > !CONFIG_KFENCE case. > >> static void __init map_mem(pgd_t *pgdp) >> { >> static const u64 direct_map_end = _PAGE_END(VA_BITS_MIN); >> @@ -543,6 +572,10 @@ static void __init map_mem(pgd_t *pgdp) >> */ >> BUILD_BUG_ON(pgd_index(direct_map_end - 1) == pgd_index(direct_map_end)); >> >> + early_kfence_pool = arm64_kfence_alloc_pool(); >> + if (early_kfence_pool) >> + memblock_mark_nomap(early_kfence_pool, KFENCE_POOL_SIZE); >> + >> if (can_set_direct_map()) >> flags |= NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; >> >> @@ -608,6 +641,17 @@ static void __init map_mem(pgd_t *pgdp) >> } >> } >> #endif >> + >> + /* Kfence pool needs page-level mapping */ >> + if (early_kfence_pool) { >> + __map_memblock(pgdp, early_kfence_pool, >> + early_kfence_pool + KFENCE_POOL_SIZE, >> + pgprot_tagged(PAGE_KERNEL), >> + NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS); >> + memblock_clear_nomap(early_kfence_pool, KFENCE_POOL_SIZE); >> + /* kfence_pool really mapped now */ >> + kfence_set_pool(early_kfence_pool); >> + } >> } >> >> void mark_rodata_ro(void) >> diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c >> index 79dd201..7ce5295 100644 >> --- a/arch/arm64/mm/pageattr.c >> +++ b/arch/arm64/mm/pageattr.c >> @@ -7,10 +7,12 @@ >> #include >> #include >> #include >> +#include >> >> #include >> #include >> #include >> +#include >> >> struct page_change_data { >> pgprot_t set_mask; >> @@ -22,12 +24,15 @@ bool rodata_full __ro_after_init = IS_ENABLED(CONFIG_RODATA_FULL_DEFAULT_ENABLED >> bool can_set_direct_map(void) >> { >> /* >> - * rodata_full, DEBUG_PAGEALLOC and KFENCE require linear map to be >> + * rodata_full and DEBUG_PAGEALLOC require linear map to be >> * mapped at page granularity, so that it is possible to >> * protect/unprotect single pages. >> + * >> + * Kfence pool requires page granularity mapping also if we init it >> + * late. >> */ >> return (rodata_enabled && rodata_full) || debug_pagealloc_enabled() || >> - IS_ENABLED(CONFIG_KFENCE); >> + (IS_ENABLED(CONFIG_KFENCE) && !early_kfence_pool); >> } >> >> static int change_page_range(pte_t *ptep, unsigned long addr, void *data) >> diff --git a/include/linux/kfence.h b/include/linux/kfence.h >> index 726857a..570d4e3 100644 >> --- a/include/linux/kfence.h >> +++ b/include/linux/kfence.h >> @@ -64,6 +64,12 @@ static __always_inline bool is_kfence_address(const void *addr) >> void __init kfence_alloc_pool(void); >> >> /** >> + * kfence_set_pool() - allows an arch to set the >> + * KFENCE pool during early init >> + */ >> +void __init kfence_set_pool(phys_addr_t addr); >> + >> +/** >> * kfence_init() - perform KFENCE initialization at boot time >> * >> * Requires that kfence_alloc_pool() was called before. This sets up the >> @@ -222,8 +228,10 @@ bool __kfence_obj_info(struct kmem_obj_info *kpp, void *object, struct slab *sla >> >> #else /* CONFIG_KFENCE */ >> >> +#define KFENCE_POOL_SIZE 0 >> static inline bool is_kfence_address(const void *addr) { return false; } >> static inline void kfence_alloc_pool(void) { } >> +static inline void kfence_set_pool(phys_addr_t addr) { } >> static inline void kfence_init(void) { } >> static inline void kfence_shutdown_cache(struct kmem_cache *s) { } >> static inline void *kfence_alloc(struct kmem_cache *s, size_t size, gfp_t flags) { return NULL; } >> diff --git a/mm/kfence/core.c b/mm/kfence/core.c >> index 5349c37..0765395 100644 >> --- a/mm/kfence/core.c >> +++ b/mm/kfence/core.c >> @@ -814,12 +814,21 @@ void __init kfence_alloc_pool(void) >> if (!kfence_sample_interval) >> return; >> >> + /* if the pool has already been initialized by arch, skip the below */ >> + if (__kfence_pool) >> + return; >> + >> __kfence_pool = memblock_alloc(KFENCE_POOL_SIZE, PAGE_SIZE); >> >> if (!__kfence_pool) >> pr_err("failed to allocate pool\n"); >> } >> >> +void __init kfence_set_pool(phys_addr_t addr) >> +{ >> + __kfence_pool = phys_to_virt(addr); >> +} > > Please move this function to the header as a static inline function, > because nothing else other than arm64 needs this, having this function > be here introduces a .text size increase for everyone. > > The function is so short that having it as a static inline function is > fine, and will save a few bytes of .text. Reasonable! Thanks! > >> static void kfence_init_enable(void) >> { >> if (!IS_ENABLED(CONFIG_KFENCE_STATIC_KEYS)) >> -- >> 2.7.4 >> Thanks, Zhenhua