From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 457EFC6FD1C for ; Tue, 14 Mar 2023 06:35:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AD3FF6B0072; Tue, 14 Mar 2023 02:35:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A84DD6B0074; Tue, 14 Mar 2023 02:35:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9255E6B0075; Tue, 14 Mar 2023 02:35:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 82DD96B0072 for ; Tue, 14 Mar 2023 02:35:42 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 38460160AE4 for ; Tue, 14 Mar 2023 06:35:42 +0000 (UTC) X-FDA: 80566542924.18.F1A12D7 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by imf20.hostedemail.com (Postfix) with ESMTP id 0460B1C000A for ; Tue, 14 Mar 2023 06:35:38 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=VrsN7ytU; spf=pass (imf20.hostedemail.com: domain of quic_zhenhuah@quicinc.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=quic_zhenhuah@quicinc.com; dmarc=pass (policy=none) header.from=quicinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678775739; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ROJMYYbxq9FJwbyRIIIbMG/dzRppxAUMWepU8mV5Dgk=; b=UQI/n6akEgOCP6y5W87Uc7NPz5NmzsGYrImvEE3z2yij/B7vNrvSxavFZvLEXB/7aKzQAi 2XLl2LBeNXgceyx2gkVbC9NN6SJbMjVOkUTwoS9EyUKOiqC8K/9uFWrb8I/MWbOuCgQbWk AxK6jariJOON5Q1YyD1gXpkPLqLFxeU= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=VrsN7ytU; spf=pass (imf20.hostedemail.com: domain of quic_zhenhuah@quicinc.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=quic_zhenhuah@quicinc.com; dmarc=pass (policy=none) header.from=quicinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678775739; a=rsa-sha256; cv=none; b=qFySTmOZohyswFWpTv5is3H92VGKLv69oDh/Vr/9DpYyTRIyXdmQsbwu7ou9b3SlYpPT6B +UQ5/cfoACdyGmxtOtmC6QoZx5f7Or88K9O112H+SN0nOzuiA8LHMtg3W6qMzDWwNHe6dJ 7H+b7nAK2KGO9GFKae9paaKi9uBFy5E= Received: from pps.filterd (m0279868.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32E3YkaK032233; Tue, 14 Mar 2023 06:35:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=qcppdkim1; bh=ROJMYYbxq9FJwbyRIIIbMG/dzRppxAUMWepU8mV5Dgk=; b=VrsN7ytUTbwOzuzDr0mGPygcBcuJOp89Cdf+rTAZdldUU0MafUApQAbsh+UYUGA307cU dCAnQe7I+ZtdKSqXRpcbDFx2h/KZAVRpSsuKVYfnTuFE/AlbVSaoBNSCf6C0DSCWaRVp 0N1oDXu9lBtBjHx1vlPnEbqjq6UYqwGd9KoT1cJCfnehkv+PrN9MD9rSzJkMDQcuOY9q /EoFUm1YPSX+reJnTlZsczkee99vLnS6sfBcrd5ZQIEj5oQAY7SuXx64bcfeJXucCdjA o25+2agZF9nGKnbIZYyORw3Q9ayov8pLHPmD5b1gj99ec0vFE5wh5NJEmrnvDr0eBNWJ oQ== Received: from nalasppmta03.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3pa9gfhgbf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 14 Mar 2023 06:35:24 +0000 Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 32E6ZNoS012097 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 14 Mar 2023 06:35:23 GMT Received: from [10.239.132.245] (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Mon, 13 Mar 2023 23:35:18 -0700 Message-ID: <8c032dab-bb59-f212-7fe3-8a069fa8dc69@quicinc.com> Date: Tue, 14 Mar 2023 14:35:15 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Subject: Re: [PATCH v7] mm,kfence: decouple kfence from page granularity mapping judgement Content-Language: en-US To: , , , , , , , , , , CC: , , , , , References: <1678771586-13332-1-git-send-email-quic_zhenhuah@quicinc.com> From: Zhenhua Huang In-Reply-To: <1678771586-13332-1-git-send-email-quic_zhenhuah@quicinc.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: Mf0eLHx-FtTmVmKdhmxguNGAIgQ7z5T1 X-Proofpoint-ORIG-GUID: Mf0eLHx-FtTmVmKdhmxguNGAIgQ7z5T1 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-13_13,2023-03-14_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 mlxscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 clxscore=1015 phishscore=0 bulkscore=0 spamscore=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303140057 X-Stat-Signature: iitfyjy7ywgzyxti6popccmtsytdu7g7 X-Rspam-User: X-Rspamd-Queue-Id: 0460B1C000A X-Rspamd-Server: rspam06 X-HE-Tag: 1678775738-776661 X-HE-Meta: U2FsdGVkX1/GHVQYYlx7K0OyN8A/4Ay7ipysopoPM2CsWimtd3yv9OYp8mnn5p8m8TXgYFd+NNoxWPtfr0h7bJvE/jeejbxZaTphb24RTsuXMRyMzDTbfzspDrtzg5lMRoBN5gpMYK+C5Tz2+FEyeRbJxFFBPDrVXSRPdjj5bUS2K9fka4DkcaiG5ita7NStHelzZmBmmkWlyT1rLh9PDDuR1piCoBcUwJLt8Z+dRUsI6SaO55EU9HfWhYEN0CJXIV2QIMf39VvTgsZlmCKE3/KdKuMC0vclrr0rQ8c1y556govWH12M0cFp8hlB075Qut7JdmZrLUqXrGj6CT5g2ndPMqrlcrl1Bxz3ql9JWjBf7e9usUu7AuNjPyxwXoXqNACgr+dadXJJh4My65S2GxwxLEkGsb36kJET0kHCyTJB2AT8W+2adL6D0WA8DSBhj4u7tvdkFxWwy9TSFjrcG8FKYGf7riOya1FTAUED5/mwxxjFeH0r3B79tnDfQXOaC1BDtqqDGiouQLRUzRPcJSU2yMhh8WCs27gVQC0MkidYO+XfmqIgnHBAuh+V/h+rFG9y7B6PkdVXsIzKNgAJ6NhNAOvyRsoQ3QzJ6d5ct+Ryy2L9XEWtiyQLvZycA9E51KkiCJYQJTLgTy78Xf9Z2s9Em5xb8pVCzdrBeZigDfPTPqHSCnXJvGOZsGGJa6CmWi1lcd1i8Ifo5AW3kRF9KNKKeAFeJMrV61CzfK+Dxm23K4HPcOU4yT7Y88ubD0ktPIzb6IYhXv4O77fmr6aZChuc3K03n0rwaeqb8bythUmxoedRpytoA+17UUu0yNescQtFNMdj2U9O77T90PvlE/yoUi1LZfOWIISG7Gt2O9dcTcHKdbOpmL9U1A1HjFVkfmuzYxWRcK9gIPGpoTkViUHqx45ch2WHj9rUlV7yjyvSS6Pz5brHG2ZLpAg4R01Mcbkp8lsgHlYr/fkFalz xCxldaSE MiW0nr6XnQE8BvWLKsRqcm2O8xz8MOjNon3RGc5Ani30Vt5K/tW70rWMPbPl/RlF94mwQv8IXrEL0/bs8jUS/aUz0gap7rjE01aINoLD3sSal/vtIiHU/P0HHiIY35odHohzXPAkpULFXJ7M+ghvA90KqRbdj66ZQwzzMIFnKf9rnGm6rK6Oi+Nn5zobXl7iazFwPnqujFqUBaU0XDdYgv4vNijlQ63IYIVqrPD0ilwyDNCQEbkaxUvmAXuOk3+6zrUMaY0uJUTaAbPifGP9GN5MUEESTiNs+s6LIxOQlhM0mbUuf/rMSAdsIXmuGToHXpg8+2u6qIscTuvJ0CrcNkXGLbq1+L/lQztNs0M7zx9DDCf76QV6HaL/YCUISUTxGOQ7OysujBCkjMnxp1YMChvqhtw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2023/3/14 13:26, Zhenhua Huang wrote: > Kfence only needs its pool to be mapped as page granularity, if it is > inited early. Previous judgement was a bit over protected. From [1], Mark > suggested to "just map the KFENCE region a page granularity". So I > decouple it from judgement and do page granularity mapping for kfence > pool only. Need to be noticed that late init of kfence pool still requires > page granularity mapping. > > Page granularity mapping in theory cost more(2M per 1GB) memory on arm64 > platform. Like what I've tested on QEMU(emulated 1GB RAM) with > gki_defconfig, also turning off rodata protection: > Before: > [root@liebao ]# cat /proc/meminfo > MemTotal: 999484 kB > After: > [root@liebao ]# cat /proc/meminfo > MemTotal: 1001480 kB > > To implement this, also relocate the kfence pool allocation before the > linear mapping setting up, arm64_kfence_alloc_pool is to allocate phys > addr, __kfence_pool is to be set after linear mapping set up. > > LINK: [1] https://lore.kernel.org/linux-arm-kernel/Y+IsdrvDNILA59UN@FVFF77S0Q05N/ > Suggested-by: Mark Rutland > Signed-off-by: Zhenhua Huang > --- > arch/arm64/mm/mmu.c | 43 +++++++++++++++++++++++++++++++++++++++++++ > arch/arm64/mm/pageattr.c | 8 ++++++-- > include/linux/kfence.h | 11 +++++++++++ > mm/kfence/core.c | 9 +++++++++ > 4 files changed, 69 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c > index 6f9d889..7f34206 100644 > --- a/arch/arm64/mm/mmu.c > +++ b/arch/arm64/mm/mmu.c > @@ -24,6 +24,7 @@ > #include > #include > #include > +#include > > #include > #include > @@ -525,6 +526,33 @@ static int __init enable_crash_mem_map(char *arg) > } > early_param("crashkernel", enable_crash_mem_map); > > +#ifdef CONFIG_KFENCE > + > +static phys_addr_t arm64_kfence_alloc_pool(void) > +{ > + phys_addr_t kfence_pool; > + > + if (!kfence_sample_interval) > + return 0; > + > + kfence_pool = memblock_phys_alloc(KFENCE_POOL_SIZE, PAGE_SIZE); > + if (!kfence_pool) > + pr_err("failed to allocate kfence pool\n"); > + > + return kfence_pool; > +} > + > +#else > + > +static phys_addr_t arm64_kfence_alloc_pool(void) > +{ > + return 0; > +} > + > +#endif > + > +phys_addr_t early_kfence_pool; I suddenly realized it would break other arch as only defined under arm64, let me relocate the declaration to arm64 headers as well. Sorry for inconvenience. > + > static void __init map_mem(pgd_t *pgdp) > { > static const u64 direct_map_end = _PAGE_END(VA_BITS_MIN); > @@ -543,6 +571,10 @@ static void __init map_mem(pgd_t *pgdp) > */ > BUILD_BUG_ON(pgd_index(direct_map_end - 1) == pgd_index(direct_map_end)); > > + early_kfence_pool = arm64_kfence_alloc_pool(); > + if (early_kfence_pool) > + memblock_mark_nomap(early_kfence_pool, KFENCE_POOL_SIZE); > + > if (can_set_direct_map()) > flags |= NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; > > @@ -608,6 +640,17 @@ static void __init map_mem(pgd_t *pgdp) > } > } > #endif > + > + /* Kfence pool needs page-level mapping */ > + if (early_kfence_pool) { > + __map_memblock(pgdp, early_kfence_pool, > + early_kfence_pool + KFENCE_POOL_SIZE, > + pgprot_tagged(PAGE_KERNEL), > + NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS); > + memblock_clear_nomap(early_kfence_pool, KFENCE_POOL_SIZE); > + /* kfence_pool really mapped now */ > + kfence_set_pool(early_kfence_pool); > + } > } > > void mark_rodata_ro(void) > diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c > index 79dd201..83f57d2 100644 > --- a/arch/arm64/mm/pageattr.c > +++ b/arch/arm64/mm/pageattr.c > @@ -7,6 +7,7 @@ > #include > #include > #include > +#include > > #include > #include > @@ -22,12 +23,15 @@ bool rodata_full __ro_after_init = IS_ENABLED(CONFIG_RODATA_FULL_DEFAULT_ENABLED > bool can_set_direct_map(void) > { > /* > - * rodata_full, DEBUG_PAGEALLOC and KFENCE require linear map to be > + * rodata_full and DEBUG_PAGEALLOC require linear map to be > * mapped at page granularity, so that it is possible to > * protect/unprotect single pages. > + * > + * Kfence pool requires page granularity mapping also if we init it > + * late. > */ > return (rodata_enabled && rodata_full) || debug_pagealloc_enabled() || > - IS_ENABLED(CONFIG_KFENCE); > + (IS_ENABLED(CONFIG_KFENCE) && !early_kfence_pool); > } > > static int change_page_range(pte_t *ptep, unsigned long addr, void *data) > diff --git a/include/linux/kfence.h b/include/linux/kfence.h > index 726857a..f1330b6 100644 > --- a/include/linux/kfence.h > +++ b/include/linux/kfence.h > @@ -18,6 +18,7 @@ > #include > > extern unsigned long kfence_sample_interval; > +extern phys_addr_t early_kfence_pool; > > /* > * We allocate an even number of pages, as it simplifies calculations to map > @@ -64,6 +65,12 @@ static __always_inline bool is_kfence_address(const void *addr) > void __init kfence_alloc_pool(void); > > /** > + * kfence_set_pool() - allows an arch to set the > + * KFENCE pool during early init > + */ > +void __init kfence_set_pool(phys_addr_t addr); > + > +/** > * kfence_init() - perform KFENCE initialization at boot time > * > * Requires that kfence_alloc_pool() was called before. This sets up the > @@ -222,8 +229,12 @@ bool __kfence_obj_info(struct kmem_obj_info *kpp, void *object, struct slab *sla > > #else /* CONFIG_KFENCE */ > > +extern phys_addr_t early_kfence_pool; > + > +#define KFENCE_POOL_SIZE 0 > static inline bool is_kfence_address(const void *addr) { return false; } > static inline void kfence_alloc_pool(void) { } > +static inline void kfence_set_pool(phys_addr_t addr) { } > static inline void kfence_init(void) { } > static inline void kfence_shutdown_cache(struct kmem_cache *s) { } > static inline void *kfence_alloc(struct kmem_cache *s, size_t size, gfp_t flags) { return NULL; } > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > index 5349c37..0765395 100644 > --- a/mm/kfence/core.c > +++ b/mm/kfence/core.c > @@ -814,12 +814,21 @@ void __init kfence_alloc_pool(void) > if (!kfence_sample_interval) > return; > > + /* if the pool has already been initialized by arch, skip the below */ > + if (__kfence_pool) > + return; > + > __kfence_pool = memblock_alloc(KFENCE_POOL_SIZE, PAGE_SIZE); > > if (!__kfence_pool) > pr_err("failed to allocate pool\n"); > } > > +void __init kfence_set_pool(phys_addr_t addr) > +{ > + __kfence_pool = phys_to_virt(addr); > +} > + > static void kfence_init_enable(void) > { > if (!IS_ENABLED(CONFIG_KFENCE_STATIC_KEYS))