From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E8E7C64EC4 for ; Thu, 9 Mar 2023 11:26:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4A486B0072; Thu, 9 Mar 2023 06:26:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DF921280001; Thu, 9 Mar 2023 06:26:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE88D6B0078; Thu, 9 Mar 2023 06:26:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id BE1AE6B0072 for ; Thu, 9 Mar 2023 06:26:32 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8DE75160F57 for ; Thu, 9 Mar 2023 11:26:32 +0000 (UTC) X-FDA: 80549131824.28.A7AF3EA Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by imf07.hostedemail.com (Postfix) with ESMTP id 5A92F4001F for ; Thu, 9 Mar 2023 11:26:29 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=SeGS5rYw; spf=pass (imf07.hostedemail.com: domain of quic_zhenhuah@quicinc.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=quic_zhenhuah@quicinc.com; dmarc=pass (policy=none) header.from=quicinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678361189; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wsbzJND1dSHOF71Db7QhoIK6N4yaqlfxD3cGUuJMWRE=; b=5WRDOHuBw5JYV5uNxaFC24lxbfdgwQZUOJxcxz95utgSOBK3v++5zFKlTa4wFKq4Et44QB LzFzQtg9lc8xT4OjJm+vICKB8GxdruaxiM8maVQKLWVpyesdI5qalu6YKfYccp4fboX5+Y dVZmXu58D1g708WXLPLVi0tX4kj/4A4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=SeGS5rYw; spf=pass (imf07.hostedemail.com: domain of quic_zhenhuah@quicinc.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=quic_zhenhuah@quicinc.com; dmarc=pass (policy=none) header.from=quicinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678361189; a=rsa-sha256; cv=none; b=1GuM9V67EY/I6BgcF0vB67/Q3ApIiSe4MLENcRYFOIcPM7i5jbpNvTE7IPnBksJ0BA4tfK 32Y1BkRoSVBoCxYVKGoovfvaopqZQIOfqVstlNGvVjgGTL2H4Ftpk+yjsOM1HoKVjbhCNa I5WuVZOAdf/Ec4ouMAbvXVHjyChbIWI= Received: from pps.filterd (m0279869.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3298QOuj025524; Thu, 9 Mar 2023 11:26:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=qcppdkim1; bh=wsbzJND1dSHOF71Db7QhoIK6N4yaqlfxD3cGUuJMWRE=; b=SeGS5rYwkOCrKFCWk2satUB4Omz7iTIkPs1qZURgNagidEEHqOe3Wayvo43/gaPBmYuM Ltyx0jM6+o8QdcdYKMk4sdl12ptR13+aC6wndhyyFORdEQLEk/BwpskUeFgvpce8Nnts 0LRX5dHlDsBxir96N6QpyKlyPBUgmFbPRm7Y5aEK6WUKS5E2b3lA1rGo57R/hhT+uucC 2d2jzZyp4GMlY/hgf0hBABGlA++MkNtT4GEAWHDM0uKgrtCeWdjymTMQhWGDb5X6nzsc rhu1VmRUPvLud+MMYCpFUZi5hnxbnrlT1NcPKRNF1eVtSOLUG663la5CwU0eWDd+SN8R aw== Received: from nalasppmta05.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3p75yq9bd5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 09 Mar 2023 11:26:10 +0000 Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 329BQ92c013812 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 9 Mar 2023 11:26:09 GMT Received: from [10.253.32.183] (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 9 Mar 2023 03:26:05 -0800 Message-ID: <3e8606e4-0585-70fa-433d-75bf115aa191@quicinc.com> Date: Thu, 9 Mar 2023 19:26:02 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.6.1 Subject: Re: [PATCH] mm,kfence: decouple kfence from page granularity mapping judgement Content-Language: en-US To: Marco Elver CC: , , , , , , , , , , , , , , References: <1678349122-19279-1-git-send-email-quic_zhenhuah@quicinc.com> <706340ef-1745-c1e4-be4d-358d5db4c05e@quicinc.com> From: Zhenhua Huang In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: Xhr9FFRlEqPZmOBQGEmXNQor7FN0hkEv X-Proofpoint-ORIG-GUID: Xhr9FFRlEqPZmOBQGEmXNQor7FN0hkEv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-09_06,2023-03-08_03,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1015 adultscore=0 spamscore=0 priorityscore=1501 impostorscore=0 suspectscore=0 lowpriorityscore=0 phishscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303090091 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5A92F4001F X-Rspam-User: X-Stat-Signature: bompeiji9793x643ez1eg1hxqx9gshe6 X-HE-Tag: 1678361189-954376 X-HE-Meta: U2FsdGVkX19NdV7edmvGFf1egHPzcdvAdNIsjI9v9VKeR0ppAkMt1czHsT4gpr6yNGtex8r1GqFWz2ZznZcbCDV24qKeYlXeJLnl0Gie7gZuUyoNz1uVsA6hVa+JS2qq3RbZbmFqSPxlcvrtWQ/uaKF6DqmqwMX1T7G9++iIGCZJPhRO+lxJIZ8o38e/o2/NURNgD3BpwSL9THjYpYTyyeLwRay9cJ6mEftlc1qEdYaQyigmbYLKArgKJqWfQW6EJESpfV/noLuT+U6Zpa2xcNZsinfM+215WEV1PimaNkREUcgi3isQpZhmkQa1wHgDAQMvuLy7y916wSCHDhwHNmactJnHgAZ39T20rBo2rLu7QRFq+555Xx08+mJ/KYlt256C5Zxzt8DR/lKgLRpuUugVIKnv0qviNTZ46DYCALKWbgDtfJXWoI9uwtrsDV379lX2Z/vYf/mZEZGgkrxNR3UszFdE9tahWblkQjtby53eJcV8qq7ip1BgUaj2W4Ta1GO9gkvNavvM1Rsm1Ln+fBJV/VDUhGKtFkFxtU6Z9XotN+n4of5o/TDXOzCeutGO+MjrTsvGgonLG36j+L1SawEx7gyh0x/XIZK5qurqJeTKciZIjEknHUynXnSnXK9EeTVEHHJhqdu6MSvve9RnmAdDCoh7te4Ks6zRMgDjrqHfhdVOSOgGHfSuYS+S+qtcFL1SzDgCdwx7dodrJrJPKiZAyg4nDDUT/Jm5gw9K0WOygIpieYL0t0BGizof5JIGR5mBrN/AqklhzC2gEvXV6G+Clv4+le+cCNPmcI60G5jV5O0HdLwyEkDzsIqXo8bFXkTYHUT4wGMya5uOYwIvK8kpeiGCysMTHXb75dZx87o4+RDUm316s0WS25EJMVtenxXTVPsf9VGyvPXtW9ucFwyzcxuyGK8xgA/TGaQBtPAY001TqNqJynkwZSQOSdpf8zkTr0L+YoXxjP+Vx5m xwQkYYQD aK/FD2bV6Iop/MoBnxQKBzHd+Txii0RaeXBzDKeUFE03JSkwd+nocvQIvbJxkFyzEX2cmy9ktH6BhoDGnm40je8HX5+cPzqlox57fqIwed7Gxk3AKS4ANUz5F2lnKre3uWs7bF/tKZGopxT8yC8i8Nd3oLD8Ua+rsIWlIts/xSST/XmY0ZcLcpgmFhIyvOtLoowAbHqWJSBCGUQVHvkTI6O5cvk4mz+jN1OQ99u7LWkzFIS1tenoyUNaTMM9jFTrXXOzz/or9Y2uPlL5Xq9FuGL5QDZBn4gml9g5E81Qd8OdNXbE/1rozfOO1ZI2u6oybsDk5m4kDPHRPAeRv4YdH4X6WS2+2sv8MoRKKBZR3rLuVtWltmfnjj3jt4tsV2SkOYDWx3AGAUZH/SN2WXrMz6OWTGw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Thanks Marco! On 2023/3/9 19:09, Marco Elver wrote: > On Thu, 9 Mar 2023 at 12:04, Zhenhua Huang wrote: >> >> Thanks Marco. >> >> On 2023/3/9 18:33, Marco Elver wrote: >>> On Thu, 9 Mar 2023 at 09:05, Zhenhua Huang wrote: >>>> >>>> Kfence only needs its pool to be mapped as page granularity, previous >>>> judgement was a bit over protected. Decouple it from judgement and do >>>> page granularity mapping for kfence pool only [1]. >>>> >>>> To implement this, also relocate the kfence pool allocation before the >>>> linear mapping setting up, kfence_alloc_pool is to allocate phys addr, >>>> __kfence_pool is to be set after linear mapping set up. >>>> >>>> LINK: [1] https://lore.kernel.org/linux-arm-kernel/1675750519-1064-1-git-send-email-quic_zhenhuah@quicinc.com/T/ >>>> Suggested-by: Mark Rutland >>>> Signed-off-by: Zhenhua Huang >>>> --- >>>> arch/arm64/mm/mmu.c | 24 ++++++++++++++++++++++++ >>>> arch/arm64/mm/pageattr.c | 5 ++--- >>>> include/linux/kfence.h | 10 ++++++++-- >>>> init/main.c | 1 - >>>> mm/kfence/core.c | 18 ++++++++++++++---- >>>> 5 files changed, 48 insertions(+), 10 deletions(-) >>>> >>>> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c >>>> index 6f9d889..bd79691 100644 >>>> --- a/arch/arm64/mm/mmu.c >>>> +++ b/arch/arm64/mm/mmu.c >>>> @@ -24,6 +24,7 @@ >>>> #include >>>> #include >>>> #include >>>> +#include >>>> >>>> #include >>>> #include >>>> @@ -532,6 +533,9 @@ static void __init map_mem(pgd_t *pgdp) >>>> phys_addr_t kernel_end = __pa_symbol(__init_begin); >>>> phys_addr_t start, end; >>>> int flags = NO_EXEC_MAPPINGS; >>>> +#ifdef CONFIG_KFENCE >>>> + phys_addr_t kfence_pool = 0; >>>> +#endif >>>> u64 i; >>>> >>>> /* >>>> @@ -564,6 +568,12 @@ static void __init map_mem(pgd_t *pgdp) >>>> } >>>> #endif >>>> >>>> +#ifdef CONFIG_KFENCE >>>> + kfence_pool = kfence_alloc_pool(); >>>> + if (kfence_pool) >>>> + memblock_mark_nomap(kfence_pool, KFENCE_POOL_SIZE); >>>> +#endif >>>> + >>>> /* map all the memory banks */ >>>> for_each_mem_range(i, &start, &end) { >>>> if (start >= end) >>>> @@ -608,6 +618,20 @@ static void __init map_mem(pgd_t *pgdp) >>>> } >>>> } >>>> #endif >>>> + >>>> + /* Kfence pool needs page-level mapping */ >>>> +#ifdef CONFIG_KFENCE >>>> + if (kfence_pool) { >>>> + __map_memblock(pgdp, kfence_pool, >>>> + kfence_pool + KFENCE_POOL_SIZE, >>>> + pgprot_tagged(PAGE_KERNEL), >>>> + NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS); >>>> + memblock_clear_nomap(kfence_pool, KFENCE_POOL_SIZE); >>>> + /* kfence_pool really mapped now */ >>>> + kfence_set_pool(kfence_pool); >>>> + } >>>> +#endif >>>> + >>>> } >>>> >>>> void mark_rodata_ro(void) >>>> diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c >>>> index 79dd201..61156d0 100644 >>>> --- a/arch/arm64/mm/pageattr.c >>>> +++ b/arch/arm64/mm/pageattr.c >>>> @@ -22,12 +22,11 @@ bool rodata_full __ro_after_init = IS_ENABLED(CONFIG_RODATA_FULL_DEFAULT_ENABLED >>>> bool can_set_direct_map(void) >>>> { >>>> /* >>>> - * rodata_full, DEBUG_PAGEALLOC and KFENCE require linear map to be >>>> + * rodata_full and DEBUG_PAGEALLOC require linear map to be >>>> * mapped at page granularity, so that it is possible to >>>> * protect/unprotect single pages. >>>> */ >>>> - return (rodata_enabled && rodata_full) || debug_pagealloc_enabled() || >>>> - IS_ENABLED(CONFIG_KFENCE); >>>> + return (rodata_enabled && rodata_full) || debug_pagealloc_enabled(); >>>> } >>>> >>>> static int change_page_range(pte_t *ptep, unsigned long addr, void *data) >>>> diff --git a/include/linux/kfence.h b/include/linux/kfence.h >>>> index 726857a..0252e74 100644 >>>> --- a/include/linux/kfence.h >>>> +++ b/include/linux/kfence.h >>>> @@ -61,7 +61,12 @@ static __always_inline bool is_kfence_address(const void *addr) >>>> /** >>>> * kfence_alloc_pool() - allocate the KFENCE pool via memblock >>>> */ >>>> -void __init kfence_alloc_pool(void); >>>> +phys_addr_t __init kfence_alloc_pool(void); >>>> + >>>> +/** >>>> + * kfence_set_pool() - KFENCE pool mapped and can be used >>>> + */ >>>> +void __init kfence_set_pool(phys_addr_t addr); >>>> >>>> /** >>>> * kfence_init() - perform KFENCE initialization at boot time >>>> @@ -223,7 +228,8 @@ bool __kfence_obj_info(struct kmem_obj_info *kpp, void *object, struct slab *sla >>>> #else /* CONFIG_KFENCE */ >>>> >>>> static inline bool is_kfence_address(const void *addr) { return false; } >>>> -static inline void kfence_alloc_pool(void) { } >>>> +static inline phys_addr_t kfence_alloc_pool(void) { return (phys_addr_t)NULL; } >>>> +static inline void kfence_set_pool(phys_addr_t addr) { } >>>> static inline void kfence_init(void) { } >>>> static inline void kfence_shutdown_cache(struct kmem_cache *s) { } >>>> static inline void *kfence_alloc(struct kmem_cache *s, size_t size, gfp_t flags) { return NULL; } >>>> diff --git a/init/main.c b/init/main.c >>>> index 4425d17..9aaf217 100644 >>>> --- a/init/main.c >>>> +++ b/init/main.c >>>> @@ -839,7 +839,6 @@ static void __init mm_init(void) >>>> */ >>>> page_ext_init_flatmem(); >>>> init_mem_debugging_and_hardening(); >>>> - kfence_alloc_pool(); >>> >>> This breaks other architectures. >> >> Nice catch. Thanks! >> >>> >>>> report_meminit(); >>>> kmsan_init_shadow(); >>>> stack_depot_early_init(); >>>> diff --git a/mm/kfence/core.c b/mm/kfence/core.c >>>> index 5349c37..dd5cdd5 100644 >>>> --- a/mm/kfence/core.c >>>> +++ b/mm/kfence/core.c >>>> @@ -809,15 +809,25 @@ static void toggle_allocation_gate(struct work_struct *work) >>>> >>>> /* === Public interface ===================================================== */ >>>> >>>> -void __init kfence_alloc_pool(void) >>>> +phys_addr_t __init kfence_alloc_pool(void) >>>> { >>> >>> You could just return here: >>> >>> if (__kfence_pool) >>> return; /* Initialized earlier by arch init code. */ >> >> Yeah. >> >>> >>> ... and see my comments below. >>> >>>> + phys_addr_t kfence_pool; >>>> if (!kfence_sample_interval) >>>> - return; >>>> + return 0; >>>> >>>> - __kfence_pool = memblock_alloc(KFENCE_POOL_SIZE, PAGE_SIZE); >>>> + kfence_pool = memblock_phys_alloc(KFENCE_POOL_SIZE, PAGE_SIZE); >>>> >>>> - if (!__kfence_pool) >>>> + if (!kfence_pool) { >>>> pr_err("failed to allocate pool\n"); >>>> + return 0; >>>> + } >>>> + >>>> + return kfence_pool; >>>> +} >>>> + >>>> +void __init kfence_set_pool(phys_addr_t addr) >>>> +{ >>>> + __kfence_pool = phys_to_virt(addr); >>>> } >>> >>> I would suggest leaving kfence_alloc_pool() to return nothing (with >>> the addition above), and just set __kfence_pool as before. >>> __kfence_pool itself is exported by include/linux/kfence.h, so if you >>> call kfence_alloc_pool() in arm64 earlier, you can access >>> __kfence_pool to get the allocated pool. >> >> Shall we add one new function like arm64_kfence_alloc_pool() ? The >> reason is linear mapping at that time not set up and we must alloc phys >> addr based on memblock. We can't use common kfence_alloc_pool().. > > Ah right - well, you can initialize __kfence_pool however you like > within arm64 init code. Just teaching kfence_alloc_pool() to do > nothing if it's already initialized should be enough. Within > arch/arm64/mm/mmu.c it might be nice to factor out some bits into a > helper like arm64_kfence_alloc_pool(), but would just stick to > whatever is simplest. Many thanks Marco. Let me conclude as following: 1. put arm64_kfence_alloc_pool() within arch/arm64/mm/mmu.c as it's arch_ specific codes. 2. leave kfence_set_pool() to set _kfence_pool within kfence driver, as it may become common part. The reason we still need #2 is because _kfence_pool only can be used after mapping set up, it must be late than pool allocation. Do you have any further suggestion? > > Thanks, > -- Marco