From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63AD8C433EF for ; Mon, 7 Mar 2022 14:09:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F026E8D0003; Mon, 7 Mar 2022 09:09:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E8B4D8D0001; Mon, 7 Mar 2022 09:09:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2C368D0003; Mon, 7 Mar 2022 09:09:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id C0EA08D0001 for ; Mon, 7 Mar 2022 09:09:00 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8AF8961E4C for ; Mon, 7 Mar 2022 14:09:00 +0000 (UTC) X-FDA: 79217771640.05.F2B6A19 Received: from mail-yb1-f174.google.com (mail-yb1-f174.google.com [209.85.219.174]) by imf22.hostedemail.com (Postfix) with ESMTP id 0C002C0011 for ; Mon, 7 Mar 2022 14:08:59 +0000 (UTC) Received: by mail-yb1-f174.google.com with SMTP id j2so31272661ybu.0 for ; Mon, 07 Mar 2022 06:08:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=0yk9BdGE0P4qhl7kTJOlzN4O3c8JJe4T1InJDUcBabo=; b=B4g1MvdpCEJjzUCjLv5igLIdZpjRxA9oIdPFfJRLD3JitCxFQCJ8h2ZzUK+yhrvVD7 tpaANJLSgE0iz66eompeUXkipYphAGsLoffZPRvxsgpqFT/k15FXqu0eJKYhaRSgoL2J juS8mtIPNt5KjOMwJ+E0nxt9mQtwiATlRz4Eoe/KtomcN/fPcf8d1e2i0Tfq8OmWauz9 cFS7+UN8tfMabPzzvp3psLtC4lN6ixS0XfJXdVj+R8/KJyamIM2GP4jp7eJypBqWlmG4 ARxusBD+GeaszzXh7b6iGIoKB/wF7lXozJOaWyxx9rmKpElGj4PGuovTFxEPk9cVzg5/ 8vQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=0yk9BdGE0P4qhl7kTJOlzN4O3c8JJe4T1InJDUcBabo=; b=hSq+DHYf3D8Ult7cKgaRgiy6jJ2azYgAHJItbr4CQaULJNLPaYJCHnQMxk4sdrHo6X FIrecUlqQD78r9GiN+gpTZ2tVuX+yni6Gt/1LBd8MapkZOBA2poZjfN8pLn+njgGoNxV czxDMcy7ugyLKsg0VVme8QhpZvKuuBvHAJMam7CgvX2WKO7mJ80ynikYGIOsluQAWWdi Fu2SQVORBgqGgR3pC3CEBR4mRKMe1QvKn/3M2mrjyhjzOzWWMJEopJbnN025/JuDUOpu qzjKaM9VrfY/PgB/vNFcPJyFXj9pxxzNU5tOiM5TVo0vSk03F7lUfwOjgKLMaw4cEi9I XOcg== X-Gm-Message-State: AOAM532VDkdQFkr0GdhLdeqUE0vPcTZ5CYXyqtwQHBzynTI8ktSvQuQo g48MRTUCbTZRc0uu4do9CsCS5fWNF1Cm5thd6LrySdGYJDg= X-Google-Smtp-Source: ABdhPJx254W9rRRb0U/Y3PdIOLgfsiCQQVMMPYuUynQoKnS4kFv+mwwwzWrUnqWtgqvZfcdq+VDkLc293umr+/lxW8Y= X-Received: by 2002:a05:6902:203:b0:628:7b6f:2845 with SMTP id j3-20020a056902020300b006287b6f2845mr8118330ybs.533.1646662139111; Mon, 07 Mar 2022 06:08:59 -0800 (PST) MIME-Version: 1.0 References: <20220307074516.6920-1-dtcccc@linux.alibaba.com> <20220307074516.6920-3-dtcccc@linux.alibaba.com> In-Reply-To: <20220307074516.6920-3-dtcccc@linux.alibaba.com> From: Marco Elver Date: Mon, 7 Mar 2022 15:08:22 +0100 Message-ID: Subject: Re: [PATCH v3 2/2] kfence: Alloc kfence_pool after system startup To: Tianchen Ding Cc: Alexander Potapenko , Dmitry Vyukov , Andrew Morton , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0C002C0011 X-Rspam-User: Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=B4g1Mvdp; spf=pass (imf22.hostedemail.com: domain of elver@google.com designates 209.85.219.174 as permitted sender) smtp.mailfrom=elver@google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: zf5pqan73zhi6y3k89y6hs5t6pn659j6 X-HE-Tag: 1646662139-158105 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, 7 Mar 2022 at 08:45, Tianchen Ding wrote: > > Allow enabling KFENCE after system startup by allocating its pool via the > page allocator. This provides the flexibility to enable KFENCE even if it > wasn't enabled at boot time. > > Signed-off-by: Tianchen Ding This looks good, thanks! Reviewed-by: Marco Elver Tested-by: Marco Elver > --- > mm/kfence/core.c | 111 ++++++++++++++++++++++++++++++++++++++--------- > 1 file changed, 90 insertions(+), 21 deletions(-) > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > index caa4e84c8b79..f126b53b9b85 100644 > --- a/mm/kfence/core.c > +++ b/mm/kfence/core.c > @@ -96,7 +96,7 @@ static unsigned long kfence_skip_covered_thresh __read_mostly = 75; > module_param_named(skip_covered_thresh, kfence_skip_covered_thresh, ulong, 0644); > > /* The pool of pages used for guard pages and objects. */ > -char *__kfence_pool __ro_after_init; > +char *__kfence_pool __read_mostly; > EXPORT_SYMBOL(__kfence_pool); /* Export for test modules. */ > > /* > @@ -537,17 +537,19 @@ static void rcu_guarded_free(struct rcu_head *h) > kfence_guarded_free((void *)meta->addr, meta, false); > } > > -static bool __init kfence_init_pool(void) > +/* > + * Initialization of the KFENCE pool after its allocation. > + * Returns 0 on success; otherwise returns the address up to > + * which partial initialization succeeded. > + */ > +static unsigned long kfence_init_pool(void) > { > unsigned long addr = (unsigned long)__kfence_pool; > struct page *pages; > int i; > > - if (!__kfence_pool) > - return false; > - > if (!arch_kfence_init_pool()) > - goto err; > + return addr; > > pages = virt_to_page(addr); > > @@ -565,7 +567,7 @@ static bool __init kfence_init_pool(void) > > /* Verify we do not have a compound head page. */ > if (WARN_ON(compound_head(&pages[i]) != &pages[i])) > - goto err; > + return addr; > > __SetPageSlab(&pages[i]); > } > @@ -578,7 +580,7 @@ static bool __init kfence_init_pool(void) > */ > for (i = 0; i < 2; i++) { > if (unlikely(!kfence_protect(addr))) > - goto err; > + return addr; > > addr += PAGE_SIZE; > } > @@ -595,7 +597,7 @@ static bool __init kfence_init_pool(void) > > /* Protect the right redzone. */ > if (unlikely(!kfence_protect(addr + PAGE_SIZE))) > - goto err; > + return addr; > > addr += 2 * PAGE_SIZE; > } > @@ -608,9 +610,21 @@ static bool __init kfence_init_pool(void) > */ > kmemleak_free(__kfence_pool); > > - return true; > + return 0; > +} > + > +static bool __init kfence_init_pool_early(void) > +{ > + unsigned long addr; > + > + if (!__kfence_pool) > + return false; > + > + addr = kfence_init_pool(); > + > + if (!addr) > + return true; > > -err: > /* > * Only release unprotected pages, and do not try to go back and change > * page attributes due to risk of failing to do so as well. If changing > @@ -623,6 +637,26 @@ static bool __init kfence_init_pool(void) > return false; > } > > +static bool kfence_init_pool_late(void) > +{ > + unsigned long addr, free_size; > + > + addr = kfence_init_pool(); > + > + if (!addr) > + return true; > + > + /* Same as above. */ > + free_size = KFENCE_POOL_SIZE - (addr - (unsigned long)__kfence_pool); > +#ifdef CONFIG_CONTIG_ALLOC > + free_contig_range(page_to_pfn(virt_to_page(addr)), free_size / PAGE_SIZE); > +#else > + free_pages_exact((void *)addr, free_size); > +#endif > + __kfence_pool = NULL; > + return false; > +} > + > /* === DebugFS Interface ==================================================== */ > > static int stats_show(struct seq_file *seq, void *v) > @@ -771,31 +805,66 @@ void __init kfence_alloc_pool(void) > pr_err("failed to allocate pool\n"); > } > > +static void kfence_init_enable(void) > +{ > + if (!IS_ENABLED(CONFIG_KFENCE_STATIC_KEYS)) > + static_branch_enable(&kfence_allocation_key); > + WRITE_ONCE(kfence_enabled, true); > + queue_delayed_work(system_unbound_wq, &kfence_timer, 0); > + pr_info("initialized - using %lu bytes for %d objects at 0x%p-0x%p\n", KFENCE_POOL_SIZE, > + CONFIG_KFENCE_NUM_OBJECTS, (void *)__kfence_pool, > + (void *)(__kfence_pool + KFENCE_POOL_SIZE)); > +} > + > void __init kfence_init(void) > { > + stack_hash_seed = (u32)random_get_entropy(); > + > /* Setting kfence_sample_interval to 0 on boot disables KFENCE. */ > if (!kfence_sample_interval) > return; > > - stack_hash_seed = (u32)random_get_entropy(); > - if (!kfence_init_pool()) { > + if (!kfence_init_pool_early()) { > pr_err("%s failed\n", __func__); > return; > } > > - if (!IS_ENABLED(CONFIG_KFENCE_STATIC_KEYS)) > - static_branch_enable(&kfence_allocation_key); > - WRITE_ONCE(kfence_enabled, true); > - queue_delayed_work(system_unbound_wq, &kfence_timer, 0); > - pr_info("initialized - using %lu bytes for %d objects at 0x%p-0x%p\n", KFENCE_POOL_SIZE, > - CONFIG_KFENCE_NUM_OBJECTS, (void *)__kfence_pool, > - (void *)(__kfence_pool + KFENCE_POOL_SIZE)); > + kfence_init_enable(); > +} > + > +static int kfence_init_late(void) > +{ > + const unsigned long nr_pages = KFENCE_POOL_SIZE / PAGE_SIZE; > +#ifdef CONFIG_CONTIG_ALLOC > + struct page *pages; > + > + pages = alloc_contig_pages(nr_pages, GFP_KERNEL, first_online_node, NULL); > + if (!pages) > + return -ENOMEM; > + __kfence_pool = page_to_virt(pages); > +#else > + if (nr_pages > MAX_ORDER_NR_PAGES) { > + pr_warn("KFENCE_NUM_OBJECTS too large for buddy allocator\n"); > + return -EINVAL; > + } > + __kfence_pool = alloc_pages_exact(KFENCE_POOL_SIZE, GFP_KERNEL); > + if (!__kfence_pool) > + return -ENOMEM; > +#endif > + > + if (!kfence_init_pool_late()) { > + pr_err("%s failed\n", __func__); > + return -EBUSY; > + } > + > + kfence_init_enable(); > + return 0; > } > > static int kfence_enable_late(void) > { > if (!__kfence_pool) > - return -EINVAL; > + return kfence_init_late(); > > WRITE_ONCE(kfence_enabled, true); > queue_delayed_work(system_unbound_wq, &kfence_timer, 0); > -- > 2.27.0 >