From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-24.8 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 014D2C433DB for ; Mon, 8 Mar 2021 11:26:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3710E6024A for ; Mon, 8 Mar 2021 11:26:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3710E6024A Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 99F338D000F; Mon, 8 Mar 2021 06:26:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 94F786B00B3; Mon, 8 Mar 2021 06:26:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A17F8D000F; Mon, 8 Mar 2021 06:26:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0103.hostedemail.com [216.40.44.103]) by kanga.kvack.org (Postfix) with ESMTP id 59D796B00B2 for ; Mon, 8 Mar 2021 06:26:20 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 155E31803A81C for ; Mon, 8 Mar 2021 11:26:20 +0000 (UTC) X-FDA: 77896478520.13.7D2AFC1 Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by imf10.hostedemail.com (Postfix) with ESMTP id AA7FB407F8F7 for ; Mon, 8 Mar 2021 11:26:16 +0000 (UTC) Received: by mail-wm1-f48.google.com with SMTP id i9so5771878wml.0 for ; Mon, 08 Mar 2021 03:26:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=C/TD0CHDy4RoQIxiEpylzxCplnrxE4ZRPieQx5Wj4OY=; b=aHi886dZS/nUhzBaf4/0j6mAsNudK+Zx4FowGzf32D+UUdxcrvHEQF+bmFOtVe3nD2 0mNTVh0tVvUSr5e/Vw97tw2bnI8TJbv5GdZG+/lOtVCKHJaBVIKN5S45Rtv+puvmu4yq X/AMpgYiJ1fTEoFNyAR2kt+FDwD4EyALwomE4R2GhLZ9oIm44D+/d31iKvubfwUcE7Jr JLNtU94jTtouvDNv5bfUBuiWm/CujWlTKJDeg0LoukempL1m68u/eTXDt9Msi+WrjO1Y uUBlTRD5pXuCraJdUIcTu1v98MDtIHPyIBEn8ll4+aX6tzoox2js1bsyvPZk7rmUopxT upeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=C/TD0CHDy4RoQIxiEpylzxCplnrxE4ZRPieQx5Wj4OY=; b=E4UoG6He97iE3tXo46cRwVkK5qMnT+TApCC6Mx3T7UM8kgVRujpTrnp8cNk1UYYuZY HV7ChkPvqUDMOGk4dtzvYssEhkdEjgbHPsvRnSrUbAbf4Ir7e/qPyVfFuxho2i7IPj4R LirJNopNreeirUubXDHDN6pKBgHbiOrgSXVDTpxptUmLqLc1CAOoZKgowPHN/c1EoBqo bi99NxaS1lchTQ2dqFDxMeilLR2ONQUDqH4V0B7X2Tha5c3mlF0IAylWbLRCoxAdeg/D bxZZ8R50bLIWA5SWyvGdk5+oI/zPZnjpw2x9wTt35Mmuyw4h5FzlLJOjnooRyM5h1g5l uH4Q== X-Gm-Message-State: AOAM532qkD9Ad0WNNaQkA3mGRUPYix9bcpqKIb7Hd+be+AUew9gD3KQK fxGB2OD2LJ748+OQU7BWXAaLuw== X-Google-Smtp-Source: ABdhPJzrQdzoKMttir4gya6hBK6b55TPbqVj8/t5HNUyrH5VP5fiMh6T4cJO3RzDVoBXNTUCIWlLCQ== X-Received: by 2002:a05:600c:4ed1:: with SMTP id g17mr13583865wmq.67.1615202777879; Mon, 08 Mar 2021 03:26:17 -0800 (PST) Received: from elver.google.com ([2a00:79e0:15:13:9d1d:b6a0:d116:531b]) by smtp.gmail.com with ESMTPSA id j4sm16663555wmo.10.2021.03.08.03.26.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Mar 2021 03:26:17 -0800 (PST) Date: Mon, 8 Mar 2021 12:26:11 +0100 From: Marco Elver To: Andrey Konovalov Cc: Alexander Potapenko , Andrew Morton , Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Peter Collingbourne , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 2/5] kasan: init memory in kasan_(un)poison for HW_TAGS Message-ID: References: <09ee2b0a0e9578885b2da1c963e9e94154d5d70a.1614989433.git.andreyknvl@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <09ee2b0a0e9578885b2da1c963e9e94154d5d70a.1614989433.git.andreyknvl@google.com> User-Agent: Mutt/2.0.5 (2021-01-21) X-Stat-Signature: huckwkfeidf636to9j5oarzjnewxypy1 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: AA7FB407F8F7 Received-SPF: none (google.com>: No applicable sender policy available) receiver=imf10; identity=mailfrom; envelope-from=""; helo=mail-wm1-f48.google.com; client-ip=209.85.128.48 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1615202776-948058 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, Mar 06, 2021 at 01:15AM +0100, Andrey Konovalov wrote: > This change adds an argument to kasan_poison() and kasan_unpoison() > that allows initializing memory along with setting the tags for HW_TAGS. > > Combining setting allocation tags with memory initialization will > improve HW_TAGS KASAN performance when init_on_alloc/free is enabled. > > This change doesn't integrate memory initialization with KASAN, > this is done is subsequent patches in this series. > > Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver > --- > lib/test_kasan.c | 4 ++-- > mm/kasan/common.c | 28 ++++++++++++++-------------- > mm/kasan/generic.c | 12 ++++++------ > mm/kasan/kasan.h | 14 ++++++++------ > mm/kasan/shadow.c | 10 +++++----- > mm/kasan/sw_tags.c | 2 +- > 6 files changed, 36 insertions(+), 34 deletions(-) > > diff --git a/lib/test_kasan.c b/lib/test_kasan.c > index e5647d147b35..d77c45edc7cd 100644 > --- a/lib/test_kasan.c > +++ b/lib/test_kasan.c > @@ -1044,14 +1044,14 @@ static void match_all_mem_tag(struct kunit *test) > continue; > > /* Mark the first memory granule with the chosen memory tag. */ > - kasan_poison(ptr, KASAN_GRANULE_SIZE, (u8)tag); > + kasan_poison(ptr, KASAN_GRANULE_SIZE, (u8)tag, false); > > /* This access must cause a KASAN report. */ > KUNIT_EXPECT_KASAN_FAIL(test, *ptr = 0); > } > > /* Recover the memory tag and free. */ > - kasan_poison(ptr, KASAN_GRANULE_SIZE, get_tag(ptr)); > + kasan_poison(ptr, KASAN_GRANULE_SIZE, get_tag(ptr), false); > kfree(ptr); > } > > diff --git a/mm/kasan/common.c b/mm/kasan/common.c > index b5e08d4cefec..316f7f8cd8e6 100644 > --- a/mm/kasan/common.c > +++ b/mm/kasan/common.c > @@ -60,7 +60,7 @@ void kasan_disable_current(void) > > void __kasan_unpoison_range(const void *address, size_t size) > { > - kasan_unpoison(address, size); > + kasan_unpoison(address, size, false); > } > > #if CONFIG_KASAN_STACK > @@ -69,7 +69,7 @@ void kasan_unpoison_task_stack(struct task_struct *task) > { > void *base = task_stack_page(task); > > - kasan_unpoison(base, THREAD_SIZE); > + kasan_unpoison(base, THREAD_SIZE, false); > } > > /* Unpoison the stack for the current task beyond a watermark sp value. */ > @@ -82,7 +82,7 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) > */ > void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); > > - kasan_unpoison(base, watermark - base); > + kasan_unpoison(base, watermark - base, false); > } > #endif /* CONFIG_KASAN_STACK */ > > @@ -108,14 +108,14 @@ void __kasan_alloc_pages(struct page *page, unsigned int order) > tag = kasan_random_tag(); > for (i = 0; i < (1 << order); i++) > page_kasan_tag_set(page + i, tag); > - kasan_unpoison(page_address(page), PAGE_SIZE << order); > + kasan_unpoison(page_address(page), PAGE_SIZE << order, false); > } > > void __kasan_free_pages(struct page *page, unsigned int order) > { > if (likely(!PageHighMem(page))) > kasan_poison(page_address(page), PAGE_SIZE << order, > - KASAN_FREE_PAGE); > + KASAN_FREE_PAGE, false); > } > > /* > @@ -251,18 +251,18 @@ void __kasan_poison_slab(struct page *page) > for (i = 0; i < compound_nr(page); i++) > page_kasan_tag_reset(page + i); > kasan_poison(page_address(page), page_size(page), > - KASAN_KMALLOC_REDZONE); > + KASAN_KMALLOC_REDZONE, false); > } > > void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object) > { > - kasan_unpoison(object, cache->object_size); > + kasan_unpoison(object, cache->object_size, false); > } > > void __kasan_poison_object_data(struct kmem_cache *cache, void *object) > { > kasan_poison(object, round_up(cache->object_size, KASAN_GRANULE_SIZE), > - KASAN_KMALLOC_REDZONE); > + KASAN_KMALLOC_REDZONE, false); > } > > /* > @@ -351,7 +351,7 @@ static inline bool ____kasan_slab_free(struct kmem_cache *cache, > } > > kasan_poison(object, round_up(cache->object_size, KASAN_GRANULE_SIZE), > - KASAN_KMALLOC_FREE); > + KASAN_KMALLOC_FREE, false); > > if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine)) > return false; > @@ -407,7 +407,7 @@ void __kasan_slab_free_mempool(void *ptr, unsigned long ip) > if (unlikely(!PageSlab(page))) { > if (____kasan_kfree_large(ptr, ip)) > return; > - kasan_poison(ptr, page_size(page), KASAN_FREE_PAGE); > + kasan_poison(ptr, page_size(page), KASAN_FREE_PAGE, false); > } else { > ____kasan_slab_free(page->slab_cache, ptr, ip, false); > } > @@ -453,7 +453,7 @@ void * __must_check __kasan_slab_alloc(struct kmem_cache *cache, > * Unpoison the whole object. > * For kmalloc() allocations, kasan_kmalloc() will do precise poisoning. > */ > - kasan_unpoison(tagged_object, cache->object_size); > + kasan_unpoison(tagged_object, cache->object_size, false); > > /* Save alloc info (if possible) for non-kmalloc() allocations. */ > if (kasan_stack_collection_enabled()) > @@ -496,7 +496,7 @@ static inline void *____kasan_kmalloc(struct kmem_cache *cache, > redzone_end = round_up((unsigned long)(object + cache->object_size), > KASAN_GRANULE_SIZE); > kasan_poison((void *)redzone_start, redzone_end - redzone_start, > - KASAN_KMALLOC_REDZONE); > + KASAN_KMALLOC_REDZONE, false); > > /* > * Save alloc info (if possible) for kmalloc() allocations. > @@ -546,7 +546,7 @@ void * __must_check __kasan_kmalloc_large(const void *ptr, size_t size, > KASAN_GRANULE_SIZE); > redzone_end = (unsigned long)ptr + page_size(virt_to_page(ptr)); > kasan_poison((void *)redzone_start, redzone_end - redzone_start, > - KASAN_PAGE_REDZONE); > + KASAN_PAGE_REDZONE, false); > > return (void *)ptr; > } > @@ -563,7 +563,7 @@ void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flag > * Part of it might already have been unpoisoned, but it's unknown > * how big that part is. > */ > - kasan_unpoison(object, size); > + kasan_unpoison(object, size, false); > > page = virt_to_head_page(object); > > diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c > index 2e55e0f82f39..53cbf28859b5 100644 > --- a/mm/kasan/generic.c > +++ b/mm/kasan/generic.c > @@ -208,11 +208,11 @@ static void register_global(struct kasan_global *global) > { > size_t aligned_size = round_up(global->size, KASAN_GRANULE_SIZE); > > - kasan_unpoison(global->beg, global->size); > + kasan_unpoison(global->beg, global->size, false); > > kasan_poison(global->beg + aligned_size, > global->size_with_redzone - aligned_size, > - KASAN_GLOBAL_REDZONE); > + KASAN_GLOBAL_REDZONE, false); > } > > void __asan_register_globals(struct kasan_global *globals, size_t size) > @@ -292,11 +292,11 @@ void __asan_alloca_poison(unsigned long addr, size_t size) > WARN_ON(!IS_ALIGNED(addr, KASAN_ALLOCA_REDZONE_SIZE)); > > kasan_unpoison((const void *)(addr + rounded_down_size), > - size - rounded_down_size); > + size - rounded_down_size, false); > kasan_poison(left_redzone, KASAN_ALLOCA_REDZONE_SIZE, > - KASAN_ALLOCA_LEFT); > + KASAN_ALLOCA_LEFT, false); > kasan_poison(right_redzone, padding_size + KASAN_ALLOCA_REDZONE_SIZE, > - KASAN_ALLOCA_RIGHT); > + KASAN_ALLOCA_RIGHT, false); > } > EXPORT_SYMBOL(__asan_alloca_poison); > > @@ -306,7 +306,7 @@ void __asan_allocas_unpoison(const void *stack_top, const void *stack_bottom) > if (unlikely(!stack_top || stack_top > stack_bottom)) > return; > > - kasan_unpoison(stack_top, stack_bottom - stack_top); > + kasan_unpoison(stack_top, stack_bottom - stack_top, false); > } > EXPORT_SYMBOL(__asan_allocas_unpoison); > > diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h > index 7fbb32234414..823a90d6a0cd 100644 > --- a/mm/kasan/kasan.h > +++ b/mm/kasan/kasan.h > @@ -331,7 +331,7 @@ static inline u8 kasan_random_tag(void) { return 0; } > > #ifdef CONFIG_KASAN_HW_TAGS > > -static inline void kasan_poison(const void *addr, size_t size, u8 value) > +static inline void kasan_poison(const void *addr, size_t size, u8 value, bool init) > { > addr = kasan_reset_tag(addr); > > @@ -344,10 +344,10 @@ static inline void kasan_poison(const void *addr, size_t size, u8 value) > if (WARN_ON(size & KASAN_GRANULE_MASK)) > return; > > - hw_set_mem_tag_range((void *)addr, size, value, false); > + hw_set_mem_tag_range((void *)addr, size, value, init); > } > > -static inline void kasan_unpoison(const void *addr, size_t size) > +static inline void kasan_unpoison(const void *addr, size_t size, bool init) > { > u8 tag = get_tag(addr); > > @@ -361,7 +361,7 @@ static inline void kasan_unpoison(const void *addr, size_t size) > return; > size = round_up(size, KASAN_GRANULE_SIZE); > > - hw_set_mem_tag_range((void *)addr, size, tag, false); > + hw_set_mem_tag_range((void *)addr, size, tag, init); > } > > static inline bool kasan_byte_accessible(const void *addr) > @@ -380,22 +380,24 @@ static inline bool kasan_byte_accessible(const void *addr) > * @addr - range start address, must be aligned to KASAN_GRANULE_SIZE > * @size - range size, must be aligned to KASAN_GRANULE_SIZE > * @value - value that's written to metadata for the range > + * @init - whether to initialize the memory range (only for hardware tag-based) > * > * The size gets aligned to KASAN_GRANULE_SIZE before marking the range. > */ > -void kasan_poison(const void *addr, size_t size, u8 value); > +void kasan_poison(const void *addr, size_t size, u8 value, bool init); > > /** > * kasan_unpoison - mark the memory range as accessible > * @addr - range start address, must be aligned to KASAN_GRANULE_SIZE > * @size - range size, can be unaligned > + * @init - whether to initialize the memory range (only for hardware tag-based) > * > * For the tag-based modes, the @size gets aligned to KASAN_GRANULE_SIZE before > * marking the range. > * For the generic mode, the last granule of the memory range gets partially > * unpoisoned based on the @size. > */ > -void kasan_unpoison(const void *addr, size_t size); > +void kasan_unpoison(const void *addr, size_t size, bool init); > > bool kasan_byte_accessible(const void *addr); > > diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c > index 63f43443f5d7..727ad4629173 100644 > --- a/mm/kasan/shadow.c > +++ b/mm/kasan/shadow.c > @@ -69,7 +69,7 @@ void *memcpy(void *dest, const void *src, size_t len) > return __memcpy(dest, src, len); > } > > -void kasan_poison(const void *addr, size_t size, u8 value) > +void kasan_poison(const void *addr, size_t size, u8 value, bool init) > { > void *shadow_start, *shadow_end; > > @@ -106,7 +106,7 @@ void kasan_poison_last_granule(const void *addr, size_t size) > } > #endif > > -void kasan_unpoison(const void *addr, size_t size) > +void kasan_unpoison(const void *addr, size_t size, bool init) > { > u8 tag = get_tag(addr); > > @@ -129,7 +129,7 @@ void kasan_unpoison(const void *addr, size_t size) > return; > > /* Unpoison all granules that cover the object. */ > - kasan_poison(addr, round_up(size, KASAN_GRANULE_SIZE), tag); > + kasan_poison(addr, round_up(size, KASAN_GRANULE_SIZE), tag, false); > > /* Partially poison the last granule for the generic mode. */ > if (IS_ENABLED(CONFIG_KASAN_GENERIC)) > @@ -344,7 +344,7 @@ void kasan_poison_vmalloc(const void *start, unsigned long size) > return; > > size = round_up(size, KASAN_GRANULE_SIZE); > - kasan_poison(start, size, KASAN_VMALLOC_INVALID); > + kasan_poison(start, size, KASAN_VMALLOC_INVALID, false); > } > > void kasan_unpoison_vmalloc(const void *start, unsigned long size) > @@ -352,7 +352,7 @@ void kasan_unpoison_vmalloc(const void *start, unsigned long size) > if (!is_vmalloc_or_module_addr(start)) > return; > > - kasan_unpoison(start, size); > + kasan_unpoison(start, size, false); > } > > static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, > diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c > index 94c2d33be333..bd0c64d4e4d9 100644 > --- a/mm/kasan/sw_tags.c > +++ b/mm/kasan/sw_tags.c > @@ -159,7 +159,7 @@ EXPORT_SYMBOL(__hwasan_storeN_noabort); > > void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) > { > - kasan_poison((void *)addr, size, tag); > + kasan_poison((void *)addr, size, tag, false); > } > EXPORT_SYMBOL(__hwasan_tag_memory); > > -- > 2.30.1.766.gb4fecdf3b7-goog >