From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-23.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9E8CC433ED for ; Tue, 11 May 2021 20:33:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 31D6861626 for ; Tue, 11 May 2021 20:33:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 31D6861626 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 43D556B0036; Tue, 11 May 2021 16:33:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 414876B006C; Tue, 11 May 2021 16:33:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C2C76B006E; Tue, 11 May 2021 16:33:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0221.hostedemail.com [216.40.44.221]) by kanga.kvack.org (Postfix) with ESMTP id 0E8B06B0036 for ; Tue, 11 May 2021 16:33:19 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D0296B9E3 for ; Tue, 11 May 2021 20:33:16 +0000 (UTC) X-FDA: 78130099992.37.9EABB6D Received: from mail-io1-f50.google.com (mail-io1-f50.google.com [209.85.166.50]) by imf21.hostedemail.com (Postfix) with ESMTP id 27E7DE000111 for ; Tue, 11 May 2021 20:33:11 +0000 (UTC) Received: by mail-io1-f50.google.com with SMTP id i7so11988496ioa.12 for ; Tue, 11 May 2021 13:33:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=h40frLs8n9UJa9r1NYPALLnPMlmcL80u1XMkItLS6OQ=; b=qjamw3isER9qbDI7iVpsOzjg6wFX/SGNHPUXYGx0z8OFU9HhjJNFWrNdt8n9M524SU +V92+yl3L/wvYiCpjvPYIRdP3TfXmW4usDHzvoEWtW450JKmcg8JUnLZCMarwaeMvwlu P9uIy4N2j2XWecIibQ7tMi0I5/06APf5+NDwNNdb2xyzakrYOJID7fm6jSfDaT+XCn6l vnN6nW/dYKvJ46xgjcC7p+6zjSD7XfgE+XFBM5O5QSsmlrSi//QOOQ85uCv+ZM06N54s aIEQx83nz1StYV3U63qdJM3ncx/QA8/s71MVwX6eGALTv5AWQnPwTAF36pFPd3aBLpRp QZOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=h40frLs8n9UJa9r1NYPALLnPMlmcL80u1XMkItLS6OQ=; b=E+UUhVX2MskzMGlJfZwqJ8yPvlPGclbUzhJU/9x0iKdEF1W6wf4dXEjdlnQ4AXcbr6 JpcyRE7irk8pVkTFnknL2dFQzNOX+FHk0t0bhKMaLLEEx1VFqaBmOCEy9hifXOfgg9Nm 9dtiR0Wb6GKHGqoESS3of54Q24NzJZkC+5gPpgGYj9tLzPjE8tE+9hRdFcOnIofFDhnT mfSSlqxkXUwGuzv7v2XFvrmb8P3RzTtLRo+962uOUCZjTv/Z2fUEDHa3ux/RpjvRYkwL 0cAR2SNOvXpmWpRXtUhH9yg6KeuBwEvLy21lzAD2YIsweLg4iwvpV4D+Qlf2ht52ULir 8K2g== X-Gm-Message-State: AOAM533lJ/uH2r/nfC+25tuMGlukMALPMtHQVHV2SkD8IxBwlUnuRL/T aT/ZA9u2Gj7a6KjQPmLZ20t1yOpjdySPYrJNzAoxUQ== X-Google-Smtp-Source: ABdhPJzk/U0AW0IEWMkbxMz3eLQF0I9sHCZakUzcUpqySnW3BkYOyB465xG3xbgv/KmEw9istTJgXLChSRGJuh90JKI= X-Received: by 2002:a02:91c1:: with SMTP id s1mr28287329jag.61.1620765195243; Tue, 11 May 2021 13:33:15 -0700 (PDT) MIME-Version: 1.0 References: <20210511073108.138837-1-pcc@google.com> <20210511073108.138837-2-pcc@google.com> <20210511125354.GC9799@arm.com> In-Reply-To: <20210511125354.GC9799@arm.com> From: Peter Collingbourne Date: Tue, 11 May 2021 13:33:03 -0700 Message-ID: Subject: Re: [PATCH 2/3] arm64: mte: handle tags zeroing at page allocation time To: Catalin Marinas Cc: Andrey Konovalov , Alexander Potapenko , Vincenzo Frascino , Andrew Morton , Evgenii Stepanov , Linux Memory Management List , Linux ARM Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 27E7DE000111 Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20161025 header.b=qjamw3is; spf=pass (imf21.hostedemail.com: domain of pcc@google.com designates 209.85.166.50 as permitted sender) smtp.mailfrom=pcc@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam04 X-Stat-Signature: 9gtwugjrgxzc8afuffii1tn9xmdh3wju Received-SPF: none (google.com>: No applicable sender policy available) receiver=imf21; identity=mailfrom; envelope-from=""; helo=mail-io1-f50.google.com; client-ip=209.85.166.50 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1620765191-438636 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 11, 2021 at 5:54 AM Catalin Marinas wrote: > > Hi Peter, > > First of all, could you please add a cover letter to your series (in > general) explaining the rationale for the patches, e.g. optimise tag > initialisation for user pages? It makes it a lot easier to review if the > overall picture is presented in the cover. Sure. It seems appropriate in cases where the series is doing a number of different things like this series, but maybe not in simpler cases (e.g. one cleanup patch followed by a main patch with a self-explanatory commit message). > On Tue, May 11, 2021 at 12:31:07AM -0700, Peter Collingbourne wrote: > > Currently, on an anonymous page fault, the kernel allocates a zeroed > > page and maps it in user space. If the mapping is tagged (PROT_MTE), > > set_pte_at() additionally clears the tags. It is, however, more > > efficient to clear the tags at the same time as zeroing the data on > > allocation. To avoid clearing the tags on any page (which may not be > > mapped as tagged), only do this if the vma flags contain VM_MTE. This > > requires introducing a new GFP flag that is used to determine whether > > to clear the tags. > > > > The DC GZVA instruction with a 0 top byte (and 0 tag) requires > > top-byte-ignore. Set the TCR_EL1.{TBI1,TBID1} bits irrespective of > > whether KASAN_HW is enabled. > > > > Signed-off-by: Peter Collingbourne > > Co-developed-by: Catalin Marinas > > Signed-off-by: Catalin Marinas > > Link: https://linux-review.googlesource.com/id/Id46dc94e30fe11474f7e54f5d65e7658dbdddb26 > > This doesn't mention that the patch adds tag clearing on free as well. > I'd actually leave this part out for a separate patch. It's not done for > tags in current mainline when kasan is disabled, AFAICT. The tag clearing on free was thought to be necessary (because clear on free implies no clear on alloc) but, upon further reflection, it isn't. This is because we clear the struct page flags, including PG_mte_tagged, on free, which means that we will set the tags if we later end up needing to reuse the page as a tagged page. This means that clear on free is quite inefficient, but no less efficient than before. As you mentioned, we can leave any improvements there to a separate patch. > > diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h > > index 012cffc574e8..a0bcaa5f735e 100644 > > --- a/arch/arm64/include/asm/page.h > > +++ b/arch/arm64/include/asm/page.h > > @@ -13,6 +13,7 @@ > > #ifndef __ASSEMBLY__ > > > > #include /* for READ_IMPLIES_EXEC */ > > +#include /* for gfp_t */ > > #include > > > > struct page; > > @@ -28,10 +29,16 @@ void copy_user_highpage(struct page *to, struct page *from, > > void copy_highpage(struct page *to, struct page *from); > > #define __HAVE_ARCH_COPY_HIGHPAGE > > > > -#define __alloc_zeroed_user_highpage(movableflags, vma, vaddr) \ > > - alloc_page_vma(GFP_HIGHUSER | __GFP_ZERO | movableflags, vma, vaddr) > > +struct page *__alloc_zeroed_user_highpage(gfp_t movableflags, > > + struct vm_area_struct *vma, > > + unsigned long vaddr); > > #define __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE > > > > +#define want_zero_tags_on_free() system_supports_mte() > > As I said above, unless essential to this patch, please move it to a > separate one. Will do. > Also, do we need this even when the kernel doesn't have kasan_hw? Yes, if we preserved PG_mte_tagged across page free/alloc then this would be needed even without KASAN. > > diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c > > index 871c82ab0a30..8127e0c0b8fb 100644 > > --- a/arch/arm64/mm/fault.c > > +++ b/arch/arm64/mm/fault.c > > @@ -921,3 +921,28 @@ void do_debug_exception(unsigned long addr_if_watchpoint, unsigned int esr, > > debug_exception_exit(regs); > > } > > NOKPROBE_SYMBOL(do_debug_exception); > > + > > +/* > > + * Used during anonymous page fault handling. > > + */ > > +struct page *__alloc_zeroed_user_highpage(gfp_t flags, > > + struct vm_area_struct *vma, > > + unsigned long vaddr) > > +{ > > + /* > > + * If the page is mapped with PROT_MTE, initialise the tags at the > > + * point of allocation and page zeroing as this is usually faster than > > + * separate DC ZVA and STGM. > > + */ > > + if (vma->vm_flags & VM_MTE) > > + flags |= __GFP_ZEROTAGS; > > + > > + return alloc_page_vma(GFP_HIGHUSER | __GFP_ZERO | flags, vma, vaddr); > > +} > > + > > +void tag_clear_highpage(struct page *page) > > +{ > > + mte_zero_clear_page_tags(page_address(page)); > > + page_kasan_tag_reset(page); > > + set_bit(PG_mte_tagged, &page->flags); > > +} > > Do we need the page_kasan_tag_reset() here? Maybe we do. Is it because > kasan_alloc_pages() is no longer calls kasan_unpoison_pages() below? Yes, otherwise the page tag will be left at an arbitrary (most likely poison) value, which would mean that any kernel-side accesses to these pages would fail (unless userspace happened to tag its memory using the page tag). page_kasan_tag_reset() sets the page tag to the TCMA tag which allows those accesses to succeed. We need to call page_kasan_tag_reset() in mte_sync_page_tags() for similar reasons. It's unrelated to no longer calling kasan_unpoison_pages() because even if we were still calling it the end result would still be that userspace's tags won't necessarily match the kernel's page tag. > > diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c > > index 45e552cb9172..34362c8d0955 100644 > > --- a/mm/kasan/hw_tags.c > > +++ b/mm/kasan/hw_tags.c > > @@ -242,7 +242,14 @@ void kasan_alloc_pages(struct page *page, unsigned int order, gfp_t flags) > > { > > bool init = !want_init_on_free() && want_init_on_alloc(flags); > > > > - kasan_unpoison_pages(page, order, init); > > + if (flags & __GFP_ZEROTAGS) { > > + int i; > > + > > + for (i = 0; i != 1 << order; ++i) > > + tag_clear_highpage(page + i); > > + } else { > > + kasan_unpoison_pages(page, order, init); > > + } > > } > > > > void kasan_free_pages(struct page *page, unsigned int order) > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > > index 6e82a7f6fd6f..7ac0f0721d22 100644 > > --- a/mm/page_alloc.c > > +++ b/mm/page_alloc.c > > @@ -1219,10 +1219,16 @@ static int free_tail_pages_check(struct page *head_page, struct page *page) > > return ret; > > } > > > > -static void kernel_init_free_pages(struct page *page, int numpages) > > +static void kernel_init_free_pages(struct page *page, int numpages, bool zero_tags) > > { > > int i; > > > > + if (zero_tags) { > > + for (i = 0; i < numpages; i++) > > + tag_clear_highpage(page + i); > > + return; > > + } > > + > > /* s390's use of memset() could override KASAN redzones. */ > > kasan_disable_current(); > > for (i = 0; i < numpages; i++) { > > This function has another loop calling clear_highpage(). Do we end up > zeroing the page twice? No because we return after the loop that calls tag_clear_highpage(). > > @@ -1314,7 +1320,8 @@ static __always_inline bool free_pages_prepare(struct page *page, > > bool init = want_init_on_free(); > > > > if (init) > > - kernel_init_free_pages(page, 1 << order); > > + kernel_init_free_pages(page, 1 << order, > > + want_zero_tags_on_free()); > > if (!skip_kasan_poison) > > kasan_poison_pages(page, order, init); > > } > > I think passing 'false' here to kernel_init_free_pages() matches the > current mainline. You could make this dependent on kasan_hw being > enabled rather than just system_supports_mte(). With kasan_hw disabled, > the kernel accesses are not checked anyway, so it's pointless to erase > the tags on free. I'll just pass false here as it's unneeded for KASAN. Peter