From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.1 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76616C2D0D2 for ; Thu, 19 Dec 2019 10:08:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1E5DF24650 for ; Thu, 19 Dec 2019 10:08:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=c-s.fr header.i=@c-s.fr header.b="AmDlgp2G" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E5DF24650 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=c-s.fr Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A879A8E0162; Thu, 19 Dec 2019 05:08:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A38888E00F5; Thu, 19 Dec 2019 05:08:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 94D548E0162; Thu, 19 Dec 2019 05:08:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0007.hostedemail.com [216.40.44.7]) by kanga.kvack.org (Postfix) with ESMTP id 7C7378E00F5 for ; Thu, 19 Dec 2019 05:08:00 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 1D190181AEF07 for ; Thu, 19 Dec 2019 10:08:00 +0000 (UTC) X-FDA: 76281465120.11.push55_b952fc5ca832 X-HE-Tag: push55_b952fc5ca832 X-Filterd-Recvd-Size: 12794 Received: from pegase1.c-s.fr (pegase1.c-s.fr [93.17.236.30]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Thu, 19 Dec 2019 10:07:59 +0000 (UTC) Received: from localhost (mailhub1-ext [192.168.12.233]) by localhost (Postfix) with ESMTP id 47dncs1fSZz9txdZ; Thu, 19 Dec 2019 11:07:57 +0100 (CET) Authentication-Results: localhost; dkim=pass reason="1024-bit key; insecure key" header.d=c-s.fr header.i=@c-s.fr header.b=AmDlgp2G; dkim-adsp=pass; dkim-atps=neutral X-Virus-Scanned: Debian amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [192.168.12.234]) (amavisd-new, port 10024) with ESMTP id FMPZ0PcMguUF; Thu, 19 Dec 2019 11:07:57 +0100 (CET) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 47dncs0W1Mz9txdX; Thu, 19 Dec 2019 11:07:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=c-s.fr; s=mail; t=1576750077; bh=/4zEamhLSd+OmHEjHewPY8apoChkej0LywPcTIUtggs=; h=Subject:From:To:References:Date:In-Reply-To:From; b=AmDlgp2GMbsJV32BkI9L4IhcM8MXtzya6CRzWBsdrYicdCf9yTi7fBsdntr7oRiu3 4Wnams1c1ClSYV25a95bXzT4UcP0EdhdiXTFXxwe5JppumUv2xBG+Fy3yPOl6nlP8f S00hCkNSWvhkYRkVdpgu1UIZ6S2IA0gzmG4yDUCY= Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 2ACE68B7AD; Thu, 19 Dec 2019 11:07:58 +0100 (CET) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id C_K90wc5tv2M; Thu, 19 Dec 2019 11:07:58 +0100 (CET) Received: from po16098vm.idsi0.si.c-s.fr (unknown [192.168.4.90]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 6F0CB8B787; Thu, 19 Dec 2019 11:07:57 +0100 (CET) Subject: Re: [PATCH v4 4/4] powerpc: Book3S 64-bit "heavyweight" KASAN support From: Christophe Leroy To: Daniel Axtens , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, kasan-dev@googlegroups.com, aneesh.kumar@linux.ibm.com, bsingharora@gmail.com References: <20191219003630.31288-1-dja@axtens.net> <20191219003630.31288-5-dja@axtens.net> <87bls4tzjn.fsf@dja-thinkpad.axtens.net> <4f2fffb3-5fb6-b5ea-a951-a7910f2439b8@c-s.fr> Message-ID: <76c5aa20-7993-9501-514d-10e0b6d882d1@c-s.fr> Date: Thu, 19 Dec 2019 10:07:57 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <4f2fffb3-5fb6-b5ea-a951-a7910f2439b8@c-s.fr> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 12/19/2019 10:05 AM, Christophe Leroy wrote: >=20 >=20 > Le 19/12/2019 =C3=A0 10:50, Daniel Axtens a =C3=A9crit=C2=A0: >> Christophe Leroy writes: >> >>> On 12/19/2019 12:36 AM, Daniel Axtens wrote: >>>> KASAN support on Book3S is a bit tricky to get right: >>>> >>>> =C2=A0=C2=A0 - It would be good to support inline instrumentation so= as to be=20 >>>> able to >>>> =C2=A0=C2=A0=C2=A0=C2=A0 catch stack issues that cannot be caught wi= th outline mode. >>>> >>>> =C2=A0=C2=A0 - Inline instrumentation requires a fixed offset. >>>> >>>> =C2=A0=C2=A0 - Book3S runs code in real mode after booting. Most not= ably a lot=20 >>>> of KVM >>>> =C2=A0=C2=A0=C2=A0=C2=A0 runs in real mode, and it would be good to = be able to=20 >>>> instrument it. >>>> >>>> =C2=A0=C2=A0 - Because code runs in real mode after boot, the offset= has to=20 >>>> point to >>>> =C2=A0=C2=A0=C2=A0=C2=A0 valid memory both in and out of real mode. >>>> >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 [ppc64 mm note: The kernel installs a= linear mapping at effective >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 address c000... onward. This is a one= -to-one mapping with=20 >>>> physical >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 memory from 0000... onward. Because o= f how memory accesses=20 >>>> work on >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 powerpc 64-bit Book3S, a kernel point= er in the linear map=20 >>>> accesses the >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 same memory both with translations on= (accessing as an 'effective >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 address'), and with translations off = (accessing as a 'real >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 address'). This works in both guests = and the hypervisor. For more >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 details, see s5.7 of Book III of vers= ion 3 of the ISA, in=20 >>>> particular >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 the Storage Control Overview, s5.7.3,= and s5.7.5 - noting that=20 >>>> this >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 KASAN implementation currently only s= upports Radix.] >>>> >>>> One approach is just to give up on inline instrumentation. This way = all >>>> checks can be delayed until after everything set is up correctly,=20 >>>> and the >>>> address-to-shadow calculations can be overridden. However, the=20 >>>> features and >>>> speed boost provided by inline instrumentation are worth trying to d= o >>>> better. >>>> >>>> If _at compile time_ it is known how much contiguous physical memory= a >>>> system has, the top 1/8th of the first block of physical memory can=20 >>>> be set >>>> aside for the shadow. This is a big hammer and comes with 3 big >>>> consequences: >>>> >>>> =C2=A0=C2=A0 - there's no nice way to handle physically discontiguou= s memory,=20 >>>> so only >>>> =C2=A0=C2=A0=C2=A0=C2=A0 the first physical memory block can be used= . >>>> >>>> =C2=A0=C2=A0 - kernels will simply fail to boot on machines with les= s memory than >>>> =C2=A0=C2=A0=C2=A0=C2=A0 specified when compiling. >>>> >>>> =C2=A0=C2=A0 - kernels running on machines with more memory than spe= cified when >>>> =C2=A0=C2=A0=C2=A0=C2=A0 compiling will simply ignore the extra memo= ry. >>>> >>>> Implement and document KASAN this way. The current implementation is= =20 >>>> Radix >>>> only. >>>> >>>> Despite the limitations, it can still find bugs, >>>> e.g. http://patchwork.ozlabs.org/patch/1103775/ >>>> >>>> At the moment, this physical memory limit must be set _even for outl= ine >>>> mode_. This may be changed in a later series - a different=20 >>>> implementation >>>> could be added for outline mode that dynamically allocates shadow at= a >>>> fixed offset. For example, see=20 >>>> https://patchwork.ozlabs.org/patch/795211/ >>>> >>>> Suggested-by: Michael Ellerman >>>> Cc: Balbir Singh # ppc64 out-of-line radix=20 >>>> version >>>> Cc: Christophe Leroy # ppc32 version >>>> Signed-off-by: Daniel Axtens >>>> >>>> --- >>>> Changes since v3: >>>> =C2=A0=C2=A0 - Address further feedback from Christophe. >>>> =C2=A0=C2=A0 - Drop changes to stack walking, it looks like the issu= e I=20 >>>> observed is >>>> =C2=A0=C2=A0=C2=A0=C2=A0 related to that particular stack, not stack= -walking generally. >>>> >>>> Changes since v2: >>>> >>>> =C2=A0=C2=A0 - Address feedback from Christophe around cleanups and = docs. >>>> =C2=A0=C2=A0 - Address feedback from Balbir: at this point I don't h= ave a good=20 >>>> solution >>>> =C2=A0=C2=A0=C2=A0=C2=A0 for the issues you identify around the limi= tations of the=20 >>>> inline implementation >>>> =C2=A0=C2=A0=C2=A0=C2=A0 but I think that it's worth trying to get t= he stack=20 >>>> instrumentation support. >>>> =C2=A0=C2=A0=C2=A0=C2=A0 I'm happy to have an alternative and more f= lexible outline mode=20 >>>> - I had >>>> =C2=A0=C2=A0=C2=A0=C2=A0 envisoned this would be called 'lightweight= ' mode as it imposes=20 >>>> fewer restrictions. >>>> =C2=A0=C2=A0=C2=A0=C2=A0 I've linked to your implementation. I think= it's best to add it=20 >>>> in a follow-up series. >>>> =C2=A0=C2=A0 - Made the default PHYS_MEM_SIZE_FOR_KASAN value 1024MB= . I think=20 >>>> most people have >>>> =C2=A0=C2=A0=C2=A0=C2=A0 guests with at least that much memory in th= e Radix 64s case so=20 >>>> it's a much >>>> =C2=A0=C2=A0=C2=A0=C2=A0 saner default - it means that if you just t= urn on KASAN without=20 >>>> reading the >>>> =C2=A0=C2=A0=C2=A0=C2=A0 docs you're much more likely to have a boot= able kernel, which=20 >>>> you will never >>>> =C2=A0=C2=A0=C2=A0=C2=A0 have if the value is set to zero! I'm happy= to bikeshed the=20 >>>> value if we want. >>>> >>>> Changes since v1: >>>> =C2=A0=C2=A0 - Landed kasan vmalloc support upstream >>>> =C2=A0=C2=A0 - Lots of feedback from Christophe. >>>> >>>> Changes since the rfc: >>>> >>>> =C2=A0=C2=A0 - Boots real and virtual hardware, kvm works. >>>> >>>> =C2=A0=C2=A0 - disabled reporting when we're checking the stack for = exception >>>> =C2=A0=C2=A0=C2=A0=C2=A0 frames. The behaviour isn't wrong, just inc= ompatible with KASAN. >>>> >>>> =C2=A0=C2=A0 - Documentation! >>>> >>>> =C2=A0=C2=A0 - Dropped old module stuff in favour of KASAN_VMALLOC. >>>> >>>> The bugs with ftrace and kuap were due to kernel bloat pushing >>>> prom_init calls to be done via the plt. Because we did not have >>>> a relocatable kernel, and they are done very early, this caused >>>> everything to explode. Compile with CONFIG_RELOCATABLE! >>>> --- >>>> =C2=A0=C2=A0 Documentation/dev-tools/kasan.rst=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 8 +- >>>> =C2=A0=C2=A0 Documentation/powerpc/kasan.txt=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 112=20 >>>> ++++++++++++++++++- >>>> =C2=A0=C2=A0 arch/powerpc/Kconfig=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 2 + >>>> =C2=A0=C2=A0 arch/powerpc/Kconfig.debug=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 |=C2=A0 21 ++++ >>>> =C2=A0=C2=A0 arch/powerpc/Makefile=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 11 ++ >>>> =C2=A0=C2=A0 arch/powerpc/include/asm/book3s/64/hash.h=C2=A0=C2=A0=C2= =A0 |=C2=A0=C2=A0 4 + >>>> =C2=A0=C2=A0 arch/powerpc/include/asm/book3s/64/pgtable.h |=C2=A0=C2= =A0 7 ++ >>>> =C2=A0=C2=A0 arch/powerpc/include/asm/book3s/64/radix.h=C2=A0=C2=A0 = |=C2=A0=C2=A0 5 + >>>> =C2=A0=C2=A0 arch/powerpc/include/asm/kasan.h=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 21 +++- >>>> =C2=A0=C2=A0 arch/powerpc/kernel/prom.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 |=C2=A0 61 +++++++++- >>>> =C2=A0=C2=A0 arch/powerpc/mm/kasan/Makefile=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 1= + >>>> =C2=A0=C2=A0 arch/powerpc/mm/kasan/init_book3s_64.c=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 |=C2=A0 70 ++++++++++++ >>>> =C2=A0=C2=A0 arch/powerpc/platforms/Kconfig.cputype=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 1 + >>>> =C2=A0=C2=A0 13 files changed, 316 insertions(+), 8 deletions(-) >>>> =C2=A0=C2=A0 create mode 100644 arch/powerpc/mm/kasan/init_book3s_64= .c >>>> >>>> diff --git a/arch/powerpc/include/asm/kasan.h=20 >>>> b/arch/powerpc/include/asm/kasan.h >>>> index 296e51c2f066..f18268cbdc33 100644 >>>> --- a/arch/powerpc/include/asm/kasan.h >>>> +++ b/arch/powerpc/include/asm/kasan.h >>>> @@ -2,6 +2,9 @@ >>>> =C2=A0=C2=A0 #ifndef __ASM_KASAN_H >>>> =C2=A0=C2=A0 #define __ASM_KASAN_H >>>> +#include >>>> +#include >>> >>> What do you need asm/pgtable.h for ? >>> >>> Build failure due to circular inclusion of asm/pgtable.h: >> >> I see there's a lot of ppc32 stuff, I clearly need to bite the bullet >> and get a ppc32 toolchain so I can squash these without chewing up any >> more of your time. I'll sort that out and send a new spin. >> >=20 > I'm using a powerpc64 toolchain to build both ppc32 and ppc64 kernels=20 > (from https://mirrors.edge.kernel.org/pub/tools/crosstool/ ) >=20 >=20 > Another thing, did you test PTDUMP stuff with KASAN ? It looks like=20 > KASAN address markers don't depend on PPC32, but are only initialised b= y=20 > populate_markers() for PPC32. >=20 > Regarding kasan.h, I think we should be able to end up with something=20 > where the definition of KASAN_SHADOW_OFFSET should only depend on the=20 > existence of CONFIG_KASAN_SHADOW_OFFSET, and where only=20 > KASAN_SHADOW_SIZE should depend on the target (ie PPC32 or BOOK3S64) > Everything else should be common. KASAN_END should be START+SIZE. >=20 > It looks like what you have called KASAN_SHADOW_SIZE is not similar to=20 > what is called KASAN_SHADOW_SIZE for PPC32, as yours only covers the=20 > SHADOW_SIZE for linear mem while PPC32 one covers the full space. >=20 More or less something like that: /* SPDX-License-Identifier: GPL-2.0 */ #ifndef __ASM_KASAN_H #define __ASM_KASAN_H #include #ifdef CONFIG_KASAN #define _GLOBAL_KASAN(fn) _GLOBAL(__##fn) #define _GLOBAL_TOC_KASAN(fn) _GLOBAL_TOC(__##fn) #define EXPORT_SYMBOL_KASAN(fn) EXPORT_SYMBOL(__##fn) #else #define _GLOBAL_KASAN(fn) _GLOBAL(fn) #define _GLOBAL_TOC_KASAN(fn) _GLOBAL_TOC(fn) #define EXPORT_SYMBOL_KASAN(fn) #endif #ifndef __ASSEMBLY__ #define KASAN_SHADOW_SCALE_SHIFT 3 #define KASAN_SHADOW_START (KASAN_SHADOW_OFFSET + \ (PAGE_OFFSET >> KASAN_SHADOW_SCALE_SHIFT)) #define KASAN_SHADOW_END (KASAN_SHADOW_START + KASAN_SHADOW_SIZE) #ifdef CONFIG_KASAN_SHADOW_OFFSET #define KASAN_SHADOW_OFFSET ASM_CONST(CONFIG_KASAN_SHADOW_OFFSET) #endif #ifdef CONFIG_PPC32 #define KASAN_SHADOW_SIZE ((-PAGE_OFFSET) >> KASAN_SHADOW_SCALE_SHIFT) #endif #ifdef CONFIG_PPC_BOOK3S_64 #define KASAN_SHADOW_SIZE (ASM_CONST(CONFIG_PHYS_MEM_SIZE_FOR_KASAN) *=20 SZ_1G) >> \ KASAN_SHADOW_SCALE_SHIFT) #endif #ifdef CONFIG_KASAN void kasan_early_init(void); void kasan_mmu_init(void); void kasan_init(void); #else static inline void kasan_init(void) { } static inline void kasan_mmu_init(void) { } #endif #endif /* __ASSEMBLY */ #endif Christophe