From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7956EC433B4 for ; Thu, 6 May 2021 22:05:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D610661090 for ; Thu, 6 May 2021 22:05:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D610661090 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 322656B0070; Thu, 6 May 2021 18:05:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2D2176B0071; Thu, 6 May 2021 18:05:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 173606B0072; Thu, 6 May 2021 18:05:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0028.hostedemail.com [216.40.44.28]) by kanga.kvack.org (Postfix) with ESMTP id EFABF6B0070 for ; Thu, 6 May 2021 18:05:25 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id AE1638249980 for ; Thu, 6 May 2021 22:05:25 +0000 (UTC) X-FDA: 78112188210.11.639E48D Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf27.hostedemail.com (Postfix) with ESMTP id 9ED7280192D9 for ; Thu, 6 May 2021 22:04:52 +0000 (UTC) IronPort-SDR: Bd6YGhpPZgz4a4Ae90h1HG24kSiRUG5zqheEFzCh9lX79ZOm/yhVnMgTHyNJUZonfzAc0DI89O f6L8trOgvoMw== X-IronPort-AV: E=McAfee;i="6200,9189,9976"; a="178149632" X-IronPort-AV: E=Sophos;i="5.82,279,1613462400"; d="scan'208";a="178149632" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2021 15:05:18 -0700 IronPort-SDR: E3B3lZ5t9BIU7x8JT3XoIkdfTP2uDRLb1wtuJN38MAl21fRXDH3sv+Vcf7oDpbQ+XVVkx4ab22 JaNUBAXX/BSw== X-IronPort-AV: E=Sophos;i="5.82,279,1613462400"; d="scan'208";a="434596777" Received: from yyu32-mobl1.amr.corp.intel.com (HELO [10.251.158.199]) ([10.251.158.199]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2021 15:05:16 -0700 Subject: Re: extending ucontext (Re: [PATCH v26 25/30] x86/cet/shstk: Handle signals for shadow stack) From: "Yu, Yu-cheng" To: Andy Lutomirski Cc: linux-arch , X86 ML , "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , LKML , "open list:DOCUMENTATION" , Linux-MM , Linux API , Arnd Bergmann , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu , Haitao Huang References: <20210427204315.24153-1-yu-cheng.yu@intel.com> <20210427204315.24153-26-yu-cheng.yu@intel.com> <8fd86049-930d-c9b7-379c-56c02a12cd77@intel.com> <5fc5dea4-0705-2aad-cf8f-7ff78a5e518a@intel.com> Message-ID: Date: Thu, 6 May 2021 15:05:15 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: <5fc5dea4-0705-2aad-cf8f-7ff78a5e518a@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 9ED7280192D9 X-Stat-Signature: zskf4hd4th8k6sj4uiqfmqugbepb156k Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none); spf=none (imf27.hostedemail.com: domain of yu-cheng.yu@intel.com has no SPF policy when checking 192.55.52.136) smtp.mailfrom=yu-cheng.yu@intel.com Received-SPF: none (intel.com>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=mga12.intel.com; client-ip=192.55.52.136 X-HE-DKIM-Result: none/none X-HE-Tag: 1620338692-743961 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 5/4/2021 1:49 PM, Yu, Yu-cheng wrote: > On 4/30/2021 11:32 AM, Yu, Yu-cheng wrote: >> On 4/30/2021 10:47 AM, Andy Lutomirski wrote: >>> On Fri, Apr 30, 2021 at 10:00 AM Yu, Yu-cheng = =20 >>> wrote: >>>> >>>> On 4/28/2021 4:03 PM, Andy Lutomirski wrote: >>>>> On Tue, Apr 27, 2021 at 1:44 PM Yu-cheng Yu = =20 >>>>> wrote: >>>>>> >>>>>> When shadow stack is enabled, a task's shadow stack states must be= =20 >>>>>> saved >>>>>> along with the signal context and later restored in sigreturn.=20 >>>>>> However, >>>>>> currently there is no systematic facility for extending a signal=20 >>>>>> context. >>>>>> There is some space left in the ucontext, but changing ucontext is= =20 >>>>>> likely >>>>>> to create compatibility issues and there is not enough space for=20 >>>>>> further >>>>>> extensions. >>>>>> >>>>>> Introduce a signal context extension struct 'sc_ext', which is=20 >>>>>> used to save >>>>>> shadow stack restore token address.=C2=A0 The extension is located= =20 >>>>>> above the fpu >>>>>> states, plus alignment.=C2=A0 The struct can be extended (such as = the=20 >>>>>> ibt's >>>>>> wait_endbr status to be introduced later), and sc_ext.total_size=20 >>>>>> field >>>>>> keeps track of total size. >>>>> >>>>> I still don't like this. >>>>> [...] >>>>> >>>>> That's where we are right now upstream.=C2=A0 The kernel has a pars= er for >>>>> the FPU state that is bugs piled upon bugs and is going to have to = be >>>>> rewritten sometime soon.=C2=A0 On top of all this, we have two upco= ming >>>>> features, both of which require different kinds of extensions: >>>>> >>>>> 1. AVX-512.=C2=A0 (Yeah, you thought this story was over a few year= s ago, >>>>> but no.=C2=A0 And AMX makes it worse.)=C2=A0 To make a long story s= hort, we >>>>> promised user code many years ago that a signal frame fit in 2048 >>>>> bytes with some room to spare.=C2=A0 With AVX-512 this is false.=C2= =A0 With AMX >>>>> it's so wrong it's not even funny.=C2=A0 The only way out of the me= ss >>>>> anyone has come up with involves making the length of the FPU state >>>>> vary depending on which features are INIT, i.e. making it more comp= act >>>>> than "compact" mode is.=C2=A0 This has a side effect: it's no longe= r >>>>> possible to modify the state in place, because enabling a feature w= ith >>>>> no space allocated will make the structure bigger, and the stack wo= n't >>>>> have room.=C2=A0 Fortunately, one can relocate the entire FPU state= , update >>>>> the pointer in mcontext, and the kernel will happily follow the >>>>> pointer.=C2=A0 So new code on a new kernel using a super-compact st= ate >>>>> could expand the state by allocating new memory (on the heap? very >>>>> awkwardly on the stack?) and changing the pointer.=C2=A0 For all we= know, >>>>> some code already fiddles with the pointer.=C2=A0 This is great, ex= cept >>>>> that your patch sticks more data at the end of the FPU block that n= o >>>>> one is expecting, and your sigreturn code follows that pointer, and >>>>> will read off into lala land. >>>>> >>>> >>>> Then, what about we don't do that at all.=C2=A0 Is it possible from = now=20 >>>> on we >>>> don't stick more data at the end, and take the relocating-fpu approa= ch? >>>> >>>>> 2. CET.=C2=A0 CET wants us to find a few more bytes somewhere, and = those >>>>> bytes logically belong in ucontext, and here we are. >>>>> >>>> >>>> Fortunately, we can spare CET the need of ucontext extension.=C2=A0 = When the >>>> kernel handles sigreturn, the user-mode shadow stack pointer is=20 >>>> right at >>>> the restore token.=C2=A0 There is no need to put that in ucontext. >>> >>> That seems entirely reasonable.=C2=A0 This might also avoid needing t= o >>> teach CRIU about CET at all. >>> >>>> >>>> However, the WAIT_ENDBR status needs to be saved/restored for signal= s. >>>> Since IBT is now dependent on shadow stack, we can use a spare bit o= f >>>> the shadow stack restore token for that. >>> >>> That seems like unnecessary ABI coupling.=C2=A0 We have plenty of bit= s in >>> uc_flags, and we have an entire reserved word in sigcontext.=C2=A0 Ho= w >>> about just sticking this bit in one of those places? >> >> Yes, I will make it UC_WAIT_ENDBR. >=20 > Personally, I think an explicit flag is cleaner than using a reserved=20 > word somewhere.=C2=A0 However, there is a small issue: ia32 has no uc_f= lags. >=20 > This series can support legacy apps up to now.=C2=A0 But, instead of cr= eating=20 > too many special cases, perhaps we should drop CET support of ia32? >=20 > Thoughts? >=20 Once we have UC_WAIT_ENDBR, IBT signal handling becomes quite simple.=20 Like the following: diff --git a/arch/x86/include/uapi/asm/ucontext.h=20 b/arch/x86/include/uapi/asm/ucontext.h index 5657b7a49f03..96375d609e11 100644 --- a/arch/x86/include/uapi/asm/ucontext.h +++ b/arch/x86/include/uapi/asm/ucontext.h @@ -49,6 +49,11 @@ */ #define UC_SIGCONTEXT_SS 0x2 #define UC_STRICT_RESTORE_SS 0x4 + +/* + * UC_WAIT_ENDBR indicates the task is in wait-ENDBR status. + */ +#define UC_WAIT_ENDBR 0x08 #endif #include diff --git a/arch/x86/kernel/ibt.c b/arch/x86/kernel/ibt.c index d2563dd4759f..da804314ddc4 100644 --- a/arch/x86/kernel/ibt.c +++ b/arch/x86/kernel/ibt.c @@ -66,3 +66,32 @@ void ibt_disable(void) ibt_set_clear_msr_bits(0, CET_ENDBR_EN); current->thread.cet.ibt =3D 0; } + +int ibt_get_clear_wait_endbr(void) +{ + u64 msr_val =3D 0; + + if (!current->thread.cet.ibt) + return 0; + + fpregs_lock(); + + if (test_thread_flag(TIF_NEED_FPU_LOAD)) + __fpregs_load_activate(); + + if (!rdmsrl_safe(MSR_IA32_U_CET, &msr_val)) + wrmsrl(MSR_IA32_U_CET, msr_val & ~CET_WAIT_ENDBR); + + fpregs_unlock(); + + return msr_val & CET_WAIT_ENDBR; +} + +int ibt_set_wait_endbr(void) +{ + if (!current->thread.cet.ibt) + return 0; + + + return ibt_set_clear_msr_bits(CET_WAIT_ENDBR, 0); +} diff --git a/arch/x86/kernel/signal.c b/arch/x86/kernel/signal.c index 66b662e57e19..5afd15419006 100644 --- a/arch/x86/kernel/signal.c +++ b/arch/x86/kernel/signal.c @@ -46,6 +46,7 @@ #include #include #include +#include #ifdef CONFIG_X86_64 /* @@ -134,6 +135,9 @@ static int restore_sigcontext(struct pt_regs *regs, */ if (unlikely(!(uc_flags & UC_STRICT_RESTORE_SS) &&=20 user_64bit_mode(regs))) force_valid_ss(regs); + + if (uc_flags & UC_WAIT_ENDBR) + ibt_set_wait_endbr(); #endif return fpu__restore_sig((void __user *)sc.fpstate, @@ -433,6 +437,9 @@ static unsigned long frame_uc_flags(struct pt_regs=20 *regs) if (likely(user_64bit_mode(regs))) flags |=3D UC_STRICT_RESTORE_SS; + if (ibt_get_clear_wait_endbr()) + flags |=3D UC_WAIT_ENDBR; + return flags; } However, this cannot handle ia32 with no SA_SIGINFO. For that, can we=20 create a synthetic token on the shadow stack? - The token points to itself with reserved bit[1] set, and cannot be=20 used for RSTORSSP. - The token only exists for ia32 with no SA_SIGINFO *AND* when the task=20 is in wait-endbr. The signal shadow stack will look like this: --> ssp before signal synthetic IBT token (for ia32 no SA_SIGINFO) shadow stack restore token sigreturn address The synthetic token is not valid in other situations. How is that? Thanks, Yu-cheng