From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36977EB64DC for ; Sun, 25 Jun 2023 23:52:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 93DD28D0002; Sun, 25 Jun 2023 19:52:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8EE518D0001; Sun, 25 Jun 2023 19:52:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 78DEC8D0002; Sun, 25 Jun 2023 19:52:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6A3F08D0001 for ; Sun, 25 Jun 2023 19:52:56 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 3183C8048F for ; Sun, 25 Jun 2023 23:52:56 +0000 (UTC) X-FDA: 80942923152.10.D0A6DB8 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf08.hostedemail.com (Postfix) with ESMTP id 3B607160009 for ; Sun, 25 Jun 2023 23:52:53 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JWW2UW2J; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of luto@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=luto@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687737173; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nVPsePdhqgwoP2kzbWw3rnebrnYDoJ8c9xk4ymDd/E4=; b=w8W8WOlqE0JD3vF0FmSIIV3a7PEuFIxJHcAxrNJuNTXTfx58eJuz3tV5GPaHFrd5MBDA0D JavIDfA3fhABMmC5BU5AtfWmwF5FfXHUrMHPT8wcQEADOp/0VVl0UckqxlzdxeJNdgD4eY REDbY6Okk1UdI2bPIhx1+W+n4tGBsx8= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JWW2UW2J; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of luto@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=luto@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687737173; a=rsa-sha256; cv=none; b=R1KV15z3uORgmq1Fi0WnM5QjFyLD8boAU9p6jUFrehFFKITPyEztP54h4BueGCdRALibXJ M4jnXFI7AfYp3zAahLdWYM0XK+mMudC0v++/I4lmd2x+cES7OIHlOM31XZDkX7gATaiTnV mdbNTh0cZF3ikCwka/+PE20+T2cgylg= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2BBF160C61 for ; Sun, 25 Jun 2023 23:52:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 381CBC43391 for ; Sun, 25 Jun 2023 23:52:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1687737171; bh=MarIaZK1/h6d3gcEiuGdnJtH9oNLc/Ovu23/aiJiCSM=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=JWW2UW2JySBQR4RQGxecvyhKgMMyEDqCnfecOWHZ9i4MZ2UEMG0MRaxT31yhC09NB ql42vu4G7fvuZ+qYXhm4wc/BK13U4gb+dYCfK2rxcQQkwLEikYyST2SbnUt9MU0FbS bju6xtvV9SSsWbi4z2+AGufTKZnmEJGVJbttWf8wk8TEXGoAWRisWj90yZDohsA6u0 UFAKBGjgVoFDCitGIc4MJ0yroygJzKccIC6I9pPAr8/195iRgAsPSvC0mYRYevENED MWbB9tM1xRBOPk79Wf/JWvbaSrnGnOoeZGhbknFS/8mEGRH5rP21r46vhkXgZrKnFq Ji+eOLeooaJ3A== Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-3f9ede60140so23402865e9.0 for ; Sun, 25 Jun 2023 16:52:51 -0700 (PDT) X-Gm-Message-State: AC+VfDzTSJzTgTaci/5gCm/Q7R3Fy1e77Ghdg++Tbe+TvibN/kPrDxi4 K8r7sQboOJKw8X+AlEIQtSmKwLjl6z6GvLtMDgqiMA== X-Google-Smtp-Source: ACHHUZ4SsLYGJn0dfSXxFKaC+b/PwcoJ8iANFo8WWh8nomaO+sSfLwffHOMAkFtQJoYhfIZGagMbWTVvbeBhHgGqx6c= X-Received: by 2002:a17:907:3ea3:b0:987:3f34:88ea with SMTP id hs35-20020a1709073ea300b009873f3488eamr23437525ejc.58.1687737148936; Sun, 25 Jun 2023 16:52:28 -0700 (PDT) MIME-Version: 1.0 References: <64837d2af3ae39bafd025b3141a04f04f4323205.camel@intel.com> <5794e4024a01e9c25f0951a7386cac69310dbd0f.camel@intel.com> In-Reply-To: From: Andy Lutomirski Date: Sun, 25 Jun 2023 16:52:16 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v9 23/42] Documentation/x86: Add CET shadow stack description To: "szabolcs.nagy@arm.com" Cc: Andy Lutomirski , "Edgecombe, Rick P" , "broonie@kernel.org" , "Xu, Pengfei" , "tglx@linutronix.de" , "linux-arch@vger.kernel.org" , "kcc@google.com" , "nadav.amit@gmail.com" , "kirill.shutemov@linux.intel.com" , "david@redhat.com" , "Schimpe, Christina" , "linux-doc@vger.kernel.org" , "peterz@infradead.org" , "corbet@lwn.net" , "nd@arm.com" , "dethoma@microsoft.com" , "jannh@google.com" , "linux-kernel@vger.kernel.org" , "debug@rivosinc.com" , "pavel@ucw.cz" , "bp@alien8.de" , "mike.kravetz@oracle.com" , "linux-api@vger.kernel.org" , "rppt@kernel.org" , "jamorris@linux.microsoft.com" , "arnd@arndb.de" , "john.allen@amd.com" , "rdunlap@infradead.org" , "bsingharora@gmail.com" , "oleg@redhat.com" , "andrew.cooper3@citrix.com" , "keescook@chromium.org" , "x86@kernel.org" , "gorcunov@gmail.com" , "Yu, Yu-cheng" , "fweimer@redhat.com" , "hpa@zytor.com" , "mingo@redhat.com" , "hjl.tools@gmail.com" , "linux-mm@kvack.org" , "Syromiatnikov, Eugene" , "Torvalds, Linus" , "akpm@linux-foundation.org" , "dave.hansen@linux.intel.com" , "Yang, Weijiang" , "Eranian, Stephane" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 3B607160009 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: xue3hmmixqtxabk7hw7uhmnegcfgd4dz X-HE-Tag: 1687737173-316685 X-HE-Meta: U2FsdGVkX1//Q1k3gZu30mpN0Z15CZe36eUho0FCbkkKoaw04gqiGGlyWaFXe9dzVIR98vTm9K3iGKJZzNZhRnFpVJpXr2Fag+j7/t2r8NVRLF+6/YgE7RjUrKwy5aRhHQSAYOoVrSraImjMjkvVEuq2OZ+yXExrjEDgKhzJTDpQiX4LZ5BurdFw5C6IZM4R70Io8PfQtB/H0ItsccYr2Zb3HQorooAl5e3ktssDX199KKDKd9vYEF3j2X6MP5ol/ptPGSlGEPL/JGtiYxv2LLx6Oi9Fu+0JL2nfp/JCseLLmIHSbAt3dnWLq7Y0KzDtx/lE6ilu05pOsGRwgWL5pqyycoTxxkKghMRB84Q8s9YOudEgEWOf9rp/1fmrzR+sF3HzHeVTWanTH0mFI8/atewZNfq6C+G1ISmBUXx6Gt6YGPJh/pJC+1mF9tdH+yfWZEU3SsPQsToD3w+ZZdzB7uV3j156TEzWfRlBuFzecwU5JVksJGss+9IOWESDSayMdGrkAfi2Bp8NE5S1/gfjob+P+/7Z79OeF8Tf8kBBw0G7JunO62Z/xsdrTnxwGeFakEJeK/2uqyhkqCKs/DL/HW7qS/WuS+xmoGwPCV7MwbA6Y5E6bTDP1KjcJ+HW6H39v3T+inaBHogZnPV6gtNkR3yQFQuJzqNgUzHEM9tizEysvvOz2gUpDR4ACkxXex0hPoqL9VWag35QsqQgYZx07oaRg3kkZ1y7ULFqs+UMtmIXb14HBfcSMax9tceJpc7JsInioGFSUDQV0EEyg0pJctDpXFscg/HY3PImFrnfRocrbOQ1UCktGJXdLkFuDC3J6hNaE3XylH6ruwGzWtTcTbyDsyhALLGHys4lHQ2LCrqob6qiqh6blRU97pqh9gGVSBKaIBhaMFMekkiE1h6m72S6+jRrIfcSzfVK5Nu6IksApHK4Vz1b7hU/IfTGPDBDKy4nqcHq4qMNmxgGL63 C0EjcSXO OtOwnOqMjx5zSBgVXsTwBgTVDuzri+IFw6wO/uYCilHORU4XtHYxgVM7azuleejL8b6Q52vQeRm+xpPA1OuwENv6yF1U/dVsGuJ9qpKvUQ5nhdzIRE41m1rc1GmZS9JmOEeZ4Kw6xaUW1L4Gt+VAkWfosJkhPHcU8LbdL7kDPCCELfEd/m2y2icQfkxY1I/0d3PVLros2C9WY0M+OWS+900lx/pJ/7fltkRGkgq+NeAvgjvCC8C3qKXFoAdDFqvRrKwmQffvCOav0haJuP7IM32zGOfHuDARGUSMqE1YhZWwb2CyQl+FW9AMfNNq7mWVg8rr2JzGIzmbmO4SPmbmSlaPE5YNr02qgOMBkHuwRv6qtNpIrkyLs0SqIdZFPiyGlVoJvjmHUcQUk7OwY5WkjpFYi3bNheQCadvx4aYcMMsUlZGPwpWMt2C2KDoVGyAGVYB3V2J0AtMkA6LLmZtcWYdfOpyljbE0aPwkZwubITlRC7Yo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Jun 22, 2023 at 9:43=E2=80=AFAM szabolcs.nagy@arm.com wrote: > > The 06/22/2023 08:26, Andy Lutomirski wrote: > > On Thu, Jun 22, 2023 at 2:28=E2=80=AFAM szabolcs.nagy@arm.com > > wrote: > > > > > > The 06/21/2023 18:54, Edgecombe, Rick P wrote: > > > > On Wed, 2023-06-21 at 12:36 +0100, szabolcs.nagy@arm.com wrote: > > > > > > The 06/20/2023 19:34, Edgecombe, Rick P wrote: > > > > > > > > I actually did a POC for this, but rejected it. The problem= is, > > > > > > > > if > > > > > > > > there is a shadow stack overflow at that point then the ker= nel > > > > > > > > > > can't > > > > > > > > push the shadow stack token to the old stack. And shadow st= ack > > > > > > > > > > overflow > > > > > > > > is exactly the alt shadow stack use case. So it doesn't rea= lly > > > > > > > > > > solve > > > > > > > > the problem. > > > > > > > > > > > > the restore token in the alt shstk case does not regress anythi= ng > > > > > > but > > > > > > makes some use-cases work. > > > > > > > > > > > > alt shadow stack is important if code tries to jump in and out = of > > > > > > signal handlers (dosemu does this with swapcontext) and for tha= t a > > > > > > restore token is needed. > > > > > > > > > > > > alt shadow stack is important if the original shstk did not > > > > > > overflow > > > > > > but the signal handler would overflow it (small thread stack, h= uge > > > > > > sigaltstack case). > > > > > > > > > > > > alt shadow stack is also important for crash reporting on shstk > > > > > > overflow even if longjmp does not work then. longjmp to a > > > > > > makecontext > > > > > > stack would still work and longjmp back to the original stack c= an > > > > > > be > > > > > > made to mostly work by an altshstk option to overwrite the top > > > > > > entry > > > > > > with a restore token on overflow (this can break unwinding thou= gh). > > > > > > > > > > > > > > There was previously a request to create an alt shadow stack for th= e > > > > purpose of handling shadow stack overflow. So you are now suggestin= g to > > > > to exclude that and instead target a different use case for alt sha= dow > > > > stack? > > > > > > that is not what i said. > > > > > > > But I'm not sure how much we should change the ABI at this point si= nce > > > > we are constrained by existing userspace. If you read the history, = we > > > > may end up needing to deprecate the whole elf bit for this and othe= r > > > > reasons. > > > > > > i'm not against deprecating the elf bit, but i think binary > > > marking will be difficult for this kind of feature no matter what > > > (code may be incompatible for complex runtime dependent reasons). > > > > > > > So should we struggle to find a way to grow the existing ABI withou= t > > > > disturbing the existing userspace? Or should we start with somethin= g, > > > > finally, and see where we need to grow and maybe get a chance at a > > > > fresh start to grow it? > > > > > > > > Like, maybe 3 people will show up saying "hey, I *really* need to u= se > > > > shadow stack and longjmp from a ucontext stack", and no one says > > > > anything about shadow stack overflow. Then we know what to do. And > > > > maybe dosemu decides it doesn't need to implement shadow stack (hig= hly > > > > likely I would think). Now that I think about it, AFAIU SS_AUTODISA= RM > > > > was created for dosemu, and the alt shadow stack patch adopted this > > > > behavior. So it's speculation that there is even a problem in that > > > > scenario. > > > > > > > > Or maybe people just enable WRSS for longjmp() and directly jump ba= ck > > > > to the setjmp() point. Do most people want fast setjmp/longjmp() at= the > > > > cost of a little security? > > > > > > > > Even if, with enough discussion, we could optimize for all > > > > hypotheticals without real user feedback, I don't see how it helps > > > > users to hold shadow stack. So I think we should move forward with = the > > > > current ABI. > > > > > > you may not get a second chance to fix a security feature. > > > it will be just disabled if it causes problems. > > > > *I* would use altshadowstack. > > > > I run a production system (that cares about correctness *and* > > performance, but that's not really relevant here -- SHSTK ought to be > > fast). And, if it crashes, I want to know why. So I handle SIGSEGV, > > etc so I have good logs if it crashes. And I want those same logs if > > I overflow the stack. > > > > That being said, I have no need for longjmp or siglongjmp for this. I > > use exit(2) to escape. > > the same crash handler that prints a log on shstk overflow should > work when a different cause of SIGSEGV is recoverable via longjmp. > to me this means that alt shstk must work with longjmp at least in > the non-shstk overflow case (which can be declared non-recoverable). Sure, but how many SIGSEGV handlers would use altshadowstack and *also, in the same handler* ever resume? Not mine. Obviously I'm only one sample. > > > For what it's worth, setjmp/longjmp is a bad API. The actual pattern > > that ought to work well (and that could be supported well by fancy > > compilers and non-C languages, as I understand it) is more like a > > function call that has two ways out. Like this (pseudo-C): > > > > void function(struct better_jmp_buf &buf, args...) > > { > > ... > > if (condition) > > better_long_jump(buf); // long jumps out! > > // could also pass buf to another function > > ... > > // could also return normally > > } > > > > better_call_with_jmp_buf(function, args); > > > > *This* could support altshadowstack just fine. And many users might > > be okay with the understanding that, if altshadowstack is on, you have > > to use a better long jump to get out (or a normal sigreturn or _exit). > > i don't understand why this would work fine when longjmp does not. > how does the shstk switch happen? Ugh, I think this may have some issues given how the ISA works. Sigh. I was imagining that better_call_with_jmp_buf would push a restore token on the shadow stack, then call the passed-in function, then, on a successful return, INCSSP over the token and continue on. better_long_jump() would RSTORSSP to the saved token. But I'm not sure how to write the token without WRUSS. What *could* be done, which would be nasty and sigaltshadowstack-specific, is to have a jump out of a signal handler provide a pointer to the signal frame (siginfo_t or ucontext pointer), and the kernel would assist it in switching the shadow stack back. Eww. --Andy > > > No one is getting an altshadowstack signal handler without code > > changes. > > assuming the same component is doing the alt shstk setup as the > longjmp. > > > siglongjmp() could support altshadowstack with help from the kernel, > > but we probably don't want to go there. > > what kind of help? maybe we need that help.. > > e.g. if the signal frame token is detected by longjmp on > the shstk then doing an rt_sigreturn with the right signal > frame context allows longjmp to continue unwinding the shstk. > however kernel sigcontext layout can change so userspace may > not know it so longjmp needs a helper, but only in the jump > across signal frame case. > > (this is a different design than what i proposed earlier, > it also makes longjmp from alt shstk work without wrss, > the downside is that longjmp across makecontext needs a > separate solution then which implies that all shstk needs > a detectable token at the end of the shstk.. so again > something that we have to get right now and cannot add > later.)