From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CBE9EB64DA for ; Wed, 19 Jul 2023 07:17:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 22F3D280033; Wed, 19 Jul 2023 03:17:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1DEAC8D0012; Wed, 19 Jul 2023 03:17:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 08157280033; Wed, 19 Jul 2023 03:17:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id EC0368D0012 for ; Wed, 19 Jul 2023 03:17:45 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C07BA401A6 for ; Wed, 19 Jul 2023 07:17:45 +0000 (UTC) X-FDA: 81027506490.21.AEEF2BC Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) by imf17.hostedemail.com (Postfix) with ESMTP id D6A5A4000B for ; Wed, 19 Jul 2023 07:17:43 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=C3xFynKw; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of jeffxu@google.com designates 209.85.160.180 as permitted sender) smtp.mailfrom=jeffxu@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689751063; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KJz9sA4DGP6CgYLkAjY0lAz1lGElKDaYsD0QJhJ6STM=; b=uce8P7ZcjZajAJwvdym7O8u3bhm/XJ4rZBOJZwPyIWNHyyrsUbY4TUb9AvFQBhetPZPQCZ IB0yMIbxyEpGuZdPJTQWciTlsUPbdQrv5tAoINqrYjB4B+GxyBBWzo8gBCwOH83xudM4g+ Hio/C13JCc8fN/rT5JOfc4GxM9BLSD0= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=C3xFynKw; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of jeffxu@google.com designates 209.85.160.180 as permitted sender) smtp.mailfrom=jeffxu@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689751063; a=rsa-sha256; cv=none; b=Te+pIDlZZlf5FUStSt42KwucZPdxVi2m9B6QyDkEUS7lqJ9B7q9pIPp3bITq5HxGBg2xc4 nqREDeSa8OTj6nUzSqXYjTGm50ogo6igL64W0ugpWarsdvWzjmwqmI258lR0bwcgDCNL31 +iu+IPetwAsW+36Ykh/9KfqXt/namsM= Received: by mail-qt1-f180.google.com with SMTP id d75a77b69052e-40371070eb7so514031cf.1 for ; Wed, 19 Jul 2023 00:17:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689751063; x=1692343063; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=KJz9sA4DGP6CgYLkAjY0lAz1lGElKDaYsD0QJhJ6STM=; b=C3xFynKwxg1DRU1+DuE5DhOV3wV/h9i3th7GM3Hn84DsD3AQDvuahPMaPfjZB92a1q 6cIdh8+OnPrq4dvRj+K/sq62IkYMUpNuHyqjtxJa/x3zG8o5bg7Qao2szql97Y0NpCQa /97J2LFKn0RnI73D9Cr12stBt9oevdC/try31WjZwCXLrgWP4DFzzGX3lnkfpogOn5iG bg0ps36g+1B4WpO9/V8EvUDmammOUTf1nISCcj9rU0r7LYuh7l7EublxkySIWZG8u7bx nSMqoCSyCX89BpM3rGcrI7i52hLLUrxMNyS0PPdUZtp9nru/EPjOGPcH/2hfjHIAXimE lDfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689751063; x=1692343063; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KJz9sA4DGP6CgYLkAjY0lAz1lGElKDaYsD0QJhJ6STM=; b=jxeUOIOTVivbeoWalNZvEnWu/F5K/yy6bp2IN6R+TpIJ+YIKcUvuv6ra5a8Npa/iWa 3JPgyt4mYTV0haHyM7VfqDqQA4o5slHTxXkLo5e7996qtDXFsn1tL847aU9uDfa4eSfi yWF2bydRqZnvdXjQSckK5pqXZkbeoILHQdkT0TAOZ5XjcjgPdo2Dk3XuL1wjmE4+XayV GyY3TTlNZjhsslBr0elgze/3R85l02ko3L2XhpTSL0LLtdzinj4rBhNxtpYLbm35hSma dycVRbz0BAdtLacF00kPeVeUP6Hm65DSpwKMaxqUuoyyI7wAzsVEfRzCcHrg28HTUp22 qGvg== X-Gm-Message-State: ABy/qLahXdhCCT2ij+2fytP3tWBv4Q5AO6iagqY+ppVxPtmJdJD+klgj mB2towwOb9k9N3SfF+AjcVNJzIqKqb+MKysRfTFr9g== X-Google-Smtp-Source: APBJJlG5E2lpIv0SvgSx7L4QeWU+mDNg17bE+ImlToYqKL1M7i7rZzz+Kp7fxMaIuOYd3Pa9EnKRQ2/MS1NsTY3vl3s= X-Received: by 2002:ac8:7e8d:0:b0:3f5:2006:50f1 with SMTP id w13-20020ac87e8d000000b003f5200650f1mr546839qtj.12.1689751062793; Wed, 19 Jul 2023 00:17:42 -0700 (PDT) MIME-Version: 1.0 References: <20230713143406.14342-1-cyphar@cyphar.com> In-Reply-To: From: Jeff Xu Date: Wed, 19 Jul 2023 00:17:06 -0700 Message-ID: Subject: Re: [RFC PATCH 0/3] memfd: cleanups for vm.memfd_noexec To: Aleksa Sarai Cc: Jeff Xu , Andrew Morton , Shuah Khan , Kees Cook , Daniel Verkamp , Luis Chamberlain , YueHaibing , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-hardening@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: D6A5A4000B X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: wfruxutypuxkxr5ux3qcq8nrnyakyfw3 X-HE-Tag: 1689751063-255461 X-HE-Meta: U2FsdGVkX1+4atBWcj4wuhXN0ptrT0+MnVUccV7On+2U7+j8FNFBQnS/1gCzaBa0eamWFU6LhCcO0pS4kojLKmwWIGRbyPsJyrMtoSewTexOlXpkkx36R76MxrVVVYg7jITqhqbo3m46Ya/gUAcEAkytP0fswtIsBsnlLmwcMFs5NXil4xEOGSPn6mguV7FPNqW76j8OYRtiHsTKKRMA8fFwy7rl0ZZjKgIRobVGmwZm7AaveudfyMlspKtpo1aPEpMkiPjlLwxaGMq162uagW2aZxj23HeG8MTv4RPBK0HB7KLdXtbYZQ+MZQZNHOFfGCW8Qw9R1lNGOKzOv4uYfVzSWA2NSBbj3sT0xqAwouGvy/C5TUOqdudMfebGLEnLZjQMN3sJ1YFVo+tmoPMwQDioKE8U9AXq398D4nL03yp7+A6Q7DW6ldLvDb491kjQri3uLR69VIsDJtJ1d5GcO5rcA2Mp+3uuC249G28/ZvkLradyhpTLPigxo3Whb1ZRSY20PEczyzOolEWohea0SqGLt9b5OdtP0D/I25ZTWpqPVgVHYdoYw+H2afDl+y7qO4cp4dKjz/Zk16nI5URjwA2jz02azm9hJGHqJqigRJ07juiipvyUuW6/Zw//yO1J0lsUkUDToWHv6WHrtkmGqpBraJn7xv0HjlbuzYuOPmcRtJ0+hWxjGaGdX6fJHPKa7rFzA6UsSwx7MbfxIGzuppkCnPLAPW55BEWPKyolaCcge6MTlQN+R/4i/Ic5JY2Yu3MSoH8YJI/uMk1PoWk/PLkXmR7WNEKtMzBf3S3lBcPKOEfc1vpuiSqf9s1EFwa/Bt5JbQaAGxcRk/TvwxnZzsqa2RU+5uRbYnwiAYTXXQBeIVYLrUPmY3ZkCdffTx7RXurYJeh8HDgmSBoGF0Ld71k6jv7NewsmTZ3anFpL/C/53GZOUUhALcZQ6ABXlm6RZj1EJgryl5rMv8wVMvH qW3RYCFR pLS6oKMwdbV62m36T/4IDDxOjQx1wsU6t/kWxLHKHqIJeDDQtNXpCLT/vtN56pQPQwXZyfF+ZlCR+elgDB0A75YRnJqGjhz3Tg1aQ1+JeDeu83Q+gAzQojCVUFyxFxqRIs8i7BTfBC3NJnOQ5b9vywOw7lZIsbFiBQgeSPhbp2jM3CGHRuRnOQqxH5m0TcHADghthroeDazucVvuMXRRu0InA5bshFmoa6/PsPZlib1X++ob5obHkw+tPq+4uA/Ro2J7G31O2CZ3iHqZtph8VM2mNyGIJkieJCLX7YkSYz5lyKu4eSPKbGB36iC9Z3fxLBrf/mvCjr/qCEKDcXXG2l62wxSUc39yTZzTRotbISOnmD+tKJZ3rYaCZZItqAY3UzB1jL3DDJvBXEQgnJl3qOCtRUnc2ys75Bgw3vku5tuYvbeppTCsWbHeJCZxHgMJGabtNZuIZL2Ia5SkfQbSlZ2B7DKotc279wx3iex20r+UZrtNmZjRgKCmiZErV89H2jf9yv2n4jZfpj7L0yv9Yk0JEfd/xH/g30MbX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jul 18, 2023 at 8:10=E2=80=AFPM Aleksa Sarai wr= ote: > > On 2023-07-17, Jeff Xu wrote: > > Hello Aleksa, > > > > Thanks for your email and patches for discussion. > > > > On Thu, Jul 13, 2023 at 7:34=E2=80=AFAM Aleksa Sarai wrote: > > > > > > It seems that the most critical issue with vm.memfd_noexec=3D2 (the f= act > > > that passing MFD_EXEC would bypass it entirely[1]) has been fixed in > > > Andrew's tree[2], but there are still some outstanding issues that ne= ed > > > to be addressed: > > > > > > * The dmesg warnings are pr_warn_once, which on most systems means t= hat > > > they will be used up by systemd or some other boot process and > > > userspace developers will never see it. The original patch posted = to > > > the ML used pr_warn_ratelimited but the merged patch had it change= d > > > (with a comment about it being "per review"), but given that the > > > current warnings are useless, pr_warn_ratelimited makes far more > > > sense. > > > > > Ya, This was discussed in [1] > > Replacing pr_warn_once with pr_warn_ratelimited won't address Peter > > Xu's observation that "ratelimited" will fill syslog [2], I'm not > > sure it is acceptable to ones who is not interested in memfd, I will > > defer this to maintainers. > > > > [1] https://lore.kernel.org/lkml/202212161233.85C9783FB@keescook/ > > [2] https://lwn.net/ml/linux-kernel/Y5yS8wCnuYGLHMj4@x1n/ > > I see Kees's point, but in that case the logging should be tied to the > sysctl being the non-default value (I can post this version next if you > prefer). The current logging setup doesn't make sense. > Is there a best practice in kernel for this problem: too much log vs too little log In other products, usually the log level or compiler flag (ifdef) are for such a situation. > > > * vm.memfd_noexec=3D2 shouldn't reject old-style memfd_create(2) sys= calls > > > because it will make it far to difficult to ever migrate. Instead = it > > > should imply MFD_EXEC. > > > > > Though the purpose of memfd_noexec=3D2 is not to help with migration - > > but to disable creation of executable memfd for the current system/pid > > namespace. > > During the migration, vm.memfd_noexe =3D 1 helps overwriting for > > unmigrated user code as a temporary measure. > > My point is that the current behaviour for =3D2 means that nobody other > than *maybe* ChromeOS will ever be able to use it because it requires > auditing every program on the system. In fact, it's possible even > ChromeOS will run into issues given that one of the arguments made for > the nosymfollow mount option was that auditing all of ChromeOS to > replace every open with RESOLVE_NO_SYMLINKS would be too much effort[1] > (which I agreed with). Maybe this is less of an issue with > memfd_create(2) (which is much newer than open(2)) but it still seems > like a lot of busy work when the =3D1 behaviour is entirely sane even in > the strict threat model that =3D2 is trying to protect against. > It can also be a container (that have all memfd_create migrated to new API) One option I considered previously was "=3D2" would do overwrite+block , and "=3D3" just block. But then I worry that applications won't have motivation to ever change their existing code, the setting will forever stay at "=3D2", making "=3D3" even more impossible to ever be used system side. > To me, using =3D1 as a migration path (and in fact, calling =3D1 a migrat= ion > path further argues that the warning for not setting _EXEC or > _NOEXEC_SEAL should be tied to =3D1) would mean finding every program tha= t > uses executable memfds and changing it to stop doing that. Not that you > use =3D1 to go and rewrite every userspace program that uses > memfd_create(2) at all, without using executable memfds (rebooting each > time to test the behaviour because we use pr_warn_once). > I tend to think logging and sysctl are orthogonal, tie them together making it more complex than necessary. If we need more logging, we should find what is the best practice in the kernel for that. > If you want to block syscalls that don't explicitly pass NOEXEC_SEAL, > there are several tools for doing this (both seccomp and LSM hooks). > > [1]: https://lore.kernel.org/linux-fsdevel/20200131212021.GA108613@google= .com/ > > > Additional functionality/features should be implemented through > > security hook and LSM, not sysctl, I think. > > This issue with =3D2 cannot be fixed in an LSM. (On the other hand, you > could implement either =3D2 behaviour with an LSM using =3D1, and the > current strict =3D2 behaviour could be implemented purely with seccomp.) > By migration, I mean a system that is not fully migrated, such a system should just use "=3D0" or "=3D1". Additional features can be implemented in SELinux/Landlock/other LSM by a motivated dev. e.g. if a system wants to limit executable memfd to specific programs or fully disable it. "=3D2" is for a system/container that is fully migrated, in that case, SELinux/Landlock/LSM can do the same, but sysctl provides a convenient alternative. Yes, seccomp provides a similar mechanism. Indeed, combining "=3D1" and seccomp (block MFD_EXEC), it will overwrite + block X mfd, which is essentially what you want, iiuc.However, I do not wish to have this implemented in kernel, due to the thinking that I want kernel to get out of business of "overwriting" eventually. > > > * The ratcheting mechanism for vm.memfd_noexec doesn't make sense as= a > > > security mechanism because a CAP_SYS_ADMIN capable user can create > > > executable binaries in a hidden tmpfs very easily, not to mention = the > > > many other things they can do. > > > > > By further limiting CAP_SYS_ADMIN, an attacker can't modify this > > sysctl even after compromising some system service with high > > privilege, YAMA has the same approach for ptrace_scope=3D3 > > Personally, I also think this behaviour from YAMA is a little goofy too, > but given that it only locks the most extreme setting and there is no > way to get around the most extreme setting, I guess it makes some sense > (not to mention it's an LSM and so there is an argument that it should > be possible to lock out privileged users from modifying it). > There are many other security sysctls, and very few have this behaviour > because it doesn't make much sense in most cases. > > > In addition, this sysctl is pid_name spaced, this means child > > pid_namespace will alway have the same or stricter security setting > > than its parent, this allows admin to maintain a tree like view. If we > > allow the child pid namespace to elevate its setting, then the > > system-wide setting is no longer meaningful. > > "no longer meaningful" is too strong of a statement imho. It is still > useful for constraining non-root processes and presumably ChromeOS > disallows random processes to do CLONE_NEWUSER (otherwise the protection > of this sysctl is pointless) so in practice for ChromeOS there is no > change in the attack surface. > > (FWIW, I think tying this to the user namespace would've made more sense > since this is about privilege restrictions, but that ship has sailed.) > The reason that this sysctl is a PID namespace is that I hope a container and host can have different sysctl values, e.g. host will allow runc's use of X mfd, while a container doesn't want X mfd. . To clarify what you meant, do you mean this: when a container is in its own pid_namespace, and has "=3D2", the programs inside the container can still use CLONE_NEWUSER to break out "=3D2" ? And what makes the user namespace a better choice than pid namespace for this sysctl ? > > The code sample shared in this patch set indicates that the attacker > > already has the ability of creating tmpfs and executing complex steps, > > at that point, it doesn't matter if the code execution is from memfd > > or not. For a safe by default system such as ChromeOS, attackers won't > > easily run arbitrary code, memfd is one of the open doors for that, so > > we are disabling executable memfd in ChromeOS. In other words: if an > > attacker can already execute the arbitrary code as sample given in > > ChromeOS, without using executable memfd, then memfd is no longer the > > thing we need to worry about, the arbitrary code execution is already > > achieved by the attacker. Even though I use ChromeOS as an example, I > > think the same type of threat model applies to any system that wants > > to disable executable memfd entirely. > > I understand the threat model this sysctl is blocking, my point is that > blocking CAP_SYS_ADMIN from modifying the setting doesn't make sense > from that threat model. An attacker that manages to trick some process > into creating a memfd with an executable payload is not going to be able > to change the sysctl setting (unless there's a confused deputy with > CAP_SYS_ADMIN, in which case you have much bigger issues). > It is the reverse. An attacker that manages to trick some CAP_SYSADMIN processes into changing this sysctl value (i.e. lower the setting to 0 if no ratcheting), will be able to continue to use mfd as part of the attack chain. In chromeOS, an attacker that can change sysctl might not necessarily gain full arbitrary code execution already. As I mentioned previously, the main threat model here is to prevent arbitrary code execution through mfd. If an attacker already gains arbitrary code execution, at that point, we no longer worry about mfd. > If a CAP_SYS_ADMIN-capable user wants to change the sysctl, blocking it > doesn't add any security because that process could create a memfd-like > fd to execute without issues. >What practical attack does this ratcheting > mechanism protect against? (This is a question you can answer with the > YAMA sysctl, but not this one AFAICS.) > > But even if you feel that allowing this in child user namespaces is > unsafe or undesirable, it's absolutely necessary that > capable(CAP_SYS_ADMIN) should be able to un-brick the running system by > changing the sysctl. The alternative is that you need to reboot your > server in order to un-set a sysctl that broke some application you run. > > Also, by the same token, this ratcheting mechanism doesn't make sense > with =3D1 *at all* because it could break programs in a way that would > require a reboot but it's not a "security setting" (and the YAMA sysctl > mentioned only locks the sysctl at the highest setting). > I think a system should use "=3D0" when it is unsure about its program's need or not need executable memfd. Technically, it is not that this sysctl breaks the user, but the admin made the mistake to set the wrong sysctl value, and an admin should know what they are doing for a sysctl. Yes. rebooting increases the steps to undo the mistake, but that could be an incentive for the admin to fully test its programs before turning on this sysctl - and avoid unexpected runtime errors. Thanks! -Jeff > > > * The memfd selftests would not exit with a non-zero error code when > > > certain tests that ran in a forked process (specifically the ones > > > related to MFD_EXEC and MFD_NOEXEC_SEAL) failed. > > > > > I will test this code and follow up. > > Thanks! > > -- > Aleksa Sarai > Senior Software Engineer (Containers) > SUSE Linux GmbH >