From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.6 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC372C433DF for ; Tue, 11 Aug 2020 03:06:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 81FB6206C3 for ; Tue, 11 Aug 2020 03:06:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Vt/xx+6q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 81FB6206C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F32886B0003; Mon, 10 Aug 2020 23:06:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EE47A6B0005; Mon, 10 Aug 2020 23:06:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF9166B0006; Mon, 10 Aug 2020 23:06:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0237.hostedemail.com [216.40.44.237]) by kanga.kvack.org (Postfix) with ESMTP id C99876B0003 for ; Mon, 10 Aug 2020 23:06:11 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 5766A181AEF09 for ; Tue, 11 Aug 2020 03:06:11 +0000 (UTC) X-FDA: 77136798942.12.rain92_3a01fe826fdf Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id 250C018007891 for ; Tue, 11 Aug 2020 03:06:11 +0000 (UTC) X-HE-Tag: rain92_3a01fe826fdf X-Filterd-Recvd-Size: 9050 Received: from mail-lf1-f65.google.com (mail-lf1-f65.google.com [209.85.167.65]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Tue, 11 Aug 2020 03:06:10 +0000 (UTC) Received: by mail-lf1-f65.google.com with SMTP id j22so5860424lfm.2 for ; Mon, 10 Aug 2020 20:06:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=lAgPe7LYxrHND6axLhyzWgoGWJo7W6UM/7KYKZFAicY=; b=Vt/xx+6qknXNsAC24DXedJ+8lw2kZx476DKnzqwxqQHYmJ/b+vSOJ6uzjOOga0I6Pr 9KJ8nvtVaX/cYg14ek3e5Prb2+LHlU5eogRzmyaTtuSoIydVzMVTUSVQWLYvUV6XSFCF gTQaoglRLjuiQYYad1i7Ky+S6jrDOMpFBu0XZspeQyMFPfMtSHSPL3J6j63TuSbtHa37 +L4QJvsOShc6cHCqwJubi8QLmUtIbE3+iA9jtZvfyfWCtZeIA0tUsZ5FrGt8bCCgA0u2 Oq+Id6wDp61B1gJ0CKJntpsULkprxmjKMo5RN2SmtSQFISDi+GEDmKNI6p/WnG6lwjAY MZ1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lAgPe7LYxrHND6axLhyzWgoGWJo7W6UM/7KYKZFAicY=; b=lttGRgDv008JS1GS+neIobHbaEiUnwQMD7xPwngQVI26ZOB8AhNhWMgFVH3Xa4p/in dTWkksnBlj0AAmVB38KaY+Q0BNVmzqqC+z4DVgqUb/VlJfiR0q79wlbyDk8zmzEOTXDk r89tf89Qm83Ud3oSIfHE0h07sA8GJCd4E5KOX9Dofm6JtP74U42IkqwrUuLT3aAg8HUY ITg7mgqf9BmYAPg8aYB99FckjL1um82/dKj1HxX28uv5daRyAW+/nDlWWQTWHzg5ESsb mYR3ln1itdiYMwyTm9+s+307CmtEqqymiFMRshlln5CrYQy+N9Ern7+CHCYe52VSy//W bQ9w== X-Gm-Message-State: AOAM5315jViQxSIWUccpPiItkyxT0nLdDOAEH3DnJiO8BocA6eGT77b7 has/UPWgxzi/ULRRPnqAVXojQUxJ15d4dkT/7L9HLA== X-Google-Smtp-Source: ABdhPJwmgkeAByVMl8IHzTNel+UQSvrkal3Iy/3KmtsWbDWQJpZEQt6WukV/7ph1+nZ7ZkmIlZpzPGBYM79Tc1xCHnA= X-Received: by 2002:ac2:5f48:: with SMTP id 8mr2044815lfz.157.1597115168800; Mon, 10 Aug 2020 20:06:08 -0700 (PDT) MIME-Version: 1.0 References: <20200429214954.44866-1-jannh@google.com> <20200429214954.44866-2-jannh@google.com> <20200505104805.GA17400@lst.de> <20200505121557.GA24052@lst.de> In-Reply-To: <20200505121557.GA24052@lst.de> From: Jann Horn Date: Tue, 11 Aug 2020 05:05:42 +0200 Message-ID: Subject: Re: [PATCH v2 1/5] binfmt_elf_fdpic: Stop using dump_emit() on user pointers on !MMU To: Christoph Hellwig Cc: Andrew Morton , Linus Torvalds , kernel list , Linux-MM , linux-fsdevel , Alexander Viro , "Eric W . Biederman" , Oleg Nesterov , Russell King , Linux ARM , Mark Salter , Aurelien Jacquiot , linux-c6x-dev@linux-c6x.org, Yoshinori Sato , Rich Felker , Linux-sh list Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 250C018007891 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 5, 2020 at 2:15 PM Christoph Hellwig wrote: > On Tue, May 05, 2020 at 01:42:12PM +0200, Jann Horn wrote: > > On Tue, May 5, 2020 at 12:48 PM Christoph Hellwig wrote: > > > On Wed, Apr 29, 2020 at 11:49:50PM +0200, Jann Horn wrote: > > > > dump_emit() is for kernel pointers, and VMAs describe userspace memory. > > > > Let's be tidy here and avoid accessing userspace pointers under KERNEL_DS, > > > > even if it probably doesn't matter much on !MMU systems - especially given > > > > that it looks like we can just use the same get_dump_page() as on MMU if > > > > we move it out of the CONFIG_MMU block. > > > > > > Looks sensible. Did you get a chance to test this with a nommu setup? > > > > Nope. Do you happen to have a recommendation for a convenient > > environment I can use with QEMU, or something like that? I'm guessing > > that just running a standard armel Debian userspace with a !mmu ARM > > kernel wouldn't work so well? > > Nommu generally needs special userspace either using uclibc-ng or musl. > When I did the RISC-V nommu work I used buildroot for my root file > systems. We haven't gotten elffdpic to work on RISC-V yet, so I can't > use that setup for testing, but it should support ARM as well. I've finally gotten around to testing this, and discovered that I actually had to change something in the patch - thanks for asking me to test this. Some notes on running ARM nommu testing: I ended up running QEMU with "-machine versatilepb". To make that work, I applied this patch: A couple of directories up, there are also a README and a kernel config for that. Note that the emulated harddrive of this board doesn't seem to work, because it's connected via PCI, and nommu generally can't use PCI; but you can boot from initramfs, and you can copy files from/to the host with netcat, since the emulated network card does work. (To avoid having to bring up the interface from userspace, you can use "ip=10.0.2.1::10.0.2.2:255.255.255.0" on the kernel cmdline if the corresponding feature is enabled in the kernel config.) The first trouble I ran into with trying to run FDPIC userspace (based on musl) was that Linux has support for running ARM userspace in "26-bit mode", which is some ARM feature from the dark ages, with no support in QEMU; and while normally Linux only tries to enable that thing when the binary explicitly requires it, the FDPIC path isn't wired up to the appropriate personality logic properly, and so you get a spectacular explosion, where eventually the kernel oopses with a message about how it's trying to load an invalid value into CPSR because first the kernel tries to return to 26-bit mode, and then, through some mysterious spooky action at a distance, the kernel (AFAICS) ends up trying to do a syscall return with the stack pointer pointing somewhere in the middle of the kernel stack (and not where the entry register frame is). Anyway, my hacky workaround for that is: diff --git a/arch/arm/include/asm/processor.h b/arch/arm/include/asm/processor.h index b9241051e5cb..d5aa409e366c 100644 --- a/arch/arm/include/asm/processor.h +++ b/arch/arm/include/asm/processor.h @@ -70,7 +70,7 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset, if (current->personality & ADDR_LIMIT_32BIT) \ regs->ARM_cpsr = USR_MODE; \ else \ - regs->ARM_cpsr = USR26_MODE; \ + { WARN(1, "setting USR26_MODE"); regs->ARM_cpsr = USR_MODE; } \ if (elf_hwcap & HWCAP_THUMB && pc & 1) \ regs->ARM_cpsr |= PSR_T_BIT; \ regs->ARM_cpsr |= PSR_ENDSTATE; \ Next up: Early on in the libc startup code, musl aborts execution by intentionally executing an undefined instruction in __set_thread_area(), because it can't figure out any working implementation of atomic cmpxchg. For the MMU case, there is a kuser helper (what x86 would call vsyscall); but for NOMMU ARM, no working implementation exists. So I gave up on musl and went with uclibc-ng (built via buildroot) instead, since uclibc-ng has support for compiling out thread support. Annoyingly, buildroot doesn't support FDPIC (at least not for nommu ARM). So I ended up telling it to build a small FLAT userspace, and used a standard ARM toolchain to build a tiny static PIE ELF binary with no reliance on libc (the FDPIC loader can actually load normal ELF mostly fine as long as it's PIE, at the cost of having to duplicate the text section for every instance) - luckily I didn't need the ELF binary to actually do anything complicated, and so working without any libc was tolerable: arm-linux-gnueabi-gcc-10 -fPIC -c -o test_crash.o test_crash.c arm-linux-gnueabi-ld -pie --no-dynamic-linker -o test_crash test_crash.o Next fun part: gdb-multiarch doesn't seem to be able to open FDPIC core dumps properly - none of the register status is available. I took apart the core dump before and after the patch in a hex editor, though, and it seems to have all the expected stuff in it. I'm guessing that maybe GDB got thrown off by struct elf_prstatus having a different layout if the core dump was generated on nommu? GDB's elf32_arm_nabi_grok_prstatus() seems to only handle the struct size for the non-FDPIC struct.