From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2235C433E0 for ; Mon, 21 Dec 2020 05:26:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 466ED22CB8 for ; Mon, 21 Dec 2020 05:26:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 466ED22CB8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 815E46B005D; Mon, 21 Dec 2020 00:26:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C5976B0068; Mon, 21 Dec 2020 00:26:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B40D6B006E; Mon, 21 Dec 2020 00:26:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0104.hostedemail.com [216.40.44.104]) by kanga.kvack.org (Postfix) with ESMTP id 510FF6B005D for ; Mon, 21 Dec 2020 00:26:02 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 07D708249980 for ; Mon, 21 Dec 2020 05:26:02 +0000 (UTC) X-FDA: 77616152964.01.fight40_100017a27454 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id DD21B1004EF4F for ; Mon, 21 Dec 2020 05:26:01 +0000 (UTC) X-HE-Tag: fight40_100017a27454 X-Filterd-Recvd-Size: 11349 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Mon, 21 Dec 2020 05:26:00 +0000 (UTC) Received: by mail-pf1-f174.google.com with SMTP id c79so5831789pfc.2 for ; Sun, 20 Dec 2020 21:26:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=UY74sHg/BA6KOJPIsWgBN2j7tXUCvLn2yKJfFPzwzzA=; b=QK2rTnltcVN5AOQJW8BoJLkPSY5P0Z+ip1xksQoAgkA+zYubkrZLwmkDEgmrPKjG9Q eMLj5uxfDqfiXgbPdsQAuRMA0MrWqzJGXWemJtaCit+PFtFpYUlK9RBUOHF8kuarI+H8 OJEg3poEd2GnwdJX2+pfolmkTEcIUANjQaShtODGCxkKiq7Eybpr3tKtxG8hIY1ZMbk8 R6c/jTfIw1jnCe/CzPH1wGno9YihLgNDslpJ070tcWNC2/EIv4XKIDufFGjg0bhPQpob J2k0FgMg0kihKXT1yzholRHV3QqHvKNfYlciO+08PBY7u9ogoX+aV3nnrZxly+JjQQes L24A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=UY74sHg/BA6KOJPIsWgBN2j7tXUCvLn2yKJfFPzwzzA=; b=iJyWzT/EtHNmAfndghRNTbQd1fE80HuX58iEAFR6mLoa7WoZ2xySsfBuDZa4WKuWFc a5et0vYLEUHKY0adZ9qaVLvGGnV7mArcHQ7L1+ce6TuE/EmkPEWqnpC0KdTUiIBdahOP V2MDNJi1amgVQGPiaN1DQ280ClY39YyGWNOY6nZXzZQI2nfrl5HqaeyZnLFi3mbRUc3V Nt34rL/PgWOohrVYW9PrMKBD9AiKuuYg+UihEfnQtC8AcuhD/rYSqOuewdeiMODxa/Nj PNyfSAkJUym+Mf7rt08BGMNzPQrsGdcMjZCwNNUArYwEK0ly33Z4LD79/j0m3/OpWPz8 jQ6w== X-Gm-Message-State: AOAM532uS1mHWTIY9vC5UzsqatEQfFpedUtMC0vlQEm5S1PMmVRqwn6W V8HaelnEksj9zTTPq/2Qg8M= X-Google-Smtp-Source: ABdhPJwovMk9JGoOBCcucNZN6nBvz6rLKaSPX1BElViDYKRoJjkij41PQF+4XXSXwuGL5EgL2OHUxw== X-Received: by 2002:a65:5588:: with SMTP id j8mr13697777pgs.245.1608528359819; Sun, 20 Dec 2020 21:25:59 -0800 (PST) Received: from [10.0.1.14] (c-24-4-128-201.hsd1.ca.comcast.net. [24.4.128.201]) by smtp.gmail.com with ESMTPSA id d10sm14924404pfn.218.2020.12.20.21.25.57 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 20 Dec 2020 21:25:58 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Re: [PATCH] mm/userfaultfd: fix memory corruption due to writeprotect From: Nadav Amit In-Reply-To: Date: Sun, 20 Dec 2020 21:25:56 -0800 Cc: Andrea Arcangeli , linux-mm , Peter Xu , lkml , Pavel Emelyanov , Mike Kravetz , Mike Rapoport , stable@vger.kernel.org, minchan@kernel.org, Andy Lutomirski , Will Deacon , Peter Zijlstra Content-Transfer-Encoding: quoted-printable Message-Id: <7986D881-3EBD-4197-A1A0-3B06BB2300B1@gmail.com> References: <20201219043006.2206347-1-namit@vmware.com> <729A8C1E-FC5B-4F46-AE01-85E00C66DFFF@gmail.com> To: Yu Zhao X-Mailer: Apple Mail (2.3608.120.23.2.4) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > On Dec 20, 2020, at 9:12 PM, Yu Zhao wrote: >=20 > On Sun, Dec 20, 2020 at 08:36:15PM -0800, Nadav Amit wrote: >>> On Dec 19, 2020, at 6:20 PM, Andrea Arcangeli = wrote: >>>=20 >>> On Sat, Dec 19, 2020 at 02:06:02PM -0800, Nadav Amit wrote: >>>>> On Dec 19, 2020, at 1:34 PM, Nadav Amit = wrote: >>>>>=20 >>>>> [ cc=E2=80=99ing some more people who have experience with similar = problems ] >>>>>=20 >>>>>> On Dec 19, 2020, at 11:15 AM, Andrea Arcangeli = wrote: >>>>>>=20 >>>>>> Hello, >>>>>>=20 >>>>>> On Fri, Dec 18, 2020 at 08:30:06PM -0800, Nadav Amit wrote: >>>>>>> Analyzing this problem indicates that there is a real bug since >>>>>>> mmap_lock is only taken for read in mwriteprotect_range(). This = might >>>>>>=20 >>>>>> Never having to take the mmap_sem for writing, and in turn never >>>>>> blocking, in order to modify the pagetables is quite an important >>>>>> feature in uffd that justifies uffd instead of mprotect. It's not = the >>>>>> most important reason to use uffd, but it'd be nice if that = guarantee >>>>>> would remain also for the UFFDIO_WRITEPROTECT API, not only for = the >>>>>> other pgtable manipulations. >>>>>>=20 >>>>>>> Consider the following scenario with 3 CPUs (cpu2 is not shown): >>>>>>>=20 >>>>>>> cpu0 cpu1 >>>>>>> ---- ---- >>>>>>> userfaultfd_writeprotect() >>>>>>> [ write-protecting ] >>>>>>> mwriteprotect_range() >>>>>>> mmap_read_lock() >>>>>>> change_protection() >>>>>>> change_protection_range() >>>>>>> ... >>>>>>> change_pte_range() >>>>>>> [ defer TLB flushes] >>>>>>> userfaultfd_writeprotect() >>>>>>> mmap_read_lock() >>>>>>> change_protection() >>>>>>> [ write-unprotect ] >>>>>>> ... >>>>>>> [ unprotect PTE logically ] >>>=20 >>> Is the uffd selftest failing with upstream or after your kernel >>> modification that removes the tlb flush from unprotect? >>=20 >> Please see my reply to Yu. I was wrong in this analysis, and I sent a >> correction to my analysis. The problem actually happens when >> userfaultfd_writeprotect() unprotects the memory. >>=20 >>> } else if (uffd_wp_resolve) { >>> /* >>> * Leave the write bit to be handled >>> * by PF interrupt handler, then >>> * things like COW could be properly >>> * handled. >>> */ >>> ptent =3D pte_clear_uffd_wp(ptent); >>> } >>>=20 >>> Upstraem this will still do pages++, there's a tlb flush before >>> change_protection can return here, so I'm confused. >>=20 >> You are correct. The problem I encountered with = userfaultfd_writeprotect() >> is during unprotecting path. >>=20 >> Having said that, I think that there are additional scenarios that = are >> problematic. Consider for instance madvise_dontneed_free() that is = racing >> with userfaultfd_writeprotect(). If madvise_dontneed_free() completed >> removing the PTEs, but still did not flush, change_pte_range() will = see >> non-present PTEs, say a flush is not needed, and then >> change_protection_range() will not do a flush, and return while >> the memory is still not protected. >>=20 >>> I don't share your concern. What matters is the PT lock, so it >>> wouldn't be one per pte, but a least an order 9 higher, but let's >>> assume one flush per pte. >>>=20 >>> It's either huge mapping and then it's likely running without other >>> tlb flushing in background (postcopy snapshotting), or it's a = granular >>> protect with distributed shared memory in which case the number of >>> changd ptes or huge_pmds tends to be always 1 anyway. So it doesn't >>> matter if it's deferred. >>>=20 >>> I agree it may require a larger tlb flush review not just mprotect >>> though, but it didn't sound particularly complex. Note the >>> UFFDIO_WRITEPROTECT is still relatively recent so backports won't >>> risk to reject so heavy as to require a band-aid. >>>=20 >>> My second thought is, I don't see exactly the bug and it's not clear >>> if it's upstream reproducing this, but assuming this happens on >>> upstream, even ignoring everything else happening in the tlb flush >>> code, this sounds like purely introduced by = userfaultfd_writeprotect() >>> vs userfaultfd_writeprotect() (since it's the only place changing >>> protection with mmap_sem for reading and note we already unmap and >>> flush tlb with mmap_sem for reading in MADV_DONTNEED/MADV_FREE = clears >>> the dirty bit etc..). Flushing tlbs with mmap_sem for reading is >>> nothing new, the only new thing is the flush after wrprotect. >>>=20 >>> So instead of altering any tlb flush code, would it be possible to >>> just stick to mmap_lock for reading and then serialize >>> userfaultfd_writeprotect() against itself with an additional >>> mm->mmap_wprotect_lock mutex? That'd be a very local change to >>> userfaultfd too. >>>=20 >>> Can you look if the rule mmap_sem for reading plus a new >>> mm->mmap_wprotect_lock mutex or the mmap_sem for writing, whenever >>> wrprotecting ptes, is enough to comply with the current tlb flushing >>> code, so not to require any change non local to uffd (modulo the >>> additional mutex). >>=20 >> So I did not fully understand your solution, but I took your point = and >> looked again on similar cases. To be fair, despite my experience with = these >> deferred TLB flushes as well as Peter Zijlstra=E2=80=99s great = documentation, I keep >> getting confused (e.g., can=E2=80=99t we somehow combine = tlb_flush_batched and >> tlb_flush_pending ?) >>=20 >> As I said before, my initial scenario was wrong, and the problem is = not >> userfaultfd_writeprotect() racing against itself. This one seems = actually >> benign to me. >>=20 >> Nevertheless, I do think there is a problem in = change_protection_range(). >> Specifically, see the aforementioned scenario of a race between >> madvise_dontneed_free() and userfaultfd_writeprotect(). >>=20 >> So an immediate solution for such a case can be resolve without = holding >> mmap_lock for write, by just adding a test for mm_tlb_flush_nested() = in >> change_protection_range(): >>=20 >> /* >> * Only flush the TLB if we actually modified any entries >> * or if there are pending TLB flushes. >> */ >> if (pages || mm_tlb_flush_nested(mm)) >> flush_tlb_range(vma, start, end); >>=20 >> To be fair, I am not confident I did not miss other problematic = cases. >>=20 >> But for now, this change, with the preserve_write change should = address the >> immediate issues. Let me know if you agree. >>=20 >> Let me know whether you agree. >=20 > The problem starts in UFD, and is related to tlb flush. But its focal > point is in do_wp_page(). I'd suggest you look at function and see > what it does before and after the commits I listed, with the following > conditions >=20 > PageAnon(), !PageKsm(), !PageSwapCache(), !pte_write(), > page_mapcount() =3D 1, page_count() > 1 or PageLocked() >=20 > when it runs against the two UFD examples you listed. Thanks for your quick response. I wanted to write a lengthy response, = but I do want to sleep on it. I presume page_count() > 1, since I have = multiple concurrent page-faults on the same address in my test, but I will check. Anyhow, before I give a further response, I was just wondering - since = you recently dealt with soft-dirty issue as I remember - isn't this = problematic COW for non-COW page scenario, in which the copy races with writes to a = page which is protected in the PTE but not in all TLB, also problematic for soft-dirty clearing?