From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40B35C433DF for ; Tue, 11 Aug 2020 21:23:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E62DC206C3 for ; Tue, 11 Aug 2020 21:23:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AkctowPp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E62DC206C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5CE626B0002; Tue, 11 Aug 2020 17:23:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 556926B0005; Tue, 11 Aug 2020 17:23:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F6A06B0006; Tue, 11 Aug 2020 17:23:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0055.hostedemail.com [216.40.44.55]) by kanga.kvack.org (Postfix) with ESMTP id 276AD6B0002 for ; Tue, 11 Aug 2020 17:23:52 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BD524282B for ; Tue, 11 Aug 2020 21:23:51 +0000 (UTC) X-FDA: 77139565062.24.ducks88_2605cf026fe6 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 901B31A4A0 for ; Tue, 11 Aug 2020 21:23:51 +0000 (UTC) X-HE-Tag: ducks88_2605cf026fe6 X-Filterd-Recvd-Size: 9988 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Tue, 11 Aug 2020 21:23:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1597181030; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=p6z1vHuJOvjpnFLCUTJl63b/aAkybtG4+Mbh9hm5Rsk=; b=AkctowPpKQP61xTh1HFummwRxmQb5o94Og9yEK99PrpcXCyN3a+VTi9Vo39sO+LSlB1Q6L Xjq7I0SnO/pdq1WjhzW7YybpIITz/5de/xdiOK6WHL14RAP03TBVFXQYCUc1dhvdoIaxMI pd9oKKHcG32kbcXkzB0Jl9nrYKXAoGc= Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-348-BT7nrAhXNBmzjGgdx0rV3g-1; Tue, 11 Aug 2020 17:23:48 -0400 X-MC-Unique: BT7nrAhXNBmzjGgdx0rV3g-1 Received: by mail-qt1-f199.google.com with SMTP id m13so84099qth.16 for ; Tue, 11 Aug 2020 14:23:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=p6z1vHuJOvjpnFLCUTJl63b/aAkybtG4+Mbh9hm5Rsk=; b=If0gy9y/6K6BhwSOOjyAQ+0udJFvlZ7mhqr1/mCDTTsxtiRGoDAgAY7Sia6AF5uSZc 5gZbXQLZSYhHnTGDFD6NaP+L359kYlJBe9eMmsJKUHReQMYUdeRDVzs1uMZ+k341exlu UB063hPOUzKHJ/WJieniuFAh06ml3+RlmlkWgV9rH6q/eft14DEfuPnqb/qMjAGF/drn HTtuH24qXDj6n+OzcTTiBN7bIbkz1bX7QKby/77FqS9WMxTcwZG3NDB+rog6gd6VZfCf ur6X8I9/lah/wEbXpRJdD7nX32Q96WN8ByEW3/Qu5YpeyOPXhTD1yJ3hJ6fwR5UzHtJR 7kKA== X-Gm-Message-State: AOAM531nf0n+DlrlCafyF6G1jHU873OkCRlZVLl/GQ1/E/lcIPilrdot VmIqD43e3PfOq8JoetjzNyWZE+52etx5KWchqk9RjJw/0Tn9NchpAXjmHNLh5SE93D4Dec1x2/D Dr0Y3/pVPKB0= X-Received: by 2002:a05:620a:7eb:: with SMTP id k11mr3165241qkk.205.1597181028222; Tue, 11 Aug 2020 14:23:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw4azdTX11/G2/Mcb6L23xIFk+cVBW364WU9IrApUJARrd8X5lfsWug3kFeSqX6zf1EY+TJfQ== X-Received: by 2002:a05:620a:7eb:: with SMTP id k11mr3165208qkk.205.1597181027753; Tue, 11 Aug 2020 14:23:47 -0700 (PDT) Received: from xz-x1 (bras-vprn-toroon474qw-lp130-11-70-53-122-15.dsl.bell.ca. [70.53.122.15]) by smtp.gmail.com with ESMTPSA id 20sm24028qtp.53.2020.08.11.14.23.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Aug 2020 14:23:46 -0700 (PDT) Date: Tue, 11 Aug 2020 17:23:45 -0400 From: Peter Xu To: Jann Horn Cc: Linux-MM , kernel list , Andrew Morton , Marty Mcfadden , "Maya B . Gokhale" , Andrea Arcangeli , Linus Torvalds , Christoph Hellwig , Oleg Nesterov , Kirill Shutemov , Jan Kara Subject: Re: [PATCH v3] mm/gup: Allow real explicit breaking of COW Message-ID: <20200811212345.GD6353@xz-x1> References: <20200811183950.10603-1-peterx@redhat.com> <20200811200256.GC6353@xz-x1> MIME-Version: 1.0 In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Rspamd-Queue-Id: 901B31A4A0 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Aug 11, 2020 at 10:22:00PM +0200, Jann Horn wrote: > On Tue, Aug 11, 2020 at 10:03 PM Peter Xu wrote: > > On Tue, Aug 11, 2020 at 09:07:17PM +0200, Jann Horn wrote: > > > On Tue, Aug 11, 2020 at 8:39 PM Peter Xu wrote: > > > > Starting from commit 17839856fd58 ("gup: document and work around "COW can > > > > break either way" issue", 2020-06-02), explicit copy-on-write behavior is > > > > enforced for private gup pages even if it's a read-only. It is achieved by > > > > always passing FOLL_WRITE to emulate a write. > > > > > > > > That should fix the COW issue that we were facing, however above commit could > > > > also break userfaultfd-wp and applications like umapsort [1,2]. > > > > > > > > One general routine of umap-like program is: userspace library will manage page > > > > allocations, and it will evict the least recently used pages from memory to > > > > external storages (e.g., file systems). Below are the general steps to evict > > > > an in-memory page in the uffd service thread when the page pool is full: > > > > > > > > (1) UFFDIO_WRITEPROTECT with mode=WP on some to-be-evicted page P, so that > > > > further writes to page P will block (keep page P clean) > > > > (2) Copy page P to external storage (e.g. file system) > > > > (3) MADV_DONTNEED to evict page P > > > > > > > > Here step (1) makes sure that the page to dump will always be up-to-date, so > > > > that the page snapshot in the file system is consistent with the one that was > > > > in the memory. However with commit 17839856fd58, step (2) can potentially hang > > > > itself because e.g. if we use write() to a file system fd to dump the page > > > > data, that will be a translated read gup request in the file system driver to > > > > read the page content, then the read gup will be translated to a write gup due > > > > to the new enforced COW behavior. This write gup will further trigger > > > > handle_userfault() and hang the uffd service thread itself. > > > > > > > > I think the problem will go away too if we replace the write() to the file > > > > system into a memory write to a mmaped region in the userspace library, because > > > > normal page faults will not enforce COW, only gup is affected. However we > > > > cannot forbid users to use write() or any form of kernel level read gup. > > > > > > > > One solution is actually already mentioned in commit 17839856fd58, which is to > > > > provide an explicit BREAK_COW scemantics for enforced COW. Then we can still > > > > use FAULT_FLAG_WRITE to identify whether this is a "real write request" or an > > > > "enfornced COW (read) request". > > > > > > > > With the enforced COW, we also need to inherit UFFD_WP bit during COW because > > > > now COW can happen with UFFD_WP enabled (previously, it cannot). > [...] > > > > @@ -1076,7 +1078,7 @@ static long __get_user_pages(struct task_struct *tsk, struct mm_struct *mm, > > > > } > > > > if (is_vm_hugetlb_page(vma)) { > > > > if (should_force_cow_break(vma, foll_flags)) > > > > - foll_flags |= FOLL_WRITE; > > > > + foll_flags |= FOLL_BREAK_COW; > > > > > > How does this interact with the FOLL_WRITE check in follow_page_pte()? > > > If we want the COW to be broken, follow_page_pte() would have to treat > > > FOLL_BREAK_COW similarly to FOLL_WRITE, right? > > > > Good point... I did checked follow_page_mask() that FOLL_COW will still be set > > correctly after applying the patch, though I forgot the FOLL_WRITE part. > > > > Does below look good to you? > > > > diff --git a/mm/gup.c b/mm/gup.c > > index 9d1f44b01165..f4f2a69c6fe7 100644 > > --- a/mm/gup.c > > +++ b/mm/gup.c > > @@ -439,7 +439,8 @@ static struct page *follow_page_pte(struct vm_area_struct *vma, > > } > > if ((flags & FOLL_NUMA) && pte_protnone(pte)) > > goto no_page; > > - if ((flags & FOLL_WRITE) && !can_follow_write_pte(pte, flags)) { > > + if ((flags & (FOLL_WRITE | FOLL_BREAK_COW)) && > > + !can_follow_write_pte(pte, flags)) { > > pte_unmap_unlock(ptep, ptl); > > return NULL; > > } > > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > > index 4f192efef37c..edbd42c9d576 100644 > > --- a/mm/huge_memory.c > > +++ b/mm/huge_memory.c > > @@ -1340,7 +1340,8 @@ struct page *follow_trans_huge_pmd(struct vm_area_struct *vma, > > > > assert_spin_locked(pmd_lockptr(mm, pmd)); > > > > - if (flags & FOLL_WRITE && !can_follow_write_pmd(*pmd, flags)) > > + if (flags & (FOLL_WRITE | FOLL_BREAK_COW) && > > + !can_follow_write_pmd(*pmd, flags)) > > goto out; > > > > /* Avoid dumping huge zero page */ > > Well, I don't see anything immediately wrong with it, at least. Not > that that means much... > > Although in addition to the normal-page path and the transhuge path, > you'll probably also have to make the same change in the hugetlb path. > I guess you may have to grep through all the uses of FOLL_WRITE, as > Linus suggested, to see if there are any other missing spots. Right. Moreover, I feel like the enforced cow is not completely done in hugetlbfs code even without this patch. E.g., it lacks the proper return of VM_FAULT_WRITE in hugetlb_cow(), and also the further convertion logics to translate that into FOLL_COW (which, iiuc, should probably be done in follow_hugetlb_page()). I quickly went over the other FOLL_WRITE references and I didn't see any other suspicious spots besides hugetlb (I only looked at the places that can be called by __get_user_pages() though; that's the only place we set FOLL_BREAK_COW after all). At the meantime we ignored the fast gups for strict breaking of cow always, so those ones seem ok too. Thanks, -- Peter Xu