From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3B0CC433E2 for ; Thu, 17 Sep 2020 19:38:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1792921D41 for ; Thu, 17 Sep 2020 19:38:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="oewQ2hrE" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1792921D41 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2397C6B0003; Thu, 17 Sep 2020 15:38:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C1086B0037; Thu, 17 Sep 2020 15:38:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 061C08E0001; Thu, 17 Sep 2020 15:38:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0143.hostedemail.com [216.40.44.143]) by kanga.kvack.org (Postfix) with ESMTP id DE9826B0003 for ; Thu, 17 Sep 2020 15:38:27 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 98FC8181AEF21 for ; Thu, 17 Sep 2020 19:38:27 +0000 (UTC) X-FDA: 77273565054.07.pull05_380072b27125 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id 7D69E1803F9AA for ; Thu, 17 Sep 2020 19:38:27 +0000 (UTC) X-HE-Tag: pull05_380072b27125 X-Filterd-Recvd-Size: 8655 Received: from mail-qv1-f68.google.com (mail-qv1-f68.google.com [209.85.219.68]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Thu, 17 Sep 2020 19:38:26 +0000 (UTC) Received: by mail-qv1-f68.google.com with SMTP id j10so1594503qvk.11 for ; Thu, 17 Sep 2020 12:38:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=CNJ7oZi6THpUP7nc6TfqkOyCuXcwBs1RipCXw7bsPco=; b=oewQ2hrEulpRw7YaRRH5ghfbNO15TIBrln0DMCA24xlcwOVILWgR7Hl6wrIO0+J6qN m69YQpqnJr2vssE1sbElucVqSkbQFz7I6/JTIRZCFGKJ0eC3gFnI/K7hANSzKUNPT1OK NEGRQibR+PiRlMNEs+ebj/Fn5J5EOg+z87uze7feR0fVMbDWoQ1YrCgrTou9j+QeIZ29 dNcvyiJDxts1AKxqRHmM5MlKKjc/81USbABQPZ6mrnYnn/o9kAIG4F0rTCU6uJn6aIos 4AO13xx2PwZajKVLyHhAJh0uhgfx/9mSotZJLJU2nFHiSC7xQmmH3sGhz9dAwp3MgZOG ZJSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=CNJ7oZi6THpUP7nc6TfqkOyCuXcwBs1RipCXw7bsPco=; b=VHVEn6bfmq7cEiGtFI49e+qbxtitnhjncXHTOYTICZVUBp9zcNINKc9y1nWV2KCOa9 vRaZDaczKLeGB0fwscqHYcTCqdXcqzCkBrWSk1Bnf+8bCyiva5Fay0yAhr2HDUxFZA8I gZPa52aQeuSv+MacjEjZ9/fJaE+/ch22hXWl1o5xXuCfGNk3Dfh5H3297W2IdwGiupsJ s45gnXjMpeyDwlHb7V2jI3nR0KMGH+nmw6+bods4IUX6XxdZgKApuXuW5y9jXsD1o8ZW kKwUdhJlM0A/Flk1/RisfXtKl6DeUfQuY4SXGTmy5JnclaPIM0nYxpyPzE1r4fgY7ZmB MSHA== X-Gm-Message-State: AOAM5333Poet0kQhSONgwc0pD3JaJSCWL1VGfP6PrZXuTaYh5qEWkhmW lps33+zRjrCI5HBB9oVtBv6aMw== X-Google-Smtp-Source: ABdhPJw9U+NS9ntsDfqB6L/1hA5PlAxaX3iFnzngGbMRUd7jg/svBVEQ+IZQB6SwuKB/NSVp1a+t5g== X-Received: by 2002:a0c:80c3:: with SMTP id 61mr30485931qvb.13.1600371506151; Thu, 17 Sep 2020 12:38:26 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-48-30.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.48.30]) by smtp.gmail.com with ESMTPSA id e7sm532752qtk.17.2020.09.17.12.38.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Sep 2020 12:38:25 -0700 (PDT) Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1kIzjc-000kpq-Ig; Thu, 17 Sep 2020 16:38:24 -0300 Date: Thu, 17 Sep 2020 16:38:24 -0300 From: Jason Gunthorpe To: Linus Torvalds Cc: Peter Xu , John Hubbard , Leon Romanovsky , Linux-MM , Linux Kernel Mailing List , "Maya B . Gokhale" , Yang Shi , Marty Mcfadden , Kirill Shutemov , Oleg Nesterov , Jann Horn , Jan Kara , Kirill Tkhai , Andrea Arcangeli , Christoph Hellwig , Andrew Morton Subject: Re: [PATCH 1/4] mm: Trial do_wp_page() simplification Message-ID: <20200917193824.GL8409@ziepe.ca> References: <20200915182933.GM1221970@ziepe.ca> <20200915191346.GD2949@xz-x1> <20200915193838.GN1221970@ziepe.ca> <20200915213330.GE2949@xz-x1> <20200915232238.GO1221970@ziepe.ca> <20200916174804.GC8409@ziepe.ca> <20200916184619.GB40154@xz-x1> <20200917112538.GD8409@ziepe.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Sep 17, 2020 at 11:11:06AM -0700, Linus Torvalds wrote: > (a) if the pinner is going to change the page, it will have to get > the pin with FOLL_WRITE in addition to FOLL_PIN > > (b) that means it will do the COW and mark the page writable and dirty Yep > (c) the whole _point_ of the FOLL_PIN is that subsequent operations > shouldn't make it non-writable any more (ie it can't be unmapped, and > we should synchronize on fork etc) It is the ideal, but FOLL_PIN has been troubled for a long time: https://lwn.net/Articles/753027/ ie writeprotect is known to happen due to writeback. I had understood that fork() will also cause write protect. Eg copy_process() copy_mm() dup_mm() dup_mmap() copy_page_range() [..] copy_one_pte() Gets to: if (is_cow_mapping(vm_flags) && pte_write(pte)) { ptep_set_wrprotect(src_mm, addr, src_pte); pte = pte_wrprotect(pte); } Which blindly write protects a FOLL_PIN page. Here src_pte will be under a pin. I *think* the failing test is basically: 1) pin_user_pages(mem, FOLL_FORCE | FOLL_WRITE) 2) pid = fork() 3) child: does a bit, then exec 4) parent: waitpid(pid) 5) *mem = 0 Here #5 is the WP triggered COW. We know a WP triggered COW is happening from the bisect and success result with MADV_DONTFORK. #2 triggers the dup_mmap() and the ptep_set_wrprotect() (From inspection, at least) This "Trial do_wp_page() simplification" patch means that when #5 goes to do COW it gets a copy instead of a re-use because the reuse_swap_page() was aborting the copy before. So, to your point, yes ideally FOLL_PIN would never write-protect pages! Looking for awhile, this now looks reasonable and doable. page_maybe_dma_pinned() was created for exactly this kind of case. I've attached a dumb sketch for the pte level (surely wrong! I have never looked at this part of the mm before!) at the end of this message. Peter, do you know this better? Does this inspire you to make a patch? :) Would really love to see this. We have such a huge expensive mess with MADV_DONTFORK, this would eliminate all that overhead. Thanks, Jason diff --git a/mm/memory.c b/mm/memory.c index 469af373ae76e1..6bc19a43da1391 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -689,6 +689,21 @@ struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, } #endif +static int duplicate_page(pte_t *newpte, struct vm_area_struct *vma, + unsigned long address, struct page *page) +{ + struct page *new_page; + + new_page = alloc_page_vma(GFP_HIGHUSER_MOVABLE, vma, address); + if (!new_page) + return -ENOMEM; + copy_user_highpage(new_page, page, address, vma); + + /* FIXME: surely more than this */ + *newpte = mk_pte(new_page, vma->vm_page_prot); + return 0; +} + /* * copy one vm_area from one task to the other. Assumes the page tables * already present in the new task to be cleared in the whole range @@ -703,6 +718,7 @@ copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, unsigned long vm_flags = vma->vm_flags; pte_t pte = *src_pte; struct page *page; + bool do_src_wp; /* pte contains position in swap or file, so copy. */ if (unlikely(!pte_present(pte))) { @@ -775,15 +791,6 @@ copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, goto out_set_pte; } - /* - * If it's a COW mapping, write protect it both - * in the parent and the child - */ - if (is_cow_mapping(vm_flags) && pte_write(pte)) { - ptep_set_wrprotect(src_mm, addr, src_pte); - pte = pte_wrprotect(pte); - } - /* * If it's a shared mapping, mark it clean in * the child @@ -800,11 +807,34 @@ copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, if (!(vm_flags & VM_UFFD_WP)) pte = pte_clear_uffd_wp(pte); + do_src_wp = is_cow_mapping(vm_flags) && pte_write(pte); page = vm_normal_page(vma, addr, pte); if (page) { get_page(page); page_dup_rmap(page, false); rss[mm_counter(page)]++; + + /* + * If a page is DMA pinned we never want to write protect it, + * copy it now. + */ + if (do_src_wp && page_maybe_dma_pinned(page)) { + do_src_wp = false; + ret = duplicate_page(&pte, vma, addr, page); + if (ret) + /* FIXME: need to restructure a bit to handle this */ + return ret; + } + } + + /* + * If it's a COW mapping, write protect it both + * in the parent and the child + * FIXME check carefully this is new order is OK + */ + if (do_src_wp) { + ptep_set_wrprotect(src_mm, addr, src_pte); + pte = pte_wrprotect(pte); } out_set_pte: