From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FD33C4727F for ; Wed, 23 Sep 2020 10:21:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D2C0C238D7 for ; Wed, 23 Sep 2020 10:21:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D2C0C238D7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nvidia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0F55A8E0001; Wed, 23 Sep 2020 06:21:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0CC316B005D; Wed, 23 Sep 2020 06:21:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 009B48E0001; Wed, 23 Sep 2020 06:21:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id DE6BB6B005C for ; Wed, 23 Sep 2020 06:21:24 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 96FD91EE6 for ; Wed, 23 Sep 2020 10:21:24 +0000 (UTC) X-FDA: 77293934088.05.grain20_0b0546427155 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 759BA1801875E for ; Wed, 23 Sep 2020 10:21:24 +0000 (UTC) X-HE-Tag: grain20_0b0546427155 X-Filterd-Recvd-Size: 4003 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Wed, 23 Sep 2020 10:21:23 +0000 (UTC) Received: from localhost (unknown [213.57.247.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 759EE20C56; Wed, 23 Sep 2020 10:21:22 +0000 (UTC) Date: Wed, 23 Sep 2020 13:21:19 +0300 From: Leon Romanovsky To: Peter Xu Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Jason Gunthorpe , Andrew Morton , Jan Kara , Michal Hocko , Kirill Tkhai , Kirill Shutemov , Hugh Dickins , Christoph Hellwig , Andrea Arcangeli , John Hubbard , Oleg Nesterov , Linus Torvalds , Jann Horn Subject: Re: [PATCH 0/5] mm: Break COW for pinned pages during fork() Message-ID: <20200923102119.GK1223944@unreal> References: <20200921211744.24758-1-peterx@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200921211744.24758-1-peterx@redhat.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Sep 21, 2020 at 05:17:39PM -0400, Peter Xu wrote: > Finally I start to post formal patches because it's growing. And also since > we've discussed quite some issues already, so I feel like it's clearer on what > we need to do, and how. > > This series is majorly inspired by the previous discussion on the list [1], > starting from the report from Jason on the rdma test failure. Linus proposed > the solution, which seems to be a very nice approach to avoid the breakage of > userspace apps that didn't use MADV_DONTFORK properly before. More information > can be found in that thread too. > > I believe the initial plan was to consider merging something like this for > rc7/rc8. However now I'm not sure due to the fact that the code change in > copy_pte_range() is probably more than expected, so it can be with some risk. > I'll leave this question to the reviewers... > > I tested it myself with fork() after vfio pinning a bunch of device pages, and > I verified that the new copy pte logic worked as expected at least in the most > general path. However I didn't test thp case yet because afaict vfio does not > support thp backed dma pages. Luckily, the pmd/pud thp patch is much more > straightforward than the pte one, so hopefully it can be directly verified by > some code review plus some more heavy-weight rdma tests. > > Patch 1: Introduce mm.has_pinned (as single patch as suggested by Jason) > Patch 2-3: Some slight rework on copy_page_range() path as preparation > Patch 4: Early cow solution for pte copy for pinned pages > Patch 5: Same as above, but for thp (pmd/pud). > > Hugetlbfs fix is still missing, but as planned, that's not urgent so we can > work upon. Comments greatly welcomed. Hi Peter, I'm ware that this series is under ongoing review and probably not final, but we tested anyway and it solves our RDMA failures. Thanks > > Thanks. > > Peter Xu (5): > mm: Introduce mm_struct.has_pinned > mm/fork: Pass new vma pointer into copy_page_range() > mm: Rework return value for copy_one_pte() > mm: Do early cow for pinned pages during fork() for ptes > mm/thp: Split huge pmds/puds if they're pinned when fork() > > include/linux/mm.h | 2 +- > include/linux/mm_types.h | 10 ++ > kernel/fork.c | 3 +- > mm/gup.c | 6 ++ > mm/huge_memory.c | 26 +++++ > mm/memory.c | 226 +++++++++++++++++++++++++++++++++++---- > 6 files changed, 248 insertions(+), 25 deletions(-) > > -- > 2.26.2 > >