From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BFF0E8FDB3 for ; Tue, 3 Oct 2023 21:04:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5BF826B00DA; Tue, 3 Oct 2023 17:04:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 548746B00DE; Tue, 3 Oct 2023 17:04:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 372F86B00DD; Tue, 3 Oct 2023 17:04:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1D63E6B00D6 for ; Tue, 3 Oct 2023 17:04:47 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C3E821604B1 for ; Tue, 3 Oct 2023 21:04:46 +0000 (UTC) X-FDA: 81305379372.05.B0D0884 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf26.hostedemail.com (Postfix) with ESMTP id 8ACA5140007 for ; Tue, 3 Oct 2023 21:04:44 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AjtA78v1; spf=pass (imf26.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696367084; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9cUoqfdOw5XZHcgjUV+sI5thBgqq8pWph++w1cw0LK0=; b=WeoUwAz+fQlI+Rq9lRwVXgiAWZ9T3rm6j0mYF2/8LpqY7dQXXAZV+nW98IRvIgxArHJ/3I KN95k5F93GEaou8HWoa4hTzS67cA8sU62orfWCSTZIa4wKHBbkTfnI1xesNp/+n4PoTEEQ /kxj+ae4R2ALV4zh1miMnsz0N3YZgs0= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AjtA78v1; spf=pass (imf26.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696367084; a=rsa-sha256; cv=none; b=zsocFfwGb0g8u/tu7CAGzuACIcBq2kSq7xxb44ravRgBrUSa/Viwc0DvTcoxjAX7+YUjm1 13CZoP9bY5vM66vtZa1jPDHklK5By4nh1YC5KNEiOn3jl4bAJnpocPhptye5KgkS5iJ4y1 u4PGUXEctgwY7l11riTFBWVE1MUkpsI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696367083; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9cUoqfdOw5XZHcgjUV+sI5thBgqq8pWph++w1cw0LK0=; b=AjtA78v1aG+o5BjkMmbkRL8mhmCpEfiFwYPWMtrUe3yPzYr0k1fV/RKhBAH6lX80I+zWKn rE8luWW4Wif+QNgTj9BEsScnB9pExi8LnOu8rFl8ruoL9HepZayNbTQjqPazMGwFFVyxer wEa90XqkLyVJSa4iJJkrm6fX0xINVOo= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-588-oVYQydJROvmDobj9Pg1JHQ-1; Tue, 03 Oct 2023 17:04:42 -0400 X-MC-Unique: oVYQydJROvmDobj9Pg1JHQ-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f5df65f9f4so9770135e9.2 for ; Tue, 03 Oct 2023 14:04:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696367081; x=1696971881; h=content-transfer-encoding:in-reply-to:subject:organization:from :content-language:references:cc:to:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9cUoqfdOw5XZHcgjUV+sI5thBgqq8pWph++w1cw0LK0=; b=JXn6DFuvv5yfApe5AGeusbbUBWh3B8ODuATq5aLoYF1vja4lrK/jG/UT24NjHBZfAE kfA/96iGUMpgJLkQ9Y89r+k/KixqcxGwEJFYL0oAvfrk/R3PjsS6gl27IL67cxuY7Nbp sZmiRC1AA1FkiJ1yJtB92zf0f72RD0gfobUkN+kuSdzLVOUiu9Vm7Zzg1PoUyCvKrRhu JREMlj++cUE4Bux12Z0vFgz/8vEc7XAVbGcjqRVZnBZfPRBUXsbwE35AsUeE7acmlWIk tWQmAG3X2+X/CIAOmC1QnMFKRik3K7gnoERVwzx54j6546+A0UkirwpN1yihZk/m7WdL XyfQ== X-Gm-Message-State: AOJu0YxnN2P05IvZLRGASLJ0msUnauT31yx0hfhxqm0n1lPI53X1qHy+ aeUjqli0i6FeRDj5wNFcdUPGt9Z1rqrEfmhjUptlNPLAS21NWvrmg5i1sTfb/OYzEccJXuaC6+M HhxxhRkOO0J8= X-Received: by 2002:a7b:cb8b:0:b0:3fb:e189:3532 with SMTP id m11-20020a7bcb8b000000b003fbe1893532mr591786wmi.20.1696367081197; Tue, 03 Oct 2023 14:04:41 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGCEM3CW7oIUIpk49usru6wH655xRCmLet2S1OOC6d3hz6/2QR8k4YhBQDCY3vQccozPA5b7Q== X-Received: by 2002:a7b:cb8b:0:b0:3fb:e189:3532 with SMTP id m11-20020a7bcb8b000000b003fbe1893532mr591768wmi.20.1696367080771; Tue, 03 Oct 2023 14:04:40 -0700 (PDT) Received: from ?IPV6:2003:cb:c736:1b00:e1fc:b01b:70f5:1985? (p200300cbc7361b00e1fcb01b70f51985.dip0.t-ipconnect.de. [2003:cb:c736:1b00:e1fc:b01b:70f5:1985]) by smtp.gmail.com with ESMTPSA id m11-20020a7bce0b000000b004064741f855sm23560wmc.47.2023.10.03.14.04.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Oct 2023 14:04:40 -0700 (PDT) Message-ID: <2e2dfe74-8185-5f10-a5ce-5f6252ad0d42@redhat.com> Date: Tue, 3 Oct 2023 23:04:38 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 To: Suren Baghdasaryan , Lokesh Gidra Cc: Peter Xu , Jann Horn , akpm@linux-foundation.org, viro@zeniv.linux.org.uk, brauner@kernel.org, shuah@kernel.org, aarcange@redhat.com, hughd@google.com, mhocko@suse.com, axelrasmussen@google.com, rppt@kernel.org, willy@infradead.org, Liam.Howlett@oracle.com, zhangpeng362@huawei.com, bgeffon@google.com, kaleshsingh@google.com, ngeoffray@google.com, jdduke@google.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, kernel-team@android.com References: <20230923013148.1390521-1-surenb@google.com> <20230923013148.1390521-3-surenb@google.com> <03f95e90-82bd-6ee2-7c0d-d4dc5d3e15ee@redhat.com> <98b21e78-a90d-8b54-3659-e9b890be094f@redhat.com> <85e5390c-660c-ef9e-b415-00ee71bc5cbf@redhat.com> <9434ef94-15e8-889c-0c31-3e875060a2f7@redhat.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH v2 2/3] userfaultfd: UFFDIO_REMAP uABI In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 8ACA5140007 X-Rspam-User: X-Stat-Signature: ok3sstgmazn8zodfyp9aw4akkb3t8pui X-Rspamd-Server: rspam01 X-HE-Tag: 1696367084-273899 X-HE-Meta: U2FsdGVkX1/316XNI8wkfhf4Gq+v0v4ZPDqhVgydBXY5XncOVGopsONDc/xLXZm+HOjufqu5x2+Lt1MFQL2Bbj2r09wFsxAFLyRK5sgG5PlIAnZ9b++ufMiiOQNRQLBQ0Ie0lKppLl/ZPyAtTBXR7KlRIDxEKdm+ypefw5ZLfUp5SB8/uRX4sIfcpZhuUbVuXwsafYlRsRq2XVNge7kOgqLT7LXA1Dx8fhMdkk/9cFQ4ELHOmfsgxWf109eqg+Amy1v/KgY1PeII+vPb0D1nwMn8kE0lZbeVpKeAKi4575MwP0615W6OzMvC1wQ+BzgOgXta85Z9GDI6kxaXzApGXSNvemAPnDAEzWirg+0w8DIiDYwnwURam48u+leh2dqHz8HEipDK/e+cNASKKMi0epwkeaknSoK+xNJPeJ14dju/or7CP9L2xbg+J7v4XSJSq2NqoJMV37eBSiDam2PhCMhhs1V1t7kJjDky2P268iVf83IJZXy2/ahth6LesSS5EqtrWLOtXA5dzWrT/c0F+Yqy+t6/Nc3ATd2HdmKhj6AWujL6R+unH6x6RiP3E15oQKZBPxU2HMiua4jvr2m+f43jcfIuqSt4niQlMZTO+IQFOPATmMc7AxQFsNfbO9xBKeaVpMQHU27vCij234mwMmmiY++jvjdjSrOcRsFU/0sDqV4BzZonZeWyt2sC3KZAgh/LGqkpyvIl/KygzqmSuCKRaGPgiV17LZPf6Je6yrnYQppiPpVe+VbS1ZBTGJtdWbkz7orm53W2VxupjEo42u+sjyrPONXJTxtB9XdIDViCQ+uvBv33q73J+z3V0jsIu9DG69TRgl9CAyIs0sFtISZz3yT/FDUXoHO8fXNMtB/uJzlmwpw7XJR2nbPGQBBuay48Jl7/kqGcPV5Bfs7f/vIMKDnh91UxJBGeq/jV++g1OtADPoNn2Y5v+gJWZ4nq9/EtabFQW4fVwbe3qeC GCbjDLhz cjQdGFFKbJD5U37DxjapU1Tr9WjFyWVvSn8PxHqKkKd2AzO+Y+J9taOcMUnCsTtQ1ol7FJ9/a3OGT0eg+5jWaFcf7oKnOKEcNLK9TdqiLm1Es+LXiA+n9C/INHqpjMCi2SR64hodjCKYzV3FQeJRMKaAr8/Aq9rkvzaFqjj5PObDO2Y23CgQYpHT2Z7X6QIw/JL7tXA89Q01zAvP+11dUPhSIjj+HnEA9wtGsmkuFot3BBTOFiM048ZeDUwYjw118rJJKJdl7PWHFwnZUtdXwxuRaAR14rctoVZWmNIWQjfrdzvALpoFZBh2827SWw7E/zq+qZqsYUmp41ePGlhMPHGa4qoiVVxOa8Ul68NsLcD1p+qU6MDMYNrbE1CUp2WOL5pzJu0XwNWRqY8+iSUyP4j6eUBaIpLOT5TT/VAe1VTwMadjKTTe7lHQg4lmnTgTJwnKF9kmjZL2hm1PA5gzluQBQFWjgXGD2D53M X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 03.10.23 22:04, Suren Baghdasaryan wrote: > On Mon, Oct 2, 2023 at 12:34 PM Lokesh Gidra wrote: >> >> On Mon, Oct 2, 2023 at 6:43 PM David Hildenbrand wrote: >>> >>> On 02.10.23 17:55, Lokesh Gidra wrote: >>>> On Mon, Oct 2, 2023 at 4:46 PM Lokesh Gidra wrote: >>>>> >>>>> On Mon, Oct 2, 2023 at 4:21 PM Peter Xu wrote: >>>>>> >>>>>> On Mon, Oct 02, 2023 at 10:00:03AM +0200, David Hildenbrand wrote: >>>>>>> In case we cannot simply remap the page, the fallback sequence (from the >>>>>>> cover letter) would be triggered. >>>>>>> >>>>>>> 1) UFFDIO_COPY >>>>>>> 2) MADV_DONTNEED >>>>>>> >>>>>>> So we would just handle the operation internally without a fallback. >>>>>> >>>>>> Note that I think there will be a slight difference on whole remap >>>>>> atomicity, on what happens if the page is modified after UFFDIO_COPY but >>>>>> before DONTNEED. >>>>>> >>>>>> UFFDIO_REMAP guarantees full atomicity when moving the page, IOW, threads >>>>>> can be updating the pages when ioctl(UFFDIO_REMAP), data won't get lost >>>>>> during movement, and it will generate a missing event after moved, with >>>>>> latest data showing up on dest. >>>>>> >>>>>> I'm not sure that means such a fallback is a problem, Suren may know >>>>>> better with the use case. >>>>> >>>>> Although there is no problem in using fallback with our use case but >>>>> as a user of userfaultfd, I'd suggest leaving it to the developer. >>>>> Failing with appropriate errno makes more sense. If handled in the >>>>> kernel, then the user may assume at the end of the operation that the >>>>> src vma is completely unmapped. And if not correctness issues, it >>>>> could lead to memory leaks. >>>> >>>> I meant that in addition to the possibility of correctness issues due >>>> to lack of atomicity, it could also lead to memory leaks, as the user >>>> may assume that src vma is empty post-operation. IMHO, it's better to >>>> fail with errno so that the user would fix the code with necessary >>>> changes (like using DONTFORK, if forking). >>> >>> Leaving the atomicity discussion out because I think this can just be >>> handled (e.g., the src_vma would always be empty post-operation): >>> >>> It might not necessarily be a good idea to only expose micro-operations >>> to user space. If the user-space fallback will almost always be >>> "UFFDIO_COPY+MADV_DONTNEED", then clearly the logical operation >>> performed is moving data, ideally with zero-copy. >>> >> IMHO, such a fallback will be useful only if it's possible that only >> some pages in the src vma fail due to this. But even then it would be >> really useful to have a flag maybe like UFFDIO_REMAP_FALLBACK_COPY to >> control if the user wants the fallback or not. OTOH, if this is >> something that can be detected for the entire src vma, then failing >> with errno is more appropriate. >> >> Given that the patch is already quite complicated, I humbly suggest >> leaving the fallback for now as a TODO. I agree about the complexity, and I hope we can reduce that further. Otherwise such things end up being a maintainance nightmare. > > Ok, I think it makes sense to implement the strict remap logic but in > a way that we can easily add copy fallback if that's needed in the I think whatever we do, we should a) never talk about any of the implementation details (mapcount, swapcount, PAE) towards the users b) make it clear from the start that we might change the decision when we fail (to the better or the worse); users should be prepared to implement backup paths. We certainly don't want such behavior to be ABI. I'd suggest documenting something like the following "The operation may fail for various reasons. Usually, remapping of pages that are not exclusive to the given process fail; once KSM might dedduplicate pages or fork() COW-shares pages during fork() with child processes, they are no longer exclusive. Further, the kernel might only perform lightweight checks for detecting whether the pages are exclusive, and return -EWHATSOEVER in case that check fails. To make the operation more likely to succeed, KSM should be disabled, fork() should be avoided or MADV_DONTFORK should be configured for the source VMA before fork()." > future. So, I'll change UFFDIO_REMAP to UFFDIO_MOVE and will return > some unique error, like EBUSY when the page is not PAE. If we need to > add a copy fallback in the future, we will add a > UFFDIO_MOVE_MODE_ALLOW_COPY flag and will implement the copy > mechanism. Does that sound good? To me, if we're talking about moving data, then zero-copy is the optimization and copy+delete would be the (slower) default. If we're talking about remapping, then there is no copy; we're remapping pages. So if we'd ever want to support the copy case, one combination would be UFFDIO_MOVE + UFFDIO_MOVE_ZERO_COPY_ONLY whereby we would fail if the latter is not specified. But just my thoughts. -- Cheers, David / dhildenb