From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AAF1C32793 for ; Wed, 18 Jan 2023 22:12:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A30BA6B0071; Wed, 18 Jan 2023 17:12:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E0A66B0073; Wed, 18 Jan 2023 17:12:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 880F66B0074; Wed, 18 Jan 2023 17:12:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 78B886B0071 for ; Wed, 18 Jan 2023 17:12:13 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 30299140CBB for ; Wed, 18 Jan 2023 22:12:13 +0000 (UTC) X-FDA: 80369318946.12.8E053CF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf08.hostedemail.com (Postfix) with ESMTP id D130216000A for ; Wed, 18 Jan 2023 22:12:10 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Be/be34c"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674079931; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MspJ78t8fpxKGkTfcthNkvOr56UzNjzkTKTIZByHk64=; b=Dwm+UpsktaloSed30CwR0b2AyoXcWyF4f/f/WBBjWYPM19ZjT5aIXx8cb3JjcSc88ZqyOp jOmq2Kj5y9XAx3Y+bsv4zOAxXdD8oyNyo33jOEe2n54ZLkaNFsNaRJxft99ChX3ouMYpbX xFz8hNgp1y0Nm2bdqiIfMpe8HnBlNXU= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Be/be34c"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674079931; a=rsa-sha256; cv=none; b=Qzc2QwbwdB5Coi+ilyCGgji6b+IM3BRqGalaEluueNdd03MS1gQzEWUWnT8GTDiHlzMaUx TIIQ2nb52QEhJRoSsJzLydEyJXOtroTW2sTJA0vzfH4ElTZxq+iSwo8x7J/HCjvdgi+3YN sgqJhpQCKoqLScXnuyEIuFISw2wk+yE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674079930; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=MspJ78t8fpxKGkTfcthNkvOr56UzNjzkTKTIZByHk64=; b=Be/be34c6o0TtDWuezxnrvwOsU4VOuTutbU3Ek/Co/+PmYpm8yq3Wjdlo+FyfBOm7WhCpu pOBsok4NanqybW4OZP8Yk9ZiwyjHxgPYBnxJXXyxxaaubLyxSFqy3LSs7csc6rkiVT4+6j /wUmWi4c0Ztr3/yakFr90TKJn7Bs7R4= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-377-QJyutptsPFaAHYa7c1_yTg-1; Wed, 18 Jan 2023 17:12:09 -0500 X-MC-Unique: QJyutptsPFaAHYa7c1_yTg-1 Received: by mail-qk1-f198.google.com with SMTP id h13-20020a05620a244d00b006fb713618b8so342609qkn.0 for ; Wed, 18 Jan 2023 14:12:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=MspJ78t8fpxKGkTfcthNkvOr56UzNjzkTKTIZByHk64=; b=IhGs3+gjPwSiwdV6U6fB/YkIuoIBPFGa8Pb/WtDVt2fCh1sNzaOWlRvbXN2I7dFf0e OrAavTMJOK3gRhWcTNqfy0vn324ztUZjGPTUUfRJVoClmzDAck4Knydmq6YK22vPTQL3 JW5dD1ZCEs/h1BXVWW/6zftFLvAVwn7ys7ui1P6ycqPqm4KjXxBIJXL8av+yV+XkjKOg OS7R/ZoFjjzUDph2jBC8Lat6aX5xaMP598O4GdcUiY9LA9pGIbBNVlH93ZzXGogYNCr/ T32PAloyhvfh58m5JRuFbOtDcYXeBK4RrxMbSi12is4VSWuWE2rJZTsTYgKQ95MvcF+P vxVg== X-Gm-Message-State: AFqh2kqTyCKZH+sPh5o8pAvh96dXsqVb7htOUOhDakahtnhmK7y9+EMj +X1LNetaOy9hYZMPmNqeznYtInJHl439a2soEl0G6YdUhYHqtmFU3WqfROdZokNP8bkcaEWDk8h LfaLtq2hSRGs= X-Received: by 2002:ac8:4696:0:b0:3b6:3334:5b95 with SMTP id g22-20020ac84696000000b003b633345b95mr10187277qto.11.1674079928307; Wed, 18 Jan 2023 14:12:08 -0800 (PST) X-Google-Smtp-Source: AMrXdXtqYoS7t97z3YRqQ/yIG2PkokwFud7oGhOXmiSrF/oiJiOgqIp1lh3R8UW5pGPmzBq5qufe2w== X-Received: by 2002:ac8:4696:0:b0:3b6:3334:5b95 with SMTP id g22-20020ac84696000000b003b633345b95mr10187245qto.11.1674079928003; Wed, 18 Jan 2023 14:12:08 -0800 (PST) Received: from x1n (bras-base-aurron9127w-grc-56-70-30-145-63.dsl.bell.ca. [70.30.145.63]) by smtp.gmail.com with ESMTPSA id z24-20020ac875d8000000b003b68c7aeebfsm342908qtq.3.2023.01.18.14.12.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Jan 2023 14:12:07 -0800 (PST) Date: Wed, 18 Jan 2023 17:12:04 -0500 From: Peter Xu To: Muhammad Usama Anjum Cc: David Hildenbrand , Andrew Morton , =?utf-8?B?TWljaGHFgiBNaXJvc8WCYXc=?= , Andrei Vagin , Danylo Mocherniuk , Paul Gofman , Cyrill Gorcunov , Alexander Viro , Shuah Khan , Christian Brauner , Yang Shi , Vlastimil Babka , "Liam R . Howlett" , Yun Zhou , Suren Baghdasaryan , Alex Sierra , Matthew Wilcox , Pasha Tatashin , Mike Rapoport , Nadav Amit , Axel Rasmussen , "Gustavo A . R . Silva" , Dan Williams , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Greg KH , kernel@collabora.com Subject: Re: [PATCH v7 0/4] Implement IOCTL to get and/or the clear info about PTEs Message-ID: References: <20230109064519.3555250-1-usama.anjum@collabora.com> MIME-Version: 1.0 In-Reply-To: <20230109064519.3555250-1-usama.anjum@collabora.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: D130216000A X-Stat-Signature: zbka1b9rtwskyh9om8wcjwdpe84fy1zs X-HE-Tag: 1674079930-257072 X-HE-Meta: U2FsdGVkX185/0AiOMby59O+wLASg2DAlMfEpEqydGPqbcVCoLiKReScdYjohbaHVU7XJbTEy8+a0zGgx7Iyuu5otZtXrWPn00m+SVLmdvnawbl9oMMt6rz1kz6pYDVPv4Hy1YAZpvUL6/tD8erbdqLJiHVa4Uq2R3G06CE/nqsREfbqAYUb8GshCxSdOBpLz+P7sUid5e+kwSJcxy8fjFRVN05mJpwSkZeJw0mBFSRyy7TVV+Q+Jqkfi0BhNXur9qxJotLkCI9cO/zRsxqAoiIVMH/0Zk9sxfrF90Cw0uYa3hMXHEggPrD9YOYLKaUCGLd24QAgAYUbbUrnp0xqPyqz8zi7XOpD2fujjmb2AZdwbwgWiDJmsV8d7Jt8cI9EExQEEn0fs1x+rq7LxQgGqeGmdKxIzRUrrj5l8W3I3c6D2nFqrvRLjQ1TeW2CZBHm5sWfGoEkPGJQ6lgA+BKNHhkuSrzagwZ+AQGT83BexxqzFYrZti5vvai+xVBDMg/GDLaH8ukyBkRtXCOBM1rz88WheW00ehXLmGBoLyrokGJ3SWwuuiy8jzBeqP40gvm189P0NSbu2Vi0NpVoKOo1bSAjGnhS0rMCbiQ5jgtj3Kk89pKa8VJw8GdJ70bxn5ytApphXeHUg74a2echwg7+bDUO5We2ZFuO9b0dWW0BPjJqxQjWteWlENH1N5RtjZmKEGP3LuWL/eLMabpmIMFHyx8eJqYAAGhXCFHws+g7tYDSim5exIT2l4l0MqAistJWvu3a4nKjy6xNNiOPbVtmY7h7nPDnphOj0WmFfFXs9fhyFbBnTY9dBML/hyCIY/84jhxK49jve578uaHKHSNYHEWmkQE/tGIR5ACKnkyeifkd3tq/k1J0K0dl2Bexfy+UlpTFQQbClhD3Jo/v46nXycfOWYkAzVqHR8agOK7urZ/hUxYSxl4nc6OHOoEHE7dBMxYaiykKBBLqGzennlS ahOq4sp1 E8yov/9vfmIyG7AApO8wNyNSwbnx7hX9pugMlsae2dL6xB7hce6CwN55E4YSc2JOXK3nDr6TOmfqMaOjztkEtr2ZTeM9g3X096NH8hjSW1EYKuaA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jan 09, 2023 at 11:45:15AM +0500, Muhammad Usama Anjum wrote: > *Changes in v7:* > - Add uffd wp async > - Update the IOCTL to use uffd under the hood instead of soft-dirty > flags > > Stop using the soft-dirty flags for finding which pages have been > written to. It is too delicate and wrong as it shows more soft-dirty > pages than the actual soft-dirty pages. There is no interest in > correcting it [A][B] as this is how the feature was written years ago. > It shouldn't be updated to changed behaviour. Peter Xu has suggested > using the async version of the UFFD WP [C] as it is based inherently > on the PTEs. > > So in this patch series, I've added a new mode to the UFFD which is > asynchronous version of the write protect. When this variant of the > UFFD WP is used, the page faults are resolved automatically by the > kernel. The pages which have been written-to can be found by reading > pagemap file (!PM_UFFD_WP). This feature can be used successfully to > find which pages have been written to from the time the pages were > write protected. This works just like the soft-dirty flag without > showing any extra pages which aren't soft-dirty in reality. > > [A] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.com > [B] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.com > [C] https://lore.kernel.org/all/Y6Hc2d+7eTKs7AiH@x1n > > *Changes in v6:* > - Updated the interface and made cosmetic changes > > *Cover Letter in v5:* > Hello, Please consider either drop the cover letter below this point or rephrase, otherwise many of them are not true anymore and it can confuse the reviewers. I have a few high level comments/questions here, please bare with me if any of them are already discussed by others in the old versions; I'd be happy to read them when there's a pointer to the relevant answers. Firstly, doc update is more than welcomed to explain the new interface first (before throwing the code..). That can be done in pagemap.rst on pagemap changes, or userfaultfd.rst on userfaultfd. Besides, can you provide more justification on the new pagemap-side interface design? It seems it came from the Windows API GetWriteWatch(), but it's definitely not exactly that. Let me spell some points out.. There're four kinds of masks (required/anyof/excluded/return). Are they all needed? Why this is a good interface design? I saw you used page_region structure to keep the information. I think you wanted to have a densed output, especially if counting in the "return mask" above it starts to make more sense. If with a very limited return mask it means many of the (continuous) page information can be merged into a single page_region struct when the kernel is scanning. However, at the meantime the other three masks (required/anyof/excluded) made me quite confused - it means you wanted to somehow filter the pages and only some of them will get collected. The thing is for a continuous page range if any of the page got skipped due to the masks (e.g. not in "required" or in "excluded") it also means it can never be merged into previous page_region either. That seems to be against the principle of having densed output. I hope you can help clarify what's the major use case here. There's also the new interface to do atomic "fetch + update" on wrprotected pages. Is that just for efficiency or is the accuracy required in some of the applications? Thanks, -- Peter Xu