From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15BE6C001E0 for ; Thu, 10 Aug 2023 19:26:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A0B686B0072; Thu, 10 Aug 2023 15:26:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C4BC6B0078; Thu, 10 Aug 2023 15:26:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85B966B007B; Thu, 10 Aug 2023 15:26:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7599D6B0072 for ; Thu, 10 Aug 2023 15:26:25 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 4680640277 for ; Thu, 10 Aug 2023 19:26:25 +0000 (UTC) X-FDA: 81109176330.10.DCDE660 Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) by imf03.hostedemail.com (Postfix) with ESMTP id 5EB3820003 for ; Thu, 10 Aug 2023 19:26:23 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=NtsWbmNF; spf=pass (imf03.hostedemail.com: domain of emmir@google.com designates 209.85.208.44 as permitted sender) smtp.mailfrom=emmir@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691695583; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7M2iPr1UJVHGFT8TQdnwZTNArsr1wGryk6ufkfsAkzI=; b=RU7RFuON8Jtv8u5Vz6O4fXZQ98UHOi7BHgR1IqNfMXMIC160tKzKuE50cmkM8zyVbnwTaC UqS/c63EwJAq9zj522jYEAHBYDvLMTU2EaGJyGqrOOZvQwflUwkZPbQApEA+V59cYYV3xr QFJM6KZzfyzw3aU5BI5OFl8kwe7vzP8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691695583; a=rsa-sha256; cv=none; b=b0C8zrPfO+liYOoUL0bbG5eyvTJL2X9bOo8aRwsvesOSwmR2dW67286R53hN1HsI5BmKqJ qNGpxgls/NH/qplsN/QStE8Q6aBp1G2voLgUWC0DFfPPEnFmJPFDsjhHiOa+cPreAepEot TH2s3RjN2StByS9ko7nK2juAZntCqCM= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=NtsWbmNF; spf=pass (imf03.hostedemail.com: domain of emmir@google.com designates 209.85.208.44 as permitted sender) smtp.mailfrom=emmir@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-ed1-f44.google.com with SMTP id 4fb4d7f45d1cf-51e24210395so2699a12.0 for ; Thu, 10 Aug 2023 12:26:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691695582; x=1692300382; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=7M2iPr1UJVHGFT8TQdnwZTNArsr1wGryk6ufkfsAkzI=; b=NtsWbmNFqBepaBH2DlfxhtQc88rTP6poWfZiNHU7zjNl3M4k9zexE6653xhn3BNt9X xAejuvrX5lIegZmwDpkL/RwZt7fUYpeMe8kN52h73ltDzKfFoNJbmWmOsFbdYEppiGlS EbghugF5nBRMsTErYdEpJj8VJQpKGT30tyUEJGBKffA62ubxCP1cg3cbJPw+DSIurZGv jZgJE+c/Ae8h/kuBGwS02qVludTmupWnD+cKqfZeX48LNl1ZUW7U8QB4+0P9pU0PGOOE DnnLio9tP0F2CjHjvJgcIt2nE+V0KVvHlQYXf7b6ts3sW3o7w0RFqC3aS1CHyplP1Yhm yCPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691695582; x=1692300382; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7M2iPr1UJVHGFT8TQdnwZTNArsr1wGryk6ufkfsAkzI=; b=ahkRyPFdPzR+WkMDwVK+dgIQW7yi3HRhIg0q32pWG6xPBMYmNjIbrevlLFP1TaKROV WHiKURmrEZNM4EuuWHEUuoxGGl2Vzn16W3jmOZVUMHd5EyNR0jS80mbDlAkgjFo6BCQ7 TXU5swVotHOcYZEhU2Y1zRAQu9DXM9VAY7W5H0X4vDfVdmBtchTLTxA9lxKSr3ZACtAr uYtRxmzQMT7jQ5pmazxrA2uFvVIpkU4QrChw8Iw5C6u3907arD4iSSE+lrOejH0qyFvz H6phG5ktFgbdSIOHfwCZ2DIMpb1EHJG+Nqb2uehm+i6TWTpZV80fJ+noQrr2aWCzFThU iTdw== X-Gm-Message-State: AOJu0YxxEU89dm4BIeqOI5AapYDqG4LfCSfsvjDXx4CpkCv3qqfFks/1 1WFNeWBDSoEk6MrACwxA0rHCUgoS+Oq14rvT1tCwuA== X-Google-Smtp-Source: AGHT+IFSthzdZHQWjZGRfN1VqKcrQ3fEjsmcl6BCv8ko21zKpNpfAA0n4Xn5zWlCsMTAGvT81+a4JzhjFClXLpQPxzA= X-Received: by 2002:a50:cdd8:0:b0:523:bdc9:48a9 with SMTP id h24-20020a50cdd8000000b00523bdc948a9mr17482edj.0.1691695581678; Thu, 10 Aug 2023 12:26:21 -0700 (PDT) MIME-Version: 1.0 References: <20230809061603.1969154-1-usama.anjum@collabora.com> <20230809061603.1969154-6-usama.anjum@collabora.com> In-Reply-To: <20230809061603.1969154-6-usama.anjum@collabora.com> From: =?UTF-8?B?TWljaGHFgiBNaXJvc8WCYXc=?= Date: Thu, 10 Aug 2023 21:26:10 +0200 Message-ID: Subject: Re: [PATCH v28 5/6] mm/pagemap: add documentation of PAGEMAP_SCAN IOCTL To: Muhammad Usama Anjum Cc: Peter Xu , David Hildenbrand , Andrew Morton , Andrei Vagin , Danylo Mocherniuk , Paul Gofman , Cyrill Gorcunov , Mike Rapoport , Nadav Amit , Alexander Viro , Shuah Khan , Christian Brauner , Yang Shi , Vlastimil Babka , "Liam R . Howlett" , Yun Zhou , Suren Baghdasaryan , Alex Sierra , Matthew Wilcox , Pasha Tatashin , Axel Rasmussen , "Gustavo A . R . Silva" , Dan Williams , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Greg KH , kernel@collabora.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: q9z73rtga5nng7qc1oxa3dpc7qgwfpnq X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 5EB3820003 X-Rspam-User: X-HE-Tag: 1691695583-629248 X-HE-Meta: U2FsdGVkX18jAd2TpQn86yM4rX7ebrJaXZs5saNlOERNgwz0+jKKqzesX1XUxHyakKWbqec4zzgC5QUtYT/bOgb91ClIAf5GRlko47iDHlNfDGd3bGJ/DLcb3TV07PoI5qVUR6FRbneD9o5Db7Yil7HTxYTGFhmzDPyYEDOkzV6SL2xhU+xDSVrtJFDcFFi/mOmEUl1WNjnywVMXGoruGsW5v50z5RqbPzRJHGWhG9iVnEGmT+7wwuI+2ApXdZfCu0nAFzXxzVwRPDpBGDecFh4zYP/w1t2U2g46ZwmiKAGFzzvzqGiuaHOe04ZI2smvKsGxpyPX1nrxq3KeHDT6VeF8OQAPxZf1C7yzQisSfBwd59yxgHmMVedwWm392HxwXRSXduNmtydntd6KqanJCJZNBuWjtoqBRAcb1POamRBn+GhLxGQSBpldS57MyFesCs3cDO/EyPv2Jo6MFZi5ZZgMTfTYjAo6M+pRUmNDnuZBFtyNoMGKfaaAXNReTQv8XY4neZCFySdmU1B95LK0QASEMhsQBThWv+uJhtmJ7bh0Q5bLPqjT2G7TfAV0JzL1qNNbSHPW5U3uNA2s9Kc/dtfoiVk5b4YX13no7f2WDkCgl5Tc/D+Ty4amwoogPt6HA1wRRsLlbzIrbXzGycHWvSsk1Zm3I2aHNa4/1SGnD/C05y+nExYbks6rHYptxLhFavEVUKyUMlSHg0QuXn1qgLD/fJtq4Vjvt2m3bRSLB6HcGNxNIBaay0rVPfVMdYKJGwH5+a+rG1f23jU8VGZZ4Ck7Te0cJ/N8tcvUGraqL5im0r/qU6GLfNKQ1aseDJvG25Z5LDpdHpWginJcr/WnPX/bubAV6qmjD5WjQgXhF+sU7tnQ+kJRTQ7dG0d1bCoIDIt0ajrLA4mQxoWKh8rIeZ0p/Y3FJlyXAEY39/6PwcxWOSOUFzx+Mk90/rmCmamgO65/JEqOgakuLOHArM5 ZYxWLIHv tdylyaNRlamMFUKWEgefX0w3dG9BuBwZYYy4ytLp4glKwOuydsHoWvROlMEKmIkn4PPYE5sU2e8yVp5RVbmlj4vuW4o5R01NNT3k06HL39bl4LLAQd0r7Jf9mOa+Gb3p9l9TzkpHBGgYX8nd6yCTiiv3LZsydy5gHxWwdFWXperaVfNuB0VQ0LRnfkJGY1wyDSKasuJb8k0dYC9oAKf/v4KlcFrWqYawhotw4UGu3ups42a27Hqg8gADG+m64pb41Tdlm1OZtoyVyvrvjIVZKg1TlWdwA3L+dCJZmbJgqPOOQMoRhimDp7xrjQDN5lTN/LV8eDm6qSsSFUIIy2FI+mYAT4RNoG0YrlKFF823g/fF8pX9WWlhSasUUMTWBoAOgMSKax9C5ViUXhYzMjCeipR2TPw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 9 Aug 2023 at 08:16, Muhammad Usama Anjum wrote: > Add some explanation and method to use write-protection and written-to > on memory range. [...] > --- a/Documentation/admin-guide/mm/pagemap.rst > +++ b/Documentation/admin-guide/mm/pagemap.rst > @@ -227,3 +227,67 @@ Before Linux 3.11 pagemap bits 55-60 were used for "= page-shift" (which is > always 12 at most architectures). Since Linux 3.11 their meaning changes > after first clear of soft-dirty bits. Since Linux 4.2 they are used for > flags unconditionally. > + > +Pagemap Scan IOCTL > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +The ``PAGEMAP_SCAN`` IOCTL on the pagemap file can be used to get or opt= ionally > +clear the info about page table entries. The following operations are su= pported > +in this IOCTL: > +- Get the information if the pages have Async Write-Protection enabled > + (``PAGE_IS_WPALLOWED``), have been written to (``PAGE_IS_WRITTEN``), f= ile mapped > + (``PAGE_IS_FILE``), present (``PAGE_IS_PRESENT``), swapped (``PAGE_IS_= SWAPPED``) > + or page has pfn zero (``PAGE_IS_PFNZERO``). A recent addition -- PAGE_IS_HUGE -- is missing. BTW, it could be easier to understand if the page categories were separated from the operation description and listed so that each has its own line and maybe a longer description where needed. > +- Find pages which have been written to and/or write protect > + (atomic ``PM_SCAN_WP_MATCHING + PM_SCAN_CHECK_WPASYNC``) the pages ato= mically. > + The (``PM_SCAN_WP_MATCHING``) is used to WP the matched pages. The > + (``PM_SCAN_CHECK_WPASYNC``) aborts the operation if non-Async-Write-Pr= otected > + pages are found. The operation the IOCTL does now is: "scan the process page tables and report memory ranges matching provided criteria '. Flags extend the operation: "PM_SCAN_WP_MATCHING write protects the memory reported" (it does it atomically, but this is just an optimization, isn't it? A process could gather the ranges, WP them, and then copy.) "PM_SCAN_CHECK_WPASYNC" aborts the scan early if a non-WP-able matching page is found. > +The ``struct pm_scan_arg`` is used as the argument of the IOCTL. > + 1. The size of the ``struct pm_scan_arg`` must be specified in the ``si= ze`` > + field. This field will be helpful in recognizing the structure if ex= tensions > + are done later. > + 2. The flags can be specified in the ``flags`` field. The ``PM_SCAN_WP_= MATCHING`` > + and ``PM_SCAN_CHECK_WPASYNC`` are the only added flags at this time.= The get > + operation is optionally performed depending upon if the output buffe= r is > + provided or not. > + 3. The range is specified through ``start`` and ``end``. > + 4. The output buffer of ``struct page_region`` array and size is specif= ied in > + ``vec`` and ``vec_len``. > + 5. The optional maximum requested pages are specified in the ``max_page= s``. > + 6. The masks are specified in ``category_mask``, ``category_anyof_mask`= `, > + ``category_inverted`` and ``return_mask``. > + 1. To find if ``PAGE_IS_WRITTEN`` flag is set for pages which have > + ``PAGE_IS_FILE`` set and ``PAGE_IS_SWAPPED`` unset, ``category_m= ask`` > + is set to ``PAGE_IS_FILE | PAGE_IS_SWAPPED``, ``category_inverte= d`` is > + set to ``PAGE_IS_SWAPPED`` and ``return_mask`` is set to ``PAGE_= IS_WRITTEN``. > + The output buffer in ``vec`` and length must be specified in ``v= ec_len``. > + 2. To find pages which have either ``PAGE_IS_FILE`` or ``PAGE_IS_SWA= PPED`` > + set, ``category_anyof_mask`` is set to ``PAGE_IS_FILE | PAGE_IS_= SWAPPED``. > + 3. To find written pages and engage write protect, ``PAGE_IS_WRITTEN= `` is > + specified in ``category_mask`` and ``return_mask``. In addition t= o > + specifying the output buffer in ``vec`` and length in ``vec_len``= , the > + ``PM_SCAN_WP_MATCHING`` is specified in ``flags`` to perform writ= e protect > + on the range as well. Could this be rewritten as examples? E.g.: Finding dirty file-backed pages: struct pm_scan_arg arg =3D { .size =3D sizeof(arg), .flags =3D 0, ... .category_mask =3D ..., .return_mask =3D ... }; ssize_t n =3D ioctl(..., &arg); Find dirty pages and write protect them in the same call: arg =3D { ... }; do { ... ioctl(...) } while(...); (The code snippets heavily commented.) > +The ``PAGE_IS_WRITTEN`` flag can be considered as the better and correct "as a better-performing alternative" > +alternative of soft-dirty flag. It doesn't get affected by housekeeping = chores > +(VMA merging) of the kernel and hence the user can find the true soft-di= rty pages > +only. This is still an optimization, e.g. in THP case there might be too many pages reported? > + This IOCTL adds the atomic way to find which pages have been written an= d > +write protect those pages again. This kind of operation is needed to eff= iciently > +find out which pages have changed in the memory. This repeats the description of PM_SCAN_WP_MATCHING -- I suggest removing this part. > +To get information about which pages have been written to or optionally = write > +protect the pages, following must be performed first in order: "PAGE_IS_WRITTEN" category is used with uffd write protect-enabled ranges to implement memory dirty tracking in userspace: > + 1. The userfaultfd file descriptor is created with ``userfaultfd`` sysc= all. > + 2. The ``UFFD_FEATURE_WP_UNPOPULATED`` and ``UFFD_FEATURE_WP_ASYNC`` fe= atures > + are set by ``UFFDIO_API`` IOCTL. > + 3. The memory range is registered with ``UFFDIO_REGISTER_MODE_WP`` mode > + through ``UFFDIO_REGISTER`` IOCTL. > + 4. Then any part of the registered memory or the whole memory region mu= st > + be write protected using ``PAGEMAP_SCAN`` IOCTL with flag ``PM_SCAN_= OP_WP`` > + or the ``UFFDIO_WRITEPROTECT`` IOCTL can be used. Both of these perf= orm the > + same operation. The former is better in terms of performance. I guess that the UFFD performance could be fixed? But this part refers to the old PM_SCAN_OP_WP, so an updated example is needed. > + 5. Now the ``PAGEMAP_SCAN`` IOCTL can be used to either just find pages= which > + have been written to and/or optionally write protect the pages as we= ll. "find the pages written to since they were last write protected", but this sounds contradicting: we look for pages that were WP but written anyway. (IOW: marking write-protected is an implementation detail - the ioctl is to find pages that changed since they were last marked.) Maybe we should call the operation "marking CLEAN" or alike? Best Regards Micha=C5=82 Miros=C5=82aw