From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C754C64EC7 for ; Tue, 28 Feb 2023 16:24:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 796CB6B0071; Tue, 28 Feb 2023 11:24:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 746E36B0073; Tue, 28 Feb 2023 11:24:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 610546B0074; Tue, 28 Feb 2023 11:24:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 51E636B0071 for ; Tue, 28 Feb 2023 11:24:27 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2DA9940649 for ; Tue, 28 Feb 2023 16:24:27 +0000 (UTC) X-FDA: 80517223374.28.C360085 Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) by imf22.hostedemail.com (Postfix) with ESMTP id 23E0CC0006 for ; Tue, 28 Feb 2023 16:24:23 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=collabora.com header.s=mail header.b=FPK6pG05; spf=pass (imf22.hostedemail.com: domain of usama.anjum@collabora.com designates 46.235.227.172 as permitted sender) smtp.mailfrom=usama.anjum@collabora.com; dmarc=pass (policy=reject) header.from=collabora.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677601464; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kGdHexJaj4sU3BfGA4UVId17FGELrqb8QQVs668r2KM=; b=pJHDBkmEYU2x7wuTm84OMgq3oOs5K5ZAzqctLx0LDUH0oYslrKE4MNPXCrnuWYc6rJXXt3 YmfH3xqq8PCcTL6TFOzhSEMHxDiS8wEUnRcOdL5MITvDz9/Nh8pBLJUoAYfcmgQ71ztpI0 L0/6VxElxKME2sbuEscSxHEcA/s8vxA= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=collabora.com header.s=mail header.b=FPK6pG05; spf=pass (imf22.hostedemail.com: domain of usama.anjum@collabora.com designates 46.235.227.172 as permitted sender) smtp.mailfrom=usama.anjum@collabora.com; dmarc=pass (policy=reject) header.from=collabora.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677601464; a=rsa-sha256; cv=none; b=BCoyjkPTEIXBe6efseurYBIjwseFh/6nhEbZAtlVkMG2yNm55gzEnKj4UtkotVqMFz4Tyg 4sx+wE79uC4IhO4g6BdE0AQdl0sqWAGLLpLViriuTnwNq76wjfgxTMS/bEp7kd5vMZrKC+ VydWeqAZFR6R1bC6QBmt/bXHkEaP0Fo= Received: from [192.168.10.12] (unknown [39.45.217.110]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) (Authenticated sender: usama.anjum) by madras.collabora.co.uk (Postfix) with ESMTPSA id 9EB17660220B; Tue, 28 Feb 2023 16:24:12 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1677601462; bh=KZI+i3dFgrsmwgaR8TVTckHR0mbIeXl+ghCN2Flz/mw=; h=Date:Cc:Subject:To:References:From:In-Reply-To:From; b=FPK6pG05bAn2CiutxJJJTWsmucf4PD5D8A0ir26sMaqyyQT7GBFaSstm0MQ4VrwHy PXW9CqMhiwm2iv8y62evwHMz/VDSGvC9kMBZURlOGRjAyg3PuWJ7wTaQOAT7J3RpBi 6xkqmhqOhALGKfQFXkau54Id/XN5TaQ6mR2MIE58RieE/S2pttFbWNWkqUSQdO7T1h pbFQ4pzeWEzHHjx4KZpsW28x0n/XaNGjNJimALW0oJYAv2pG/8+NOHV96oOX1lD4yg 2h0au7iBmUn1jv2x9tWKcFsMadHhmkcoIL1YxdUcdPkbOabuo0QmY+VxtsZGhcFe29 mpAmzzHzYagPQ== Message-ID: Date: Tue, 28 Feb 2023 21:24:07 +0500 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.2 Cc: Muhammad Usama Anjum , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , Andrew Morton , Mike Rapoport , Axel Rasmussen , Nadav Amit , David Hildenbrand , "kernel@collabora.com" Subject: Re: [PATCH v2] mm/uffd: UFFD_FEATURE_WP_UNPOPULATED Content-Language: en-US To: Peter Xu References: <20230227230044.1596744-1-peterx@redhat.com> From: Muhammad Usama Anjum In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: o458o5afgsgjp6gy573zscyu1t6keqgu X-Rspamd-Queue-Id: 23E0CC0006 X-HE-Tag: 1677601463-446839 X-HE-Meta: U2FsdGVkX18PelYMUKGxqq+HUJDMmGKXx7ZeKk/g5II/8nvhz4xiU5dPdMR5uiANXGJEs5znSM+wSMUNep8j7+DsFO2FSfmCWppoldVObeSCsNm889BMJ5x3xkQCIappFYh/yTAjHviGhGC/hDuHWjf6+Kav+VEe4QnnxlrKX1Jevp1jP3F+4kEakeOHr+K/13czLsDOTmsCFLkm+kAk4GmC+1hgfzuq/eFQw28m+YcncPpq9yRs8pYMuxlJLSCUiWMB+75i9rVLHu2r/Gjm7RkvQJs1dg6EzDmIKBI7l1QNTBSeOGjVMRoWibReAn86RdHjIuHbrQacMOmVHJBQeES5o7FiHr2yAnMdm5XEMBFfkJjeHsd/Di/lKlO2iSC7uW74wo+z9mC/LQoo9iuSSWiXKjl6swG+mbXJFq73RitAvwBVa2p/sSUTopZ/J2s84ijzYB4MfS+R+MVknk8nJOaONhqdKSYKDSUghmcY1utoZ6aIlV5lbHrJmTvmjxSJb30M+snoE/o9Olm5yycUZuhf5XnD3oByEKQyD6Ug+t98T+ZoE4ROI9ajxr8ncTPjc9W9Y0kJJ0m7Yaf7lZOuBE/xVa0MW763OTqfHb6oPCGiPccTki08NcvvbG9X0fn3BowCSHAUYX1IrdNg9hNUwA813PUsD7YZJUDoagTeXjmE3aYo2VoiF/6P2wE4TTJ8zcPsLnkjOKFz+AtJp62/DoyZJCVh4FqmFQjzilATh37wYy8dodWmxPNW2tISn5gDyn4jxzzK4YqRtm0qlv+o2oiB8qma6k5O6lsIoESGT2UCvVXvuN11VebfB7zLgMEqx/7uv/rVgaTGA6yzIshr918NjxojKdGB9a2ivvGt0tUn+efNGxPL/UsrV9iVQ9zAr6GWTaxRvZee99HhUsej5FB9ib5aE50OFgmGPjVu/Dz4abJcrdkIMVqiK+zWj3NgQJDI45VJn/ud465/C9E pOE30V3t ec61e/dgt7p/o1m06YjdDn3RvHDpMqRAu7tVEbcPE+dzrjI4X5cgy6zEU3DAmIpanqomEGLxBccIjgtX3MhaU1wIGlKQpoliPqYkQnSzq5EnhIy//Ptfmqi5kHWs7lnk5JHETuDJsq158m7VN7ZCKY1WTluTAwu0I2wpEL0mtkcNR9AwD7kzjLdDXkGZKZvJPy5Ui8dXuq5zsVB41EZy7M8nPEt0d1zIEdi+15dC7RSszTGwhmJ8kWbLbRPSHqakBOZHoCKHopE41kVVmhQjeNeT9a/6nYaZqhIr1hD09ZmsdnVqGDMLy3VZ3ZVVhzb0I3QCHjnWFvz/ktCw7kiqft0Ml6dpwpKhWryQu9Xq/8kGrWXnbdjo2hHOx/M4W4PPkEWcQymHmf3EIJpTaWpkjjZekKD4q2EfbVbxwfhbfKGHKicfT/3qfZ4a5yxapnmH712Oo00RBgZxNlfci/BsKT19jI5f+Uw1G336zwOBLdMgJDb/4q6o/b9NWjnlEo54RBwhr5K5nLB7Uj9k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2/28/23 8:58 PM, Peter Xu wrote: > On Tue, Feb 28, 2023 at 12:21:45PM +0500, Muhammad Usama Anjum wrote: >> Hi Peter, > > Hi, Muhammad, > >> >> Thank you so much for sending. >> >> On 2/28/23 5:36 AM, Peter Xu wrote: >>> On Mon, Feb 27, 2023 at 06:00:44PM -0500, Peter Xu wrote: >>>> This is a new feature that controls how uffd-wp handles none ptes. When >>>> it's set, the kernel will handle anonymous memory the same way as file >>>> memory, by allowing the user to wr-protect unpopulated ptes. >>>> >>>> File memories handles none ptes consistently by allowing wr-protecting of >>>> none ptes because of the unawareness of page cache being exist or not. For >>>> anonymous it was not as persistent because we used to assume that we don't >>>> need protections on none ptes or known zero pages. >>>> >>>> One use case of such a feature bit was VM live snapshot, where if without >>>> wr-protecting empty ptes the snapshot can contain random rubbish in the >>>> holes of the anonymous memory, which can cause misbehave of the guest when >>>> the guest OS assumes the pages should be all zeros. >>>> >>>> QEMU worked it around by pre-populate the section with reads to fill in >>>> zero page entries before starting the whole snapshot process [1]. >>>> >>>> Recently there's another need raised on using userfaultfd wr-protect for >>>> detecting dirty pages (to replace soft-dirty in some cases) [2]. In that >>>> case if without being able to wr-protect none ptes by default, the dirty >>>> info can get lost, since we cannot treat every none pte to be dirty (the >>>> current design is identify a page dirty based on uffd-wp bit being cleared). >>>> >>>> In general, we want to be able to wr-protect empty ptes too even for >>>> anonymous. >>>> >>>> This patch implements UFFD_FEATURE_WP_UNPOPULATED so that it'll make >>>> uffd-wp handling on none ptes being consistent no matter what the memory >>>> type is underneath. It doesn't have any impact on file memories so far >>>> because we already have pte markers taking care of that. So it only >>>> affects anonymous. >>>> >>>> The feature bit is by default off, so the old behavior will be maintained. >>>> Sometimes it may be wanted because the wr-protect of none ptes will contain >>>> overheads not only during UFFDIO_WRITEPROTECT (by applying pte markers to >>>> anonymous), but also on creating the pgtables to store the pte markers. So >>>> there's potentially less chance of using thp on the first fault for a none >>>> pmd or larger than a pmd. >>>> >>>> The major implementation part is teaching the whole kernel to understand >>>> pte markers even for anonymously mapped ranges, meanwhile allowing the >>>> UFFDIO_WRITEPROTECT ioctl to apply pte markers for anonymous too when the >>>> new feature bit is set. >>>> >>>> Note that even if the patch subject starts with mm/uffd, there're a few >>>> small refactors to major mm path of handling anonymous page faults. But >>>> they should be straightforward. >>>> >>>> So far, add a very light smoke test within the userfaultfd kselftest >>>> pagemap unit test to make sure anon pte markers work. >>>> >>>> [1] https://lore.kernel.org/all/20210401092226.102804-4-andrey.gruzdev@virtuozzo.com/ >>>> [1] https://lore.kernel.org/all/Y+v2HJ8+3i%2FKzDBu@x1n/ >>>> >>>> Signed-off-by: Peter Xu >>>> --- >>>> v1->v2: >>>> - Use pte markers rather than populate zero pages when protect [David] >>>> - Rename WP_ZEROPAGE to WP_UNPOPULATED [David] >>> >>> Some very initial performance numbers (I only ran in a VM but it should be >>> similar, unit is "us") below as requested. The measurement is about time >>> spent when wr-protecting 10G range of empty but mapped memory. It's done >>> in a VM, assuming we'll get similar results on bare metal. >>> >>> Four test cases: >>> >>> - default UFFDIO_WP >>> - pre-read the memory, then UFFDIO_WP (what QEMU does right now) >>> - pre-fault using MADV_POPULATE_READ, then default UFFDIO_WP >>> - UFFDIO_WP with WP_UNPOPULATED >>> >>> Results: >>> >>> Test DEFAULT: 2 >>> Test PRE-READ: 3277099 (pre-fault 3253826) >>> Test MADVISE: 2250361 (pre-fault 2226310) >>> Test WP-UNPOPULATE: 20850 >>> >>> I'll add these information into the commit message when there's a new >>> version. >> I'm hitting a bug where I'm unable to write to the memory after adding this >> patch and wp the memory. I'm hitting this case in your test and my tests as >> well. Please apply the following diff to your test to reproduce on your end: >> >> --- uffd_wp_perf.c.orig 2023-02-28 12:09:38.971820791 +0500 >> +++ uffd_wp_perf.c 2023-02-28 12:13:11.077827160 +0500 >> @@ -114,6 +114,7 @@ >> start1 = get_usec(); >> } >> wp_range(uffd, buffer, SIZE, true); >> + buffer[0] = 'a'; >> if (start1 == 0) >> printf("%"PRIu64"\n", get_usec() - start); >> else > > This is expected, because the test didn't start any fault resolving thread, > so the write will block until someone unprotects the page. Ohh.. sorry. Wrong reproducer. > > But it shouldn't happen to your use case if you applied both WP_UNPOPULATED > & WP_ASYNC. I'm using both WP_UNPOPULATED and ASYNC. The program gets stuck at right time: 1..57 ok 1 sanity_tests_sd wrong flag specified ok 2 sanity_tests_sd wrong mask specified ok 3 sanity_tests_sd wrong return mask specified ok 4 sanity_tests_sd mixture of correct and wrong flag ok 5 sanity_tests_sd PM_SCAN_OP_WP cannot be used without get ok 6 sanity_tests_sd Clear area with larger vec size ^C Program received signal SIGINT, Interrupt. 0x000000000040220c in sanity_tests_sd () at pagemap_ioctl.c:198 198 mem[i]++; (gdb) bt #0 0x000000000040220c in sanity_tests_sd () at pagemap_ioctl.c:198 #1 0x0000000000404e14 in main () at pagemap_ioctl.c:846 () /proc/$PID/stack is empty. Not sure why. I can see stack trace of other applications, but not this one's. Let me send better reproducer for you. > > Could you try "cat /proc/$PID/stack" to see where does your thread stuck > at? > -- BR, Muhammad Usama Anjum