From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 912A5C0755A for ; Mon, 27 Nov 2023 10:35:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 311876B02E7; Mon, 27 Nov 2023 05:35:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 29A986B02EB; Mon, 27 Nov 2023 05:35:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13C7F6B02EC; Mon, 27 Nov 2023 05:35:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id F14066B02E7 for ; Mon, 27 Nov 2023 05:35:49 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D04398014B for ; Mon, 27 Nov 2023 10:35:49 +0000 (UTC) X-FDA: 81503378418.23.53B7119 Received: from mail-ua1-f46.google.com (mail-ua1-f46.google.com [209.85.222.46]) by imf02.hostedemail.com (Postfix) with ESMTP id 1BCA480007 for ; Mon, 27 Nov 2023 10:35:47 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=F5eJfLhs; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.46 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701081348; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GN2NHS0ADz42vQgBk5jmGpz9KoJL/tRq0nS2+wspS0I=; b=mxfJtayyfSLwj5tyA3C6Yd4aanw+Fu9dZCW4BVkrkKRz3p4ap1bgxu21u4iPbxwbMy9yWM +QxoIlWSPkL/CYVkazedDQ578WnCJQKwtR0vgDYE8bl1Dk1QbScx0hobrbhp3ut483cwJc x/kJV2Fa3Oj3GfnID9HR0EntpkbJI38= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=F5eJfLhs; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.46 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701081348; a=rsa-sha256; cv=none; b=AcA9088lmqpYtEkqzgXZqr9yqiUjR4svOTC6EU+k+/7qPH4e0RGp+N2+ghv26tANzBUuR9 YKaAGOOZanfBc1TmYPjEH+OxtG8JEE1Ws+0kLH0zxEutr86MzIRjMEX2HKwjAnS/PQPESe GYoCqywdnazcuuFFuch5dBqNjYKnC4c= Received: by mail-ua1-f46.google.com with SMTP id a1e0cc1a2514c-7c45fa55391so1183402241.2 for ; Mon, 27 Nov 2023 02:35:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701081347; x=1701686147; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=GN2NHS0ADz42vQgBk5jmGpz9KoJL/tRq0nS2+wspS0I=; b=F5eJfLhssfG1iMeYI3biBdknbV9Q4RpZ+jC5OImfRn+ez/+jdUO2bu1OwmJoKCXuxt Vv7vCAJeCVt6OLbye2TGPTTDfUis4eBSMEFDu8zGxgMpDhFUCdmi3eOIG1rSGX/ATn+n cGxLVCPDL2hViE6itdmBErRroO0zrlBxHGfJtetUkehP6OvPEAkw32j8GsSU1aDMApu0 BO8mpr94T+Hj/Z3MV0D1BtanpFaRIT9FETMu5nlZh5YvjOklWf/D324AHtivMiCukQ2w hqTCQLy+Ein85QYqyjE3TVTtAXzgbiXmdQPxEBWgStSfvDqsY2wTjj6aFLkONTtjnRra FDBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701081347; x=1701686147; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GN2NHS0ADz42vQgBk5jmGpz9KoJL/tRq0nS2+wspS0I=; b=au2KrkzGMJ/Hp7LpH5QiJu5lrTu1RjMHdw7Cj9pcT5DRUtkbfyzJ8lJ4gIrJxcsDHb cM3jeH90zWa//svlb+eAaQjCjx7M1qWVPu9T8lX1gtUIncn8hXyrf/BO1bKSOLz3cMS+ 4M5MdCT7wcXwUB/YDGV3H8kA1nAhoFrU9RofsWo1UO/X7YLXzd/3ZGYiAyF+G5HisSa3 hlbYI9zu9qjFn5zDpYTlgg7cGYFfQ8MMKVOIdxEHpxoAvJHuG3Z/MpWsNVo+X65T+nUc yhjcCuYUKPu6YpnmyhPGGxhiR0+3KiCnMFanXmZ9uhETgQNWTb+/WkDTzua8pMrG0qnc yS4g== X-Gm-Message-State: AOJu0YyoNTDoAwmdkwQ+wj1LfmbxRRDUlICM/RsRlIpsWaXN1IFxit+l o0gZ50JZUF6FwS/T6RBrkandCBXIeSGeTMTjOi4= X-Google-Smtp-Source: AGHT+IGNqLxrdpuDVZuDBdZvRc0DGZ+sapq9bI1ucbL3A67bCeygEGVywRqbAdJU8SE8CN+G+VEewgfYqtFxTdwpCp0= X-Received: by 2002:a67:b107:0:b0:462:ba4d:ccd4 with SMTP id w7-20020a67b107000000b00462ba4dccd4mr10287500vsl.30.1701081347189; Mon, 27 Nov 2023 02:35:47 -0800 (PST) MIME-Version: 1.0 References: <20231115163018.1303287-1-ryan.roberts@arm.com> <20231127031813.5576-1-v-songbaohua@oppo.com> <234021ba-73c2-474a-82f9-91e1604d5bb5@arm.com> In-Reply-To: <234021ba-73c2-474a-82f9-91e1604d5bb5@arm.com> From: Barry Song <21cnbao@gmail.com> Date: Mon, 27 Nov 2023 23:35:35 +1300 Message-ID: Subject: Re: [PATCH v2 00/14] Transparent Contiguous PTEs for User Mappings To: Ryan Roberts Cc: akpm@linux-foundation.org, andreyknvl@gmail.com, anshuman.khandual@arm.com, ardb@kernel.org, catalin.marinas@arm.com, david@redhat.com, dvyukov@google.com, glider@google.com, james.morse@arm.com, jhubbard@nvidia.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, mark.rutland@arm.com, maz@kernel.org, oliver.upton@linux.dev, ryabinin.a.a@gmail.com, suzuki.poulose@arm.com, vincenzo.frascino@arm.com, wangkefeng.wang@huawei.com, will@kernel.org, willy@infradead.org, yuzenghui@huawei.com, yuzhao@google.com, ziy@nvidia.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 1BCA480007 X-Stat-Signature: fjxeekirfi3sxhpfxfxhcdemtj3apnpj X-HE-Tag: 1701081347-176602 X-HE-Meta: U2FsdGVkX1/ZbHg9r6WVIT1euwc93KH4iKM9ADh9Sza4Cxe4/khkMrfD5hl2msYrFh6cAYLEmN3cOLI8L58ea97X1TvfsLPhgyIRIGssOP2HMfzKN5C3Buvzlw6eD1iDGFINL5ABKTSe21oZTucI4mNYvj2ADyAJ1K2nRQck07T0oi7MCs67uHkNqtdSHdIK6sRwCq61x/RCgemgQNcxsPmu9XINigBtskO3pmskcT3oecbKDtc06CRmowT3YKgvhVuPt9xLd6idW3f78qPLaF/JbyMyJb3vb7w05Gwq4kcnBWEOgoO9SurtQVM5tRHHMMaIk03l3wlqSASskQlc51uOGpP00Nc+9wBk2dpmmzV8jjeQ0xRmtq0SRJ6+uVS2HNdff+RrVRy3Pq/tcKFxTuaLm2n4Gc2A73MYFDFj6GdqMDSOOcPt0D/78UN23JSIenz6dk00Nihuy5xJsXZEt+WV2TXagZJ7OS0rAK1wMSdsnD04dlMN8vAqfg8TDfGbuS/UKminYHm+l1yV4OWBKoPQOCJUwJWrRyi9m0kXEgK1TdSy40KSJ8JWTpKLOB0u79v0U8XWnWQi83tY52DMgSrHStRtltA66A5cSEVfOxHFXhIT3sP3l9AMfG/KgHj6UqjrThA7v2bKSDc5SFGEMXa1vxXzmKYEl3fgoY3RdESXF9xpqvhMEO13kUaOSbjMJeaim9pKE6c10sC7BHwhGrV6ZEAqSpGdeV0xRW+MO8tY8W8xWiqaxJFxZ5mZWyxVEucTDD/N4EZCAdLiQWifNN6Xqpd6aP9o00oMsZDFrnabuV4niuOZehuaxiBmxsphJKlXgiZEXpeX3g7gcohevzHTNDRINBo9ikZ3Ixg4pc6sJqa4ClBtj5QAQgTC9KD1obegsUplfJx62xLafd6Xnu7n4gNd/P4fdyG3yX4EFKMTdPaGA98YvXTnd97ABlKrNmHzCoaBciq3gKVzqO/ FBKWlN5z fDfvEcUz9ksewmvqBf/E9vCnO8ZYh/c9JEk5WXCpq3gDZso8ruRw62w3I4ktk3p0eYHwSDg+Q/xUWAB75MtQqCNMUjFZeZ54v0O6PkFVphh5ldN9frKclBGTC5JuiZUxtunnWCvZKpeqTy3TGuyVJhnX/bFzQVihqkMmOthEiu+7fnNh9GXrpE4mlY9TpwZEa/Qq3W455kUcVZUzhbx+nPxcJh7PWEcPhj2HaWr+WhYktimMfIVdqrKizVPZ2zXaUfXNdpP6uxrWirbVfkeaHSVGLiNqtNYixqZ5TwsNEjAxk8eWHf1Sp9A2etc2CLoPikCiYBVhSIXHKRR9qfdLpruvchZfynRK6EAesuRvBbdr6Nf0bZDpIZb6ecE7CgPr+EUiLtKAH+02kAwki31f0VI1LdKnARIqoe3yxfavUDLnOs+bjNJsN0eeCZzNgfkKnuKGJ3IaFgdhWU044Yi3pHSYmwjg2ptkyavzkzFdZFgm46iU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000041, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Nov 27, 2023 at 10:15=E2=80=AFPM Ryan Roberts wrote: > > On 27/11/2023 03:18, Barry Song wrote: > >> Ryan Roberts (14): > >> mm: Batch-copy PTE ranges during fork() > >> arm64/mm: set_pte(): New layer to manage contig bit > >> arm64/mm: set_ptes()/set_pte_at(): New layer to manage contig bit > >> arm64/mm: pte_clear(): New layer to manage contig bit > >> arm64/mm: ptep_get_and_clear(): New layer to manage contig bit > >> arm64/mm: ptep_test_and_clear_young(): New layer to manage contig bi= t > >> arm64/mm: ptep_clear_flush_young(): New layer to manage contig bit > >> arm64/mm: ptep_set_wrprotect(): New layer to manage contig bit > >> arm64/mm: ptep_set_access_flags(): New layer to manage contig bit > >> arm64/mm: ptep_get(): New layer to manage contig bit > >> arm64/mm: Split __flush_tlb_range() to elide trailing DSB > >> arm64/mm: Wire up PTE_CONT for user mappings > >> arm64/mm: Implement ptep_set_wrprotects() to optimize fork() > >> arm64/mm: Add ptep_get_and_clear_full() to optimize process teardown > > > > Hi Ryan, > > Not quite sure if I missed something, are we splitting/unfolding CONTPT= ES > > in the below cases > > The general idea is that the core-mm sets the individual ptes (one at a t= ime if > it likes with set_pte_at(), or in a block with set_ptes()), modifies its > permissions (ptep_set_wrprotect(), ptep_set_access_flags()) and clears th= em > (ptep_clear(), etc); This is exactly the same interface as previously. > > BUT, the arm64 implementation of those interfaces will now detect when a = set of > adjacent PTEs (a contpte block - so 16 naturally aligned entries when usi= ng 4K > base pages) are all appropriate for having the CONT_PTE bit set; in this = case > the block is "folded". And it will detect when the first PTE in the block > changes such that the CONT_PTE bit must now be unset ("unfolded"). One of= the > requirements for folding a contpte block is that all the pages must belon= g to > the *same* folio (that means its safe to only track access/dirty for thec= ontpte > block as a whole rather than for each individual pte). > > (there are a couple of optimizations that make the reality slightly more > complicated than what I've just explained, but you get the idea). > > On that basis, I believe all the specific cases you describe below are al= l > covered and safe - please let me know if you think there is a hole here! > > > > > 1. madvise(MADV_DONTNEED) on a part of basepages on a CONTPTE large fol= io > > The page will first be unmapped (e.g. ptep_clear() or ptep_get_and_clear(= ), or > whatever). The implementation of that will cause an unfold and the CONT_P= TE bit > is removed from the whole contpte block. If there is then a subsequent > set_pte_at() to set a swap entry, the implementation will see that its no= t > appropriate to re-fold, so the range will remain unfolded. > > > > > 2. vma split in a large folio due to various reasons such as mprotect, > > munmap, mlock etc. > > I'm not sure if PTEs are explicitly unmapped/remapped when splitting a VM= A? I > suspect not, so if the VMA is split in the middle of a currently folded c= ontpte > block, it will remain folded. But this is safe and continues to work corr= ectly. > The VMA arrangement is not important; it is just important that a single = folio > is mapped contiguously across the whole block. I don't think it is safe to keep CONTPTE folded in a split_vma case. as otherwise, copy_ptes in your other patch might only copy a part of CONTPES. For example, if page0-page4 and page5-page15 are splitted in split_vma, in fork, while copying pte for the first VMA, we are copying page0-page4, this will immediately cause inconsistent CONTPTE. as we have to make sure all CONTPTEs are atomically mapped in a PTL. > > > > > 3. try_to_unmap_one() to reclaim a folio, ptes are scanned one by one > > rather than being as a whole. > > Yes, as per 1; the arm64 implementation will notice when the first entry = is > cleared and unfold the contpte block. > > > > > In hardware, we need to make sure CONTPTE follow the rule - always 16 > > contiguous physical address with CONTPTE set. if one of them run away > > from the 16 ptes group and PTEs become unconsistent, some terrible > > errors/faults can happen in HW. for example > > Yes, the implementation obeys all these rules; see contpte_try_fold() and > contpte_try_unfold(). the fold/unfold operation is only done when all > requirements are met, and we perform it in a manner that is conformant to= the > architecture requirements (see contpte_fold() - being renamed to > contpte_convert() in the next version). > > Thanks for the review! > > Thanks, > Ryan > > > > > case0: > > addr0 PTE - has no CONTPE > > addr0+4kb PTE - has CONTPTE > > .... > > addr0+60kb PTE - has CONTPTE > > > > case 1: > > addr0 PTE - has no CONTPE > > addr0+4kb PTE - has CONTPTE > > .... > > addr0+60kb PTE - has swap > > > > Unconsistent 16 PTEs will lead to crash even in the firmware based on > > our observation. > > Thanks Barry