From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09986C4167B for ; Mon, 27 Nov 2023 22:54:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81FA76B02A3; Mon, 27 Nov 2023 17:54:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7D0F76B02AD; Mon, 27 Nov 2023 17:54:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 696EE6B02E5; Mon, 27 Nov 2023 17:54:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 56AAA6B02A3 for ; Mon, 27 Nov 2023 17:54:05 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 2EAF61A00E8 for ; Mon, 27 Nov 2023 22:54:05 +0000 (UTC) X-FDA: 81505238850.14.D514C95 Received: from mail-ua1-f44.google.com (mail-ua1-f44.google.com [209.85.222.44]) by imf21.hostedemail.com (Postfix) with ESMTP id F21861C0257 for ; Mon, 27 Nov 2023 22:54:02 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iEMLjxvR; spf=pass (imf21.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.44 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701125643; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=a4jTo4Nl5QOfj+i8ClgVWtaOb4p2NtBPY55Cf4vOdgU=; b=jfAkp/9TrUFnwwZq3qQkKwcGa9Pwqm9Z/a94JjEljrtlIROTOLOKHBIJC2ePMJz80yBymb lUwY6Vic34QBKvxqWsYmggC5aBuPoMoOjKhl+MozTysXzmNePMbm3qDtIHjH/irxOWPfxN QJh+BYeBHg8SEm2nDxwj1GUSKSTZzsE= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iEMLjxvR; spf=pass (imf21.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.44 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701125643; a=rsa-sha256; cv=none; b=V1Y9jJubm0A7u0Mk3yGyrk9NSRy28SgILgHxl56bfzMiQQmOJ2ZTKEMl607gTxQqVNc+c4 MUuY6GmNo1O6+OrRIw+MMfmEvxoUr8Ww0blSrPCRTaRIiVSws8BT8WbMvaPgyOQ4G5cmxn hP4YSKlMbADby3FC1MxpVu0wS75byGY= Received: by mail-ua1-f44.google.com with SMTP id a1e0cc1a2514c-7c45fa55391so1339473241.2 for ; Mon, 27 Nov 2023 14:54:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701125642; x=1701730442; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=a4jTo4Nl5QOfj+i8ClgVWtaOb4p2NtBPY55Cf4vOdgU=; b=iEMLjxvRjtdQcZM8dRmiZsfOKdKP37XBIfgTlDKrl3xMBwXEc80C0WiJGLs6SvYAtr szchCUCkXwoCkJzLwFXE6l8KRqhURRgwSWMVgPVabysuIpHl1KpFJ8lQkKkEsnGwxXKe QA7osSjJ0jL9/hjrZPMcTJZoJm4EDy6SBoqQl7CwPLC9OhtKvj3Yl98IZrMTaUmnonoa 6e1ai4VMbQzhugQdslUXbvoMFuMBeV9b01JlsQXc6vyfG0kROXT5sPUeykgInUNFW6fm tfP20+ofRo9U3i96UvD3HpP20P6kwDdJsR0s4FpFZm0RENa7gtm6BQJOodTp/UY5dEfn TOKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701125642; x=1701730442; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a4jTo4Nl5QOfj+i8ClgVWtaOb4p2NtBPY55Cf4vOdgU=; b=qfFGz1pdcPvZeuddKOPRzSptjRD127QhtjZzv75UW6/BqvGi1ETG9XndSBdFWI5LU4 X+oz2ELMmVKlmiEx6o7dtuAauEWAu7HZMEFAp0pedq/Q9ol1ZRVmpTXxsO9qVghyW7fm VWFks0vU9neCq6pEp3WhudvktdI/AepUpzZeWLNd8jEMFB5nLM5Ad0VfvO9ciAnvvTGs x2zbgdos6LLUxk8H7j6WZQvCleKLg70UkNtVwUzDRD0toeO9usssLkEOPm1PNNdgyXLb yc1qPNNNGNuMJl/3ElDS93hSLEOlHZCDwXOWwLQcKckuDUKUOa64kuuq02YS9VGGx76P Kwcg== X-Gm-Message-State: AOJu0Yy+zx1TEbA6id9m9DHkPCJHamTS6y4Nycr7C/NEFlii9DiZtsuA BTaq/pmHmRC+ibsImMCEXN5XrD4SfA1oKt5yJM4= X-Google-Smtp-Source: AGHT+IGekE2LN0Cg4u5BOtKEQozDVxDXNwhTBdKonUMT/PJ3qDvYv4NiD6tzxfknhNrzUH5mW9r5TH2UE5goMY4OYik= X-Received: by 2002:a67:fb03:0:b0:462:d592:b078 with SMTP id d3-20020a67fb03000000b00462d592b078mr12636292vsr.32.1701125642060; Mon, 27 Nov 2023 14:54:02 -0800 (PST) MIME-Version: 1.0 References: <20231115163018.1303287-1-ryan.roberts@arm.com> <20231127031813.5576-1-v-songbaohua@oppo.com> <234021ba-73c2-474a-82f9-91e1604d5bb5@arm.com> In-Reply-To: From: Barry Song <21cnbao@gmail.com> Date: Tue, 28 Nov 2023 11:53:50 +1300 Message-ID: Subject: Re: [PATCH v2 00/14] Transparent Contiguous PTEs for User Mappings To: Ryan Roberts Cc: akpm@linux-foundation.org, andreyknvl@gmail.com, anshuman.khandual@arm.com, ardb@kernel.org, catalin.marinas@arm.com, david@redhat.com, dvyukov@google.com, glider@google.com, james.morse@arm.com, jhubbard@nvidia.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, mark.rutland@arm.com, maz@kernel.org, oliver.upton@linux.dev, ryabinin.a.a@gmail.com, suzuki.poulose@arm.com, vincenzo.frascino@arm.com, wangkefeng.wang@huawei.com, will@kernel.org, willy@infradead.org, yuzenghui@huawei.com, yuzhao@google.com, ziy@nvidia.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: F21861C0257 X-Rspam-User: X-Stat-Signature: 8dbex59p8o8jds8p9y8p689c77c5chm8 X-Rspamd-Server: rspam01 X-HE-Tag: 1701125642-851322 X-HE-Meta: U2FsdGVkX1+WzGPbrfh7SXeeJSe2Xz3ndQgriqdIufsrxSQpZuVYrQe2pLCpMxRjEfzuvT3gnXkc0rxLqIlGBD5hDXHY5KG0+h473fSjkrg5LfHPZ2ZRdjV2fVzoN09wasnkWfsqqxDIRmVvSf4rSZb3Q/jsWZJqG6NeB2x12F10LWbjv7tO+CG8cf3PATEk7k4bwCPcCLRSCYpRsbmwYA0lhnrHTAHj7jCRXpEPGWs7nkBaNgARQHbW+x0FW44cFUilZjj3qTsoQmAn2CPzcCn5VQKKY4P4glyXQEuVn8YBeUid2mn5Fm+n65IA7G9ZSfxogRQZbZmqXLweilFclLCdsWukXrq2vZqRw39y8Gt0F8ANBpo7ui5rJcqKNJkF2grd5sUny8/iM3sGjSDp3aq4DDwv/wveE7FrAWhNwhgpTbfWPO74SiQYvFo0BsVjZ8VjVTuwK0cOwMYZQzDS6u2aP+QPt5iuaOM4XdXWYXhl2hJMZUgdA1NU2Agwr9kUtgT9aD4Dk/cCApoJaGc3udQKoPBs1eqcsdoipO4IXorouvxgTOcSG/K9GWgHicPwmTV60QEVU5o7V7gRuO7YiXZL75CZlzoDDt2KLaemFuACng0JOAwf+tWhQ8o6IXHg8A33rVfujAOCU2ONtc8yGML0Y6hcfu93FxCaGJtoo+NhlbZIfq84q7EbWkC5xbgWhZIUYykpzvpVVPxlLxGygxJbjOkfZXQqXQb/Q66rM8w6TIEdKwsGYde3VVKrwtmN9uE5gi0GgRXiUCmWW7LCCHWsjAAtikrhR3si+t+oWCbO37JJH51H6lwMYSutyyVDsod2wjoLBmxv0dukbvwSIFCzLqR5MZR93Re+O6sNlk9KgYZD4QiHQJKIrxtnt10eUOJmdUn6MpXGfvGPyHFJ710IBVubP7tdyvnLVk1ZoqDTYLM3AKcfqlzcQbUwhwkThG4Z4s74Pg/fG7qkCfz tnIM7+We sZIs9jmc8uoW6FUjUZR8466lwwouDfmDZMcTeCDVP0OItjud2uKX+CMLvDOKaBV1QSoxz2ixtK6/nSlIjfj2dLSCfEWs9qj3UJBr6M0+V4sZoExh16LVxaGDrQgR4nzWS2cX3tDILcnW3uosHdVDRQTEtfC7Ft7tXGAOAmbx66IbrLwrvM6CocEO33gptwFVG/KjNEIqqcC88Fif/IHUzOkbzEpuA/muPJ+06VAFAabmXJ3qqVXC+aon7FKnXkk7OhR24ec9McdK/SzVy4NJxuf6NmYIGX/wbHvGuRfaDqZmE5MYmVmkFfaLX9QfsKGaiskvn5A9yBXL/l3b5Ri5Lr5QfTSY6HoqKWoq8TTflz9uJXXp6/sUj7NERpWHhaGb6mxo1vdvRrYEGPHoD93d0xqeuDC2MUUYyrDOxbY1quOowRkKHwScgWFtRWFrvD8Yj4IDb/e9brJXeWTX3fiD3ee/l5sFTR0HGQAFUYiAljbbrZZi/CCiW6gFM7ZTHGKZRmob4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Nov 28, 2023 at 12:11=E2=80=AFAM Ryan Roberts wrote: > > On 27/11/2023 10:35, Barry Song wrote: > > On Mon, Nov 27, 2023 at 10:15=E2=80=AFPM Ryan Roberts wrote: > >> > >> On 27/11/2023 03:18, Barry Song wrote: > >>>> Ryan Roberts (14): > >>>> mm: Batch-copy PTE ranges during fork() > >>>> arm64/mm: set_pte(): New layer to manage contig bit > >>>> arm64/mm: set_ptes()/set_pte_at(): New layer to manage contig bit > >>>> arm64/mm: pte_clear(): New layer to manage contig bit > >>>> arm64/mm: ptep_get_and_clear(): New layer to manage contig bit > >>>> arm64/mm: ptep_test_and_clear_young(): New layer to manage contig = bit > >>>> arm64/mm: ptep_clear_flush_young(): New layer to manage contig bit > >>>> arm64/mm: ptep_set_wrprotect(): New layer to manage contig bit > >>>> arm64/mm: ptep_set_access_flags(): New layer to manage contig bit > >>>> arm64/mm: ptep_get(): New layer to manage contig bit > >>>> arm64/mm: Split __flush_tlb_range() to elide trailing DSB > >>>> arm64/mm: Wire up PTE_CONT for user mappings > >>>> arm64/mm: Implement ptep_set_wrprotects() to optimize fork() > >>>> arm64/mm: Add ptep_get_and_clear_full() to optimize process teardo= wn > >>> > >>> Hi Ryan, > >>> Not quite sure if I missed something, are we splitting/unfolding CONT= PTES > >>> in the below cases > >> > >> The general idea is that the core-mm sets the individual ptes (one at = a time if > >> it likes with set_pte_at(), or in a block with set_ptes()), modifies i= ts > >> permissions (ptep_set_wrprotect(), ptep_set_access_flags()) and clears= them > >> (ptep_clear(), etc); This is exactly the same interface as previously. > >> > >> BUT, the arm64 implementation of those interfaces will now detect when= a set of > >> adjacent PTEs (a contpte block - so 16 naturally aligned entries when = using 4K > >> base pages) are all appropriate for having the CONT_PTE bit set; in th= is case > >> the block is "folded". And it will detect when the first PTE in the bl= ock > >> changes such that the CONT_PTE bit must now be unset ("unfolded"). One= of the > >> requirements for folding a contpte block is that all the pages must be= long to > >> the *same* folio (that means its safe to only track access/dirty for t= hecontpte > >> block as a whole rather than for each individual pte). > >> > >> (there are a couple of optimizations that make the reality slightly mo= re > >> complicated than what I've just explained, but you get the idea). > >> > >> On that basis, I believe all the specific cases you describe below are= all > >> covered and safe - please let me know if you think there is a hole her= e! > >> > >>> > >>> 1. madvise(MADV_DONTNEED) on a part of basepages on a CONTPTE large f= olio > >> > >> The page will first be unmapped (e.g. ptep_clear() or ptep_get_and_cle= ar(), or > >> whatever). The implementation of that will cause an unfold and the CON= T_PTE bit > >> is removed from the whole contpte block. If there is then a subsequent > >> set_pte_at() to set a swap entry, the implementation will see that its= not > >> appropriate to re-fold, so the range will remain unfolded. > >> > >>> > >>> 2. vma split in a large folio due to various reasons such as mprotect= , > >>> munmap, mlock etc. > >> > >> I'm not sure if PTEs are explicitly unmapped/remapped when splitting a= VMA? I > >> suspect not, so if the VMA is split in the middle of a currently folde= d contpte > >> block, it will remain folded. But this is safe and continues to work c= orrectly. > >> The VMA arrangement is not important; it is just important that a sing= le folio > >> is mapped contiguously across the whole block. > > > > I don't think it is safe to keep CONTPTE folded in a split_vma case. as > > otherwise, copy_ptes in your other patch might only copy a part > > of CONTPES. > > For example, if page0-page4 and page5-page15 are splitted in split_vma= , > > in fork, while copying pte for the first VMA, we are copying page0-page= 4, > > this will immediately cause inconsistent CONTPTE. as we have to > > make sure all CONTPTEs are atomically mapped in a PTL. > > No that's not how it works. The CONT_PTE bit is not blindly copied from p= arent > to child. It is explicitly managed by the arch code and set when appropri= ate. In > the case above, we will end up calling set_ptes() for page0-page4 in the = child. > set_ptes() will notice that there are only 5 contiguous pages so it will = map > without the CONT_PTE bit. Ok. cool. alternatively, in the code I shared to you, we are doing an unfol= d immediately when split_vma happens within a large anon folio, so we disallo= w CONTPTE to cross two VMAs to avoid all kinds of complexity afterwards. https://github.com/OnePlusOSS/android_kernel_oneplus_sm8550/blob/oneplus/sm= 8550_u_14.0.0_oneplus11/mm/huge_memory.c #ifdef CONFIG_CONT_PTE_HUGEPAGE void vma_adjust_cont_pte_trans_huge(struct vm_area_struct *vma, unsigned long start, unsigned long end, long adjust_next) { /* * If the new start address isn't hpage aligned and it could * previously contain an hugepage: check if we need to split * an huge pmd. */ if (start & ~HPAGE_CONT_PTE_MASK && (start & HPAGE_CONT_PTE_MASK) >=3D vma->vm_start && (start & HPAGE_CONT_PTE_MASK) + HPAGE_CONT_PTE_SIZE <=3D vma->= vm_end) split_huge_cont_pte_address(vma, start, false, NULL); .... } #endif In your approach, you are still holding CONTPTE crossing two VMAs. but it s= eems ok. I can't have a case which might fail in my brain right now. only running the code on a large amount of real hardware will tell :-) > > > > >> > >>> > >>> 3. try_to_unmap_one() to reclaim a folio, ptes are scanned one by one > >>> rather than being as a whole. > >> > >> Yes, as per 1; the arm64 implementation will notice when the first ent= ry is > >> cleared and unfold the contpte block. > >> > >>> > >>> In hardware, we need to make sure CONTPTE follow the rule - always 16 > >>> contiguous physical address with CONTPTE set. if one of them run away > >>> from the 16 ptes group and PTEs become unconsistent, some terrible > >>> errors/faults can happen in HW. for example > >> > >> Yes, the implementation obeys all these rules; see contpte_try_fold() = and > >> contpte_try_unfold(). the fold/unfold operation is only done when all > >> requirements are met, and we perform it in a manner that is conformant= to the > >> architecture requirements (see contpte_fold() - being renamed to > >> contpte_convert() in the next version). > >> > >> Thanks for the review! > >> > >> Thanks, > >> Ryan > >> > >>> > >>> case0: > >>> addr0 PTE - has no CONTPE > >>> addr0+4kb PTE - has CONTPTE > >>> .... > >>> addr0+60kb PTE - has CONTPTE > >>> > >>> case 1: > >>> addr0 PTE - has no CONTPE > >>> addr0+4kb PTE - has CONTPTE > >>> .... > >>> addr0+60kb PTE - has swap > >>> > >>> Unconsistent 16 PTEs will lead to crash even in the firmware based on > >>> our observation. > >>> > > Thanks Barry