From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8739C4167B for ; Tue, 28 Nov 2023 11:53:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5FDEF6B0302; Tue, 28 Nov 2023 06:53:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5AE586B0304; Tue, 28 Nov 2023 06:53:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 49D8C6B0306; Tue, 28 Nov 2023 06:53:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 380E66B0302 for ; Tue, 28 Nov 2023 06:53:00 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0F6F0A0199 for ; Tue, 28 Nov 2023 11:53:00 +0000 (UTC) X-FDA: 81507201720.03.65AAD4C Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf13.hostedemail.com (Postfix) with ESMTP id 409AA20007 for ; Tue, 28 Nov 2023 11:52:58 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf13.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701172378; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BLrVwSbuBFPb8E8WOxCPBl4ryH3MDrpEoMlSFtN1Djc=; b=NslMUUK3iOKytHtEN8O4bjILBBvb/zhxdjMitdHx+iz6oZGAumv2xDyD++DhwZfSamc0cT 8axZIz3vq44DAn289UN4GK9LkUEvV7xsMazFgoV2y00aOAD+bJqa0hQ8cDaQtWYKOGdmvo 0jVSh8B2gU2Xm6Ww8MRN+TVzAehAlZw= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf13.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701172378; a=rsa-sha256; cv=none; b=AVhxohMyA9YiO9Z5GdDZ/a2CbOwM3X1l6y5J4ccKyXXuIJDs56iGW24G79qq/DLJ4YLkwP LPlTkfOzLSMecELn93y8Z7gozj2xm94ejI4Ja0simQq0z5RTDNIgc/yauVONT73K6qmVdg NI6NJSruBoeChHqfR7vseJZXBNj0zeA= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D554BC15; Tue, 28 Nov 2023 03:53:44 -0800 (PST) Received: from [10.1.33.188] (XHFQ2J9959.cambridge.arm.com [10.1.33.188]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 45B893F73F; Tue, 28 Nov 2023 03:52:54 -0800 (PST) Message-ID: <1537e042-1e73-4920-a1db-1eb32d66e32d@arm.com> Date: Tue, 28 Nov 2023 11:52:53 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 00/14] Transparent Contiguous PTEs for User Mappings Content-Language: en-GB To: Barry Song <21cnbao@gmail.com> Cc: akpm@linux-foundation.org, andreyknvl@gmail.com, anshuman.khandual@arm.com, ardb@kernel.org, catalin.marinas@arm.com, david@redhat.com, dvyukov@google.com, glider@google.com, james.morse@arm.com, jhubbard@nvidia.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, mark.rutland@arm.com, maz@kernel.org, oliver.upton@linux.dev, ryabinin.a.a@gmail.com, suzuki.poulose@arm.com, vincenzo.frascino@arm.com, wangkefeng.wang@huawei.com, will@kernel.org, willy@infradead.org, yuzenghui@huawei.com, yuzhao@google.com, ziy@nvidia.com References: <20231115163018.1303287-1-ryan.roberts@arm.com> <20231127031813.5576-1-v-songbaohua@oppo.com> <234021ba-73c2-474a-82f9-91e1604d5bb5@arm.com> From: Ryan Roberts In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 409AA20007 X-Stat-Signature: ue6z54rodp76t7czy7xoe41ji8izrsth X-HE-Tag: 1701172378-644647 X-HE-Meta: U2FsdGVkX18j9PUescek7m/kOCsmUlp98VosWJ5764pLNIxLvCex87KI400L3qRZQ9v33mYcIxjCushpJlSwILDkvuqOpeoAjtYs9wctYiL94SKNqwrc59aoWO7NFQeSI8NkUt+m4T1GT6/vgUSdqbeQWnfrRk4hho5jA4VM2nZCnY3nVc1VGO0M/ztHbZWHxfH5La+8rWZYvPT2DI2ulN1dwkgDf/zNkp7svBJAt8CdPQdwa2/Zu62MlAjK2MW1Mw2ZI6bVXP0bthevWdtVUd4wZ69xfgo3Ow9xTx8b/Y/MNmMxVPPRn3sm+lNNwcF+Y4uFnzXi6fl5feIpzFKwFz0rZVJ9v0DrGVdOa5YSC7xuATRMwoF91qIJkCchVhJOF3wdpPEqEgu35APnAH+3O89R7EIL5JKVDzpuh5f1u/53In7cDn0lYqiaqYl9uTy0TjsZcaU3MWILbf/Li3THxoGh60Ye9LTYljKRCkn22St7xaz99R2OYcDRCFOhcP4sQJQGroa7UMmnjT5COsZx7sW2z10epBavkx8emb9BFFk7JZtWI34WKhptV8Ql8rY1V8P4Q4Wadu4BGlEEaNzgedtwkXJd5qZKwJd3xjcBU7cqL7HNBBJ+A887pdqLVsifw21cHwwguf7bn2wB3wFQ8v2Es393bZ7kDdMg6bImPA/PcI/VPjKAT1DRSUD5kG1oD0QwM9fspFvheGPMa2jY5dwmOZkf/Wxk1n51bV1CEs2VRsbG0Gisi6EkDl57j5ZsxvJoIj9V4Q0QSV1bCNSShXwrss8bN1QDSC0kh9e3p5kH5wJJz/vavBNmKJbJombaoIOFNI8zEPuJZOs4mzi8kbpRaxLkUGLrizCsVaomMo5ngDEwCKRCFg0zpw40dv8I8+7bYKbeoQOxBExvecg1Xc+xB2XDZm9vDIjBRgVUSD471YgtOIZ1uCzaaprX9XP3qAOjeIhw2wK1rKL0a5v B6Ai+Fzg FZlcWHPDX25onujsCoBGLbxTHH3/d0+azbu7nLTmD0D6FMPsyedV+cX6cRXR9Grm7X2rFNbdrILyFX+ZuyQW/TcBmWcYtQ2p8kukZxpQsR7IESZtz3kjkjjmBqarRFj+oNZgMk2nHjf5WFGGnwjcmVVpU3mwItKC6Ay5KxUQ2eYX1CKpW3liuoO20Dgjd8590ERTGdz4vVtgbRZpdRva1rXEffS3Y1UjJfRTDJbRecVMBzAQG1Fk4FVzZDz9tWV3egWfX+60Yq6N6Y06cHrGv8CdZo6jvhop0ZDQhiHRsTh6dG/w= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 27/11/2023 22:53, Barry Song wrote: > On Tue, Nov 28, 2023 at 12:11 AM Ryan Roberts wrote: >> >> On 27/11/2023 10:35, Barry Song wrote: >>> On Mon, Nov 27, 2023 at 10:15 PM Ryan Roberts wrote: >>>> >>>> On 27/11/2023 03:18, Barry Song wrote: >>>>>> Ryan Roberts (14): >>>>>> mm: Batch-copy PTE ranges during fork() >>>>>> arm64/mm: set_pte(): New layer to manage contig bit >>>>>> arm64/mm: set_ptes()/set_pte_at(): New layer to manage contig bit >>>>>> arm64/mm: pte_clear(): New layer to manage contig bit >>>>>> arm64/mm: ptep_get_and_clear(): New layer to manage contig bit >>>>>> arm64/mm: ptep_test_and_clear_young(): New layer to manage contig bit >>>>>> arm64/mm: ptep_clear_flush_young(): New layer to manage contig bit >>>>>> arm64/mm: ptep_set_wrprotect(): New layer to manage contig bit >>>>>> arm64/mm: ptep_set_access_flags(): New layer to manage contig bit >>>>>> arm64/mm: ptep_get(): New layer to manage contig bit >>>>>> arm64/mm: Split __flush_tlb_range() to elide trailing DSB >>>>>> arm64/mm: Wire up PTE_CONT for user mappings >>>>>> arm64/mm: Implement ptep_set_wrprotects() to optimize fork() >>>>>> arm64/mm: Add ptep_get_and_clear_full() to optimize process teardown >>>>> >>>>> Hi Ryan, >>>>> Not quite sure if I missed something, are we splitting/unfolding CONTPTES >>>>> in the below cases >>>> >>>> The general idea is that the core-mm sets the individual ptes (one at a time if >>>> it likes with set_pte_at(), or in a block with set_ptes()), modifies its >>>> permissions (ptep_set_wrprotect(), ptep_set_access_flags()) and clears them >>>> (ptep_clear(), etc); This is exactly the same interface as previously. >>>> >>>> BUT, the arm64 implementation of those interfaces will now detect when a set of >>>> adjacent PTEs (a contpte block - so 16 naturally aligned entries when using 4K >>>> base pages) are all appropriate for having the CONT_PTE bit set; in this case >>>> the block is "folded". And it will detect when the first PTE in the block >>>> changes such that the CONT_PTE bit must now be unset ("unfolded"). One of the >>>> requirements for folding a contpte block is that all the pages must belong to >>>> the *same* folio (that means its safe to only track access/dirty for thecontpte >>>> block as a whole rather than for each individual pte). >>>> >>>> (there are a couple of optimizations that make the reality slightly more >>>> complicated than what I've just explained, but you get the idea). >>>> >>>> On that basis, I believe all the specific cases you describe below are all >>>> covered and safe - please let me know if you think there is a hole here! >>>> >>>>> >>>>> 1. madvise(MADV_DONTNEED) on a part of basepages on a CONTPTE large folio >>>> >>>> The page will first be unmapped (e.g. ptep_clear() or ptep_get_and_clear(), or >>>> whatever). The implementation of that will cause an unfold and the CONT_PTE bit >>>> is removed from the whole contpte block. If there is then a subsequent >>>> set_pte_at() to set a swap entry, the implementation will see that its not >>>> appropriate to re-fold, so the range will remain unfolded. >>>> >>>>> >>>>> 2. vma split in a large folio due to various reasons such as mprotect, >>>>> munmap, mlock etc. >>>> >>>> I'm not sure if PTEs are explicitly unmapped/remapped when splitting a VMA? I >>>> suspect not, so if the VMA is split in the middle of a currently folded contpte >>>> block, it will remain folded. But this is safe and continues to work correctly. >>>> The VMA arrangement is not important; it is just important that a single folio >>>> is mapped contiguously across the whole block. >>> >>> I don't think it is safe to keep CONTPTE folded in a split_vma case. as >>> otherwise, copy_ptes in your other patch might only copy a part >>> of CONTPES. >>> For example, if page0-page4 and page5-page15 are splitted in split_vma, >>> in fork, while copying pte for the first VMA, we are copying page0-page4, >>> this will immediately cause inconsistent CONTPTE. as we have to >>> make sure all CONTPTEs are atomically mapped in a PTL. >> >> No that's not how it works. The CONT_PTE bit is not blindly copied from parent >> to child. It is explicitly managed by the arch code and set when appropriate. In >> the case above, we will end up calling set_ptes() for page0-page4 in the child. >> set_ptes() will notice that there are only 5 contiguous pages so it will map >> without the CONT_PTE bit. > > Ok. cool. alternatively, in the code I shared to you, we are doing an unfold > immediately when split_vma happens within a large anon folio, so we disallow > CONTPTE to cross two VMAs to avoid all kinds of complexity afterwards. > > https://github.com/OnePlusOSS/android_kernel_oneplus_sm8550/blob/oneplus/sm8550_u_14.0.0_oneplus11/mm/huge_memory.c > > #ifdef CONFIG_CONT_PTE_HUGEPAGE > void vma_adjust_cont_pte_trans_huge(struct vm_area_struct *vma, > unsigned long start, > unsigned long end, > long adjust_next) > { > /* > * If the new start address isn't hpage aligned and it could > * previously contain an hugepage: check if we need to split > * an huge pmd. > */ > if (start & ~HPAGE_CONT_PTE_MASK && > (start & HPAGE_CONT_PTE_MASK) >= vma->vm_start && > (start & HPAGE_CONT_PTE_MASK) + HPAGE_CONT_PTE_SIZE <= vma->vm_end) > split_huge_cont_pte_address(vma, start, false, NULL); > > .... > } > #endif > > In your approach, you are still holding CONTPTE crossing two VMAs. but it seems > ok. I can't have a case which might fail in my brain right now. only Yes, I'm dealing with the CONT_PTE bit at the pgtable level, not at the VMA level. > running the code on > a large amount of real hardware will tell :-) Indeed - is this something you might be able to help with? :) > >> >>> >>>> >>>>> >>>>> 3. try_to_unmap_one() to reclaim a folio, ptes are scanned one by one >>>>> rather than being as a whole. >>>> >>>> Yes, as per 1; the arm64 implementation will notice when the first entry is >>>> cleared and unfold the contpte block. >>>> >>>>> >>>>> In hardware, we need to make sure CONTPTE follow the rule - always 16 >>>>> contiguous physical address with CONTPTE set. if one of them run away >>>>> from the 16 ptes group and PTEs become unconsistent, some terrible >>>>> errors/faults can happen in HW. for example >>>> >>>> Yes, the implementation obeys all these rules; see contpte_try_fold() and >>>> contpte_try_unfold(). the fold/unfold operation is only done when all >>>> requirements are met, and we perform it in a manner that is conformant to the >>>> architecture requirements (see contpte_fold() - being renamed to >>>> contpte_convert() in the next version). >>>> >>>> Thanks for the review! >>>> >>>> Thanks, >>>> Ryan >>>> >>>>> >>>>> case0: >>>>> addr0 PTE - has no CONTPE >>>>> addr0+4kb PTE - has CONTPTE >>>>> .... >>>>> addr0+60kb PTE - has CONTPTE >>>>> >>>>> case 1: >>>>> addr0 PTE - has no CONTPE >>>>> addr0+4kb PTE - has CONTPTE >>>>> .... >>>>> addr0+60kb PTE - has swap >>>>> >>>>> Unconsistent 16 PTEs will lead to crash even in the firmware based on >>>>> our observation. >>>>> >>> > > Thanks > Barry