From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BE3CC77B7C for ; Thu, 11 May 2023 12:59:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 669656B007B; Thu, 11 May 2023 08:59:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 61A226B007D; Thu, 11 May 2023 08:59:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 509966B007E; Thu, 11 May 2023 08:59:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 40A386B007B for ; Thu, 11 May 2023 08:59:12 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id EF9261A086C for ; Thu, 11 May 2023 12:59:11 +0000 (UTC) X-FDA: 80777979702.07.F6D64BB Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id D961D80009 for ; Thu, 11 May 2023 12:59:08 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683809949; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references; bh=9+PfJTehcmlWlYRsYJFPvFBtSHXC9eHCVpIu3zhE+M0=; b=Ki97oRel0qNkdQuIUyWRA860Fjnq6Y0l+whX3z0ko/l5s0irBj9fEziBBy5GYRLgiYV6UD pnfA21QbVVZIuuDLatbjJ668UROR20WGDZeihPvm/JE2wQTWSH3Jnux3Rl7OEvd25oyNFM ZFGrpxp4QUUR1n4lfLiwMtiBX3UkyXk= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683809949; a=rsa-sha256; cv=none; b=BtNaycXZw6CyA2XSITkOaYXfdoF6jKheMizHBMrrcPwbz7oTTnopQwfyJDQJR9B+raZb5F WVrVHCkfGoM2NK1a+YmlOW0VWhxp58aYlNAyh/GG8ZQwRr/Wd0LrGdQVPtmU6UGtlUwDMN roeD+A9LAAuzFCN040hiw8IzJ0vSahU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1CFA4113E; Thu, 11 May 2023 05:59:52 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 74F303F5A1; Thu, 11 May 2023 05:59:06 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , SeongJae Park Cc: Ryan Roberts , linux-kernel@vger.kernel.org, linux-mm@kvack.org, damon@lists.linux.dev Subject: Date: Thu, 11 May 2023 13:58:43 +0100 Message-Id: <20230511125848.78621-1-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Stat-Signature: erh9wu4pwwrcowr7je9gakkkrub8nudh X-Rspam-User: X-Rspamd-Queue-Id: D961D80009 X-Rspamd-Server: rspam07 X-HE-Tag: 1683809948-970719 X-HE-Meta: U2FsdGVkX18Y9SpztnTubszGC4KHKiuRvvIsMJSsMJ2v5C6GCtcgASBC0z/6rBY0jqc0PH4JBwDRPZ1eOxR6ueg8E/Rs/y6iBly0ChwredZvrVPwzMeiNxGF2/eeHfA+HvoqekBSbtDlyU+EUgop5msbqN+ozb2L2wneNzfnZgX7A/sBTKUY+febkwqLluW5NCMZm2257Rbuk+Oy8OuOCuyARS57bGz/Ntsew+FY+dECsG8JiARye+aDM2T2or525UU0CBYOrA5auNjktI4FDV+KxkvhAIR7cBMHlP6XrkIjgITom0S9k6flZ/FrCFs8K/JZcCcCEiOcXDMA5VUYaWecE5rU+I4X7EzRIMXF5RAIKAkOanWP4FcDhx7WFHMnwo+pEK+x+508O5QquRiZYcJOfdRp3zo2gJ+5b8hnUO0Axr0XPXtqZwA5L42qBvZIuPLkjKIfYQ/Bf4seME94jhkrkc7xFhVa5EmfXoAVAH2IlU81UMtElianYkBFULEfhucBLhqDz/PRVJFhaOTNA6sYphlv3bUUuo/z6D58y7IOvFYLHc8mMSXi9Ld4E+obEZLxRMM7iTmrca9H7l4Ie2or+BmWuoTpeFRxgumUnQ+P123FmTs+see92q54JFQ6TBUkCGbJDSd2Njfrv9y/yqmxSdP9JDpP4L7bSMA0IoHwkrIY1Z1EK71V73Zj5mCJr2qhYWa4/1VDix+uWAVKImWNm7/9OTFxwHrgIR+I0S3bIDGrMXO/yxB7+pmpQISTOC+4nLtCMdvN9RHinTFRSqCgeTuUbeMz0bSiO74nROYqad01X6IZ8l1Us+P3PSm60GWY3GwW0cTgcSc9yZO3bpD78Lg8iU5/5s+OhyDwtYFSW8ZxbLjnVFfg6xtRC+cdh25ZdlvvvjQof1Y7goZFDdHTqDMxouKr+wjW6OH8TItLcTBUNVkkl/HtlJvRpoVYBF++RKq1oGrmuHRIZXU M1+rNcgD 8rkAVzc6ju6aevcxL2MHscWMHvfi6YkbNmODsQn4gGSw4/Zbsn0Fzc893sIxjpkK7r4DzIPiWogvwFMqrehVv9cXMJjYez7lfYjoZb6Hnyp9gi1c1wSBKDBkKaq2PHuZCfV3oQzJXnOvEe8Hcwi43C7sLrAKLCarpcvBR3K7FFQGTTJC0TFULh8fJFDXHAKl/nRPWDSljW1j0M+mZCVXbosaNIWkfbBPNIPE0GvQoOh4MaTG2oVOqSuOYGauPPpNokhJT+kTqcOfcwmu3oP3jsWWpmqk1jMjdmSRp X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Date: Thu, 11 May 2023 11:38:28 +0100 Subject: [PATCH v1 0/5] Encapsulate PTE contents from non-arch code Hi All, This series improves the encapsulation of pte entries by disallowing non-arch code from directly dereferencing pte_t pointers. Instead code must use a new helper, `pte_t ptep_deref(pte_t *ptep)`. By default, this helper does a direct dereference of the pointer, so generated code should be exactly the same. But it's presence sets us up for arch code being able to override the default to "virtualize" the ptes without needing to maintain a shadow table. I intend to take advantage of this for arm64 to enable use of its "contiguous bit" to coalesce multiple ptes into a single tlb entry, reducing pressure and improving performance. I have an RFC for the first part of this work at [1]. The cover letter there also explains the second part, which this series is enabling. I intend to post an RFC for the contpte changes in due course, but it would be good to get the ball rolling on this enabler. There are 2 reasons that I need the encapsulation: - Prevent leaking the arch-private PTE_CONT bit to the core code. If the core code reads a pte that contains this bit, it could end up calling set_pte_at() with the bit set which would confuse the implementation. So we can always clear PTE_CONT in ptep_deref() (and ptep_get()) to avoid a leaky abstraction. - Contiguous ptes have a single access and dirty bit for the contiguous range. So we need to "mix-in" those bits when the core is dereferencing a pte that lies in the contig range. There is code that dereferences the pte then takes different actions based on access/dirty (see e.g. write_protect_page()). While ptep_get() and ptep_get_lockless() already exist, both of them are implemented using READ_ONCE() by default. While we could use ptep_get() instead of the new ptep_deref(), I didn't want to risk performance regression. Alternatively, all call sites that currently use ptep_get() that need the lockless behaviour could be upgraded to ptep_get_lockless() and ptep_get() could be downgraded to a simple dereference. That would be cleanest, but is a much bigger (and likely error prone) change because all the arch code would need to be updated for the new definitions of ptep_get(). The series is split up as follows: patchs 1-2: Fix bugs where code was _setting_ ptes directly, rather than using set_pte_at() and friends. patch 3: Fix highmem unmapping issue I spotted while doing the work. patch 4: Introduce the new ptep_deref() helper with default implementation. patch 5: Convert all direct dereferences to use ptep_deref(). [1] https://lore.kernel.org/linux-mm/20230414130303.2345383-1-ryan.roberts@arm.com/ Thanks, Ryan Ryan Roberts (5): mm: vmalloc must set pte via arch code mm: damon must atomically clear young on ptes and pmds mm: Fix failure to unmap pte on highmem systems mm: Add new ptep_deref() helper to fully encapsulate pte_t mm: ptep_deref() conversion .../drm/i915/gem/selftests/i915_gem_mman.c | 8 +- drivers/misc/sgi-gru/grufault.c | 2 +- drivers/vfio/vfio_iommu_type1.c | 7 +- drivers/xen/privcmd.c | 2 +- fs/proc/task_mmu.c | 33 +++--- fs/userfaultfd.c | 6 +- include/linux/hugetlb.h | 2 +- include/linux/mm_inline.h | 2 +- include/linux/pgtable.h | 13 ++- kernel/events/uprobes.c | 2 +- mm/damon/ops-common.c | 18 ++- mm/damon/ops-common.h | 4 +- mm/damon/paddr.c | 6 +- mm/damon/vaddr.c | 14 ++- mm/filemap.c | 2 +- mm/gup.c | 21 ++-- mm/highmem.c | 12 +- mm/hmm.c | 2 +- mm/huge_memory.c | 4 +- mm/hugetlb.c | 2 +- mm/hugetlb_vmemmap.c | 6 +- mm/kasan/init.c | 9 +- mm/kasan/shadow.c | 10 +- mm/khugepaged.c | 24 ++-- mm/ksm.c | 22 ++-- mm/madvise.c | 6 +- mm/mapping_dirty_helpers.c | 4 +- mm/memcontrol.c | 4 +- mm/memory-failure.c | 6 +- mm/memory.c | 103 +++++++++--------- mm/mempolicy.c | 6 +- mm/migrate.c | 14 ++- mm/migrate_device.c | 14 ++- mm/mincore.c | 2 +- mm/mlock.c | 6 +- mm/mprotect.c | 8 +- mm/mremap.c | 2 +- mm/page_table_check.c | 4 +- mm/page_vma_mapped.c | 26 +++-- mm/pgtable-generic.c | 2 +- mm/rmap.c | 32 +++--- mm/sparse-vmemmap.c | 8 +- mm/swap_state.c | 4 +- mm/swapfile.c | 16 +-- mm/userfaultfd.c | 4 +- mm/vmalloc.c | 11 +- mm/vmscan.c | 14 ++- virt/kvm/kvm_main.c | 9 +- 48 files changed, 302 insertions(+), 236 deletions(-) -- 2.25.1