From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD5C6C47258 for ; Thu, 25 Jan 2024 16:45:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 64FAF6B0098; Thu, 25 Jan 2024 11:45:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D8EB8D0002; Thu, 25 Jan 2024 11:45:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 453146B00AB; Thu, 25 Jan 2024 11:45:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 30ACD6B0098 for ; Thu, 25 Jan 2024 11:45:37 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id EE8EE1C1721 for ; Thu, 25 Jan 2024 16:45:36 +0000 (UTC) X-FDA: 81718409472.14.C507AAE Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf05.hostedemail.com (Postfix) with ESMTP id 3861C100020 for ; Thu, 25 Jan 2024 16:45:35 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of alexandru.elisei@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=alexandru.elisei@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706201135; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O83xQ4m/aJM+qp7cmwtNg+48CMmBJybDD269x+AFojc=; b=dcy6gf6kWDNaeGRrbJRzouA0+qJiVEkS9g/xz8+OYtQsoId8DOwG0kn5aUPxHCVOjkAW/E lwbmN0Z9hUOu2N5XKKTygkTR4GupnTg1pal11kzkHgN0lzpgrSLviAd3PQ7YvF6Ed2POz/ 6GLnXAU0Qc8tYgfsBOjlSft8pxFdCXw= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of alexandru.elisei@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=alexandru.elisei@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706201135; a=rsa-sha256; cv=none; b=yphkySWZ4mqQAhsinqJQTW1JRSkiCpCMidSoSQuyTKYSp2lusCHXMDRWSFYysFlaiM5jBg abK3bsGRYMWRKWQWEkMZ0miljn37k/3SgmjH+fqpJdcUKmG7VDlNK74kiayynErkVt3Rra gFTt5AvvwsT46RoBqXH9Wh4ErkF8kqA= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0743516F3; Thu, 25 Jan 2024 08:46:19 -0800 (PST) Received: from e121798.cable.virginm.net (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1E2EF3F5A1; Thu, 25 Jan 2024 08:45:28 -0800 (PST) From: Alexandru Elisei To: catalin.marinas@arm.com, will@kernel.org, oliver.upton@linux.dev, maz@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, arnd@arndb.de, akpm@linux-foundation.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mhiramat@kernel.org, rppt@kernel.org, hughd@google.com Cc: pcc@google.com, steven.price@arm.com, anshuman.khandual@arm.com, vincenzo.frascino@arm.com, david@redhat.com, eugenis@google.com, kcc@google.com, hyesoo.yu@samsung.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH RFC v3 29/35] arm64: mte: copypage: Handle tag restoring when missing tag storage Date: Thu, 25 Jan 2024 16:42:50 +0000 Message-Id: <20240125164256.4147-30-alexandru.elisei@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240125164256.4147-1-alexandru.elisei@arm.com> References: <20240125164256.4147-1-alexandru.elisei@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 3861C100020 X-Stat-Signature: d7azxybhiz9ybdye7ms59tho6tgdi346 X-Rspam-User: X-HE-Tag: 1706201135-527137 X-HE-Meta: U2FsdGVkX1/AsqldcD6bkORTrJ+wGwi8+Cy794uZ5vnwono8Ef5zM4fjaHzChuvUFlD5wnzLQKhgdCsCu/dHqcdQ0i74rFzQjOiQz1Pq/3P7BEJOCye7v1EW13jKSDcOhBpt9x1gFAqNrpJ0XdKkC/LG8KY+VEWwe2s5tEBb4CH4OqH4Qn2iNp8wzSbWoRksb2hTqWQuqQAVZj0rTBYavN5HCMksGEEhbTHGjcQ/gdZukTXAIN3qJsm8z77Qvis1IfZAp8obp4feOZW+r4h3o8Q6txAjbUivZ6fAnsSptuLhZ+KAa8OedZvsQTrB5ludKWaKeIFu8LFmDeeCwk49Fwgh3/MAJoZTwLZpAzI1yQKieH5c3XL3WPQpUZXSxScRh/Vw/dqCXFYzB0/2ttbEMhvq4g7buyjfIB1f4myDOx0z1qomOUFv9IgGIa3k1o60TAoO0L6APwaeW1lXsPEWz6Dz8SwZrcQwjHxCklRpWyRAZcJRZKSXz3UKHesG/ldEcg3Bljg+zPArPDx8AgVlz8XE4+QlwWLeDsQeD0d/j7hDEOwGwFAOj5NiLi2b9Xj7c5dnQrdUDiCSUcZWXpDCjG40koBAdhnyy9zwqVoKccn0Ycdicd+ABXJpRE9cfuh6azG8e87HnqEsn99CKO4J2PTimGSfLB3rLVhIZnomPtMOwFyz3jxhrXoGRLQPIysyKhLNrJMBKWOcCYo9wRTZ21mMhf5pS/Jjg5EsWO8rMvOa69tjzHGhKhi8a2TewhFha79aqPHCKkvcNv/zUtB3lXYUiNF6TJ1HJS3uJiiy6T5X7ZNHpkXNhyxbyei1cYEH+MB3/pjrbwSLTzzO+5jjpF+/bLp57COAMCQeLsONQGQFVexUNOVwaITFoFIMKS7CCj1LialHZKGsVnt0apj16f0vNMSU2JYEharZfQjrbyuRAUlOqMnRx+5/Ep5RVHtXtChpWoMam4tDUgHY+rf KwH5jCl9 8zPaK+31Rzq4ht5mmPWHe64XdBcGlaZ8Gx3VUrgJ8dLRen+FEq2Mbgxc5b5MAiSBCGiLRXpH6gGpKnc+AtARjjM6dCfCc5tAo2w+R9kmY4HoL8/kvJt91xbMVzW3elejxtQW6x4sjhuii5fXewI221R+o8q4+h1bYHX9DRKVvlT9b7OGGwjMb2zFDtQ6GDlNhc9LC X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are several situations where copy_highpage() can end up copying tags to a page which doesn't have its tag storage reserved. One situation involves migration racing with mprotect(PROT_MTE): VMA is initially untagged, migration starts and destination page is allocated as untagged, mprotect(PROT_MTE) changes the VMA to tagged and userspace accesses the source page, thus making it tagged. The migration code then calls copy_highpage(), which will copy the tags from the source page (now tagged) to the destination page (allocated as untagged). Yes another situation can happen during THP collapse. The huge page that will replace the HPAGE_PMD_NR contiguous mapped pages is allocated with __GFP_TAGGED not set. copy_highpage() will copy the tags from the pages being replaced to the huge page which doesn't have tag storage reserved. The situation gets even more complicated when the replacement huge page is a tag storage page. The tag storage huge page will be migrated after a fault on access, but the tags from the original pages must be copied over to the huge page that will be replacing the tag storage huge page. Signed-off-by: Alexandru Elisei --- arch/arm64/mm/copypage.c | 56 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/arch/arm64/mm/copypage.c b/arch/arm64/mm/copypage.c index a7bb20055ce0..e991ccb43fb7 100644 --- a/arch/arm64/mm/copypage.c +++ b/arch/arm64/mm/copypage.c @@ -13,6 +13,59 @@ #include #include #include +#include + +#ifdef CONFIG_ARM64_MTE_TAG_STORAGE +static inline bool try_transfer_saved_tags(struct page *from, struct page *to) +{ + void *tags; + bool saved; + + VM_WARN_ON_ONCE(!preemptible()); + + if (page_mte_tagged(from)) { + if (page_tag_storage_reserved(to)) + return false; + + tags = mte_allocate_tag_buf(); + if (WARN_ON(!tags)) + return true; + + mte_copy_page_tags_to_buf(page_address(from), tags); + saved = mte_save_tags_for_pfn(tags, page_to_pfn(to)); + if (!saved) + mte_free_tag_buf(tags); + + return saved; + } + + tags_by_pfn_lock(); + tags = mte_erase_tags_for_pfn(page_to_pfn(from)); + tags_by_pfn_unlock(); + + if (likely(!tags)) + return false; + + if (page_tag_storage_reserved(to)) { + WARN_ON_ONCE(!try_page_mte_tagging(to)); + mte_copy_page_tags_from_buf(page_address(to), tags); + set_page_mte_tagged(to); + mte_free_tag_buf(tags); + return true; + } + + saved = mte_save_tags_for_pfn(tags, page_to_pfn(to)); + if (!saved) + mte_free_tag_buf(tags); + + return saved; +} +#else +static inline bool try_transfer_saved_tags(struct page *from, struct page *to) +{ + return false; +} +#endif void copy_highpage(struct page *to, struct page *from) { @@ -24,6 +77,9 @@ void copy_highpage(struct page *to, struct page *from) if (kasan_hw_tags_enabled()) page_kasan_tag_reset(to); + if (tag_storage_enabled() && try_transfer_saved_tags(from, to)) + return; + if (system_supports_mte() && page_mte_tagged(from)) { /* It's a new page, shouldn't have been tagged yet */ WARN_ON_ONCE(!try_page_mte_tagging(to)); -- 2.43.0