From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADCF0C3ABC3 for ; Mon, 12 May 2025 10:22:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4FCC16B00E3; Mon, 12 May 2025 06:22:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4AB8B6B00E4; Mon, 12 May 2025 06:22:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 370AE6B00E5; Mon, 12 May 2025 06:22:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 172BF6B00E3 for ; Mon, 12 May 2025 06:22:57 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 49692E1608 for ; Mon, 12 May 2025 10:22:57 +0000 (UTC) X-FDA: 83433867594.01.702DD78 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf15.hostedemail.com (Postfix) with ESMTP id 6CCA1A000B for ; Mon, 12 May 2025 10:22:55 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747045375; a=rsa-sha256; cv=none; b=XGZZH5g6Kv67kPgZHAqnsmKiqn/CetuTiIQFqdSWEMQW7+/i/23+fbpaLC1lctA58OHF4o CqowvoQXDvxFf6VHsz3810p8KpW1tKLY7wE223dgEf8cBrhBO+ZJ9JxAJCxOLpafOMehyr UpLTT/ng9Zgkr0KKHEDaP8mBltxQJFY= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747045375; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references; bh=kS54aYfvY/Lc/pq+XKDAHStSOWbUNQbDoyfez/W8dLI=; b=kcBwMfJOjWWFwVghJMH39kPMJJ0hD7y2/rQC9D9M3XKxLVf2Pw7noHADkbY2w1XV137Zpp qyPE7A2wfEQ7pq6If+YcMUWlpI+KzKboqbulrCNVqsGzVyvaBn/71t2cVf+X4P3PfkpZOi 2nP2d87A6a6snvh8h0XuvGIcbU56vJ8= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5F05D150C; Mon, 12 May 2025 03:22:43 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5014F3F673; Mon, 12 May 2025 03:22:52 -0700 (PDT) From: Ryan Roberts To: Catalin Marinas , Will Deacon , Pasha Tatashin , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , David Hildenbrand , "Matthew Wilcox (Oracle)" , Mark Rutland , Anshuman Khandual , Alexandre Ghiti , Kevin Brodsky Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, syzbot+5c0d9392e042f41d45c5@syzkaller.appspotmail.com Subject: [PATCH] arm64/mm: Disable barrier batching in interrupt contexts Date: Mon, 12 May 2025 11:22:40 +0100 Message-ID: <20250512102242.4156463-1-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6CCA1A000B X-Rspam-User: X-Stat-Signature: efmnr3gd47g7nuyd4duezi64aphy5ciq X-HE-Tag: 1747045375-422020 X-HE-Meta: U2FsdGVkX1/TTMHr0BY8QE5QRTZABiXsmW00Vqzcs/WMqMMYWkeJWvLMjS4jGHae31KPXa7PmpR9lep7ME6zZrz01dOaidKPM4Hdo7fUOqH51hKH4G6UHBNQN1/4vQN54oxGuUNXp5k2LUqvLZfedfa5QGzblEkHCPuuHQppophu7dL6Nqw2s9cOXhDkEvFE2d97q32TteSaPzwgXGpCPakstbreStWsC26PWy1eCxXK/vCBfpD5iAGgiso3t3T5SlnSqbT2RXiAkW0yIozn0XH+Rx95TIudI/IxWIDUoBYo8ytveNMAwrr9iPvJsEXPQpaIty0daEreGS21TR67Ex37OgeDF/dRLJk6e1KKMXXjPH7Ay8vKFPG9lAm4eCNLtmUlSWZ0y3bFHu3lMz4tN26dcvQhYjxotNUvlyFl2cpA4d079hTD4Hdaw2ppcJ1VcIgkD3p6dTsoIvQteQBiZSPvl9Jc+DcfZCRFhS+bxQI774CuupEUn80/GPNvkeGGvjqFu4OVD0qNAArvrtQmOsbImnbuzDFTEjHjHdi6TNC0mM0cgVhZimpnxyKgwL0gcOKB/KwyUtfAE+BQQDoT9doGAUODyIsKE++uBC3fpoLnhoIKo+axt8ziA1XfQrirmfCk4xilpo/dG7DUs6jLd8tocCpv5XK+6PdMk+JxGYsk5z9MHc56ZMgTKWV/11BO1nJ9Qd5rJoViwiEyXPazEFAJcf3DShQ1yqZX/1kaC8KrNS/QsB65vkCqcSBh4kNE429SMpuTjyw5wDNU1uTe5HZGY/fV8UkSfV9H+uV7xyBM1SREfogPTMSUHtWw8pvg2GCg5AzNVE4mcWk3AgPYsKh3wX+nUWhrs6aMA11BQbIsEpUTnxkGVuLktnKwCkhA7b2RirKw+Xf1wmV0nsxeAi9g5JQrq7l+AmWnldVOTygXyXsCvYfUE9/7ZByO5Em/GIJCs3GHXgi8zCuKQtg 4KFFAaCm 63qPJjPC53pZWlPWgMf7CLoVZLhX76oHPLT37nXLL8ddSb/raXeuK6WvzpCebtuwlWf2msU7Jht2KILukiYckdwYbUaZiN2p9DLVmagjc7DaX7XM+9iVYnn4koHgqz0vjZFHZsCpvshjnpDZ56BlJyqRt52GltqkaqI1/xri/eUARzPZtA2DvgxKKp0RlleZR6hysh6YOOpnYY3rDuQepdpIiww3b+ye3ce0o1fg+DqOCAAFw85X3axrLeAFlCyoSWy8nfRtOniGrvbUdXv/a9+4jV7NTFFYlHvpcDf/Z7/dPxSqf/cqgY8kBd1fy5s6VSvs95r9FUAs3ykUMletzOELP66zs6CdcSp2YBCkOgva40WM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Commit 5fdd05efa1cd ("arm64/mm: Batch barriers when updating kernel mappings") enabled arm64 kernels to track "lazy mmu mode" using TIF flags in order to defer barriers until exiting the mode. At the same time, it added warnings to check that pte manipulations were never performed in interrupt context, because the tracking implementation could not deal with nesting. But it turns out that some debug features (e.g. KFENCE, DEBUG_PAGEALLOC) do manipulate ptes in softirq context, which triggered the warnings. So let's take the simplest and safest route and disable the batching optimization in interrupt contexts. This makes these users no worse off than prior to the optimization. Additionally the known offenders are debug features that only manipulate a single PTE, so there is no performance gain anyway. There may be some obscure case of encrypted/decrypted DMA with the dma_free_coherent called from an interrupt context, but again, this is no worse off than prior to the commit. Some options for supporting nesting were considered, but there is a difficult to solve problem if any code manipulates ptes within interrupt context but *outside of* a lazy mmu region. If this case exists, the code would expect the updates to be immediate, but because the task context may have already been in lazy mmu mode, the updates would be deferred, which could cause incorrect behaviour. This problem is avoided by always ensuring updates within interrupt context are immediate. Fixes: 5fdd05efa1cd ("arm64/mm: Batch barriers when updating kernel mappings") Reported-by: syzbot+5c0d9392e042f41d45c5@syzkaller.appspotmail.com Closes: https://lore.kernel.org/linux-arm-kernel/681f2a09.050a0220.f2294.0006.GAE@google.com/ Signed-off-by: Ryan Roberts --- Hi Will, I've tested before and after with KFENCE enabled and it solves the issue. I've also run all the mm-selftests which all continue to pass. Catalin suggested a Fixes patch targetting the SHA as it is in for-next/mm was the preferred approach, but shout if you want something different. I'm hoping that with this fix we can still make it for this cycle, subject to not finding any more issues. Thanks, Ryan arch/arm64/include/asm/pgtable.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index ab4a1b19e596..e65083ec35cb 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -64,7 +64,11 @@ static inline void queue_pte_barriers(void) { unsigned long flags; - VM_WARN_ON(in_interrupt()); + if (in_interrupt()) { + emit_pte_barriers(); + return; + } + flags = read_thread_flags(); if (flags & BIT(TIF_LAZY_MMU)) { @@ -79,7 +83,9 @@ static inline void queue_pte_barriers(void) #define __HAVE_ARCH_ENTER_LAZY_MMU_MODE static inline void arch_enter_lazy_mmu_mode(void) { - VM_WARN_ON(in_interrupt()); + if (in_interrupt()) + return; + VM_WARN_ON(test_thread_flag(TIF_LAZY_MMU)); set_thread_flag(TIF_LAZY_MMU); @@ -87,12 +93,18 @@ static inline void arch_enter_lazy_mmu_mode(void) static inline void arch_flush_lazy_mmu_mode(void) { + if (in_interrupt()) + return; + if (test_and_clear_thread_flag(TIF_LAZY_MMU_PENDING)) emit_pte_barriers(); } static inline void arch_leave_lazy_mmu_mode(void) { + if (in_interrupt()) + return; + arch_flush_lazy_mmu_mode(); clear_thread_flag(TIF_LAZY_MMU); } -- 2.43.0