From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40C69C61D85 for ; Wed, 22 Nov 2023 01:08:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9B8C6B0523; Tue, 21 Nov 2023 20:08:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AFCCA6B0525; Tue, 21 Nov 2023 20:08:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C5476B0526; Tue, 21 Nov 2023 20:08:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8A3906B0523 for ; Tue, 21 Nov 2023 20:08:21 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6352CB5D6C for ; Wed, 22 Nov 2023 01:08:21 +0000 (UTC) X-FDA: 81483804402.26.62374A6 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf27.hostedemail.com (Postfix) with ESMTP id 9064F40010 for ; Wed, 22 Nov 2023 01:08:19 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="mQy/FkxA"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf27.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700615299; a=rsa-sha256; cv=none; b=4MmsP/3ScNJJNPbEAVIopC+Hl1riKzjuJ0dzQ9+FXDD3gz3KhjMPILj8G7VVvcQmfOea1I hGmt2dq4YQVyjG6/fzKdKoEnmmAoeh59O33uzNAvjywQgwH//ZE3wHex7i8kG1FYRKEk3X Cbw3gTybuBVbL42WYpgQQEpqLdv51Q4= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="mQy/FkxA"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf27.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700615299; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/6X+mOMZoGJuf/KhFjronMx0sFEsDU8bO3mA0YdbfIg=; b=c2gMZgnFoO/vngeDYLzJzEvatDxIVJGzwdMkdvEX4zIDp0eiSBLRv1rlrJuAoI9h3wals0 3VuloG7XGYbJfyLMFi5HosPgz5JKNtbr6nSOA2jrMEDzjyZgp5RZ38BWoMNLSdBJXUia98 sC8E+FHRNIayC/wrSt8dXlRthtsEkMg= Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-6c39ad730aaso5094507b3a.0 for ; Tue, 21 Nov 2023 17:08:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1700615298; x=1701220098; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/6X+mOMZoGJuf/KhFjronMx0sFEsDU8bO3mA0YdbfIg=; b=mQy/FkxAiftK/O9lFSeGaJzSklfNbVj59N0GfHgioeHp0iuZpeolNfQ0lJsjCJFk2j RxSWKnhXsWcC0aPinb4C/dItnvAetuNgAUvSpQEPoEu6u+c58F/6jfr/4cXtLRwYR1Ku iuXi4cu+R7JEYD33hn8fXY8YvZKwAzEL0nBJ8VcRMZ/dKJRodjAOkhTtv1NOaJkB0V1f O+dbTwXbxVyg8rg3aIi4MiSzClNTHDDWDJNE634tP87hw+cebsLlbp2T2+IWZyoJ6n1k GFZsnm/GUrPVE5ZcLvR7RdnP7wYKM6NLmMmb1M/qwR62PWEcKpqlGzpr2uga9A2dSa++ UKTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700615298; x=1701220098; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/6X+mOMZoGJuf/KhFjronMx0sFEsDU8bO3mA0YdbfIg=; b=U2BrcwnF/kStpiKSplaPXiwbau2uSckYk93MV1ZmflMbT0eusWMzhFjxlQyydwa5Ih 8GazFqF1e65JpI4zfgD358pE8bXoF1ylIS3kE6WX9V6C0VdKHF3slp3QgJv45INjGXNB vwMQJG90oRXI1jQfU5bgb/mx5pzgA9oOQN85C7XmQCvtl0/qAAE6WRz1jH96zdrt9bhW DSf8LrFK+PD6erJ6dtJFR6QSRNfaGjzSu6wC6MpxsNt6xGSV94R0rsFqLOPY3gqRr0GZ +KuMLdTuY2K2lPtQe8VTF5QiFMrXd/BvgQGxACIOz7YG+5yMOquwEWglvU0TzQh1VXQL ibMQ== X-Gm-Message-State: AOJu0Yxh8e2my1iiOEz83y7yCmaSBW30lETFq+Vl71+B480Hr9Y9s8rG 5d1N7Sqq/FM6ip/aCvPCrf/WgBz2pccpfYtbkmM= X-Google-Smtp-Source: AGHT+IF0MPSUCqiTsnl1eQc+NOEkZdfylg4Cinzvui2qig7vBCcLaky4/GlXwa1fL7VF+JjfOGRcVA== X-Received: by 2002:a05:6a20:431a:b0:18a:da5a:3825 with SMTP id h26-20020a056a20431a00b0018ada5a3825mr948843pzk.34.1700615298368; Tue, 21 Nov 2023 17:08:18 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id bn5-20020a056a00324500b006be047268d5sm8713961pfb.174.2023.11.21.17.08.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Nov 2023 17:08:18 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v3 1/8] riscv: mm: Combine the SMP and UP TLB flush code Date: Tue, 21 Nov 2023 17:07:12 -0800 Message-ID: <20231122010815.3545294-2-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231122010815.3545294-1-samuel.holland@sifive.com> References: <20231122010815.3545294-1-samuel.holland@sifive.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9064F40010 X-Stat-Signature: 7zakapz85rjgqayadybe7iiimfppa7x1 X-HE-Tag: 1700615299-707683 X-HE-Meta: U2FsdGVkX1/uoZwuEszcnn549q57eR2vlmJUxVfZg+nKwuENnWUQ0c7idP+TPOZqYi6cevmnWSGbJvv7INEjA2/ccNkeIVrILxDdGQpm9FBWKU+Xq6aBN3d5pynsqIEWbDuJ78tJYM5JW60x//3+ko2qjsbXk72CoF/pWBF+zzSL+YV4Uh84vPo2L/8vof6x+nQby06jq3yKmmour4TSxnJ+j7Y71rkvjIWMt5ikCgPtwFjApBpKbN1TPDIgH+pCVXBpgQuJBj6Ne3Nk5ZRj4UiuZ08zrghbyrPp2/LRbipH39dnv/BXByDr2gXAyGdjkKDrhTWobq6t/7OckqyYmX30BqI7Vkgzca4KbTdM4LengEUNw9Be1quA0ynVbOvDZEF/ye3QHimYIFHNSGtbWWMBiyo9U+46TdN+67a0UD4dl9bMTg9VHYkwGmM4HmHFVPQkHeAtsBDcGMc+PXEtGynGQucvU68Im8ClZTDX9m59XfQPcoPmGUk1OC3fIVkrpQhigkY0yhKsG0JFjnx8ySr59itlMYxNjcIrWktTtiCTw+EB+g+qSi0QY/utIuhWb2CPhiDv/M+3sXIq7RonZ53xZVQUBPLE7GSKtTOCjN5LdSI3BMw3l/2gcWyGpQXETrBZjvF/vPKWx2ancgC7SQmmnfpVFoLWelyiz6MuLofu+CDWSxQXjQOe5PI74W5o4PvptakA0IroX/S56mv/AKX+1t57OabYtIaE+GQO6CRXMoNB3ib5kJryPk7iyI9gYYzBtiac6CdRQ4XSJCmk2k6Fvy/1V6IyLzoq8zD2zrFXyZ8b4Ls9XBdd3UMDaDstzdgqFvS26EDMXVDcaL8NN1w7WJdqe2bPkiDSi3g8sA+uzNFsy92FQeULXssB/UF1f0rNBFH2kZ53Y8vJ4pe+m+2bFc9Vi9zJe7ntazIYWr8VITxhwrnwstXrpqr+6ybu+Qs+W8FYDjL2BN68cae rgw/GJI+ ovpu3Q8MIHZHvCQlqCM50OtuoxVUCJEB5Ll4ZOacUs6daECz8rL8RDf17g9oBn6D/iHBjXdjbKARm0CVT51rrb6vn3jZtjktWCN5yByzkpSSUts/ELnMcGD/Ahmntj14OeEjNoDPkkt9p0L7qF0wbsOp2T+3HfBhz29sBN0c10OuuWlx+kK+QGSTS5LZYKm9P4sKbNksSIf9afG9JK2RUVSD7UrxTkRg5pgZXeUhSh+LFhcXERXyjB2DxbP0UTMV/hZZux+ZkLIZHuNq9ZmN9zlW7EZ9sb7iYG5vehN129XMoC7D/f1rIFI4J3Dwx0ZzQyihE6241TydAYtI7AtlfaH9MBk74vAltpRO9hfcVaxK7h5LkfrZhjycg+kgAyXni3hQwrpXRC/l/tjU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In SMP configurations, all TLB flushing narrower than flush_tlb_all() goes through __flush_tlb_range(). Do the same in UP configurations. This allows UP configurations to take advantage of recent improvements to the code in tlbflush.c, such as support for huge pages and flushing multiple-page ranges. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Move the SMP/UP merge earlier in the series to avoid build issues - Make a copy of __flush_tlb_range() instead of adding ifdefs inside - local_flush_tlb_all() is the only function used on !MMU (smpboot.c) arch/riscv/include/asm/tlbflush.h | 33 +++++++------------------------ arch/riscv/mm/Makefile | 5 +---- arch/riscv/mm/tlbflush.c | 13 ++++++++++++ 3 files changed, 21 insertions(+), 30 deletions(-) diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 8f3418c5f172..317a1811aa51 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -27,13 +27,12 @@ static inline void local_flush_tlb_page(unsigned long addr) { ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); } -#else /* CONFIG_MMU */ -#define local_flush_tlb_all() do { } while (0) -#define local_flush_tlb_page(addr) do { } while (0) -#endif /* CONFIG_MMU */ -#if defined(CONFIG_SMP) && defined(CONFIG_MMU) +#ifdef CONFIG_SMP void flush_tlb_all(void); +#else +#define flush_tlb_all() local_flush_tlb_all() +#endif void flush_tlb_mm(struct mm_struct *mm); void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, unsigned long end, unsigned int page_size); @@ -46,26 +45,8 @@ void flush_tlb_kernel_range(unsigned long start, unsigned long end); void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); #endif -#else /* CONFIG_SMP && CONFIG_MMU */ - -#define flush_tlb_all() local_flush_tlb_all() -#define flush_tlb_page(vma, addr) local_flush_tlb_page(addr) - -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - local_flush_tlb_all(); -} - -/* Flush a range of kernel pages */ -static inline void flush_tlb_kernel_range(unsigned long start, - unsigned long end) -{ - local_flush_tlb_all(); -} - -#define flush_tlb_mm(mm) flush_tlb_all() -#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() -#endif /* !CONFIG_SMP || !CONFIG_MMU */ +#else /* CONFIG_MMU */ +#define local_flush_tlb_all() do { } while (0) +#endif /* CONFIG_MMU */ #endif /* _ASM_RISCV_TLBFLUSH_H */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 3a4dfc8babcf..96e65c571ce8 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -13,15 +13,12 @@ endif KCOV_INSTRUMENT_init.o := n obj-y += init.o -obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o +obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o tlbflush.o obj-y += cacheflush.o obj-y += context.o obj-y += pgtable.o obj-y += pmem.o -ifeq ($(CONFIG_MMU),y) -obj-$(CONFIG_SMP) += tlbflush.o -endif obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_KASAN) += kasan_init.o diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index e6659d7368b3..22d7ed5abf8e 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -66,6 +66,7 @@ static inline void local_flush_tlb_range_asid(unsigned long start, local_flush_tlb_range_threshold_asid(start, size, stride, asid); } +#ifdef CONFIG_SMP static void __ipi_flush_tlb_all(void *info) { local_flush_tlb_all(); @@ -138,6 +139,18 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, if (mm) put_cpu(); } +#else +static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, + unsigned long size, unsigned long stride) +{ + unsigned long asid = FLUSH_TLB_NO_ASID; + + if (mm && static_branch_unlikely(&use_asid_allocator)) + asid = atomic_long_read(&mm->context.id) & asid_mask; + + local_flush_tlb_range_asid(start, size, stride, asid); +} +#endif void flush_tlb_mm(struct mm_struct *mm) { -- 2.42.0