From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1A07C27C7C for ; Fri, 20 Jan 2023 14:21:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F8296B00A9; Fri, 20 Jan 2023 09:20:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 082D36B00AA; Fri, 20 Jan 2023 09:20:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA4D56B00AC; Fri, 20 Jan 2023 09:20:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C6FCA6B00AA for ; Fri, 20 Jan 2023 09:20:55 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 6173F120E73 for ; Fri, 20 Jan 2023 14:20:55 +0000 (UTC) X-FDA: 80375388870.03.1909249 Received: from fx305.security-mail.net (smtpout30.security-mail.net [85.31.212.35]) by imf22.hostedemail.com (Postfix) with ESMTP id 1C48CC001F for ; Fri, 20 Jan 2023 14:20:52 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kalray.eu header.s=sec-sig-email header.b=5qPYtkX0; dkim=fail ("body hash did not verify") header.d=kalray.eu header.s=32AE1B44-9502-11E5-BA35-3734643DEF29 header.b=ha+njJPr; spf=pass (imf22.hostedemail.com: domain of ysionneau@kalray.eu designates 85.31.212.35 as permitted sender) smtp.mailfrom=ysionneau@kalray.eu; dmarc=pass (policy=quarantine) header.from=kalray.eu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674224453; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=t1Z0n5PX0iol5aakTpvq5Mow1IdNST7Cqz+38HiiLxI=; b=xwH/fU+sttWnSmVIxClSEMIvodyE1PuLhtXzayn94QWk81hJajmCT9KLN3hgCJd8PM9oeA kA1//fm+l897bUMuQ8TmFxAhTi625NEQABLMYlmBnJ2uEQLPDOPtogwYdRlTHeeuNMsTn+ mTw3+kYoF5QON3EtkKloQet5CbWHecI= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kalray.eu header.s=sec-sig-email header.b=5qPYtkX0; dkim=fail ("body hash did not verify") header.d=kalray.eu header.s=32AE1B44-9502-11E5-BA35-3734643DEF29 header.b=ha+njJPr; spf=pass (imf22.hostedemail.com: domain of ysionneau@kalray.eu designates 85.31.212.35 as permitted sender) smtp.mailfrom=ysionneau@kalray.eu; dmarc=pass (policy=quarantine) header.from=kalray.eu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674224453; a=rsa-sha256; cv=none; b=aNsJin/poFaDwWDLgxwJ4SAsojCjC7klXF6X8cOF4+EPpRBGZty2umS7ClkoM3Su0Le0FF BtrDtYEZKkQy0+jpMW94T2fJzjNVwib8HMptzAHuYye5dPjylRZQzYaJLIhmHfCh6Rf1Rn M1hco6a4Ym88CcH5qyMC2UmZBuv7B8o= Received: from localhost (fx305.security-mail.net [127.0.0.1]) by fx305.security-mail.net (Postfix) with ESMTP id B0B183100BF for ; Fri, 20 Jan 2023 15:20:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674224451; bh=bNmYslkbuquxKAj+Znp8eugUmqYemSNASVZgyAJPx68=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=5qPYtkX0BRHONeyuqbqEmsiHgumuz8FkQAVqIyDA11qd/VtB2qmF8n0TJIg2fXAFr aHoXe/zN1x+C0zG0OYOLjzhuHntMXYEuv4G9vHWfr5pMrM/MEotKxgjT27YWQ3qqi6 3gakPv+MFebrkracTyBwyRW6RutDiel6wOsH9wU8= Received: from fx305 (fx305.security-mail.net [127.0.0.1]) by fx305.security-mail.net (Postfix) with ESMTP id 6109130FEBB; Fri, 20 Jan 2023 15:20:49 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx305.security-mail.net (Postfix) with ESMTPS id D624F30FC92; Fri, 20 Jan 2023 15:20:44 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 4AEEB27E0454; Fri, 20 Jan 2023 15:10:37 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 3349D27E045E; Fri, 20 Jan 2023 15:10:37 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id XV8A6iMCRbpC; Fri, 20 Jan 2023 15:10:37 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 9550C27E0457; Fri, 20 Jan 2023 15:10:36 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <911c.63caa33c.d37a7.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 3349D27E045E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223837; bh=3OWgg5n0jIy60CSKPsLXfDN5oeIG50h2WPbisKqx88I=; h=From:To:Date:Message-Id:MIME-Version; b=ha+njJPrX4jOZJ1/7CAya1KsiiCG+CEgarQekDqLqd2GgjUmheDnGnk7n4MPvjURn ZvRTniHv/MFR3PJRWqisBvTz8xd3DrHxIqcwqD8l4XW6bT/pQz3cSyEDB+1mKQh7op qOvusDQL3LvcPjBMP4WfmkcjLkmthT/TRFDXAjV0= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhpw6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 26/31] kvx: Add multi-processor (SMP) support Date: Fri, 20 Jan 2023 15:09:57 +0100 Message-ID: <20230120141002.2442-27-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=utf-8 X-ALTERMIMEV2_out: done X-Stat-Signature: 67im6fu4maqj7js8htbfcsbuisboemb1 X-Rspam-User: X-Rspamd-Queue-Id: 1C48CC001F X-Rspamd-Server: rspam06 X-HE-Tag: 1674224452-767987 X-HE-Meta: U2FsdGVkX1+01k66eLA0paWqdo1rqw7LSeUVUAAfR1BOYDPZkEBhotoTjjZxXNCrKFBaHRgaRs4mn5DKbbFRshjG4BePGG+Ph31zokCfxbWpEAoNHxEpd0hFFcqwyBFpu6/QM6VD13neQ7zQV1afavvpg/ovn0ns7oqfOEDSBY2kFqLWd2OG+SC4SIztR2mNFVre6Jo71n3cFYiHPBnJX6PXFU54lf3tj286st57kENjAg07fmXquBNb1BjLrpB8IGZZ35KqLIyp7ttHPJ1IOBA0o9MTdFG7v/66+Wrv1K7enjDW2MGxQwQGj4e3/zj7CLVPAomnCbAMMOzD5tKiMXMaYcxV4K0ETjqAOcIIGIXT++DueoFFN2ASk9BX3LpfqqNKwv03NUcp3FhtFbf5Tcwe26KNDzRvDUumjFBZ+h8Z+l23MKdT9NQYOLZ5QUVCXPfD6TbxBdnDBiiI11StrMgLFlEAHwTAcIRKIa//l2lkZ6jYLqR6eV1CrLHhoCADjNbf/T5jRyUa/FVW1cY+xXVaxwt1i5Ui6ZgK4ryp9e1UaCm6xarT0q2UvW4bUD0hGcvXIKgm0Knu1diGVuqGvzTatNOnEJ9k4YcAFdOHF9PZXZBrUVgigZUrjW191wTSxLT9UBgzEkM6Xd/9ELyWkuSBFVxLlLVybvMsn1iIFzxSphGBtVYR2atb13LYVJ902si90l1A73Y3ya708mJ9KkL3VWuHr5jg5JkscZblVETUbWUXcqTIg71qjWYY8RJHF3f//XmgBsbpOU341qZlfKDF8CKwLIwmUmLRroZGw64dkzCY6P5bpAytYJjVB8XYyUqzWEpPdG2FZvDWLGYIp3AUBUiMsXjk+MYizBSCx1rWq9KpzFhAKCKWh6SHKUARzblnza3f0oYDFhix5qjkVEze4uDGiBPea5lYFU8Z69IkBit5gDQ5sKrydmOmsiupM2NKTQckn2j3TapuHp8 AC7rkeQ9 Wanadeq2al5KD2dsOJVVogn1p6l9Wytmc0Ga2IKw5S25BEx/zw92ac7kse1louz6QNYfWwQ1UH/BhexWqelCSymTbMBaMJ4z5TERqH2f11oJMyg0+0HlDjCvIRmygY3xjvpTSV9T5KxBFygOtH6hP+7uI94dliJYXf4SzpioPM5TCLJ2m9XTjg7Pxanl2JgJCLzv65m7YEYzbyfS2GjWOmkbejnbJE2s8Bv2DkVt/tspRbQNdk2t1HZdApX6oIiqOhOzcuRE/kxUcJ9tvc4jODRp0GDUcMGLwPcFbZokCTZanra/gOIsJQizokbNNnuxDwfLjmzzv9PLEYZ5lPxdahbemVEYsxQvVvA65zvsZR3F9jNJ7iHhne378nKGew2uqV5PqKGxzbAOgdzcZidFHXUYHjLgOb16d98vCGbZ1xEYqbmK0T455ky38XmiUfIWZDzfEJQPloPDFA4ecDBp9bBBJnl1DWhDhuXqL814Sxbf4QdQ2ha0GyDd2LWp7GO9sWzOlHT3p/5EXOw5huaqsqrdeLvuVWLtc0SV+6VZ2u2/jXBjC5RtcEGfymXc8PEL8jJEoT76EMXJ5w2P+wz41dYqnY+xSMj8sjcrMoaxUUZ5jZimPZ/Lk7dQVuk7/H6SHvtyKjOpZySv0hnvNlIUqCqv24l+z/nJ0fZ+itO1uJdRrl3y5JLKNJZL1WF0Sk0SGbLT4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Coolidge v1 SoC has 5 clusters of 17 kvx cores: - 16 application cores aka PE - 1 privileged core, the Resource Manager, aka RM. Linux can run in SMP config on the 16 cores of a Cluster. Memory coherency between all cores is guarenteed by the L2 cache. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Julien Hascoet Signed-off-by: Julien Hascoet Co-developed-by: Louis Morhet Signed-off-by: Louis Morhet Co-developed-by: Luc Michel Signed-off-by: Luc Michel Co-developed-by: Marius Gligor Signed-off-by: Marius Gligor Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: - removed L2 cache driver - removed ipi and pwr-ctrl driver (splitted in their own patch) arch/kvx/include/asm/smp.h | 42 ++++++++++++ arch/kvx/kernel/smp.c | 110 ++++++++++++++++++++++++++++++++ arch/kvx/kernel/smpboot.c | 127 +++++++++++++++++++++++++++++++++++++ include/linux/cpuhotplug.h | 2 + 4 files changed, 281 insertions(+) create mode 100644 arch/kvx/include/asm/smp.h create mode 100644 arch/kvx/kernel/smp.c create mode 100644 arch/kvx/kernel/smpboot.c diff --git a/arch/kvx/include/asm/smp.h b/arch/kvx/include/asm/smp.h new file mode 100644 index 000000000000..e4fd4d001b2c --- /dev/null +++ b/arch/kvx/include/asm/smp.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SMP_H +#define _ASM_KVX_SMP_H + +#include +#include + +#include + +#ifdef CONFIG_SMP + +/* Hook for the generic smp_call_function_many() routine. */ +void arch_send_call_function_ipi_mask(struct cpumask *mask); + +/* Hook for the generic smp_call_function_single() routine. */ +void arch_send_call_function_single_ipi(int cpu); + +void __init setup_processor(void); + +void smp_init_cpus(void); + +irqreturn_t ipi_call_interrupt(int irq, void *dev_id); + +#define raw_smp_processor_id() ((int) \ + ((kvx_sfr_get(PCR) & KVX_SFR_PCR_PID_MASK) \ + >> KVX_SFR_PCR_PID_SHIFT)) + +#define flush_cache_vmap(start, end) do { } while (0) +#define flush_cache_vunmap(start, end) do { } while (0) + +#else + +void smp_init_cpus(void) {} + +#endif /* CONFIG_SMP */ + +#endif diff --git a/arch/kvx/kernel/smp.c b/arch/kvx/kernel/smp.c new file mode 100644 index 000000000000..ed4c35a8c4bc --- /dev/null +++ b/arch/kvx/kernel/smp.c @@ -0,0 +1,110 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +enum ipi_message_type { + IPI_RESCHEDULE, + IPI_CALL_FUNC, + IPI_IRQ_WORK, + IPI_MAX +}; + +/* A collection of single bit ipi messages. */ +static struct { + unsigned long bits ____cacheline_aligned; +} ipi_data[NR_CPUS] __cacheline_aligned; + +static void send_ipi_message(const struct cpumask *mask, + enum ipi_message_type operation) +{ + unsigned long flags; + int cpu; + + /* Set operation that must be done by receiver */ + for_each_cpu(cpu, mask) + set_bit(operation, &ipi_data[cpu].bits); + + /* Commit the write before sending IPI */ + smp_wmb(); + + local_irq_save(flags); + + kvx_ipi_send(mask); + + local_irq_restore(flags); +} + +void arch_send_call_function_ipi_mask(struct cpumask *mask) +{ + send_ipi_message(mask, IPI_CALL_FUNC); +} + +void arch_send_call_function_single_ipi(int cpu) +{ + send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC); +} + +#ifdef CONFIG_IRQ_WORK +void arch_irq_work_raise(void) +{ + send_ipi_message(cpumask_of(smp_processor_id()), IPI_IRQ_WORK); +} +#endif + +static void ipi_stop(void *unused) +{ + local_cpu_stop(); +} + +void smp_send_stop(void) +{ + struct cpumask targets; + + cpumask_copy(&targets, cpu_online_mask); + cpumask_clear_cpu(smp_processor_id(), &targets); + + smp_call_function_many(&targets, ipi_stop, NULL, 0); +} + +void smp_send_reschedule(int cpu) +{ + send_ipi_message(cpumask_of(cpu), IPI_RESCHEDULE); +} + +irqreturn_t ipi_call_interrupt(int irq, void *dev_id) +{ + unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits; + + while (true) { + unsigned long ops = xchg(pending_ipis, 0); + + if (ops == 0) + return IRQ_HANDLED; + + if (ops & (1 << IPI_RESCHEDULE)) + scheduler_ipi(); + + if (ops & (1 << IPI_CALL_FUNC)) + generic_smp_call_function_interrupt(); + + if (ops & (1 << IPI_IRQ_WORK)) + irq_work_run(); + + BUG_ON((ops >> IPI_MAX) != 0); + } + + return IRQ_HANDLED; +} diff --git a/arch/kvx/kernel/smpboot.c b/arch/kvx/kernel/smpboot.c new file mode 100644 index 000000000000..987a6f014163 --- /dev/null +++ b/arch/kvx/kernel/smpboot.c @@ -0,0 +1,127 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Julian Vetter + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +void *__cpu_up_stack_pointer[NR_CPUS]; +void *__cpu_up_task_pointer[NR_CPUS]; + +void __init smp_prepare_boot_cpu(void) +{ +} + +int __cpu_up(unsigned int cpu, struct task_struct *tidle) +{ + __cpu_up_stack_pointer[cpu] = task_stack_page(tidle) + THREAD_SIZE; + __cpu_up_task_pointer[cpu] = tidle; + /* We need to be sure writes are committed */ + smp_mb(); + + kvx_pwr_ctrl_cpu_poweron(cpu); + while (!cpu_online(cpu)) + cpu_relax(); + + return 0; +} + +void __init smp_cpus_done(unsigned int max_cpus) +{ +} + +void __init smp_init_cpus(void) +{ + struct cpumask cpumask; + struct device_node *cpu; + const __be32 *reg; + u32 cpu_num; + unsigned int nr_cpus = 0; + + cpumask_clear(&cpumask); + + for_each_of_cpu_node(cpu) { + if (!of_device_is_available(cpu)) + continue; + + reg = of_get_property(cpu, "reg", NULL); + if (!reg) + continue; + + cpu_num = be32_to_cpup(reg); + if (cpu_num >= nr_cpu_ids) + continue; + + nr_cpus++; + cpumask_set_cpu(cpu_num, &cpumask); + } + + pr_info("%d possible cpus\n", nr_cpus); + init_cpu_possible(&cpumask); +} + +void __init smp_prepare_cpus(unsigned int max_cpus) +{ + if (num_present_cpus() <= 1) + init_cpu_present(cpu_possible_mask); +} + +int __init setup_smp(void) +{ + int ret; + + ret = kvx_pwr_ctrl_probe(); + if (ret) + panic("Failed to probe power controller !"); + + ret = kvx_ipi_ctrl_probe(ipi_call_interrupt); + if (ret) + panic("Failed to probe IPI controller !"); + + return 0; +} + +early_initcall(setup_smp); + +/* + * C entry point for a secondary processor. + */ +void __init start_kernel_secondary(void) +{ + struct mm_struct *mm = &init_mm; + unsigned int cpu = smp_processor_id(); + + setup_processor(); + kvx_mmu_early_setup(); + + /* All kernel threads share the same mm context. */ + mmgrab(mm); + current->active_mm = mm; + cpumask_set_cpu(cpu, mm_cpumask(mm)); + + notify_cpu_starting(cpu); + set_cpu_online(cpu, true); + trace_hardirqs_off(); + + local_flush_tlb_all(); + + local_irq_enable(); + cpu_startup_entry(CPUHP_AP_ONLINE_IDLE); +} diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index f61447913db9..f5a484547b15 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -152,6 +152,7 @@ enum cpuhp_state { CPUHP_AP_IRQ_RISCV_STARTING, CPUHP_AP_IRQ_LOONGARCH_STARTING, CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING, + CPUHP_AP_IRQ_KVX_STARTING, CPUHP_AP_ARM_MVEBU_COHERENCY, CPUHP_AP_MICROCODE_LOADER, CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING, @@ -189,6 +190,7 @@ enum cpuhp_state { CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING, CPUHP_AP_KVM_ARM_VGIC_STARTING, CPUHP_AP_KVM_ARM_TIMER_STARTING, + CPUHP_AP_KVX_TIMER_STARTING, /* Must be the last timer callback */ CPUHP_AP_DUMMY_TIMER_STARTING, CPUHP_AP_ARM_XEN_STARTING, -- 2.37.2