From: Jinjiang Tu <tujinjiang@huawei.com>
To: Yang Shi <yang@os.amperecomputing.com>, <catalin.marinas@arm.com>,
<will@kernel.org>, <ryan.roberts@arm.com>,
<akpm@linux-foundation.org>, <david@redhat.com>,
<lorenzo.stoakes@oracle.com>, <ardb@kernel.org>,
<dev.jain@arm.com>, <scott@os.amperecomputing.com>,
<cl@gentwo.org>
Cc: <linux-arm-kernel@lists.infradead.org>,
<linux-kernel@vger.kernel.org>, <linux-mm@kvack.org>
Subject: Re: [PATCH v8 0/5] arm64: support FEAT_BBM level 2 and large block mapping when rodata=full
Date: Mon, 16 Mar 2026 15:35:43 +0800 [thread overview]
Message-ID: <0b2a4ae5-fc51-4d77-b177-b2e9db74f11d@huawei.com> (raw)
In-Reply-To: <20250917190323.3828347-1-yang@os.amperecomputing.com>
在 2025/9/18 3:02, Yang Shi 写道:
> On systems with BBML2_NOABORT support, it causes the linear map to be mapped
> with large blocks, even when rodata=full, and leads to some nice performance
> improvements.
Hi,
I find this feature is incompatible with realm. The calltrace is as follows:
[ 0.000000][ T0] ------------[ cut here ]------------
[ 0.000000][ T0] WARNING: CPU: 0 PID: 0 at arch/arm64/mm/pageattr.c:56 pageattr_pmd_entry+0x60/0x78
[ 0.000000][ T0] Modules linked in:
[ 0.000000][ T0] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.6.0 #16
[ 0.000000][ T0] Hardware name: linux,dummy-virt (DT)
[ 0.000000][ T0] pstate: 800000c5 (Nzcv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 0.000000][ T0] pc : pageattr_pmd_entry+0x60/0x78
[ 0.000000][ T0] lr : walk_pmd_range.isra.0+0x170/0x1f0
[ 0.000000][ T0] sp : ffffcb90a0f337d0
[ 0.000000][ T0] x29: ffffcb90a0f337d0 x28: 0000000000000000 x27: ffff0000035e0000
[ 0.000000][ T0] x26: ffffcb90a0f338f8 x25: ffff00001fff60d0 x24: ffff0000035d0000
[ 0.000000][ T0] x23: 0400000000000001 x22: 0c00000000000001 x21: ffff0000035dffff
[ 0.000000][ T0] x20: ffffcb909fe3b7f0 x19: ffff0000035e0000 x18: ffffffffffffffff
[ 0.000000][ T0] x17: 7220303030303178 x16: 307e303030306435 x15: ffffcb90a0f334c8
[ 0.000000][ T0] x14: 0000000000000000 x13: 205d305420202020 x12: 5b5d303030303030
[ 0.000000][ T0] x11: 00000000ffff7fff x10: 00000000ffff7fff x9 : ffffcb909f1e27d8
[ 0.000000][ T0] x8 : 00000000000bffe8 x7 : c0000000ffff7fff x6 : 0000000000000001
[ 0.000000][ T0] x5 : 0000000000000001 x4 : 0078000083400705 x3 : ffffcb90a0f338f8
[ 0.000000][ T0] x2 : 0000000000010000 x1 : ffff0000035d0000 x0 : ffff00001fff60d0
[ 0.000000][ T0] Call trace:
[ 0.000000][ T0] pageattr_pmd_entry+0x60/0x78
[ 0.000000][ T0] walk_pud_range+0x124/0x190
[ 0.000000][ T0] walk_pgd_range+0x158/0x1b0
[ 0.000000][ T0] walk_kernel_page_table_range_lockless+0x58/0x98
[ 0.000000][ T0] update_range_prot+0xb8/0x108
[ 0.000000][ T0] __change_memory_common+0x30/0x1a8
[ 0.000000][ T0] __set_memory_enc_dec.part.0+0x170/0x260
[ 0.000000][ T0] realm_set_memory_decrypted+0x6c/0xb0
[ 0.000000][ T0] set_memory_decrypted+0x38/0x58
[ 0.000000][ T0] its_alloc_pages_node+0xc4/0x140
[ 0.000000][ T0] its_probe_one+0xbc/0x3c0
[ 0.000000][ T0] its_of_probe.isra.0+0x130/0x220
[ 0.000000][ T0] its_init+0x160/0x2f8
[ 0.000000][ T0] gic_init_bases+0x1fc/0x318
[ 0.000000][ T0] gic_of_init+0x2a0/0x300
[ 0.000000][ T0] of_irq_init+0x238/0x4b8
[ 0.000000][ T0] irqchip_init+0x20/0x50
[ 0.000000][ T0] init_IRQ+0x1c/0x100
[ 0.000000][ T0] start_kernel+0x1ec/0x4f0
[ 0.000000][ T0] __primary_switched+0xbc/0xd0
[ 0.000000][ T0] ---[ end trace 0000000000000000 ]---
[ 0.000000][ T0] ------------[ cut here ]------------
[ 0.000000][ T0] Failed to decrypt memory, 16 pages will be leaked
realm feature relies on rodata=full to dynamically update kernel page table prot.
In init_IRQ(), realm_set_memory_decrypted() is called to update kernel page table prot.
At this time, secondary cpus aren't booted, BBML2 noabort feature isn't initializated,
and system_supports_bbml2_noabort() still returns false. As a result,
split_kernel_leaf_mapping() is skipped, leading to WARN_ON_ONCE((next - addr) != PMD_SIZE)
in pageattr_pmd_entry().
Before setup_system_features(), we don't know if all cpus support BBML2 noabort, and we
couldn't split kernel page table, in case another cpu that doesn't support BBML2 noabort
is running.
How could we fix this issue?
1. force pte mapping if realm feature is enabled? Although force_pte_mapping()
return true if is_realm_world() return true, arm64_rsi_init() is called after
map_mem(). So is_realm_world() still return false during map_mem(). Thus
realm feature relies on rodata=full. If we fix by this solution, we need
to add a new cmdline to force pte mapping.
2. If we could try to split kernel page table before setup_system_features()?
Thanks.
>
> Ryan tested v7 on an AmpereOne system (a VM with 12G RAM) in all 3 possible
> modes by hacking the BBML2 feature detection code:
>
> - mode 1: All CPUs support BBML2 so the linear map uses large mappings
> - mode 2: Boot CPU does not support BBML2 so linear map uses pte mappings
> - mode 3: Boot CPU supports BBML2 but secondaries do not so linear map
> initially uses large mappings but is then repainted to use pte mappings
>
> In all cases, mm selftests run and no regressions are observed. In all cases,
> ptdump of linear map is as expected. Because there are just some cleanups
> between v7 and v8, so I kept using Ryan's test result:
>
> Mode 1:
> =======
> ---[ Linear Mapping start ]---
> 0xffff000000000000-0xffff000000200000 2M PMD RW NX SHD AF BLK UXN MEM/NORMAL-TAGGED
> 0xffff000000200000-0xffff000000210000 64K PTE RW NX SHD AF CON UXN MEM/NORMAL-TAGGED
> 0xffff000000210000-0xffff000000400000 1984K PTE ro NX SHD AF UXN MEM/NORMAL
> 0xffff000000400000-0xffff000002400000 32M PMD ro NX SHD AF BLK UXN MEM/NORMAL
> 0xffff000002400000-0xffff000002550000 1344K PTE ro NX SHD AF UXN MEM/NORMAL
> 0xffff000002550000-0xffff000002600000 704K PTE RW NX SHD AF CON UXN MEM/NORMAL-TAGGED
> 0xffff000002600000-0xffff000004000000 26M PMD RW NX SHD AF BLK UXN MEM/NORMAL-TAGGED
> 0xffff000004000000-0xffff000040000000 960M PMD RW NX SHD AF CON BLK UXN MEM/NORMAL-TAGGED
> 0xffff000040000000-0xffff000140000000 4G PUD RW NX SHD AF BLK UXN MEM/NORMAL-TAGGED
> 0xffff000140000000-0xffff000142000000 32M PMD RW NX SHD AF CON BLK UXN MEM/NORMAL-TAGGED
> 0xffff000142000000-0xffff000142120000 1152K PTE RW NX SHD AF CON UXN MEM/NORMAL-TAGGED
> 0xffff000142120000-0xffff000142128000 32K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000142128000-0xffff000142159000 196K PTE ro NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000142159000-0xffff000142160000 28K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000142160000-0xffff000142240000 896K PTE RW NX SHD AF CON UXN MEM/NORMAL-TAGGED
> 0xffff000142240000-0xffff00014224e000 56K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff00014224e000-0xffff000142250000 8K PTE ro NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000142250000-0xffff000142260000 64K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000142260000-0xffff000142280000 128K PTE RW NX SHD AF CON UXN MEM/NORMAL-TAGGED
> 0xffff000142280000-0xffff000142288000 32K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000142288000-0xffff000142290000 32K PTE ro NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000142290000-0xffff0001422a0000 64K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff0001422a0000-0xffff000142465000 1812K PTE ro NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000142465000-0xffff000142470000 44K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000142470000-0xffff000142600000 1600K PTE RW NX SHD AF CON UXN MEM/NORMAL-TAGGED
> 0xffff000142600000-0xffff000144000000 26M PMD RW NX SHD AF BLK UXN MEM/NORMAL-TAGGED
> 0xffff000144000000-0xffff000180000000 960M PMD RW NX SHD AF CON BLK UXN MEM/NORMAL-TAGGED
> 0xffff000180000000-0xffff000181a00000 26M PMD RW NX SHD AF BLK UXN MEM/NORMAL-TAGGED
> 0xffff000181a00000-0xffff000181b90000 1600K PTE RW NX SHD AF CON UXN MEM/NORMAL-TAGGED
> 0xffff000181b90000-0xffff000181b9d000 52K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000181b9d000-0xffff000181c80000 908K PTE ro NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000181c80000-0xffff000181c90000 64K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000181c90000-0xffff000181ca0000 64K PTE RW NX SHD AF CON UXN MEM/NORMAL-TAGGED
> 0xffff000181ca0000-0xffff000181dbd000 1140K PTE ro NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000181dbd000-0xffff000181dc0000 12K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000181dc0000-0xffff000181e00000 256K PTE RW NX SHD AF CON UXN MEM/NORMAL-TAGGED
> 0xffff000181e00000-0xffff000182000000 2M PMD RW NX SHD AF BLK UXN MEM/NORMAL-TAGGED
> 0xffff000182000000-0xffff0001c0000000 992M PMD RW NX SHD AF CON BLK UXN MEM/NORMAL-TAGGED
> 0xffff0001c0000000-0xffff000300000000 5G PUD RW NX SHD AF BLK UXN MEM/NORMAL-TAGGED
> 0xffff000300000000-0xffff008000000000 500G PUD
> 0xffff008000000000-0xffff800000000000 130560G PGD
> ---[ Linear Mapping end ]---
>
> Mode 3:
> =======
> ---[ Linear Mapping start ]---
> 0xffff000000000000-0xffff000000210000 2112K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000000210000-0xffff000000400000 1984K PTE ro NX SHD AF UXN MEM/NORMAL
> 0xffff000000400000-0xffff000002400000 32M PMD ro NX SHD AF BLK UXN MEM/NORMAL
> 0xffff000002400000-0xffff000002550000 1344K PTE ro NX SHD AF UXN MEM/NORMAL
> 0xffff000002550000-0xffff000143a61000 5264452K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000143a61000-0xffff000143c61000 2M PTE ro NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000143c61000-0xffff000181b9a000 1015012K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000181b9a000-0xffff000181d9a000 2M PTE ro NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000181d9a000-0xffff000300000000 6261144K PTE RW NX SHD AF UXN MEM/NORMAL-TAGGED
> 0xffff000300000000-0xffff008000000000 500G PUD
> 0xffff008000000000-0xffff800000000000 130560G PGD
> ---[ Linear Mapping end ]---
>
>
> Performance Testing
> ===================
> * Memory use after boot
> Before:
> MemTotal: 258988984 kB
> MemFree: 254821700 kB
>
> After:
> MemTotal: 259505132 kB
> MemFree: 255410264 kB
>
> Around 500MB more memory are free to use. The larger the machine, the
> more memory saved.
>
> * Memcached
> We saw performance degradation when running Memcached benchmark with
> rodata=full vs rodata=on. Our profiling pointed to kernel TLB pressure.
> With this patchset we saw ops/sec is increased by around 3.5%, P99
> latency is reduced by around 9.6%.
> The gain mainly came from reduced kernel TLB misses. The kernel TLB
> MPKI is reduced by 28.5%.
>
> The benchmark data is now on par with rodata=on too.
>
> * Disk encryption (dm-crypt) benchmark
> Ran fio benchmark with the below command on a 128G ramdisk (ext4) with
> disk encryption (by dm-crypt).
> fio --directory=/data --random_generator=lfsr --norandommap \
> --randrepeat 1 --status-interval=999 --rw=write --bs=4k --loops=1 \
> --ioengine=sync --iodepth=1 --numjobs=1 --fsync_on_close=1 \
> --group_reporting --thread --name=iops-test-job --eta-newline=1 \
> --size 100G
>
> The IOPS is increased by 90% - 150% (the variance is high, but the worst
> number of good case is around 90% more than the best number of bad
> case). The bandwidth is increased and the avg clat is reduced
> proportionally.
>
> * Sequential file read
> Read 100G file sequentially on XFS (xfs_io read with page cache
> populated). The bandwidth is increased by 150%.
>
> Additionally Ryan also ran this through a random selection of benchmarks on
> AmpereOne. None show any regressions, and various benchmarks show statistically
> significant improvement. I'm just showing those improvements here:
>
> +----------------------+----------------------------------------------------------+-------------------------+
> | Benchmark | Result Class | Improvement vs 6.17-rc1 |
> +======================+==========================================================+=========================+
> | micromm/vmalloc | full_fit_alloc_test: p:1, h:0, l:500000 (usec) | (I) -9.00% |
> | | kvfree_rcu_1_arg_vmalloc_test: p:1, h:0, l:500000 (usec) | (I) -6.93% |
> | | kvfree_rcu_2_arg_vmalloc_test: p:1, h:0, l:500000 (usec) | (I) -6.77% |
> | | pcpu_alloc_test: p:1, h:0, l:500000 (usec) | (I) -4.63% |
> +----------------------+----------------------------------------------------------+-------------------------+
> | mmtests/hackbench | process-sockets-30 (seconds) | (I) -2.96% |
> +----------------------+----------------------------------------------------------+-------------------------+
> | mmtests/kernbench | syst-192 (seconds) | (I) -12.77% |
> +----------------------+----------------------------------------------------------+-------------------------+
> | pts/perl-benchmark | Test: Interpreter (Seconds) | (I) -4.86% |
> +----------------------+----------------------------------------------------------+-------------------------+
> | pts/pgbench | Scale: 1 Clients: 1 Read Write (TPS) | (I) 5.07% |
> | | Scale: 1 Clients: 1 Read Write - Latency (ms) | (I) -4.72% |
> | | Scale: 100 Clients: 1000 Read Write (TPS) | (I) 2.58% |
> | | Scale: 100 Clients: 1000 Read Write - Latency (ms) | (I) -2.52% |
> +----------------------+----------------------------------------------------------+-------------------------+
> | pts/sqlite-speedtest | Timed Time - Size 1,000 (Seconds) | (I) -2.68% |
> +----------------------+----------------------------------------------------------+-------------------------+
>
> Changes since v7 [1]
> ====================
> - Rebased on v6.17-rc6 and Shijie's rodata series (https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/commit/?id=bfbbb0d3215f)
> which has been picked up by Will.
> - Patch 1: Fixed pmd_leaf/pud_leaf issue since the code may need to change
> permission for invalid entries per Jinjiang Tu.
> - Patch 1: Removed pageattr_pgd_entry and pageattr_p4d_entry per Ryan.
> - Used (-1ULL) instead of -1 per Catalin.
> - Added comment about arm64 lazy mmu allow sleeping per Ryan.
> - Squashed patch #4 in v7 into patch #3.
> - Squashed patch #6 in v7 into patch #4.
> - Added patch #5 to fix a arm64 kprobes bug. It guarantees set_memory_rox()
> is called before vfree(). It can go into separately or with this series
> together.
> - Collected all the R-bs and A-bs.
>
> Changes since v6 [2]
> ====================
> - Patch 1: Minor refactor to implement walk_kernel_page_table_range() in terms
> of walk_kernel_page_table_range_lockless(). Also lead to adding *pmd argument
> to the lockless variant for consistency (per Catalin).
> - Misc function/variable renames to improve clarity and consistency.
> - Share same syncrhonization flag between idmap_kpti_install_ng_mappings and
> wait_linear_map_split_to_ptes, which allows removal of bbml2_ptes[] to save
> ~20K from kernel image.
> - Only take pgtable_split_lock and enter lazy mmu mode once for both splits.
> - Only walk the pgtable once for the common "split single page" case.
> - Bypass split to contpmd and contpte when spllitting linear map to ptes.
>
> [1] https://lore.kernel.org/linux-arm-kernel/20250829115250.2395585-1-ryan.roberts@arm.com/
> [2] https://lore.kernel.org/linux-arm-kernel/20250805081350.3854670-1-ryan.roberts@arm.com/
>
>
> Dev Jain (1):
> arm64: Enable permission change on arm64 kernel block mappings
>
> Ryan Roberts (1):
> arm64: mm: split linear mapping if BBML2 unsupported on secondary CPUs
>
> Yang Shi (3):
> arm64: cpufeature: add AmpereOne to BBML2 allow list
> arm64: mm: support large block mapping when rodata=full
> arm64: kprobes: call set_memory_rox() for kprobe page
>
> arch/arm64/include/asm/cpufeature.h | 2 +
> arch/arm64/include/asm/mmu.h | 3 +
> arch/arm64/include/asm/pgtable.h | 5 ++
> arch/arm64/kernel/cpufeature.c | 12 +++-
> arch/arm64/kernel/probes/kprobes.c | 12 ++++
> arch/arm64/mm/mmu.c | 422 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----
> arch/arm64/mm/pageattr.c | 123 ++++++++++++++++++++++++---------
> arch/arm64/mm/proc.S | 27 ++++++--
> include/linux/pagewalk.h | 3 +
> mm/pagewalk.c | 36 ++++++----
> 10 files changed, 581 insertions(+), 64 deletions(-)
>
>
next prev parent reply other threads:[~2026-03-16 7:35 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-17 19:02 Yang Shi
2025-09-17 19:02 ` [PATCH v8 1/5] arm64: Enable permission change on arm64 kernel block mappings Yang Shi
2025-09-17 19:02 ` [PATCH v8 2/5] arm64: cpufeature: add AmpereOne to BBML2 allow list Yang Shi
2025-09-17 19:02 ` [PATCH v8 3/5] arm64: mm: support large block mapping when rodata=full Yang Shi
2025-11-01 16:14 ` Guenter Roeck
2025-11-02 10:31 ` Ryan Roberts
2025-11-02 12:11 ` Ryan Roberts
2025-11-02 15:13 ` Guenter Roeck
2025-11-02 17:46 ` Guenter Roeck
2025-11-02 17:49 ` Guenter Roeck
2025-11-02 17:52 ` Guenter Roeck
2025-11-03 0:47 ` Yang Shi
2025-11-03 10:07 ` Ryan Roberts
2025-11-03 16:21 ` Yang Shi
2025-11-03 5:53 ` Dev Jain
2025-09-17 19:02 ` [PATCH v8 4/5] arm64: mm: split linear mapping if BBML2 unsupported on secondary CPUs Yang Shi
2026-02-02 7:18 ` Arnd Bergmann
2026-02-02 7:43 ` Ard Biesheuvel
2026-02-02 8:11 ` Arnd Bergmann
2025-09-17 19:02 ` [PATCH v8 5/5] arm64: kprobes: call set_memory_rox() for kprobe page Yang Shi
2025-09-18 12:48 ` Catalin Marinas
2025-09-18 15:05 ` Yang Shi
2025-09-18 15:30 ` Ryan Roberts
2025-09-18 15:50 ` Yang Shi
2025-09-18 15:32 ` Catalin Marinas
2025-09-18 15:48 ` Yang Shi
2025-09-18 21:10 ` [PATCH v8 0/5] arm64: support FEAT_BBM level 2 and large block mapping when rodata=full Will Deacon
2025-09-19 10:08 ` Ryan Roberts
2025-09-19 11:27 ` Will Deacon
2025-09-19 11:49 ` Ryan Roberts
2025-09-19 11:56 ` Will Deacon
2025-09-19 12:00 ` Ryan Roberts
2025-09-19 18:44 ` Will Deacon
2025-09-23 7:15 ` Ryan Roberts
2025-09-19 14:55 ` Yang Shi
2026-03-16 7:35 ` Jinjiang Tu [this message]
2026-03-16 15:47 ` Ryan Roberts
2026-03-17 0:15 ` Yang Shi
2026-03-17 2:06 ` Jinjiang Tu
2026-03-17 9:07 ` Ryan Roberts
2026-03-17 17:03 ` Yang Shi
2026-03-18 8:29 ` Jinjiang Tu
2026-03-18 9:17 ` Ryan Roberts
2026-03-19 1:22 ` Jinjiang Tu
2026-03-17 17:12 ` Yang Shi
2026-03-17 8:47 ` Kevin Brodsky
2026-03-17 9:13 ` Ryan Roberts
2026-03-17 9:29 ` Kevin Brodsky
2026-03-17 11:45 ` Ryan Roberts
2026-03-17 12:43 ` Kevin Brodsky
2026-03-17 15:05 ` Ryan Roberts
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0b2a4ae5-fc51-4d77-b177-b2e9db74f11d@huawei.com \
--to=tujinjiang@huawei.com \
--cc=akpm@linux-foundation.org \
--cc=ardb@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=cl@gentwo.org \
--cc=david@redhat.com \
--cc=dev.jain@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lorenzo.stoakes@oracle.com \
--cc=ryan.roberts@arm.com \
--cc=scott@os.amperecomputing.com \
--cc=will@kernel.org \
--cc=yang@os.amperecomputing.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox