Re: [PATCH] riscv: make ZONE_DMA32 optional

[Drew Fustini <drew@pdp7.com>]

On Fri, Sep 20, 2024 at 01:58:23AM -0700, Palmer Dabbelt wrote:
> On Tue, 27 Aug 2024 16:10:20 PDT (-0700), dfustini@tenstorrent.com wrote:
> > On Tue, Aug 27, 2024 at 02:36:11PM +0300, Vladimir Kondratiev wrote:
> > > It is not necessary any RISCV platform has ZONE_DMA32.
> > > 
> > > Example - if platform has no DRAM in [0..4G] region,
> > > it will report failure like below each boot.
> > > 
> > > [    0.088709] swapper/0: page allocation failure: order:7, mode:0xcc4(GFP_KERNEL|GFP_DMA32), nodemask=(null),cpuset=/
> > > [    0.088832] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.11.0-rc5 #30
> > > [    0.088864] Call Trace:
> > > [    0.088869] [<ffffffff800059f2>] dump_backtrace+0x1c/0x24
> > > [    0.088910] [<ffffffff805f328c>] show_stack+0x2c/0x38
> > > [    0.088957] [<ffffffff805fd800>] dump_stack_lvl+0x52/0x74
> > > [    0.088987] [<ffffffff805fd836>] dump_stack+0x14/0x1c
> > > [    0.089010] [<ffffffff801a23a8>] warn_alloc+0xf4/0x176
> > > [    0.089041] [<ffffffff801a3052>] __alloc_pages_noprof+0xc28/0xcb4
> > > [    0.089067] [<ffffffff80086eda>] atomic_pool_expand+0x62/0x1f8
> > > [    0.089090] [<ffffffff8080d674>] __dma_atomic_pool_init+0x46/0x9e
> > > [    0.089115] [<ffffffff8080d762>] dma_atomic_pool_init+0x96/0x11c
> > > [    0.089139] [<ffffffff80002146>] do_one_initcall+0x5c/0x1b2
> > > [    0.089158] [<ffffffff8080127c>] kernel_init_freeable+0x214/0x274
> > > [    0.089190] [<ffffffff805fefd8>] kernel_init+0x1e/0x10a
> > > [    0.089209] [<ffffffff8060748a>] ret_from_fork+0xe/0x1c
> > > 
> > > Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
> > > ---
> > >  arch/riscv/Kconfig | 2 +-
> > >  mm/Kconfig         | 2 +-
> > >  2 files changed, 2 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > index 0f3cd7c3a436..94a573112625 100644
> > > --- a/arch/riscv/Kconfig
> > > +++ b/arch/riscv/Kconfig
> > > @@ -50,6 +50,7 @@ config RISCV
> > >  	select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
> > >  	select ARCH_HAS_UBSAN
> > >  	select ARCH_HAS_VDSO_DATA
> > > +	select ARCH_HAS_ZONE_DMA_SET if 64BIT
> > >  	select ARCH_KEEP_MEMBLOCK if ACPI
> > >  	select ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE	if 64BIT && MMU
> > >  	select ARCH_OPTIONAL_KERNEL_RWX if ARCH_HAS_STRICT_KERNEL_RWX
> > > @@ -200,7 +201,6 @@ config RISCV
> > >  	select THREAD_INFO_IN_TASK
> > >  	select TRACE_IRQFLAGS_SUPPORT
> > >  	select UACCESS_MEMCPY if !MMU
> > > -	select ZONE_DMA32 if 64BIT
> > > 
> > >  config CLANG_SUPPORTS_DYNAMIC_FTRACE
> > >  	def_bool CC_IS_CLANG
> > > diff --git a/mm/Kconfig b/mm/Kconfig
> > > index b72e7d040f78..97c85da98e89 100644
> > > --- a/mm/Kconfig
> > > +++ b/mm/Kconfig
> > > @@ -1032,7 +1032,7 @@ config ZONE_DMA
> > >  config ZONE_DMA32
> > >  	bool "Support DMA32 zone" if ARCH_HAS_ZONE_DMA_SET
> > >  	depends on !X86_32
> > > -	default y if ARM64
> > > +	default y if ARM64 || (RISCV && 64BIT)
> > > 
> > >  config ZONE_DEVICE
> > >  	bool "Device memory (pmem, HMM, etc...) hotplug support"
> > > --
> > > 2.37.3
> > > 
> > 
> > Reviewed-by: Drew Fustini <dfustini@tenstorrent.com>
> > 
> > Thanks for sending this patch as I've also encountered that annoying
> > error on systems with DRAM above 4GB.
> > 
> > I tested this patch by changing the qemu virt machine to have DRAM
> > starting at 2^32:
> > 
> > diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> > index cef41c150aaf..3033a2560edb 100644
> > --- a/hw/riscv/virt.c
> > +++ b/hw/riscv/virt.c
> > @@ -87,7 +87,7 @@ static const MemMapEntry virt_memmap[] = {
> >      [VIRT_IMSIC_S] =      { 0x28000000, VIRT_IMSIC_MAX_SIZE },
> >      [VIRT_PCIE_ECAM] =    { 0x30000000,    0x10000000 },
> >      [VIRT_PCIE_MMIO] =    { 0x40000000,    0x40000000 },
> > -    [VIRT_DRAM] =         { 0x80000000,           0x0 },
> > +    [VIRT_DRAM] =        { 0x100000000,           0x0 },
> >  };
> > 
> >  /* PCIe high mmio is fixed for RV32 */
> 
> IIRC the ZONE_DMA32 stuff existed for some of the early SiFive systems,
> where the expansion daughterboard's PCIe controller (via a Xilinx FPGA)
> could only handle 32-bit DMA addreses.  I think there's a similar quirk in
> the Microsemi PCIe controller on the PolarFire boards, but Conor would know
> for sure.

I don't think this patch would affect those systems that need ZONE_DMA2.
I believe it just makes it possible to disable it in the kernel config.
The platform I'm working on has no memory below 4GB and all the PCIe
devices that I care about are not 32-bit constrained. Therefore I just
want to be able to turn it off in my .config.

Thanks,
Drew