Re: mm: percpu: increase PERCPU_MODULE_RESERVE to avoid allocation failure

[Dennis Zhou <dennis@kernel.org>]

Hi Andrew,

On Tue, Apr 22, 2025 at 05:02:09PM -0700, Andrew Morton wrote:
> On Tue, 22 Apr 2025 11:39:30 +0000 gaoxu <gaoxu2@honor.com> wrote:
> 
> > In android16-6.12, enabling CONFIG_MEM_ALLOC_PROFILING causes some modules
> > to fail to load during boot because of failed percpu memory allocation.
> 
> Which modules?  If they're in-tree modules then we should fix this
> issue in -stable kernels also.
> 
> If they're out-of-tree modules then what argument is there for altering
> the mainline kernel?
> 
> > [811:modprobe]percpu: allocation failed, size=5200 align=8 atomic=0, alloc
> > from reserved chunk failed
> > [811:modprobe]Call trace:
> > [811:modprobe] dump_backtrace+0xfc/0x17c
> > [811:modprobe] show_stack+0x18/0x28
> > [811:modprobe] dump_stack_lvl+0x40/0xc0
> > [811:modprobe] dump_stack+0x18/0x24
> > [811:modprobe] pcpu_alloc_noprof+0x96c/0xb58
> > [811:modprobe] percpu_modalloc+0x50/0xec
> > [811:modprobe] load_module+0x1158/0x153c
> > [811:modprobe] __arm64_sys_finit_module+0x23c/0x340
> > [811:modprobe] invoke_syscall+0x58/0x10c
> > [811:modprobe] el0_svc_common+0xa8/0xdc
> > [811:modprobe] do_el0_svc+0x1c/0x28
> > [811:modprobe] el0_svc+0x40/0x90
> > [811:modprobe] el0t_64_sync_handler+0x70/0xbc
> > [811:modprobe] el0t_64_sync+0x1a8/0x1ac
> > [811:modprobe]ipam: Could not allocate 5200 bytes percpu data
> > 
> > Increase PERCPU_MODULE_RESERVE to resolve this issue.
> > 
> > ...
> >
> > --- a/include/linux/percpu.h
> > +++ b/include/linux/percpu.h
> > @@ -16,7 +16,7 @@
> >  /* enough to cover all DEFINE_PER_CPUs in modules */
> >  #ifdef CONFIG_MODULES
> >  #ifdef CONFIG_MEM_ALLOC_PROFILING
> > -#define PERCPU_MODULE_RESERVE		(8 << 13)
> > +#define PERCPU_MODULE_RESERVE		(8 << 14)
> >  #else
> >  #define PERCPU_MODULE_RESERVE		(8 << 10)
> >  #endif
> 
> PERCPU_MODULE_RESERVE is a pretty unpleasant thing.  It appears that it
> gives us the choice between either wasting memory or failing module
> loading.  But I expect that something more dynamic would be a ton of work.

From Tj's commit back in 2009... 6b19b0c24004

+/*
+ * On x86_64 symbols referenced from code should be reachable using
+ * 32bit relocations.  Reserve space for static percpu variables in
+ * modules so that they are always served from the first chunk which
+ * is located at the percpu segment base.  On x86_32, anything can
+ * address anywhere.  No need to reserve space in the first chunk.
+ */

I'm not too sure where our x86_64 32 bit support is. If that is no
longer true then we can likely fold the reserved region back into the
dynamic region.

Given the above, there's not really an opportunity to do this after the
system has booted hence why it's baked into the first chunk.

Thanks,
Dennis