Re: Ipmi modules and linux-4.19.1

[Tejun Heo <tj@kernel.org>]

Hello, Angel.

(cc'ing Paul for SRCU)

On Thu, Dec 20, 2018 at 09:55:10AM +0200, Angel Shtilianov wrote:
> Hi everybody.
> A couple of days I've decided to migrate several servers on
> linux-4.19. What I've observed is that I have no /dev/ipmi. After
> taking a look into the boot log I've found that ipmi modules are
> complaining about percpu memory allocation failures:
> https://pastebin.com/MCDssZzV
...
> -#define PERCPU_DYNAMIC_RESERVE         (28 << 10)
> +#define PERCPU_DYNAMIC_RESERVE         (28 << 11)

So, you prolly just needed to bump this number.  The reserved percpu
area is used to accommodate static percpu variables used by modules.
They are special because code generation assumes static symbols aren't
too far from the program counter.  The usual dynamic percpu area is
way high up in vmalloc area, so if we put static percpu allocations
there, they go out of range for module symbol relocations.

The reserved area has some issues.

1. The area is not dynamically mapped, meaning that however much we
   reserve is hard allocated on boot for future module uses, so we
   don't can't increase it willy-nilly.

2. There is no mechanism to adjust the size dynamically.  28k is just
   a number I pulled out of my ass after looking at some common
   configs like a decade ago, so it being low now isn't too
   surprising.  Provided that we can't make it run-time dynamic (and I
   can't think of a way to do that), the right thing to do would be
   sizing it during build with some buffer and allow it to be
   overridden boot time.  This is definitely doable.

BTW, ipmi's extra usage, 8k, is coming from the use of static SRCU.
Paul, that's quite a bit of percpu memory to reserve statically.
Would it be possible to make srcu_struct init dynamic so that it can
use the normal percpu_alloc?  That way, this problem can be completely
side-stepped and it only occupies percpu memory which tends to be
pretty expensive unless ipmi is actually initialized.

Thanks.

-- 
tejun