Re: [PATCH bpf-next v2 0/5] execmem_alloc for BPF programs

[Song Liu <song@kernel.org>]

On Tue, Nov 15, 2022 at 1:09 PM Luis Chamberlain <mcgrof@kernel.org> wrote:
>
> On Mon, Nov 14, 2022 at 05:30:39PM -0800, Song Liu wrote:
> > On Mon, Nov 7, 2022 at 2:41 PM Song Liu <song@kernel.org> wrote:
> > >
> >
> > [...]
> >
> > >
> > >
> > > This set enables bpf programs and bpf dispatchers to share huge pages with
> > > new API:
> > >   execmem_alloc()
> > >   execmem_alloc()
> > >   execmem_fill()
> > >
> > > The idea is similar to Peter's suggestion in [1].
> > >
> > > execmem_alloc() manages a set of PMD_SIZE RO+X memory, and allocates these
> > > memory to its users. execmem_alloc() is used to free memory allocated by
> > > execmem_alloc(). execmem_fill() is used to update memory allocated by
> > > execmem_alloc().
> >
> > Sigh, I just realized this thread made through linux-mm@kvack.org, but got
> > dropped by bpf@vger.kernel.org, so I guess I will have to resend v3.
>
> I don't know what is going on with the bpf list but whatever it is, is silly.
> You should Cc the right folks to ensure proper review if the bpf list is
> the issue.
>
> > Currently, I have got the following action items for v3:
> > 1. Add unify API to allocate text memory to motivation;
> > 2. Update Documentation/x86/x86_64/mm.rst;
> > 3. Allow none PMD_SIZE allocation for powerpc.
>
> - I am really exausted of asking again for real performance tests,
>   you keep saying you can't and I keep saying you can, you are not
>   trying hard enough. Stop thinking about your internal benchmark which
>   you cannot publish. There should be enough crap out which you can use.
>
> - A new selftest or set of selftests which demonstrates gain in
>   performance

I didn't mean to not show the result with publically available. I just
thought the actual benchmark was better (and we do use that to
demonstrate the benefit of a lot of kernel improvement).

For something publically available, how about the following:

Run 100 instances of the following benchmark from bpf selftests:
  tools/testing/selftests/bpf/bench -w2 -d100 -a trig-kprobe
which loads 7 BPF programs, and triggers one of them.

Then use perf to monitor TLB related counters:
   perf stat -e iTLB-load-misses,itlb_misses.walk_completed_4k, \
        itlb_misses.walk_completed_2m_4m -a

The following results are from a qemu VM with 32 cores.

Before bpf_prog_pack:
  iTLB-load-misses: 350k/s
  itlb_misses.walk_completed_4k: 90k/s
  itlb_misses.walk_completed_2m_4m: 0.1/s

With bpf_prog_pack (current upstream):
  iTLB-load-misses: 220k/s
  itlb_misses.walk_completed_4k: 68k/s
  itlb_misses.walk_completed_2m_4m: 0.2/s

With execmem_alloc (with this set):
  iTLB-load-misses: 185k/s
  itlb_misses.walk_completed_4k: 58k/s
  itlb_misses.walk_completed_2m_4m: 1/s

Do these address your questions with this?

>
> - Extensions maybe of lib/test_vmalloc.c or whatever is appropriate to
>   test correctness

I will look into this.

>
> - Enhance commit logs to justify the *why*, one of which to hightight is
>   providing an API for memory semantics for special memory pages

And this.

Thanks,
Song