[RFC PATCH bpf-next v2 00/11] mm, bpf: Add BPF into /proc/meminfo

[Yafang Shao <laoar.shao@gmail.com>]

Currently there's no way to get BPF memory usage, while we can only
estimate the usage by bpftool or memcg, both of which are not reliable.

- bpftool
  `bpftool {map,prog} show` can show us the memlock of each map and
  prog, but the memlock is vary from the real memory size. The memlock
  of a bpf object is approximately
  `round_up(key_size + value_size, 8) * max_entries`,
  so 1) it can't apply to the non-preallocated bpf map which may
  increase or decrease the real memory size dynamically. 2) the element
  size of some bpf map is not `key_size + value_size`, for example the
  element size of htab is
  `sizeof(struct htab_elem) + round_up(key_size, 8) + round_up(value_size, 8)`
  That said the differece between these two values may be very great if
  the key_size and value_size is small. For example in my verifaction,
  the size of memlock and real memory of a preallocated hash map are,

  $ grep BPF /proc/meminfo
  BPF:                 350 kB  <<< the size of preallocated memalloc pool

  (create hash map)

  $ bpftool map show
  41549: hash  name count_map  flags 0x0
        key 4B  value 4B  max_entries 1048576  memlock 8388608B

  $ grep BPF /proc/meminfo
  BPF:               82284 kB
 
  So the real memory size is $((82284 - 350)) which is 81934 kB 
  while the memlock is only 8192 kB. 

- memcg  
  With memcg we only know that the BPF memory usage is less than
  memory.kmem.usage_in_bytes (or memory.current in v2). Furthermore, we
  only know that the BPF memory usage is less than $MemTotal if the BPF
  object is charged into root memcg :)

So we need a way to get the BPF memory usage especially there will be
more and more bpf programs running on the production environment. The
memory usage of BPF memory is not trivial, which deserves a new item in
/proc/meminfo.

There're some ways to calculate the BPF memory usage. They all have pros
and cons.

- Option 1: Annotate BPF memory allocation only
  It is how I implemented in RFC v1. You can look into the detail and
  discussion on it via the link below[1]. 
  - pros
    We only need to annotate the BPF memory allocation, and then we can
    find these allocated memory in the free path automatically. So it is
    very easy to use, and we don't need to worry about the stat leak.
  - cons
    We must store the information of these allocated memory, in
    particular the allocated slab objects. So it takes extra memory. If
    we introduce a new member into struct page or add this member into
    page_ext, it will take at least 0.2% of the total memory on 64bit
    system, that is not acceptible.
    One way to reduce this memory overhead is to introduce dynamic page
    extension, but it will take great effort and it may not worth it.

- Option 2: Annotate both allocation and free
  It is similar to how I implemented in an earlier version[2].
  - pros
    There's almost no memory overhead.
  - cons    
    All the memory allocation and free must use the BPF helpers, but
    can't use the generic helpers like kfree/vfree/percpu_free. So if
    the user forget to use the helpers we introduced to allocate or
    free BPF memory, there will be stat leak.
    It is not easy to annotate some derferred allocation, in particular
    the kfree_rcu(). So the user have to use call_rcu() instead of
    kfree_rcu(). Another risk is that if we introduce other deferred
    free helpers in the future, this BPF statistic may break easily.
    
- Option 3: Calculate the memory size via the pointer
  It is how I implement in this patchset.    
  After allocating some BPF memory, we get the full size from the
  pointer and add it; Before freeing the BPF memory, we get the full
  size from the pointer and sub it.
  - pros    
    No memory overhead.    
    No code churn in MM core allocation and free path.
    The impementation is quite clear and easy to maintain.
  - cons
    The calculation is not embedded in the MM allocation/free path, so
    there will be some redundant code to execute to get the size via
    pointer.    
    BPF memory allocation and free must use the helpers we introduced,
    otherwise there will be stat leak.

I perfer the option 3. Its cons can be justified.    
- bpf_map_free should be paired with bpf_map_alloc, that's reasonable.
- Regarding the possible extra cpu cycles it may take, the user should
  not allocate and free memory in the critical path if it is latency
  sensitive. 

[1]. https://lwn.net/Articles/917647/
[2]. https://lore.kernel.org/linux-mm/20220921170002.29557-1-laoar.shao@gmail.com/

v1->v2: don't use page_ext (Vlastimil, Hyeonggon)

Yafang Shao (11):
  mm: percpu: count memcg relevant memory only when kmemcg is enabled
  mm: percpu: introduce percpu_size()
  mm: slab: rename obj_full_size()
  mm: slab: introduce ksize_full()
  mm: vmalloc: introduce vsize()
  mm: util: introduce kvsize()
  bpf: introduce new helpers bpf_ringbuf_pages_{alloc,free}
  bpf: use bpf_map_kzalloc in arraymap
  bpf: use bpf_map_kvcalloc in bpf_local_storage
  bpf: add and use bpf map free helpers
  bpf: introduce bpf memory statistics

 fs/proc/meminfo.c              |   4 ++
 include/linux/bpf.h            | 115 +++++++++++++++++++++++++++++++++++++++--
 include/linux/percpu.h         |   1 +
 include/linux/slab.h           |  10 ++++
 include/linux/vmalloc.h        |  15 ++++++
 kernel/bpf/arraymap.c          |  20 +++----
 kernel/bpf/bpf_cgrp_storage.c  |   2 +-
 kernel/bpf/bpf_inode_storage.c |   2 +-
 kernel/bpf/bpf_local_storage.c |  24 ++++-----
 kernel/bpf/bpf_task_storage.c  |   2 +-
 kernel/bpf/cpumap.c            |  13 +++--
 kernel/bpf/devmap.c            |  10 ++--
 kernel/bpf/hashtab.c           |   8 +--
 kernel/bpf/helpers.c           |   2 +-
 kernel/bpf/local_storage.c     |  12 ++---
 kernel/bpf/lpm_trie.c          |  14 ++---
 kernel/bpf/memalloc.c          |  19 ++++++-
 kernel/bpf/ringbuf.c           |  75 ++++++++++++++++++---------
 kernel/bpf/syscall.c           |  54 ++++++++++++++++++-
 mm/percpu-internal.h           |   4 +-
 mm/percpu.c                    |  35 +++++++++++++
 mm/slab.h                      |  19 ++++---
 mm/slab_common.c               |  52 +++++++++++++------
 mm/slob.c                      |   2 +-
 mm/util.c                      |  15 ++++++
 net/core/bpf_sk_storage.c      |   4 +-
 net/core/sock_map.c            |   2 +-
 net/xdp/xskmap.c               |   2 +-
 28 files changed, 422 insertions(+), 115 deletions(-)

-- 
1.8.3.1