Re: [PATCH] proc/meminfo: add MemKernel counter

[Konstantin Khlebnikov <koct9i@gmail.com>]

On Wed, May 22, 2019 at 8:04 PM Roman Gushchin <guro@fb.com> wrote:
>
> On Wed, May 22, 2019 at 07:09:22PM +0300, Konstantin Khlebnikov wrote:
> > On 22.05.2019 18:52, Michal Hocko wrote:
> > > On Wed 22-05-19 17:40:09, Konstantin Khlebnikov wrote:
> > > > Some kinds of kernel allocations are not accounted or not show in meminfo.
> > > > For example vmalloc allocations are tracked but overall size is not shown
> > > > for performance reasons. There is no information about network buffers.
> > > >
> > > > In most cases detailed statistics is not required. At first place we need
> > > > information about overall kernel memory usage regardless of its structure.
> > > >
> > > > This patch estimates kernel memory usage by subtracting known sizes of
> > > > free, anonymous, hugetlb and caches from total memory size: MemKernel =
> > > > MemTotal - MemFree - Buffers - Cached - SwapCached - AnonPages - Hugetlb.
> > >
> > > Why do we need to export something that can be calculated in the
> > > userspace trivially? Also is this really something the number really
> > > meaningful? Say you have a driver that exports memory to the userspace
> > > via mmap but that memory is not accounted. Is this really a kernel
> > > memory?
> > >
> >
> > It may be trivial right now but not fixed.
> > Adding new kinds of memory may change this definition.
>
> Right, and it's what causes me to agree with Michal here, and leave it
> to the userspace calculation.
>
> The real meaning of the counter is the size of the "gray zone",
> basically the memory which we have no clue about.

Well, all kernel memory is a gray zone for normal programmers.
They have direct control only over anon and file-cache.

I want to invent simple metrics for 'system' memory usage.
It's about the same as separation cpu time to user and system.

> If we'll add accounting of some new type of memory, which now in this
> gray zone (say, xfs buffers), we probably should exclude it too.
> And this means that definition of this counter will change.

I'm not very familiar with xfs internals, never digged into it.
I've excluded buffers because this is simply file-cache for block devices.
Filesystems use it as cache for metadata. But userspace has direct access to it.

>
> So IMO the definition is way too implementation-defined to be a part
> of procfs API.
>

Ok. User/kernel memory separation could be redefined in more
abstract manner depending on the data access.