Re: [Ksummit-discuss] [CORE TOPIC] Kernel tinification: shrinking the kernel and avoiding size regressions

[Steven Rostedt <rostedt@goodmis.org>]

On Fri, 9 May 2014 09:48:19 -0500 (CDT)
Christoph Lameter <cl@linux.com> wrote:

> Static branching means that it is removed from the code path but the
> overall code size still is increased because the function need to be
> somewhere. And usually the additional functions are mixed with other
> functions that are essential. Which means increased need for TLB entries
> to do the virtual mappings. Plus there are noop holes here and there that
> increase the size of the function still.
> 
> One improvement would be to sort the functions by functionality. All the
> important functions in the first 2M of the code covered by one huge tlb
> f.e.

I thought pretty much all of kernel core memory is mapped in by huge
tlbs? At least for kernel core code (not modules), the size should not
impact tlbs.

> 
> Maybe we could reduce the number of cachelines used by critical functions
> too? Arent there some tools that can automatize this in gcc?

As I believe James has mentioned. This only helps if we keep the
critical functions tight in a cacheline. I did some benchmarks moving
the tracepoint code more out of line to help in cachelines, and I
haven't seen anything above the noise. Which is the reason I haven't
pushed that work further.

Size may not be as important as having reuse of code. Perhaps if you
can tweak several functions to call one helper function, which may
actually increase the total size of the kernel, but having more helper
functions that live in cache longer may be of benefit.


> 
> Syscalls are often essential to performance in particular if one wants to
> use the I/O services of the kernel instead of relying on something like
> RDMA that bypasses the kernel.
> 
> In general the ability to reduce the size of the kernel to a minimum is a
> desirable feature. I still see deployments of older kernels in the
> financial industry because they have a higher performance and lower
> latency. The only way to get those guys would be to keep the kernel size
> and the size of the data touched the same.

I actually wonder if that performance is really on "size" of the kernel
and not just less features. Usually with features, we add more function
calls and branches, which I believe may be the culprit of the slowdowns
we are seeing.

-- Steve