Re: [PATCH 0/7] minmax: reduce compilation time

[Lorenzo Stoakes <lorenzo.stoakes@oracle.com>]

On Fri, Jul 26, 2024 at 07:56:31PM GMT, Lorenzo Stoakes wrote:
> On Fri, Jul 26, 2024 at 11:24:09AM GMT, Linus Torvalds wrote:
> > On Fri, 26 Jul 2024 at 11:13, Lorenzo Stoakes
> > <lorenzo.stoakes@oracle.com> wrote:
> > >
> > > 5,447,539       ./arch/x86/xen/setup.o.pre
> >
> > Can you perhaps do some kind of "max expansion" on all the
> > preprocessor files (you seem to have done it by changing the ".c.o"
> > rule to just spit it out as "o.pre", which sounds fine).
>
> Yeah I simply hacked in a gcc -E, ugly but effective, e.g. prepending
> cmd_cc_o_c in scripts/Makefile.build with:
>
> cmd_cc_o_c = $(CC) $(c_flags) -E $< > $@.pre; \
>
> (with credit to Arnd, who came up with a timing patch using a similar
> technique, I just super trivially adapted to do this instead).
>
> I therefore generate a bunch of .pre files, + run this to get total
> numbers:
>
> find . -iname '*.pre' -exec du -cb {} + | grep total$ | cut -f1 | paste -sd+ - | bc
>
> So I have the numbers for all of the files. Did you want me to post the
> numbers somewhere, or were you after someting cleverer, not sure what you
> mean by "max expansion"?
>
> >
> > For example, this trivial patch seems to fix the setup.c expansion by
> > about an order of magnitude (ie 50M -> 5M).
> >
> > Entirely untested, but looks ObviouslyCorrect(tm) to me.
> >
> >                      Linus
> >
> >                 Linus
>
> Yeah this has a big impact, with just this patch we go from 51,407,944 to
> 4,351,747 locally for me, so match your results.
>
> Patch looks obviously correct to me, I wonder if we should just apply that
> now and take out one of the worst offenders while we figure out how to fix
> the broader problem?
>
> Reviewed-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> fwiw on that :)

A simple comparison pre-revert vs. post-revert gives some ideas for other
low-hanging fruit:

  389180./drivers/block/loop.o.pre
  409741./net/ipv4/tcp_output.o.pre
  462230./net/ipv4/tcp_input.o.pre
  470714./mm/page_alloc.o.pre
  570698./block/bio.o.pre
  600206./net/ipv6/proc.o.pre
  761333./arch/x86/kernel/cpu/cacheinfo.o.pre
  1315265./drivers/gpu/drm/i915/display/intel_pmdemand.o.pre
  1334256./mm/compaction.o.pre
  45960405./arch/x86/xen/setup.o.pre

I might try an allmodconfig with David's patches to see if that pulls out any
other really obvious cases.