Re: [TECH TOPIC] Kernel documentation

[Jonathan Corbet <corbet@lwn.net>]

Jani Nikula <jani.nikula@intel.com> writes:

> It should be more feasible to build the documentation. Make it faster,
> reduce the warnings.
>
> Some ideas to make it faster:
>
> - Bump the minimum Sphinx version requirement if it helps the speed. I
>   don't think it needs to be as conservative as the compiler.

Alas, newer versions of Sphinx are slower, not faster; 2.4 takes about
half the time to build the docs that 5.x does.

A while back, I went into Sphinx with a hatchet and managed to take
about 20% off the build time.  The C domain stuff builds a data
structure of incredible complexity, then just tosses much of it away.
I've never had the time to figure out why they do that or to try to get
my hack job into a condition where I'd be willing to show it to my dog,
much less the Sphinx developers.

I wish we had an active presence in the Sphinx community, but I've never
been able to make that happen myself.

> - Cache kernel-doc results per document. A bunch of .rst files use
>   multiple kernel-doc directives for the same source file to better
>   control the documentation order [1]. Each directive causes the same
>   source to be parsed. (I'm not sure how bad the effect is though.)

That would help, but I don't think this is our biggest problem.

> - Simplify the rst output kernel-doc produces. For example, use rst
>   native field lists for parameter and member descriptions instead of
>   hand-crafting them. See [2]. Drop the "definition" part from
>   structures, as nobody relies on it anyway. If necessary, add links to
>   source instead.

Seems worth a try.

> - Default to Sphinx parallel build.

We *do* default to that, or so I thought.  Much of the build doesn't
actually parallelize, though.

> - Consider splitting the whole documentation to multiple smaller
>   projects, and linking between them using intersphinx. (This may be a
>   tall order.)

This would be nice.  I looked into it a little while back and ran into
some roadblocks; I'll need to go back to my notes to remind myself of
where the problems were.

> Some ideas to reduce warnings:
>
> - W=1 already includes kernel-doc warnings for .c. In i915 we've added
>   that to the regular build as well as a separate target to test
>   headers, and use kernel-doc -Werror for development. Try to get more
>   folks on board.
>
> - Add more warning levels to kernel-doc similar to compilers, and reduce
>   the default warnings. For example, I'm not sure it's necessary to warn
>   about each undocumented parameter/member by default. That could be a
>   verbose option. Bump up the warnings after we've fixed the more
>   glaring issues.

This seems like a good idea.

> - For more verbose checking without Sphinx, it should be possible to
>   lint the rst produced by kernel-doc (originating from source), and
>   check that as part of the build. But that's clearly W=2 stuff or on a
>   subsystem/driver basis.
>
> - Making the Sphinx build faster would also get more people on board
>   fixing the warnings too.

This, I think, is the key point.  The build speed is a real pain point
that impedes contribution.

jon