Re: [LSF/MM/BPF TOPIC] mm documentation

[Lorenzo Stoakes <lstoakes@gmail.com>]

On Sun, May 21, 2023 at 04:50:57PM -0700, David Rientjes wrote:
> On Tue, 16 May 2023, Jonathan Corbet wrote:
>
> > >> > And if we are going to participate in projects like Outreachy and Google
> > >> > Season of Docs, we can suggest a project "Convert Lorenzo's book to kernel
> > >> > documentation" alongside "Write more MM documentation" project.
> > >>
> > >> Yeah, this could be a good starting point actually, as rather than starting
> > >> from zero, people would have some material that they can cross-check.
> > >
> > > I'm going to suggest "MM docs" for the next Outreachy round and I'll ping
> > > you then about the "Convert Lorenzo's book to kernel documentation"
> > > project.
> > >
> > > As for participation in the Google Season of Docs, maybe this should be
> > > more broad than only mm docs.
> > > Jon, what do you think?
> >
> > Sorry, I'm still catching up from travel and have a lot of digging out
> > to do...
> >
> > Converting relevant bits of the book to RST seems like a great project.
> > In theory pandoc can do that, but I've never tried it and can imagine
> > that a fair amount of tweaking is required.
> >
> > Season of docs definitely seems worth looking into, but we've missed the
> > bus for this year.  Something to keep in mind next January.
> >
>
> I think both of these programs are useful for importing mm documentation
> that is up-to-date at the time that it's imported.
>
> I'm concerned that the documentation will quickly become out of date,
> however.  If mm documentation were imported before the introduction of
> folios or per-vma locking, for example, it would likely need large scale
> changes.
>

Yeah, this is one of the ways in which the book differs quite a bit from
the documentation process - I get to 'cheat' in that I have frozen the
target on v6, skip all arches but x86-64 and also don't cover quite a few
things for the sake of brevity/feasibility.

Key purpose is to a. describe core algorithms and concepts that should
(probably) not change too much [relevant to mm docs] and b. describe code
that might become/already is obsolete in considerable detail, intending to
give a smaller delta to book/code + provide basis for reader to go to code
[not relevant to mm docs].

Obviously kernel documentation cannot take this approach.

> Maybe the idea is that any documentation, no matter how outdated, is
> better than no documentation.  This may not always be the case, however:
> how often have people become frustrated that the documentation that does
> exist doesn't actually reflect the current state of the kernel anymore?
>

My thinking on book/doc overlap was precisely the area for which docs can
be written without too much concern about obsolescence - core
concepts/algorithms that are highly unlikely to change any time soon.

This kind of area seems like a good early target for docs, and Mike has
already gone down this road with the (excellent) node/zone documentation.

I do want to reiterate what I raised at LSF/MM, which is that documentation
is the ultimate bikeshed bait and we do have to try to reach a bar of 'good
enough' to prevent endless nitpicks slowing documentation patches to a
glacial pace.

Perhaps providing broad strokes descriptions of e.g. page cache/reclaim can
set the groundwork and momentum for future changes. Adjusting a document to
tweak a description of how reclaim works in scenario XYZ is a lot less
daunting than having to write something from scratch.

> Mike, I think one of the goals you were trying to achieve at LSF/MM/BPF
> was how this documentation would not only get originally created/imported,
> but also how it would stay relatively up-to-date.  To keep the
> documentation current, I think the burden would likely need to fall on the
> kernel hakcers who are actually developing the code?
>

I will let Mike comment on this obviously but from my perspective and as
discussed at LSF/MM, process seems to me to be absolutely key to this - new
features such as per-VMA locks or maple trees should proactively _require_
documentation in my view.

Equally changes to logic that would render documentation out of date should
require changes there also.

As a hobbyist my constraints differ quite a bit from you guys working full
time on this so I realise there might be difficulty justifying time on such
things, but maintainers have the ability to insist + establish new
requirements :)

Just my 2 English pence however, as the process side is something that
maintainers need to figure out.

I feel like this is a game of two halves - the core algos are the
low-hanging fruit, as they can be written at our leisure. The detailed
stuff that is in constant flux really requires a change in process to be
viable.