Re: [RFC v2 01/22] kernel/api: introduce kernel API specification framework

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

Em Tue, 1 Jul 2025 10:23:03 -0400
Sasha Levin <sashal@kernel.org> escreveu:

> On Tue, Jul 01, 2025 at 12:20:58AM +0200, Mauro Carvalho Chehab wrote:
> >Em Mon, 30 Jun 2025 13:53:55 -0600
> >Jonathan Corbet <corbet@lwn.net> escreveu:
> >  
> >> Sasha Levin <sashal@kernel.org> writes:
> >>  
> >> > Add a comprehensive framework for formally documenting kernel APIs with
> >> > inline specifications. This framework provides:
> >> >
> >> > - Structured API documentation with parameter specifications, return
> >> >   values, error conditions, and execution context requirements
> >> > - Runtime validation capabilities for debugging (CONFIG_KAPI_RUNTIME_CHECKS)
> >> > - Export of specifications via debugfs for tooling integration
> >> > - Support for both internal kernel APIs and system calls
> >> >
> >> > The framework stores specifications in a dedicated ELF section and
> >> > provides infrastructure for:
> >> > - Compile-time validation of specifications
> >> > - Runtime querying of API documentation
> >> > - Machine-readable export formats
> >> > - Integration with existing SYSCALL_DEFINE macros
> >> >
> >> > This commit introduces the core infrastructure without modifying any
> >> > existing APIs. Subsequent patches will add specifications to individual
> >> > subsystems.
> >> >
> >> > Signed-off-by: Sasha Levin <sashal@kernel.org>
> >> > ---
> >> >  Documentation/admin-guide/kernel-api-spec.rst |  507 ++++++  
> >>
> >> You need to add that file to index.rst in that directory or it won't be
> >> pulled into the docs build.
> >>
> >> Wouldn't it be nice to integrate all this stuff with out existing
> >> kerneldoc mechanism...? :)  
> >
> >+1
> >
> >Having two different mechanisms (kapi and kerneldoc) makes a lot harder
> >to maintain kAPI.  
> 
> I hated the idea of not reusing kerneldoc.
> 
> My concern with kerneldoc was that I can't manipulate the
> information it stores in the context of a kernel build. So for example,
> I wasn't sure how I can expose information stored within kerneldoc via
> debugfs on a running system (or how I can store it within the vmlinux
> for later extraction from the binary built kernel).
> 
> I did some research based on your proposal, and I think I was incorrect
> with the assumption above. I suppose we could do something like the
> following:
> 
> 1. Add new section patterns to doc_sect regex in to include API
> specification sections: api-type, api-version, param-type, param-flags,
> param-constraint, error-code, capability, signal, lock-req, since...
>   
> 2. Create new output module (scripts/lib/kdoc/kdoc_apispec.py?) to
> generate C macro invocations from parsed data.
> 
> Which will generate output like:
> 
>     DEFINE_KERNEL_API_SPEC(function_name)
>         KAPI_DESCRIPTION("...") 
>         KAPI_PARAM(0, "name", "type", "desc")
>             KAPI_PARAM_TYPE(KAPI_TYPE_INT)
>             KAPI_PARAM_FLAGS(KAPI_PARAM_IN)
>         KAPI_PARAM_END
>     KAPI_END_SPEC 

> 3. And then via makefile we can: 
>     - Generate API specs from kerneldoc comments
>     - Include generated specs conditionally based on CONFIG_KERNEL_API_SPEC
> 
> Allowing us to just have these in the relevant source files:
>     #ifdef CONFIG_KERNEL_API_SPEC
>     #include "socket.apispec.h"
>     #endif
> 
> 
> In theory, all of that will let us have something like the following in
> kerneldoc:
> 
> - @api-type: syscall
> - @api-version: 1
> - @context-flags: KAPI_CTX_PROCESS | KAPI_CTX_SLEEPABLE
> - @param-type: family, KAPI_TYPE_INT
> - @param-flags: family, KAPI_PARAM_IN
> - @param-range: family, 0, 45
> - @param-mask: type, SOCK_TYPE_MASK | SOCK_CLOEXEC | SOCK_NONBLOCK
> - @error-code: -EAFNOSUPPORT, "Address family not supported"
> - @error-condition: -EAFNOSUPPORT, "family < 0 || family >= NPROTO"
> - @capability: CAP_NET_RAW, KAPI_CAP_GRANT_PERMISSION
> - @capability-allows: CAP_NET_RAW, "Create SOCK_RAW sockets"
> - @since: 2.0
> - @return-type: KAPI_TYPE_FD
> - @return-check: KAPI_RETURN_ERROR_CHECK
> 
> How does it sound? I'm pretty excited about the possiblity to align this
> with kerneldoc. Please poke holes in the plan :)

Sounds like a plan!

We did something somewhat similar on IGT. 

The python classes there were written with the goal to document
tests, so its examples are related to test docs, but I wrote it
to be generic.

There, all fields comes form a JSON file like this:

	https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/blob/master/tests/intel/xe_test_config.json?ref_type=heads

which describes what fields will be used. It also lists file
patterns that will use it. The fields allow hierarchical
grouping, with could be interesting for some types of fields.

From the json example (I dropped the optional field description
from the example, to make it cleaner):

	"Category": {
	    "Mega feature": {
            	"Sub-category": {},
	    }
	...
 	"Test category": {},
	"Issue": {},
	...

The hierarchical part is useful to properly order kapi content
without the need to add multiple Sphinx markups to manually reorder
the output inside the .rst files.

(*) I would avoid hardcoding the fields/structures, as eventually
    we may need more flexibility to add fields and/or having some
    fields that are specific, for instance, to debugfs or sysfs.

The python class it uses is at:
	https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/blob/master/scripts/test_list.py?ref_type=heads

and caller is at:
	https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/blob/master/scripts/igt_doc.py?ref_type=heads

Eventually you may find something useful there. If so, feel free to
pick from it.

Regards,
Mauro