RE: [Lsf-pc] [LSF/MM/BPF TOPIC] Machine Learning (ML) library in Linux kernel

[Viacheslav Dubeyko <Slava.Dubeyko@ibm.com>]

On Wed, 2026-02-11 at 10:55 +0100, Jan Kara wrote:
> On Tue 10-02-26 21:02:12, Viacheslav Dubeyko wrote:
> > On Tue, 2026-02-10 at 14:47 +0100, Jan Kara wrote:
> > > On Mon 09-02-26 22:28:59, Viacheslav Dubeyko via Lsf-pc wrote:
> > > > The idea is to have ML model running in user-space and kernel subsystem can
> > > > interact with ML model in user-space. As the next step, I am considering two
> > > > real-life use-cases: (1) GC subsystem of LFS file system, (2) ML-based DAMON
> > > > approach. So, for example, GC can be represented by ML model in user-space. GC
> > > > can request data (segments state) from kernel-space and ML model in user-space
> > > > can do training or/and inference. As a result, ML model in user-space can select
> > > > victim segments and instruct kernel-space logic of moving valid data from victim
> > > > segment(s) into clean/current one(s). 
> > > 
> > > To be honest I'm skeptical about how generic this can be. Essentially
> > > you're describing a generic interface to offload arbitrary kernel decision
> > > to userspace. ML is a userspace bussiness here and not really relevant for
> > > the concept AFAICT. And we already have several ways of kernel asking
> > > userspace to do something for it and unless it is very restricted and well
> > > defined it is rather painful, prone to deadlocks, security issues etc.
> > 
> > Scepticism is normal reaction. :) So, nothing wrong is to be sceptical.
> > 
> > I believe it can be pretty generic from the data flow point of view. Probably,
> > different kernel subsystems could require different ways of interaction with
> > user-space. However, if we are talking about data flow but NOT execution flow,
> > then it could be generic enough. And if it can be generic, then we can suggest
> > generic way of extending any kernel subsystem by ML support.
> > 
> > I don't think that we need to consider the ML library appraoch like "kernel
> > asking userspace to do something". Rather it needs to consider the model like
> > "kernel share data with user-space and user-space recommends something to
> > kernel". So, user-space agent (ML model) can request data from kernel space or
> > kernel subsystem can notify the user-space agent that data is available. And
> > it's up to kernel subsystem implementation which data could be shared with user-
> > space. So, ML model can be trained in user-space and, then, share
> > recommendations (or eBPF code, for example) with kernel space. Finally, it's up
> > to kernel subsystem how and when to apply these recommendations on kernel side.
> 
> I guess I have to see some examples. Because so far it sounds so generic
> that I'm failing to see a value in this :)

I completely see your point. And I am not going to push anything abstract one. I
am going to implement ML-based approach for several real-life use-cases. So, I
will have something real or I will fail. :)

> 
> > > So by all means if you want to do GC decisions for your filesystem in
> > > userspace by ML, be my guest, it does make some sense although I'd be wary
> > > of issues where we need to writeback dirty pages to free memory which may
> > > now depend on your userspace helper to make a decision which may need the
> > > memory to do the decision... But I don't see why you need all the ML fluff
> > > around it when it seems like just another way to call userspace helper and
> > > why some of the existing methods would not suffice.
> > > 
> > 
> > OK. I see. :) You understood GC like a subsystem that helps to kernel
> > memory subsystem to manage the writeback dirty memory pages. :) It's
> > potential direction and I like your suggestion. :) But I meant something
> > different because I consider of LFS file system's GC subsystem. So, if we
> > are using Copy-On-Write (COW) policy, then we have segments or erase
> > blocks with a mixture of valid and invalid logical blocks after update
> > operations. And we need GC subsystem to clean old segments by means of
> > moving valid logical blocks from exhausted segments into clean/current
> > ones. The problem here is to find an efficient algorithm of selecting
> > victim segments with smallest amount of valid blocks with the goal of
> > decreasing write amplification. So, file system needs to share the
> > metadata details (segments state, for example), ML model can share the
> > recommendations, and kernel code of file system can finally move valid
> > blocks in the background.
> 
> No, I actually meant the LFS file system GC as you talk about it. But I was
> just too terse about my concerns: As you said an LFS with COW needs to
> select a new position to write each block. When there is no free block
> available, it has to select partially used erase block (some logical blocks
> in it became invalid) to reuse.
> 

I assume that you imply F2FS here. Because, I cannot imagine how LFS file system
(like NILFS2) can do something like this. If it's LFS file system, then you add
logs into the current segment(s). Even if some logical blocks have been
invalidated into this segment, then you add another log into the head/tail of
current segment until complete exhaustion of it. And it needs to allocate the
completely clean/free segment to be current and receive the logs. So, you need
to take completely exhausted segment for cleaning by GC. If you have pure COW
file system, then you cannot write anything in likewise segment until complete
invalidation + "erase"/clean. So, GC moves valid blocks from completely
exhausted segment into the current one(s). It's responsibility of GC to
guarantee that file system is not running out of free physical space if file
system still has free logical blocks. And if we are running out free physical
space, then operation stops because of GC failure to keep enough clean segments.

>  And for this selection you want to use ML
> AFAIU. Hence we have a dependency folio writeback -> COW block allocation ->
> GC to make some block free -> ML decision.
> 

Usually, GC works in the background. So, ML model in user-space get segments
state metadata from file system. Then, it selects one or several segments and
recommends to file system of moving valid blocks for the selected segment(s) ID
+ maximal amount of valid blocks for single operation. Background process of
file system checks that these logical blocks of exhausted segment are still
valid and initiates operation of moving into the current segment by adding
another log.

Finally, we have two flows: (1) regular file system operations: folio writeback
-> COW block allocation -> add log into current segment; (2) GC operations: ML
GC decision -> recommendation of moving valid blocks for segment -> check that
logical block is still valid -> read block content (ignore logical block if we
have folio in page cache) -> add log into current segment -> update metadata.

Thanks,
Slava.

>  And now you have to be really
> careful so that "ML decision" doesn't even indirectly depend on folio
> writeback to complete. And bear in mind that e.g. if the code doing "ML
> decision" dirties some mmaped file pages it *will* block waiting for page
> writeback to complete to get the system below the limit of dirty pages.
> This is the kind of deadlock I'm talking about that is hard to avoid when
> offloading kernel decisions to userspace (and yes, I've seen these kind of
> deadlocks in practice in various shapes and forms with various methods when
> kernel depended on userspace to make forward progress).
> 
> 								Honza