Re: [RFC PATCH 0/4] Add support for File Based Memory Management

[David Hildenbrand <david@redhat.com>]

On 23.11.24 13:23, Lorenzo Stoakes wrote:
> + VMA guys, it's important to run scripts/get_maintainers.pl on your
> changes so the right people are pinged :)
> 
> On Fri, Nov 22, 2024 at 02:38:26PM -0600, Bijan Tabatabai wrote:
>> This patch set implements file based memory management (FBMM) [1], a
>> research project from the University of Wisconsin-Madison where a process's
>> memory can be transparently managed by memory managers which are written as
>> filesystems. When using FBMM, instead of using the traditional anonymous
>> memory path, a process's memory is managed by mapping files from a memory
>> management filesystem (MFS) into its address space. The MFS implements the
>> memory management related callback functions provided by the VFS to
>> implement the desired memory management functionality. After presenting
>> this work at a conference, a handful of people asked if we were going to
>> upstream the work, so we decided to see if the Linux community would be
>> interested in this functionality as well.
>>
> 
> While it's a cool project, I don't think it's upstreamable in its current
> form - it essentially bypasses core mm functionality and 'does mm'
> somewhere else (which strikes me, in effect, as the entire purpose of the
> series).
> 
> mm is a subsystem that is in constant flux with many assumptions that one
> might make about it being changed, which make it wholly unsuited to having
> its functionality exported like this.
> 
> So in in effect it, by its nature, has to export internals somewhere else,
> and that somewhere else now assumes things about mm that might change at
> any point, additionally bypassing a great deal of highly sensitive and
> purposeful logic.
> 
> This series also adds a lot of if (fbmm) { ... } changes to core logic
> which is really not how we want to do things. hugetlbfs does this kind of
> thing, but it is more or less universally seen as a _bad thing_ and
> something we are trying to refactor.
> 
> So any upstreamable form of this would need to a. be part of mm, b. use
> existing extensible mechanisms or create them, and c. not have _core_ mm
> tasks or activities be performed 'elsewhere'.
> 
> Sadly I think the latter part may make a refactoring in this direction
> infeasible, as it seems to me this is sort of the point of this.
> 
> This also means it's not acceptable to export highly sensitive mm internals
> as you do in patch 3/4. Certainly in 1/4, as a co-maintainer of the mmap
> logic, I can't accept the changes you suggest to brk() and mmap(), sorry.
> 
> There are huge subtleties in much of mm, including very very sensitive lock
> mechanisms, and keeping such things within mm means we can have confidence
> they work, and that fixes resolve issues.
> 
> I hope this isn't too discouraging, the fact you got this functioning is
> amazing and as an out-of-tree research and experimentation project it looks
> really cool, but for me, I don't think this is for upstream.

I agreed with this sentiment. It looks like something a research OS 
might want to consider as it's way of dealing with anonymous memory in 
general, but nothing on squeezes into an existing MM implementation.

I'm also not 100% sure on statements like "Providing this transparency" 
-- what about fork() and COW? What about memory statistics? 
"Transparency" is a strong word :)

-- 
Cheers,

David / dhildenb