[LSF/MM/BPF TOPIC] Opening HugeTLB up for more generic use

[LSF/MM/BPF TOPIC] Opening HugeTLB up for more generic use

[Ackerley Tng]

Hi all,

I would like to propose to open HugeTLB up for more generic use.

Motivation
==========

This proposal was motivated by guest_memfd needing to provide huge
pages. guest_memfd wraps existing allocators to provide memory for
backing KVM virtual machines, and wants to use HugeTLB as a source of
huge pages.

Proposal details
================

The proposal is to (re)think the in-memory filesystems as memory
providers around memory sources:

| Memory Provider                | Memory Source            |
|--------------------------------|--------------------------|
| Anonymous (mmap(MAP_ANONYMOUS) | Buddy allocator          |
| Tmpfs/shmem                    | Buddy allocator          |
| HugeTLBfs                      | HugeTLB                  |
| guest_memfd                    | Buddy allocator, HugeTLB |

While working on HugeTLB and exploring HugeTLBfs vs HugeTLB, my
impression is that these two are overly coupled together and could
benefit from better conceptual separation, as well as code separation
between mm/hugetlb.c and fs/hugetlbfs/inode.c.

Status
======

I've been working on guest_memfd HugeTLB support for a while now, and
Google is in the process of qualifying guest_memfd HugeTLB
support. Here [1] is an earlier version of what's being qualified.

Here [2] is an RFC patch series, split out from [1] and updated, that
refactors the HugeTLB allocation routine for more generic use. I hope
this will serve as a starting point for this discussion.

Goals of the discussion
=======================

+ Find out what the community thinks of opening up HugeTLB as a more
  generic source of huge pages, not coupled to HugeTLBfs
    + Get feedback on guest_memfd using HugeTLB
+ Find out what the plan is for HugeTLB(fs) testing
    + Is the plan to continue maintaining tests in libhugetlbfs?
    + Should the tests be migrated as kernel selftests?
+ Gather key components that would need refactoring

Requested attendees
===================

+ David Hildenbrand
+ Muchun Song
+ Oscar Salvador
+ Peter Xu

References
==========

[1] https://lore.kernel.org/all/cover.1726009989.git.ackerleytng@google.com/T/
[2] https://lore.kernel.org/all/cover.1770854662.git.ackerleytng@google.com/T/
[3] https://github.com/libhugetlbfs/libhugetlbfs/


Thanks,

Ackerley