Re: [PATCH v8 mm-new 04/12] mm: thp: add support for BPF based THP order selection

[Yafang Shao <laoar.shao@gmail.com>]

On Fri, Sep 26, 2025 at 11:13 PM Usama Arif <usamaarif642@gmail.com> wrote:
>
>
>
> On 26/09/2025 10:33, Yafang Shao wrote:
> > This patch introduces a new BPF struct_ops called bpf_thp_ops for dynamic
> > THP tuning. It includes a hook bpf_hook_thp_get_order(), allowing BPF
> > programs to influence THP order selection based on factors such as:
> > - Workload identity
> >   For example, workloads running in specific containers or cgroups.
> > - Allocation context
> >   Whether the allocation occurs during a page fault, khugepaged, swap or
> >   other paths.
> > - VMA's memory advice settings
> >   MADV_HUGEPAGE or MADV_NOHUGEPAGE
> > - Memory pressure
> >   PSI system data or associated cgroup PSI metrics
> >
> > The kernel API of this new BPF hook is as follows,
> >
> > /**
> >  * thp_order_fn_t: Get the suggested THP order from a BPF program for allocation
> >  * @vma: vm_area_struct associated with the THP allocation
> >  * @type: TVA type for current @vma
> >  * @orders: Bitmask of available THP orders for this allocation
> >  *
> >  * Return: The suggested THP order for allocation from the BPF program. Must be
> >  *         a valid, available order.
> >  */
> > typedef int thp_order_fn_t(struct vm_area_struct *vma,
> >                          enum tva_type type,
> >                          unsigned long orders);
> >
> > Only a single BPF program can be attached at any given time, though it can
> > be dynamically updated to adjust the policy. The implementation supports
> > anonymous THP, shmem THP, and mTHP, with future extensions planned for
> > file-backed THP.
> >
> > This functionality is only active when system-wide THP is configured to
> > madvise or always mode. It remains disabled in never mode. Additionally,
> > if THP is explicitly disabled for a specific task via prctl(), this BPF
> > functionality will also be unavailable for that task.
> >
> > This BPF hook enables the implementation of flexible THP allocation
> > policies at the system, per-cgroup, or per-task level.
> >
> > This feature requires CONFIG_BPF_THP_GET_ORDER_EXPERIMENTAL to be
> > enabled. Note that this capability is currently unstable and may undergo
> > significant changes—including potential removal—in future kernel versions.
> >
> > Suggested-by: David Hildenbrand <david@redhat.com>
> > Suggested-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
> > Signed-off-by: Yafang Shao <laoar.shao@gmail.com>
> > ---
> >  MAINTAINERS             |   1 +
> >  include/linux/huge_mm.h |  23 +++++
> >  mm/Kconfig              |  12 +++
> >  mm/Makefile             |   1 +
> >  mm/huge_memory_bpf.c    | 204 ++++++++++++++++++++++++++++++++++++++++
> >  5 files changed, 241 insertions(+)
> >  create mode 100644 mm/huge_memory_bpf.c
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index ca8e3d18eedd..7be34b2a64fd 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -16257,6 +16257,7 @@ F:    include/linux/huge_mm.h
> >  F:   include/linux/khugepaged.h
> >  F:   include/trace/events/huge_memory.h
> >  F:   mm/huge_memory.c
> > +F:   mm/huge_memory_bpf.c
> >  F:   mm/khugepaged.c
> >  F:   mm/mm_slot.h
> >  F:   tools/testing/selftests/mm/khugepaged.c
> > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> > index a635dcbb2b99..fea94c059bed 100644
> > --- a/include/linux/huge_mm.h
> > +++ b/include/linux/huge_mm.h
> > @@ -56,6 +56,7 @@ enum transparent_hugepage_flag {
> >       TRANSPARENT_HUGEPAGE_DEFRAG_REQ_MADV_FLAG,
> >       TRANSPARENT_HUGEPAGE_DEFRAG_KHUGEPAGED_FLAG,
> >       TRANSPARENT_HUGEPAGE_USE_ZERO_PAGE_FLAG,
> > +     TRANSPARENT_HUGEPAGE_BPF_ATTACHED,      /* BPF prog is attached */
> >  };
> >
> >  struct kobject;
> > @@ -269,6 +270,23 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
> >                                        enum tva_type type,
> >                                        unsigned long orders);
> >
> > +#ifdef CONFIG_BPF_THP_GET_ORDER_EXPERIMENTAL
> > +
> > +unsigned long
> > +bpf_hook_thp_get_orders(struct vm_area_struct *vma, enum tva_type type,
> > +                     unsigned long orders);
> > +
> > +#else
> > +
> > +static inline unsigned long
> > +bpf_hook_thp_get_orders(struct vm_area_struct *vma, enum tva_type type,
> > +                     unsigned long orders)
> > +{
> > +     return orders;
> > +}
> > +
> > +#endif
> > +
> >  /**
> >   * thp_vma_allowable_orders - determine hugepage orders that are allowed for vma
> >   * @vma:  the vm area to check
> > @@ -290,6 +308,11 @@ unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma,
> >  {
> >       vm_flags_t vm_flags = vma->vm_flags;
> >
> > +     /* The BPF-specified order overrides which order is selected. */
> > +     orders &= bpf_hook_thp_get_orders(vma, type, orders);
> > +     if (!orders)
> > +             return 0;
> > +
> >       /*
> >        * Optimization to check if required orders are enabled early. Only
> >        * forced collapse ignores sysfs configs.
> > diff --git a/mm/Kconfig b/mm/Kconfig
> > index bde9f842a4a8..fd7459eecb2d 100644
> > --- a/mm/Kconfig
> > +++ b/mm/Kconfig
> > @@ -895,6 +895,18 @@ config NO_PAGE_MAPCOUNT
> >
> >         EXPERIMENTAL because the impact of some changes is still unclear.
> >
> > +config BPF_THP_GET_ORDER_EXPERIMENTAL
> > +     bool "BPF-based THP order selection (EXPERIMENTAL)"
> > +     depends on TRANSPARENT_HUGEPAGE && BPF_SYSCALL
> > +
> > +     help
> > +       Enable dynamic THP order selection using BPF programs. This
> > +       experimental feature allows custom BPF logic to determine optimal
> > +       transparent hugepage allocation sizes at runtime.
> > +
> > +       WARNING: This feature is unstable and may change in future kernel
> > +       versions.
> > +
>
> I am assuming this series opens up the possibility of additional hooks being added in
> the future. Instead of naming this BPF_THP_GET_ORDER_EXPERIMENTAL, should we
> name it BPF_THP? Otherwise we will end up with 1 Kconfig option per hook, which
> is quite bad.

makes sense.

>
> Also It would be really nice if we dont put "EXPERIMENTAL" in the name of the defconfig.
> If its decided that its not experimental anymore without any change to the code needed,
> renaming the defconfig will break it for everyone.

makes sense to me.
Lorenzo, what do you think ?

>
>
> >  endif # TRANSPARENT_HUGEPAGE
> >
> >  # simple helper to make the code a bit easier to read
> > diff --git a/mm/Makefile b/mm/Makefile
> > index 21abb3353550..62ebfa23635a 100644
> > --- a/mm/Makefile
> > +++ b/mm/Makefile
> > @@ -99,6 +99,7 @@ obj-$(CONFIG_MIGRATION) += migrate.o
> >  obj-$(CONFIG_NUMA) += memory-tiers.o
> >  obj-$(CONFIG_DEVICE_MIGRATION) += migrate_device.o
> >  obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += huge_memory.o khugepaged.o
> > +obj-$(CONFIG_BPF_THP_GET_ORDER_EXPERIMENTAL) += huge_memory_bpf.o
> >  obj-$(CONFIG_PAGE_COUNTER) += page_counter.o
> >  obj-$(CONFIG_MEMCG_V1) += memcontrol-v1.o
> >  obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o
> > diff --git a/mm/huge_memory_bpf.c b/mm/huge_memory_bpf.c
> > new file mode 100644
> > index 000000000000..b59a65d70a93
> > --- /dev/null
> > +++ b/mm/huge_memory_bpf.c
> > @@ -0,0 +1,204 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * BPF-based THP policy management
> > + *
> > + * Author: Yafang Shao <laoar.shao@gmail.com>
> > + */
> > +
> > +#include <linux/bpf.h>
> > +#include <linux/btf.h>
> > +#include <linux/huge_mm.h>
> > +#include <linux/khugepaged.h>
> > +
> > +/**
> > + * @thp_order_fn_t: Get the suggested THP order from a BPF program for allocation
> > + * @vma: vm_area_struct associated with the THP allocation
> > + * @type: TVA type for current @vma
> > + * @orders: Bitmask of available THP orders for this allocation
> > + *
> > + * Return: The suggested THP order for allocation from the BPF program. Must be
> > + *         a valid, available order.
> > + */
> > +typedef int thp_order_fn_t(struct vm_area_struct *vma,
> > +                        enum tva_type type,
> > +                        unsigned long orders);
> > +
> > +struct bpf_thp_ops {
> > +     thp_order_fn_t __rcu *thp_get_order;
> > +};
> > +
> > +static struct bpf_thp_ops bpf_thp;
> > +static DEFINE_SPINLOCK(thp_ops_lock);
> > +
> > +unsigned long bpf_hook_thp_get_orders(struct vm_area_struct *vma,
> > +                                   enum tva_type type,
> > +                                   unsigned long orders)
> > +{
> > +     thp_order_fn_t *bpf_hook_thp_get_order;
> > +     int bpf_order;
> > +
> > +     /* No BPF program is attached */
> > +     if (!test_bit(TRANSPARENT_HUGEPAGE_BPF_ATTACHED,
> > +                   &transparent_hugepage_flags))
> > +             return orders;
> > +
> > +     rcu_read_lock();
> > +     bpf_hook_thp_get_order = rcu_dereference(bpf_thp.thp_get_order);
> > +     if (!bpf_hook_thp_get_order)
>
> Should we warn over here if we are going to out? TRANSPARENT_HUGEPAGE_BPF_ATTACHED
> being set + !bpf_hook_thp_get_order shouldnt be possible, right?

will add a warning in the next version.

-- 
Regards
Yafang