Re: [RFC PATCH -mm 0/4] mm, security, bpf: Fine-grained control over memory policy adjustments with lsm bpf

[Yafang Shao <laoar.shao@gmail.com>]

On Mon, Nov 13, 2023 at 4:32 AM Paul Moore <paul@paul-moore.com> wrote:
>
> On Sun, Nov 12, 2023 at 2:35 AM Yafang Shao <laoar.shao@gmail.com> wrote:
> >
> > Background
> > ==========
> >
> > In our containerized environment, we've identified unexpected OOM events
> > where the OOM-killer terminates tasks despite having ample free memory.
> > This anomaly is traced back to tasks within a container using mbind(2) to
> > bind memory to a specific NUMA node. When the allocated memory on this node
> > is exhausted, the OOM-killer, prioritizing tasks based on oom_score,
> > indiscriminately kills tasks. This becomes more critical with guaranteed
> > tasks (oom_score_adj: -998) aggravating the issue.
> >
> > The selected victim might not have allocated memory on the same NUMA node,
> > rendering the killing ineffective. This patch aims to address this by
> > disabling MPOL_BIND in container environments.
> >
> > In the container environment, our aim is to consolidate memory resource
> > control under the management of kubelet. If users express a preference for
> > binding their memory to a specific NUMA node, we encourage the adoption of
> > a standardized approach. Specifically, we recommend configuring this memory
> > policy through kubelet using cpuset.mems in the cpuset controller, rather
> > than individual users setting it autonomously. This centralized approach
> > ensures that NUMA nodes are globally managed through kubelet, promoting
> > consistency and facilitating streamlined administration of memory resources
> > across the entire containerized environment.
> >
> > Proposed Solutions
> > =================
> >
> > - Introduce Capability to Disable MPOL_BIND
> >   Currently, any task can perform MPOL_BIND without specific capabilities.
> >   Enforcing CAP_SYS_RESOURCE or CAP_SYS_NICE could be an option, but this
> >   may have unintended consequences. Capabilities, being broad, might grant
> >   unnecessary privileges. We should explore alternatives to prevent
> >   unexpected side effects.
> >
> > - Use LSM BPF to Disable MPOL_BIND
> >   Introduce LSM hooks for syscalls such as mbind(2), set_mempolicy(2), and
> >   set_mempolicy_home_node(2) to disable MPOL_BIND. This approach is more
> >   flexibility and allows for fine-grained control without unintended
> >   consequences. A sample LSM BPF program is included, demonstrating
> >   practical implementation in a production environment.
>
> Without looking at the patchset in any detail yet, I wanted to point
> out that we do have some documented guidelines for adding new LSM
> hooks:
>
> https://github.com/LinuxSecurityModule/kernel/blob/main/README.md#new-lsm-hook-guidelines
>
> I just learned that there are provisions for adding this to the
> MAINTAINERS file, I'll be doing that shortly.  My apologies for not
> having it in there sooner.

Thanks for your information. I will learn it carefully.

-- 
Regards
Yafang