Re: [RFC PATCH -mm 0/4] mm, security, bpf: Fine-grained control over memory policy adjustments with lsm bpf

[Casey Schaufler <casey@schaufler-ca.com>]

On 11/13/2023 12:50 AM, Ondrej Mosnacek wrote:
> On Mon, Nov 13, 2023 at 4:17 AM Yafang Shao <laoar.shao@gmail.com> wrote:
>> On Mon, Nov 13, 2023 at 12:45 AM Casey Schaufler <casey@schaufler-ca.com> wrote:
>>> On 11/11/2023 11:34 PM, Yafang Shao wrote:
>>>> Background
>>>> ==========
>>>>
>>>> In our containerized environment, we've identified unexpected OOM events
>>>> where the OOM-killer terminates tasks despite having ample free memory.
>>>> This anomaly is traced back to tasks within a container using mbind(2) to
>>>> bind memory to a specific NUMA node. When the allocated memory on this node
>>>> is exhausted, the OOM-killer, prioritizing tasks based on oom_score,
>>>> indiscriminately kills tasks. This becomes more critical with guaranteed
>>>> tasks (oom_score_adj: -998) aggravating the issue.
>>> Is there some reason why you can't fix the callers of mbind(2)?
>>> This looks like an user space configuration error rather than a
>>> system security issue.
>> It appears my initial description may have caused confusion. In this
>> scenario, the caller is an unprivileged user lacking any capabilities.
>> While a privileged user, such as root, experiencing this issue might
>> indicate a user space configuration error, the concerning aspect is
>> the potential for an unprivileged user to disrupt the system easily.
>> If this is perceived as a misconfiguration, the question arises: What
>> is the correct configuration to prevent an unprivileged user from
>> utilizing mbind(2)?"
>>
>>>> The selected victim might not have allocated memory on the same NUMA node,
>>>> rendering the killing ineffective. This patch aims to address this by
>>>> disabling MPOL_BIND in container environments.
>>>>
>>>> In the container environment, our aim is to consolidate memory resource
>>>> control under the management of kubelet. If users express a preference for
>>>> binding their memory to a specific NUMA node, we encourage the adoption of
>>>> a standardized approach. Specifically, we recommend configuring this memory
>>>> policy through kubelet using cpuset.mems in the cpuset controller, rather
>>>> than individual users setting it autonomously. This centralized approach
>>>> ensures that NUMA nodes are globally managed through kubelet, promoting
>>>> consistency and facilitating streamlined administration of memory resources
>>>> across the entire containerized environment.
>>> Changing system behavior for a single use case doesn't seem prudent.
>>> You're introducing a bunch of kernel code to avoid fixing a broken
>>> user space configuration.
>> Currently, there is no mechanism in place to proactively prevent an
>> unprivileged user from utilizing mbind(2). The approach adopted is to
>> monitor mbind(2) through a BPF program and trigger an alert if its
>> usage is detected. However, beyond this monitoring, the only recourse
>> is to verbally communicate with the user, advising against the use of
>> mbind(2). As a result, users will question why mbind(2) isn't outright
>> prohibited in the first place.
> Is there a reason why you can't use syscall filtering via seccomp(2)?
> AFAIK, all the mainstream container tooling already has support for
> specifying seccomp filters for containers.

That looks like a practical solution from here.