From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 638A5C369C2 for ; Mon, 28 Apr 2025 03:36:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9E6136B0012; Sun, 27 Apr 2025 23:36:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 997156B0022; Sun, 27 Apr 2025 23:36:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 810F46B0023; Sun, 27 Apr 2025 23:36:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 5A07D6B0012 for ; Sun, 27 Apr 2025 23:36:51 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2026981371 for ; Mon, 28 Apr 2025 03:36:51 +0000 (UTC) X-FDA: 83382041022.02.DAFD361 Received: from out-189.mta1.migadu.com (out-189.mta1.migadu.com [95.215.58.189]) by imf25.hostedemail.com (Postfix) with ESMTP id 4C916A0002 for ; Mon, 28 Apr 2025 03:36:49 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=BzNdfSQO; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf25.hostedemail.com: domain of roman.gushchin@linux.dev designates 95.215.58.189 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745811409; a=rsa-sha256; cv=none; b=qSdKgxQzFm+jfh1xc4vNYGPRuKY0Id6K4IGdDlVlcmL1idIyf5XbG6WSxoH5K+2FLYWKOz DxRnST22Y1L5jpetq2V0C9qDSBi854m6BvFjHhHp1I31is7DnqSow2v8YCDRCcMPyjEB1L JxYaznUU7IGtvBKV8e1rKq0FydgXz9Y= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=BzNdfSQO; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf25.hostedemail.com: domain of roman.gushchin@linux.dev designates 95.215.58.189 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745811409; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6cq3axG+82N0/3Jqyh4bOxzur9jfJxES+/ChQNHJI1A=; b=7PZDKtKnlOvUzeysJfaWZFCh6zdCz0vSpXo6PWy47RiuRij/ZyKdKPPLhjSsNoVtLd09l2 t4IAxn1G8p33lBI41986mYXPCDTPKABs8t/593oMnXNt+foaSX3gtVPgQbQOHVGpOnFjOp F8xXndP8+R4aVoOwCbDA9PRpnGQvWoE= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1745811407; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6cq3axG+82N0/3Jqyh4bOxzur9jfJxES+/ChQNHJI1A=; b=BzNdfSQODHW9rGTR8urNbCRUaXaCdcBN8wO7z4a9fsNytWmNoBhUr5ItTnxyymIIp3zkDv DiJJv5G0JaBfEE25qmuiGofwvLam4KGIWiAB5naNyvr+J34xIjVbkqjvUZ3/DBkBJIY40/ hDpA2TgnZb/q4LtMRPvg7UFQfW0MHpI= From: Roman Gushchin To: linux-kernel@vger.kernel.org Cc: Andrew Morton , Alexei Starovoitov , Johannes Weiner , Michal Hocko , Shakeel Butt , Suren Baghdasaryan , David Rientjes , Josh Don , Chuyi Zhou , cgroups@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, Roman Gushchin Subject: [PATCH rfc 05/12] mm: introduce bpf kfuncs to deal with memcg pointers Date: Mon, 28 Apr 2025 03:36:10 +0000 Message-ID: <20250428033617.3797686-6-roman.gushchin@linux.dev> In-Reply-To: <20250428033617.3797686-1-roman.gushchin@linux.dev> References: <20250428033617.3797686-1-roman.gushchin@linux.dev> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 4C916A0002 X-Stat-Signature: fx1ds1knaj3db7wakg3bqhfws9z1hit8 X-HE-Tag: 1745811409-853868 X-HE-Meta: U2FsdGVkX1+ELZhJfkmbyGToCwpwdMUP1CKNkKqriC+DC4qb87b0tadj0atFXMeaMhG3cD/iSaSi0Wqim4lagb+LY08GQjsagBBhbYH0IB0hwQlxR9VNqa/MtuIqfilHSf3QA7rW6frhBQVrlXAg6xLJQHKRx4hDTad62RIjKmDUhmDOqy27yYdAEMS+izF+vByc1udeD9oY9oPD2k4QVhu2+UqZM0TeOsm2HvN+Dt1bTI9r20qe7RKQSkaEPbtNP1kGnhykFRPqysl3nIYkSwEqRBjbQZJTtZ1lCWKs/9lzGhvqlNWaOIMa0FgeoqnUgNo76qs9mS7fu3OCGWXh8edeAvCSd5b1T05BDIhvH4adl7khHGp/H9DLZ/9P+a7peMiQ4ICSCh3iwPrXNFinlPy1SMYMniDNEHQ1sHglkVm3sbePp9s/Acot4cP9A59nkddhzRRnYCf8RlXR2VIuc6a4/931F1/1FHnI+quNILHYS5OAZlfEfK8bpA1AL4feYPHN3XKM+0JnlV9aTmg63PA2DWpf9E5vw6NXkMPrtzPOonUR+lpvBYXeRMBvG/gUm+iZLicuRnDNwRAfpv2WfYP4+oFdO+ZG2cmVEGuUR1KLluJOw6iykusdz9oK60tEdquyqwulpiVzl3u/snSn3avQIswxAMxoUdDFWr3iPhkcee7M3nvm09UMAeBG1+bBNPaZxqeaKKA73c/R7zPAV3v1MRnpE6wUwKtncTN0mkJAS60rzQDKKtCeM2rOlYdqc2o6g2WXMuTi5NrReMLBr7WzSDq6eQn6QfnWOZdsqPc9ZElkfy/y0EokiUIHpYRtWAyBCFJ8Vmo/35V04lIp8eSU8nsTekeqdcQZLtageLylq5ERPQgJL9l7nH9wfKpsizM3Ylk23T3COPR7affXh9hheWhra2YPpl7N58GE0b4kdbaxTsKQV//s1FYmidVaMd05VY9pI5UJV+8Y/3k oQQOtgin eI4HQtRBPqswN2r9GhPSd3CKrkYNwbdrUUM2O2J3rEwnoqq3ZwlXc56j/Jg7KJVifpCO2q/mbhHo6DpdTGSrLuQ3qJkL1qbfbS7j4l75Qs8VqXMYcdALQTtqSDZ/V6WZ1FLFxVq4XN15esJ5QcMUjYm+TFGO5DCBS7QRd2DQdnEOxh9A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To effectively operate with memory cgroups in bpf there is a need to convert css pointers to memcg pointers. A simple container_of cast which is used in the kernel code can't be used in bpf because from the verifier's point of view that's a out-of-bounds memory access. Introduce helper get/put kfuncs which can be used to get a refcounted memcg pointer from the css pointer: - bpf_get_mem_cgroup, - bpf_put_mem_cgroup. bpf_get_mem_cgroup() can take both memcg's css and the corresponding cgroup's "self" css. It allows it to be used with the existing cgroup iterator which iterates over cgroup tree, not memcg tree. Signed-off-by: Roman Gushchin --- include/linux/memcontrol.h | 2 + mm/Makefile | 3 ++ mm/bpf_memcontrol.c | 101 +++++++++++++++++++++++++++++++++++++ 3 files changed, 106 insertions(+) create mode 100644 mm/bpf_memcontrol.c diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 53364526d877..a2ecd9caacfb 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -932,6 +932,8 @@ static inline void mod_memcg_page_state(struct page *page, rcu_read_unlock(); } +unsigned long memcg_events(struct mem_cgroup *memcg, int event); +unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap); unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx); unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx); unsigned long lruvec_page_state_local(struct lruvec *lruvec, diff --git a/mm/Makefile b/mm/Makefile index e7f6bbf8ae5f..3eedba68e8cb 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -105,6 +105,9 @@ obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o ifdef CONFIG_SWAP obj-$(CONFIG_MEMCG) += swap_cgroup.o endif +ifdef CONFIG_BPF_SYSCALL +obj-$(CONFIG_MEMCG) += bpf_memcontrol.o +endif obj-$(CONFIG_CGROUP_HUGETLB) += hugetlb_cgroup.o obj-$(CONFIG_GUP_TEST) += gup_test.o obj-$(CONFIG_DMAPOOL_TEST) += dmapool_test.o diff --git a/mm/bpf_memcontrol.c b/mm/bpf_memcontrol.c new file mode 100644 index 000000000000..dacdf53735e5 --- /dev/null +++ b/mm/bpf_memcontrol.c @@ -0,0 +1,101 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Memory Controller-related BPF kfuncs and auxiliary code + * + * Author: Roman Gushchin + */ + +#include +#include + +__bpf_kfunc_start_defs(); + +__bpf_kfunc struct mem_cgroup * +bpf_get_mem_cgroup(struct cgroup_subsys_state *css) +{ + struct mem_cgroup *memcg = NULL; + bool rcu_unlock = false; + + if (!root_mem_cgroup) + return NULL; + + if (root_mem_cgroup->css.ss != css->ss) { + struct cgroup *cgroup = css->cgroup; + int ssid = root_mem_cgroup->css.ss->id; + + rcu_read_lock(); + rcu_unlock = true; + css = rcu_dereference_raw(cgroup->subsys[ssid]); + } + + if (css && css_tryget(css)) + memcg = container_of(css, struct mem_cgroup, css); + + if (rcu_unlock) + rcu_read_unlock(); + + return memcg; +} + +__bpf_kfunc void bpf_put_mem_cgroup(struct mem_cgroup *memcg) +{ + css_put(&memcg->css); +} + +__bpf_kfunc unsigned long bpf_mem_cgroup_events(struct mem_cgroup *memcg, int event) +{ + + if (event < 0 || event >= NR_VM_EVENT_ITEMS) + return (unsigned long)-1; + + return memcg_events(memcg, event); +} + +__bpf_kfunc unsigned long bpf_mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) +{ + return mem_cgroup_usage(memcg, swap); +} + +__bpf_kfunc unsigned long bpf_mem_cgroup_page_state(struct mem_cgroup *memcg, int idx) +{ + if (idx < 0 || idx >= MEMCG_NR_STAT) + return (unsigned long)-1; + + return memcg_page_state(memcg, idx); +} + +__bpf_kfunc void bpf_mem_cgroup_flush_stats(struct mem_cgroup *memcg) +{ + mem_cgroup_flush_stats(memcg); +} + +__bpf_kfunc_end_defs(); + +BTF_KFUNCS_START(bpf_memcontrol_kfuncs) +BTF_ID_FLAGS(func, bpf_get_mem_cgroup, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_put_mem_cgroup, KF_RELEASE) + +BTF_ID_FLAGS(func, bpf_mem_cgroup_events, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_mem_cgroup_usage, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_mem_cgroup_page_state, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_mem_cgroup_flush_stats, KF_TRUSTED_ARGS) + +BTF_KFUNCS_END(bpf_memcontrol_kfuncs) + +static const struct btf_kfunc_id_set bpf_memcontrol_kfunc_set = { + .owner = THIS_MODULE, + .set = &bpf_memcontrol_kfuncs, +}; + +static int __init bpf_memcontrol_init(void) +{ + int err; + + err = register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, + &bpf_memcontrol_kfunc_set); + if (err) + pr_warn("error while registering bpf memcontrol kfuncs: %d", err); + + return err; +} +late_initcall(bpf_memcontrol_init); -- 2.49.0.901.g37484f566f-goog