From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2952C369D9 for ; Tue, 29 Apr 2025 02:42:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D77A76B0024; Mon, 28 Apr 2025 22:42:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D297C6B0025; Mon, 28 Apr 2025 22:42:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC9526B0026; Mon, 28 Apr 2025 22:42:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9E9F66B0024 for ; Mon, 28 Apr 2025 22:42:05 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id CD9471602C5 for ; Tue, 29 Apr 2025 02:42:05 +0000 (UTC) X-FDA: 83385531810.12.AB2890A Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf11.hostedemail.com (Postfix) with ESMTP id DFFE740003 for ; Tue, 29 Apr 2025 02:42:03 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=DJcO8j2q; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745894523; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OqeGyt7gPoidax5gVsRj8CtbtpGY7PbbgW35mARwMvU=; b=1EmitKnqIIqBCAfm9irgbroV3y5ej2a9zJ2mMxTbACLrJNc38sz09tff/El+2EBHmeNa5T NaT+sOTUHfuMENeFHXx+Q0bbuXYKmWdSYcPWyy7JLnwhDqAIBKfPmuMEh26ImBUbyYq75/ BOTCFNsOkIafCwxvjQKgCIyyaSLJIwg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745894523; a=rsa-sha256; cv=none; b=1Vij1RqOYx1t76PqK8VDXkZFnvGF9k7ogKbeUoi8Uvc1uQVL52yw6dH4elfeklflaMDClI B8REb2UfHYE+wUGDGj5s/S1AvNk9LibA1hf9IjuO+u+aYaS3Iw8k2BpC38hRkAqx52JefG NNnyHGzZSE5QmWsneC9Ubn6uwaomdsw= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=DJcO8j2q; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-227d6b530d8so59527505ad.3 for ; Mon, 28 Apr 2025 19:42:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1745894523; x=1746499323; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OqeGyt7gPoidax5gVsRj8CtbtpGY7PbbgW35mARwMvU=; b=DJcO8j2qN/GnUH3Irq2ILxLfHhf1UeZlxL+r/uWcrRN6hUeJBC6sXnTkbRa4VaZWTg TL1VRyDdYxQVuw1M4Qn4+olY3Jx/TbofrVpWvQ1VNMYHO2GiRH1z2X6hTJPMdStWYVmj gMBJTIgesQdrEMjU459t5HD9sRlyYuskmMSRZaRM9jgvDunD1zwknezKgRqdPZnyo2Yy fpmykWKiludGUl6rRuDXoq10wWiDEcgRAGT0IDjoNrIbLjeUq1OWW5K3ogejSVoLpEDo jin/EFCDFG/ZScbx7/cfH9v++2CZwyhUWjyHYC9EL2QU/MTEoHp6CFOiFUqCbpFNuHvP cT5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745894523; x=1746499323; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OqeGyt7gPoidax5gVsRj8CtbtpGY7PbbgW35mARwMvU=; b=PEA8chOMr7+gwKbkUbcr7gjc8rYFhix8LBN6Ww1BN0BmyJjOSYavxVL3xtKAnd9ZMJ OII9j+T9xJXP8SCK1ls+YOM0GaFEwHpRgRtVt4TGWXE3j9ohlM9Nt87JUZxt8Ud1tYNv IzEfE28lVnA8WLOTbWq2kZdyn21gTai+4PNGqNilatlLUgfhO/ghTzqam4dIzBhHkZgz VNfeZPHL7qNPiBaqxrn2FcRQy+2ZRx25mYueXm7ic7sZ3s3HglFc36dVH6ipUm0SB7pe Hlznw4wE/ZF6nvuY5vy5wPuRQWXQlToTYGzuxDlhUxj71urunsNhPC6Z98Ys0O0isDOq MjzQ== X-Forwarded-Encrypted: i=1; AJvYcCWTvvHzT6akaTPGMYamVsgyJTHyDthnnEuhGGm4xiY3hl7ccjjJ/vTUEhMsrgNYnbPbjVCmOVzgVQ==@kvack.org X-Gm-Message-State: AOJu0YxpEQHMkFZtpzyCnVFSe7pNn161CDs8VBvTODNgT38zQJsSHuqT qsBz4UxkVrL0DTVAUkuyW0rwyly2vR++zbQAYUFgfgs5uRIzcVNAKKcEz9jo3jM= X-Gm-Gg: ASbGncs5nz03uBsZllQVCcQZBXk1ABS6igf9debjqSGEehSpRmGmRR18ADfmbLvtay5 MYkAxMx834sjYtSK1dd9VeWxzIZvzUlFswFFifi+I9c9UtQorNzZ3PDp2hZGi/W3d0dqFak/jFL PVA6NapYjpEXpDrNJpAN6Rw8/rfpSDv+4vCWRQUOQQVO3oQGpTIMSF94F3MDLEJMCdBqs11i5Jn +kQ5EEF1WQsTXeEd3udXh3gugquV9Bgw/AYZZ1GEcFeGzmWu77ZrU6Aw8kRkEAVKeAoKu7LcHXb y32bWKR+8ZC8tgTmd1sA4pfJotP5I/LC6Ll4ociwQaK5qWH295jHxO79yM/ADmW3Ig== X-Google-Smtp-Source: AGHT+IGD53soZVsbv8HzCNlv/Jg6Ocr9dlSnbrRgmsfIxJvtLS7McOWhLaZgNgxSsqj3V/A0kWYWEg== X-Received: by 2002:a17:903:950:b0:223:501c:7576 with SMTP id d9443c01a7336-22de6e9edfdmr23058635ad.12.1745894522628; Mon, 28 Apr 2025 19:42:02 -0700 (PDT) Received: from localhost.localdomain ([39.144.106.153]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-309ef097cb7sm9893211a91.22.2025.04.28.19.41.58 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 28 Apr 2025 19:42:02 -0700 (PDT) From: Yafang Shao To: akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org Cc: bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH 3/4] mm: add BPF hook for THP adjustment Date: Tue, 29 Apr 2025 10:41:38 +0800 Message-Id: <20250429024139.34365-4-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20250429024139.34365-1-laoar.shao@gmail.com> References: <20250429024139.34365-1-laoar.shao@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: DFFE740003 X-Rspam-User: X-Stat-Signature: mh95g3qkni8t3dtikzbb3bumftdm1jxy X-HE-Tag: 1745894523-355936 X-HE-Meta: U2FsdGVkX18VU+p19336cKAisnCJ3ZyNN/Yo/BghvAXuYx4J23Qg6oZBpbDbuptTdoCR5DdX6QhbRT6U468/pkVTdlSA9iHq17pzjMNEV8vmMIwdvFz+Lp8u6cPrjywOQxXY/VNM+hUTl8X0z48FEdn3ScFmPyfa/Jna1XW9UwcXChV9np/EI/rgrPN7XU/YXc7t/HKur6396g9zv+xpXOV1Z2Uf3iNZ0D9lgeXLr4xDiVqdLVpt5xK9utng2q6MpbW9H/hegz/nbeEsjkg9CgSjtG0Nbig/S9Y5gm8BsscrlPPBU9kW7GBFAC1tLn38bGULvXhXs4TFJIgOHua9gXe4kzPvCZsFfJt/GmcvMm/dpfhZ33WCy4kCf2NwqGILp6tCnx/HHzL+ZlGewviRGl/I9NtHyGZFJ0M0IVp5Nv/OF59fq8w74GkhcAGDJaAM1qPjeY1NfDtitt8xeMLgCX3+3ir54xh8bMsLtkR/3mw8A9E+e7OpwaDMhpDqxwKAc/52eHV25xPFVd40cHZlq/LK5yJmidqHJBoUKstOaENLoz1Nj8/rDt4MrjWWu5hojgLvq6aflnA+vAW4ktog+5fEF/sHSIsMs4zIRzEpK5LiYYVs0My5wvFqBjg1z9EYKFPMDDbGGke9/s7e6zUIhWCcBHbPfE/10vcK76IRXSI0Fe4N5RHrKj8D3j7z3dbInlkgJ9rjIlEqgyuTsAasNDKVXK117wF46PQ/qYp0tCbBmjHlyzN+9DwLsWWTvGAmPWeJKEL86gPP5KMqSql2oqDSC2Z+NE+9wO/6ZpMDkz2J0nq6LyN7z5t+lhsFzp79qgvWeDFQpHE9hiOgDBUW6RxiAfAg3hJN3hsAbLU2SX0MFB7WtNPe78av9Kco22Cq7MsnH/by+/oGRei3PUhG1KjPo7q/OEitH/vSP5JSfRx/s0UWMdsYGwB4l5gHfY0pkwv+8JegRfpptsJHbOV NJ24AYW+ mYe/34VRsQ8Cx/wbLwgj/I+hU05SBDJoVfg01VZqN8H0Tt8Lxe+Tsnw1tXv2wIlOcZAfQ+H2+tURbd8x5qIOFayAGjBGlME1ibPpCuL3tL7x3uUwCSTD9VQHE5yL0hz/JSbgS3CIq/AvZQO4+8wiU77X4SsecNFWk1InKH8lkBOr5JKL9UD+wqGllCkBzLHBOmx/vnXpC61Gw0LHYuvOQdJD91Ihe3vATQSgh96oOcCQGeUgZ9AnJRGcigWMwMOHGXcmkTSYdFQhJ95wyDtlPsI7nSkbOebYiXEll5fgeEX4fX9rQC1xmHAnx8qyIFNDqBlKnc8WxIcyHngg7K2m7B9ExXTfkBhuMiTkuNjoMoWumOByDVWSK4ehdTQL1S/B8IGLpJJ2JQ6Hl2DTLX2o4EgYEQllgWcRU9HBlGPEOo2z4W7HlUYo13dWiY4bVHcczKO1WKwad2hgKdrN63ldtoyUuhmJNgGeUyNlp3S24pNPfDh1dDL6+O7iCcb7tKiuPvR1G X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We will use the @vma parameter in BPF programs to determine whether THP can be used. The typical workflow is as follows: 1. Retrieve the mm_struct from the given @vma. 2. Obtain the task_struct associated with the mm_struct It depends on CONFIG_MEMCG. 3. Adjust THP behavior dynamically based on task attributes E.g., based on the task’s cgroup Signed-off-by: Yafang Shao --- mm/Makefile | 3 +++ mm/bpf.c | 36 ++++++++++++++++++++++++++++++++++++ mm/bpf.h | 21 +++++++++++++++++++++ mm/internal.h | 3 +++ 4 files changed, 63 insertions(+) create mode 100644 mm/bpf.c create mode 100644 mm/bpf.h diff --git a/mm/Makefile b/mm/Makefile index e7f6bbf8ae5f..97055da04746 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -99,6 +99,9 @@ obj-$(CONFIG_MIGRATION) += migrate.o obj-$(CONFIG_NUMA) += memory-tiers.o obj-$(CONFIG_DEVICE_MIGRATION) += migrate_device.o obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += huge_memory.o khugepaged.o +ifdef CONFIG_BPF_SYSCALL +obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += bpf.o +endif obj-$(CONFIG_PAGE_COUNTER) += page_counter.o obj-$(CONFIG_MEMCG_V1) += memcontrol-v1.o obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o diff --git a/mm/bpf.c b/mm/bpf.c new file mode 100644 index 000000000000..72eebcdbad56 --- /dev/null +++ b/mm/bpf.c @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Author: Yafang Shao + */ + +#include +#include + +__bpf_hook_start(); + +/* Checks if this @vma can use THP. */ +__weak noinline int +mm_bpf_thp_vma_allowable(struct vm_area_struct *vma) +{ + /* At present, fmod_ret exclusively uses 0 to signify that the return + * value remains unchanged. + */ + return 0; +} + +__bpf_hook_end(); + +BTF_SET8_START(mm_bpf_fmod_ret_ids) +BTF_ID_FLAGS(func, mm_bpf_thp_vma_allowable) +BTF_SET8_END(mm_bpf_fmod_ret_ids) + +static const struct btf_kfunc_id_set mm_bpf_fmodret_set = { + .owner = THIS_MODULE, + .set = &mm_bpf_fmod_ret_ids, +}; + +static int __init bpf_mm_kfunc_init(void) +{ + return register_btf_fmodret_id_set(&mm_bpf_fmodret_set); +} +late_initcall(bpf_mm_kfunc_init); diff --git a/mm/bpf.h b/mm/bpf.h new file mode 100644 index 000000000000..e03a38084b08 --- /dev/null +++ b/mm/bpf.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +#ifndef __MM_BPF_H +#define __MM_BPF_H + +#define MM_BPF_ALLOWABLE (1) +#define MM_BPF_NOT_ALLOWABLE (-1) + +#define MM_BPF_ALLOWABLE_HOOK(func, args...) { \ + int ret = func(args); \ + \ + if (ret == MM_BPF_ALLOWABLE) \ + return 1; \ + if (ret == MM_BPF_NOT_ALLOWABLE) \ + return 0; \ +} + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +int mm_bpf_thp_vma_allowable(struct vm_area_struct *vma); +#endif + +#endif diff --git a/mm/internal.h b/mm/internal.h index aa698a11dd68..c8bf405fa581 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -21,6 +21,7 @@ /* Internal core VMA manipulation functions. */ #include "vma.h" +#include "bpf.h" struct folio_batch; @@ -1632,6 +1633,7 @@ static inline bool reclaim_pt_is_enabled(unsigned long start, unsigned long end, */ static inline bool hugepage_global_enabled(struct vm_area_struct *vma) { + MM_BPF_ALLOWABLE_HOOK(mm_bpf_thp_vma_allowable, vma); return transparent_hugepage_flags & ((1<