From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5398C83F1A for ; Thu, 17 Jul 2025 18:21:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E1466B00CC; Thu, 17 Jul 2025 14:21:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 68B516B00CF; Thu, 17 Jul 2025 14:21:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57A2A6B00D1; Thu, 17 Jul 2025 14:21:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 476C16B00CC for ; Thu, 17 Jul 2025 14:21:49 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id BAF84C014E for ; Thu, 17 Jul 2025 18:21:48 +0000 (UTC) X-FDA: 83674575096.06.5E860BB Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by imf29.hostedemail.com (Postfix) with ESMTP id C1CB4120003 for ; Thu, 17 Jul 2025 18:21:46 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gnjKsVbu; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf29.hostedemail.com: domain of ameryhung@gmail.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=ameryhung@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752776506; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Owo8/pepQaG8tCLdms7r8I0XLUvKhXbhFJD1zAzOisI=; b=cUfmr6UkRbLIa0qs3JKdCqPwuTQNxtfeiZwr48WcRb2nyXYKpHoVoO6iexiONvrQUGFCN1 /V4sUSmCh6/vGKTlEqSl4UvaIr5afLtt66DwRHY8uU8XnhIXhVRAUHapp+kM3ZXxyIdTf6 2yKfXLinosh8lNzkhpixPalUtPq6hqQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752776506; a=rsa-sha256; cv=none; b=wo9YosYChUxnv/X22cxVe6POSLQyOgP86Oda7Rw1EgUv7j1/RzBWOBrY96HXYwCfqYg06o Lo2UZ4CPGF1G5e9Npc/iRTjROdCY9BCR0NjtFWAD2eoU4FkJjl16rq31y46OIXxLYej8WP oLUBzY+xzUuei8P4W7DZfw3meEJM9zk= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gnjKsVbu; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf29.hostedemail.com: domain of ameryhung@gmail.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=ameryhung@gmail.com Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-74b50c71b0aso836068b3a.0 for ; Thu, 17 Jul 2025 11:21:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1752776505; x=1753381305; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=Owo8/pepQaG8tCLdms7r8I0XLUvKhXbhFJD1zAzOisI=; b=gnjKsVbuk9kGqWo6345DxjiG8nH8OXvyub+sCN+9DKD6ej8lYRVEAtjjGfzddy376C D5ogEAQyuT5mnShMkseTMRfu6kIWNvgiwmYQNDJjbaO2hTRuAN0XWmRpNkprfUV4BM5v ujDweYkgZRqXtHZPXykxZ8PYcRDa8d8M04c1ZPffhjdEumNWjR2qcLiu63b/iV/ERa0q Kv1IbrTxEO9RJ0+y0VJ3GI0YMiGhNuvAdu3TJM6dii2fd1qWDvPtrpLNZjaIoUdOfh/j 1CXgWVdCZi4NC6e+QAE6cTrXfAdILtFC99ED3kO4lS7gGrV5ZIHmVUsqxckBDmHCy0S5 n6VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752776505; x=1753381305; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Owo8/pepQaG8tCLdms7r8I0XLUvKhXbhFJD1zAzOisI=; b=EVE6sSUxOtKdGfZmy1AglF7iwa7TFZEbHQHBf3+prF04s2AzyvDlI4xZ+pnn+DHFYl cqJ+xuy/LSXn1lLPOdssk8ACXEex57FqtGB28iw2aa6jhTFigwj68AzolmefNbtwa9Hi 0kpqYdQ0AdPlLE+rFpTwhqACRBZjffiPxB3FikY7KdhaXN/xlqMb0EloAUHvfAZfhsFA KGS8BJv/3w8jylzBZPf8S0XQH8HuoHmG9EiFdN8BoWgudvu5sxbwSl2wJyfgLzPLQA5L FnrDbOoYGfW/glcHUR0j9M1Tu5dnmlQTR3/9rOrlM/jDpdzf5MxQS4VM5RUKAI4ldzjw o1bQ== X-Forwarded-Encrypted: i=1; AJvYcCV9b4M5fNovWplS6USAR3SMW7KWa0Tp6MeTZ/P9nM5Pfnx6D1b9QH8dWgG7ecXTlnFlRk9c1FBSxQ==@kvack.org X-Gm-Message-State: AOJu0YxEfj3GEsejAQbr6hIwZibcR227pwL8xYIe8jl7Dse1wIQMla+u PZUQx6N3EipGraIqTwnU+F3MY2lF2LmEyQJrXcn5JirHC39kGUaOFlP/ X-Gm-Gg: ASbGncviNCj+QpFblhNtR9U8+AjoAHCHv87tOmN4u2/cD8sQaVlcbmD+0JcmtYErIOR VZTnDY5MFE1LSlAKO3q2Hb3Z6vWuYJGGGav+TOZ8xTYUwzWX6waGfYH7zGoYTEG1HCMs+ILAmlk 3aroNJjl9u/slQVcdOxUNMSzGkID5+fyqgr32jP9izxCjDQZ6q6+IDcNyoxc3U5XCjDPVVthfNB A092rtpHs+0FPJ0rZ5o56MxVFiWPYsyEh7ilZ+GXuIJsgFyAu5l8v5/IByN4QG9BM2qGss8LyzI EqpiJfFMVxFApTBFL4wJITu472crCX1M9zT/qLwA2V+Iwtwnv84bZfb/5zMkh+8bvxIHFDirQri 5FhB82B3XreMNzoNga7A7XRgOV5AedKsTdBU9udhPxY1ZoW6mCDreSzSIcPwnV/lHkmDp4VY6Bx C1 X-Google-Smtp-Source: AGHT+IGIDQ8ZOjuWdcX5onexleEqNk5wiDXmBBzs6kP/LngPXZMeA3Vsi09NDta52vhcFJRoLXFutQ== X-Received: by 2002:a05:6a21:62c1:b0:216:1476:f5c with SMTP id adf61e73a8af0-23812b50d6cmr12577497637.25.1752776503749; Thu, 17 Jul 2025 11:21:43 -0700 (PDT) Received: from ?IPV6:2a03:83e0:1156:a:49c:e9dc:5730:d2af? ([2620:10d:c090:500::4:ec62]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74eb9dd5ab5sm16627521b3a.30.2025.07.17.11.21.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Jul 2025 11:21:43 -0700 (PDT) Message-ID: Date: Thu, 17 Jul 2025 11:21:38 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH v3 4/5] mm: thp: add bpf thp struct ops To: Yafang Shao , akpm@linux-foundation.org, david@redhat.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, npache@redhat.com, ryan.roberts@arm.com, dev.jain@arm.com, hannes@cmpxchg.org, usamaarif642@gmail.com, gutierrez.asier@huawei-partners.com, willy@infradead.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org Cc: bpf@vger.kernel.org, linux-mm@kvack.org References: <20250608073516.22415-1-laoar.shao@gmail.com> <20250608073516.22415-5-laoar.shao@gmail.com> Content-Language: en-US From: Amery Hung In-Reply-To: <20250608073516.22415-5-laoar.shao@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Stat-Signature: 79rrhjpkn6c94ckj1f3hyfga6w1njbo9 X-Rspamd-Queue-Id: C1CB4120003 X-Rspamd-Server: rspam10 X-Rspam-User: X-HE-Tag: 1752776506-608235 X-HE-Meta: U2FsdGVkX1+jNBK1lSp6RFoh7eU7kAAMfxVU3BGR3MD1Lh6Uwek45kk2mmSafpdcAZuE76Z7vRKh/dGG74KZH8/bBR2cJLd7lIzxdJ/pxSatKP1LC7+UL4jMhSjnYZwRikCLGW5MpmQKErjYxYHdFv7kvr++cvYk1ebfK17RTaeCH1zLdODsU9SGc/EABcwIk/wFxmSQ7Z5C4c/hLccFPBBrM3UVdJIKKc04BCeQ8bRDsu4cQ8kj/OOAbAD8wZN1/tD+DgKgFe9NwGxnNYUxiFPavb7bTjZqRD+cRYU+X8yA+rumMskk6+0DruFCzvCPFNDSd5a/WvLN4NXHmw02oZHYWe8LwEhhewuQcDzNL9MpyjaroZ5cjEZjEQJuzrsqKJT6pmTlQdT42MjuWY7OHcu9gThkBuJJtl47s9hQpwugRxuCoOb7Lnjbe+9nuS8fInBUDPtmf7/MD3UbMzv/f0l6JnVA/u66r8O+sqoO3lOsZ5SPcNQvfgHwdz1a+uoGBHBAPuJhaJZoCfU6Sjd3WyAs/fBbZhYeHBDKb8+sH+8YvQ6ASCHTe6LK83V9BV8vqYzughQfqcdrigHT8zpipwQ/MgOz9p9FZAVeIKXqJJGO3OtWsKkhndWscbee6o9WUygyddzJF2H2GnfvOGVhroOCp8mXAF+1HfrpEDSIY0+va4EFZthuXNywEsgHojHGSLE1WOE67RVjq4HZEsv9VvFpepckhBo4O6ES84bYtXf4lB3mgUij0hhXmyxbZYFkyUseiNYViIMO30J/4IOxCqTyYHTSUgceAhlJelFsDMdN+/H83PsNLXmhEn7VDudJObAF8GyPSS0+0FmW5It7RNVz1hif6cGug1A1olPocyRRO2cxhuRSw6ed7Cfd5gjwhBppM/fQiCnGG1UdQM6pzYE383U/GerT/rgeBNedCzGE8zpJUdjwf4T5tvMiwEom1iUJcSR3iVMGO+t0/L1 D/bbJhxZ CQEmgKoGCqz5Soy038y2GorDd8ufTXmTSPFunPMPRnP05FZ0e1sJfxFGgNkuP4qQwODA1j26VpL653rT5Rmf5krKBskHFMpf8q6AIdnnX89Qr7s+9vbAOn92SNWENokbBQw8HmYnEwpcNZnhIUiy3esDW5tSBOesu4ZG52A9gvQsXDzGTSVqYkkCAFQEEXTbzzn8NqIQxTBJjj5DPefTUd1nVjRisDf13s3pwO1QW8hobkvirWcOG02wxAHwxWwe/HNEYBk6ojOgr9v5wf8E1uqC5BFoVvLr+T755SpHPh8ixOATeg5WVW250FCD8s+7g5DBWNX2jyfOmMvyqULkBTibj64h9hXTxVJYtMEGCF2t+MlBcyS8GvkRfA1sdyhB9kF3M2QNhsnT/+Bk5izEhNeXvHulskqqLEOganVtuTGgUDDUPl4U+/fVMd4KPAor/jFnq8K2n277tWNzNyxNSsXPpV428y64iI1WxvKKeNTG6dyEhmBvfLUxU4owCzllFi4fg3NOqlmr0CupNanyLdw9gFGRyYw7oN5IMVa4DDgmtxR6W1nwq6DdDJMgkw47LPow3fJN/ERcmQsIpDIYwa17o5OvTtYeHsQ4XQRa/2OlfpyXq29QyLYVrdQ+BNr/igUdi X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 6/8/25 12:35 AM, Yafang Shao wrote: > A new bpf_thp struct ops is introduced to provide finer-grained control > over THP allocation policy. The struct ops includes two APIs for > determining the THP allocator and reclaimer behavior: > > - THP allocator > > int (*allocator)(unsigned long vm_flags, unsigned long tva_flags); > > The BPF program returns either THP_ALLOC_CURRENT or THP_ALLOC_KHUGEPAGED, > indicating whether THP allocation should be performed synchronously > (current task) or asynchronously (khugepaged). > > The decision is based on the current task context, VMA flags, and TVA > flags. > > - THP reclaimer > > int (*reclaimer)(bool vma_madvised); > > The BPF program returns either RECLAIMER_CURRENT or RECLAIMER_KSWAPD, > determining whether memory reclamation is handled by the current task or > kswapd. > > The decision depends on the current task and VMA flags. > > Signed-off-by: Yafang Shao > --- > include/linux/huge_mm.h | 13 +-- > mm/Makefile | 3 + > mm/bpf_thp.c | 184 ++++++++++++++++++++++++++++++++++++++++ > 3 files changed, 190 insertions(+), 10 deletions(-) > create mode 100644 mm/bpf_thp.c > > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h > index 6a40ebf25f5c..0d02c9b56a85 100644 > --- a/include/linux/huge_mm.h > +++ b/include/linux/huge_mm.h > @@ -54,6 +54,7 @@ enum transparent_hugepage_flag { > TRANSPARENT_HUGEPAGE_DEFRAG_REQ_MADV_FLAG, > TRANSPARENT_HUGEPAGE_DEFRAG_KHUGEPAGED_FLAG, > TRANSPARENT_HUGEPAGE_USE_ZERO_PAGE_FLAG, > + TRANSPARENT_HUGEPAGE_BPF_ATTACHED, /* BPF prog is attached */ > }; > > struct kobject; > @@ -192,16 +193,8 @@ static inline bool hugepage_global_always(void) > > #define THP_ALLOC_KHUGEPAGED (1 << 1) > #define THP_ALLOC_CURRENT (1 << 2) > -static inline int bpf_thp_allocator(unsigned long vm_flags, > - unsigned long tva_flags) > -{ > - return THP_ALLOC_KHUGEPAGED | THP_ALLOC_CURRENT; > -} > - > -static inline gfp_t bpf_thp_gfp_mask(bool vma_madvised) > -{ > - return 0; > -} > +int bpf_thp_allocator(unsigned long vm_flags, unsigned long tva_flags); > +gfp_t bpf_thp_gfp_mask(bool vma_madvised); > > static inline int highest_order(unsigned long orders) > { > diff --git a/mm/Makefile b/mm/Makefile > index 1a7a11d4933d..e5f41cf3fd61 100644 > --- a/mm/Makefile > +++ b/mm/Makefile > @@ -99,6 +99,9 @@ obj-$(CONFIG_MIGRATION) += migrate.o > obj-$(CONFIG_NUMA) += memory-tiers.o > obj-$(CONFIG_DEVICE_MIGRATION) += migrate_device.o > obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += huge_memory.o khugepaged.o > +ifdef CONFIG_BPF_SYSCALL > +obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += bpf_thp.o > +endif > obj-$(CONFIG_PAGE_COUNTER) += page_counter.o > obj-$(CONFIG_MEMCG_V1) += memcontrol-v1.o > obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o > diff --git a/mm/bpf_thp.c b/mm/bpf_thp.c > new file mode 100644 > index 000000000000..894d6cb93107 > --- /dev/null > +++ b/mm/bpf_thp.c > @@ -0,0 +1,184 @@ > +// SPDX-License-Identifier: GPL-2.0 > + > +#include > +#include > +#include > +#include > + > +#define RECLAIMER_CURRENT (1 << 1) > +#define RECLAIMER_KSWAPD (1 << 2) > +#define RECLAIMER_BOTH (RECLAIMER_CURRENT | RECLAIMER_KSWAPD) > + > +struct bpf_thp_ops { > + /** > + * @allocator: Specifies whether the THP allocation is performed > + * by the current task or by khugepaged. > + * @vm_flags: Flags for the VMA in the current allocation context > + * @tva_flags: Flags for the TVA in the current allocation context > + * > + * Rerurn: > + * - THP_ALLOC_CURRENT: THP was allocated synchronously by the calling > + * task's context. > + * - THP_ALLOC_KHUGEPAGED: THP was allocated asynchronously by the > + * khugepaged kernel thread. > + * - 0: THP allocation is disallowed in the current context. > + */ > + int (*allocator)(unsigned long vm_flags, unsigned long tva_flags); > + /** > + * @reclaimer: Specifies the entity performing page reclaim: > + * - current task context > + * - kswapd > + * - none (no reclaim) > + * @vma_madvised: MADV flags for this VMA (e.g., MADV_HUGEPAGE, MADV_NOHUGEPAGE) > + * > + * Return: > + * - RECLAIMER_CURRENT: Direct reclaim by the current task if THP > + * allocation fails. > + * - RECLAIMER_KSWAPD: Wake kswapd to reclaim memory if THP allocation fails. > + * - RECLAIMER_ALL: Both current and kswapd will perform the reclaim > + * - 0: No reclaim will be attempted. > + */ > + int (*reclaimer)(bool vma_madvised); > +}; > + > +static struct bpf_thp_ops bpf_thp; > + > +int bpf_thp_allocator(unsigned long vm_flags, unsigned long tva_flags) > +{ > + int allocator; > + > + /* No BPF program is attached */ > + if (!(transparent_hugepage_flags & (1< + return THP_ALLOC_KHUGEPAGED | THP_ALLOC_CURRENT; > + > + if (current_is_khugepaged()) > + return THP_ALLOC_KHUGEPAGED | THP_ALLOC_CURRENT; > + if (!bpf_thp.allocator) > + return THP_ALLOC_KHUGEPAGED | THP_ALLOC_CURRENT; > + > + allocator = bpf_thp.allocator(vm_flags, tva_flags); > + if (!allocator) > + return 0; The check seems redundant. Is it? > + /* invalid return value */ > + if (allocator & ~(THP_ALLOC_KHUGEPAGED | THP_ALLOC_CURRENT)) > + return THP_ALLOC_KHUGEPAGED | THP_ALLOC_CURRENT; > + return allocator; > +} > + > +gfp_t bpf_thp_gfp_mask(bool vma_madvised) > +{ > + int reclaimer; > + > + if (!(transparent_hugepage_flags & (1< + return 0; > + > + if (!bpf_thp.reclaimer) > + return 0; > + > + reclaimer = bpf_thp.reclaimer(vma_madvised); > + switch (reclaimer) { > + case RECLAIMER_CURRENT: > + return GFP_TRANSHUGE | __GFP_NORETRY; > + case RECLAIMER_KSWAPD: > + return GFP_TRANSHUGE_LIGHT | __GFP_KSWAPD_RECLAIM; > + case RECLAIMER_BOTH: > + return GFP_TRANSHUGE | __GFP_KSWAPD_RECLAIM | __GFP_NORETRY; > + default: > + return 0; > + } > +} > + > +static bool bpf_thp_ops_is_valid_access(int off, int size, > + enum bpf_access_type type, > + const struct bpf_prog *prog, > + struct bpf_insn_access_aux *info) > +{ > + return bpf_tracing_btf_ctx_access(off, size, type, prog, info); > +} > + > +static const struct bpf_func_proto * > +bpf_thp_get_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) > +{ > + return bpf_base_func_proto(func_id, prog); > +} > + > +static const struct bpf_verifier_ops thp_bpf_verifier_ops = { > + .get_func_proto = bpf_thp_get_func_proto, > + .is_valid_access = bpf_thp_ops_is_valid_access, > +}; > + > +static int bpf_thp_reg(void *kdata, struct bpf_link *link) > +{ > + struct bpf_thp_ops *ops = kdata; > + > + /* TODO: add support for multiple attaches */ > + if (test_and_set_bit(TRANSPARENT_HUGEPAGE_BPF_ATTACHED, > + &transparent_hugepage_flags)) > + return -EOPNOTSUPP; I think returning -EBUSY if the struct_ops is already attached is a better choice > + bpf_thp.allocator = ops->allocator; > + bpf_thp.reclaimer = ops->reclaimer; > + return 0; > +} > + > +static void bpf_thp_unreg(void *kdata, struct bpf_link *link) > +{ > + clear_bit(TRANSPARENT_HUGEPAGE_BPF_ATTACHED, &transparent_hugepage_flags); > + bpf_thp.allocator = NULL; > + bpf_thp.reclaimer = NULL; > +} > + > +static int bpf_thp_check_member(const struct btf_type *t, > + const struct btf_member *member, > + const struct bpf_prog *prog) > +{ > + return 0; > +} > + [...] > +static int bpf_thp_init_member(const struct btf_type *t, > + const struct btf_member *member, > + void *kdata, const void *udata) > +{ > + return 0; > +} > + > +static int bpf_thp_init(struct btf *btf) > +{ > + return 0; > +} > + > +static int allocator(unsigned long vm_flags, unsigned long tva_flags) > +{ > + return 0; > +} > + > +static int reclaimer(bool vma_madvised) > +{ > + return 0; > +} > + > +static struct bpf_thp_ops __bpf_thp_ops = { > + .allocator = allocator, > + .reclaimer = reclaimer, > +}; > + > +static struct bpf_struct_ops bpf_bpf_thp_ops = { > + .verifier_ops = &thp_bpf_verifier_ops, > + .init = bpf_thp_init, > + .check_member = bpf_thp_check_member, nit. check_member doesn't need to be defined if it does not do anything. > + .init_member = bpf_thp_init_member, > + .reg = bpf_thp_reg, > + .unreg = bpf_thp_unreg, > + .name = "bpf_thp_ops", > + .cfi_stubs = &__bpf_thp_ops, > + .owner = THIS_MODULE, > +}; > + > +static int __init bpf_thp_ops_init(void) > +{ > + int err = register_bpf_struct_ops(&bpf_bpf_thp_ops, bpf_thp_ops); > + > + if (err) > + pr_err("bpf_thp: Failed to register struct_ops (%d)\n", err); > + return err; > +} > +late_initcall(bpf_thp_ops_init);