From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB35EC369D5 for ; Tue, 29 Apr 2025 02:41:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B3FED6B0011; Mon, 28 Apr 2025 22:41:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AED176B0012; Mon, 28 Apr 2025 22:41:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 968856B0022; Mon, 28 Apr 2025 22:41:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 761056B0011 for ; Mon, 28 Apr 2025 22:41:56 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 803EC587DA for ; Tue, 29 Apr 2025 02:41:56 +0000 (UTC) X-FDA: 83385531432.07.CA25960 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf04.hostedemail.com (Postfix) with ESMTP id A2E6E40005 for ; Tue, 29 Apr 2025 02:41:54 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=U0qA0DCe; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf04.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745894514; a=rsa-sha256; cv=none; b=prOyzY/mcNw4PYkR5N2lPWuvhJRXUtIXIYmfF/J0fFzz8PfpclMdsjUuVrUOJIl1f6QI6s IreiGaxdhWY2L9DfqXS7SH7ltBhnbMGD8XsahGOqybcfagfTWqT2WMd7//sL/yhKIdhwRR bOskyDAgygcUzCG8C2iDnblJ+zpi6vs= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=U0qA0DCe; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf04.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745894514; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=waacX/lXsmFSOHeLxxTGTjSLf8mXp5UdS3vwhftA+Os=; b=LVwLdapRksOad8ISXnrIIAkFaw1jdMpuTCVx33tsk3IBI4vfNjoojsPmHNVEhG6iGhighW z4dIx3ZKYLw4BvrN/cDaHGDgN0f4FX7b2Zh7/UKdwT9ZHvU5KkEMj7zGjvVH3X6eg5nnEC h53L5hvfdcYrX2KwL68nujPDXWtR1AY= Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-3085f827538so7513131a91.0 for ; Mon, 28 Apr 2025 19:41:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1745894513; x=1746499313; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=waacX/lXsmFSOHeLxxTGTjSLf8mXp5UdS3vwhftA+Os=; b=U0qA0DCeY1GyaeZ6jXF9RLKw48H4TXaw2pOjyy6xIFcbNb4rGETx++nLfY2LIpOyIb cS6kca13ep4pDnCD+7x1KvKfXjihJ/4PqGqg6LJeWJI+mzEt/VE6WRY+avA7pQjtScq0 icn6Igj31bmXEDDXHmTEv9nQRSrRB1DNRmuHojfajYgHZHCLOoqqA04NuqJSdm+zOo3B UOrvaXSW8kxbmaMdF0u3COEVNiNYMZh7N/7zLKk2/jqVWsfxNtJsQKpCVEtFNmJae0x7 YQL4ZfI6i5u8MKKv28rr2aMlfdmLTUavZz+EVY4+kK4Ly0cBiMM0+MXsbMeN8CrF+FRt BDTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745894513; x=1746499313; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=waacX/lXsmFSOHeLxxTGTjSLf8mXp5UdS3vwhftA+Os=; b=INORg1n/Z9LowKNG1LKEAC69NlqKI6q5lWGxxqZ1wiuIZVs4tDGfe9gn6pult/Cl4Z p7rAsUDIseF/ojOqYLoXHNYXyQUvbKW40JE7LwwBhCGgPkudbxlb0K1TpXW/jqV+XeTJ fw8eKs+QXwLD7BIG+QZ2La3NhyimO/YZ3VmYD1x/dcb2N3G+uSTH257pxXLk2Mp+QCDq 1axLMAdwGoBCjqzn/ezmAl017RyMgTROZAs0vubl0XH7uChMo0Pokk7EUVNwNH7QPtdd 4YNGQBW47OS8XjVYV2qiStdI8wnQ39Z/G9sBOR1x1ga0VYVArsYo2YrBlHlH/GJHLDCs J8Yw== X-Forwarded-Encrypted: i=1; AJvYcCUaYuswkBCwqUa6CuoMY7Q5TghvVPQBlqsJTo4Q2meT5rs6sRQgQ9jEb5ECAFahlnqpuTGPPR/qSA==@kvack.org X-Gm-Message-State: AOJu0YxKln41OkQJsPQAtuQjCgaiXrDpfP1rdssjTqPQn5lICZlIm7My dJAE5WeZnXTokBGtmjvbIlG9wcjhbgxmdYpDOaSUzxFPdfsdWhfh X-Gm-Gg: ASbGncsJAyipLDHKTCnhY7pp0d9xS+eL0zYA3+4qrvlTzgALI0sLlfWpU2B894555P8 Gnjs3rFz7Gp/alqyuQ7FOTwHwO6ttKkOGjFUArrFodKnp1fsJgxIgAm3TjS+D9/8wJEb4oVdI16 YWw30O2bhrtt6QQy3hUNBoy4LlCI109CBxbNUo8LIguai3sIfMInCZZgvWSZ1AHY0q3INoiX49C PyBW8o7vYPonGHUoSisgVG0LOsv2i1HqUFMdWkmUnUHht+t0mXBfIuJlBxXmdOmvSRMFkSa/0Lt nPRbh3OK9E7C6YrwXm2a5bqcEammMaJAqv40lkz459bYeHg2ianfjXKshsqnXRrxOg== X-Google-Smtp-Source: AGHT+IF7e4pA8Y55vBVIkd/Z4QppQKJkTuySaI50+TMICaQepHZ9gdzqB0a1MJO32aNBlUBY2KnCiA== X-Received: by 2002:a17:90b:2dc1:b0:2ff:6f88:b04a with SMTP id 98e67ed59e1d1-30a0133a0bemr17925694a91.15.1745894513455; Mon, 28 Apr 2025 19:41:53 -0700 (PDT) Received: from localhost.localdomain ([39.144.106.153]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-309ef097cb7sm9893211a91.22.2025.04.28.19.41.49 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 28 Apr 2025 19:41:53 -0700 (PDT) From: Yafang Shao To: akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org Cc: bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH 1/4] mm: move hugepage_global_{enabled,always}() to internal.h Date: Tue, 29 Apr 2025 10:41:36 +0800 Message-Id: <20250429024139.34365-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20250429024139.34365-1-laoar.shao@gmail.com> References: <20250429024139.34365-1-laoar.shao@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: A2E6E40005 X-Stat-Signature: gpbuqdp1b7dwtuas83b7dz9igyeyukni X-Rspam-User: X-HE-Tag: 1745894514-529822 X-HE-Meta: U2FsdGVkX1+I6LYHXZ1fPmk9cZI2bkSixj2+0ROkfw477yvt8gDUMOpHHXP4ahTG5CdBamAtCjs+q5MrdlR1o1dGJq977NwhzrhRnANTZwbIk+uwjDbDa9eBMfUjy+lxOh7o95qRIKBiJrUIX32hAnpoIrUChwGQdAkDCMsohOPOVtcn/i9uqdH4ZEdW4nZZlh5C8Er4BWjC2vADZpG5nsmLHvOjLeBbkPLMxvcZIGaCc3OUYZNeFiDwrWpBMgvrKxaJxgwFCNa1+ZC5wkXD3uo5zEVYxKOlptQcLtGdOmbO1+Rcwl/Yx/670cjWAzw1SYaEJuoUPLs+uhKZxh76eKK+v81dOZYDxeBSSLLE/6m1KPo13muh1G+KOyi3Yj8v9zGsnimtpG26B0hJvyqFgBA9w+ufFUxS5we+bBKikpVahiRuGog8UbsTtS10zO1cl3ujWjCG3Zvg2y8WYJoJJUxhNK0U7KfVh8vOdPP0TgTapJddJNlehmXmgNs/WlGdTWKDLkN6WtojOxwwi1QqZ/lE6W4wafoSp2N+6rUGiRbJFQeS49eKdCuH5uea1GEgP03gzeY9j479Y0gb7mjtojPrTn6OcN6RszHJx1+dz/Lai91+8DVGU8OpHHK4bOs2lZkS52/vWIAzqHn/wR8sn186KqUwjmVn0hdbepl7jKHuvkSE/9qZiJ8qB+Y1jK8sj27dSDv0FrwqGGTIFKbpbDbgR5hisF4np6XYYL9+yIYfY54pOwqFnEBglaXrwmvTlG86Ef24vr8y0YrDJS7l4qzFK+Hn+OTEubW/EemPfiEAMHBMkzRS7FUUcmmJexprCyXXJz+jEQaRNKt1XKEQpkV+nKB0gUoesn4TMjWL9oRN0rmY1RqyHLe+aq09Qey9XgrdOZxz14aNkikJNDDfNR+WTXurz4xzevx2kI31ozASxKd6z0wokvl2TitYYESb9Fr3MFm1NAmLOrdDP8y SVwNK9xV CGaS3KBX1hVdIjDQcEnitwM3ZMooHaYa3u6Q30UsDoo0fN6jrVhvdk0J81684yv7DGE9NVOPrOUD98lJoXAXtGw3toiZhPInPN04kyc+NXKjQg9P/hbfe1xQqxPhSoXoic5kPwAGfIv+J8Te7iepJFUKwUNys1CanZKkYE0hCQHDp62wiL2D09A92RPGTJlcFbTKpPbQakvx7fNRw0FJIYHqyKwxl7ErlDB1WVZf1atZheETqy7pmHT02A9Kdoqszt/3gvD2y9KmhA1DBTyyd9Ql9c6IwNjdjf/DPFWhOU53fgTZLlj0W8kEjbnSsGkaLCyJkOjzG4/rIWMx3HdyX4b/87lA2JOAq0e+2rQEthCt+C4OXeVVF3Lb+U8RahP3YVbC0IBq+I2MxQ4Eaut6qOt0adkqLjsujvaAMOXIDec+YZiCi7k6HAB27IpAKXJRhBSbt4Si8aN/sx/JSqmg71db5WCMzsVz+LNgKjR//rl66YxzWtxwBBzFkd4aOTi8CpY8Pt+P2vGwYoDCgtEHDTcR5QQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The functions hugepage_global_{enabled,always}() are currently only used in mm/huge_memory.c, so we can move them to mm/internal.h. They will also be exposed for BPF hooking in a future change. Signed-off-by: Yafang Shao --- include/linux/huge_mm.h | 54 +---------------------------------------- mm/huge_memory.c | 46 ++++++++++++++++++++++++++++++++--- mm/internal.h | 14 +++++++++++ 3 files changed, 57 insertions(+), 57 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index e893d546a49f..5e92db48fc99 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -177,19 +177,6 @@ extern unsigned long huge_anon_orders_always; extern unsigned long huge_anon_orders_madvise; extern unsigned long huge_anon_orders_inherit; -static inline bool hugepage_global_enabled(void) -{ - return transparent_hugepage_flags & - ((1<vm_flags - * @tva_flags: Which TVA flags to honour - * @orders: bitfield of all orders to consider - * - * Calculates the intersection of the requested hugepage orders and the allowed - * hugepage orders for the provided vma. Permitted orders are encoded as a set - * bit at the corresponding bit position (bit-2 corresponds to order-2, bit-3 - * corresponds to order-3, etc). Order-0 is never considered a hugepage order. - * - * Return: bitfield of orders allowed for hugepage in the vma. 0 if no hugepage - * orders are allowed. - */ -static inline unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma, unsigned long vm_flags, unsigned long tva_flags, - unsigned long orders) -{ - /* Optimization to check if required orders are enabled early. */ - if ((tva_flags & TVA_ENFORCE_SYSFS) && vma_is_anonymous(vma)) { - unsigned long mask = READ_ONCE(huge_anon_orders_always); - - if (vm_flags & VM_HUGEPAGE) - mask |= READ_ONCE(huge_anon_orders_madvise); - if (hugepage_global_always() || - ((vm_flags & VM_HUGEPAGE) && hugepage_global_enabled())) - mask |= READ_ONCE(huge_anon_orders_inherit); - - orders &= mask; - if (!orders) - return 0; - } - - return __thp_vma_allowable_orders(vma, vm_flags, tva_flags, orders); -} + unsigned long orders); struct thpsize { struct kobject kobj; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 2a47682d1ab7..39afa14af2f2 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -98,10 +98,10 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) return !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); } -unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, - unsigned long vm_flags, - unsigned long tva_flags, - unsigned long orders) +static unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, + unsigned long vm_flags, + unsigned long tva_flags, + unsigned long orders) { bool smaps = tva_flags & TVA_SMAPS; bool in_pf = tva_flags & TVA_IN_PF; @@ -208,6 +208,44 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, return orders; } +/** + * thp_vma_allowable_orders - determine hugepage orders that are allowed for vma + * @vma: the vm area to check + * @vm_flags: use these vm_flags instead of vma->vm_flags + * @tva_flags: Which TVA flags to honour + * @orders: bitfield of all orders to consider + * + * Calculates the intersection of the requested hugepage orders and the allowed + * hugepage orders for the provided vma. Permitted orders are encoded as a set + * bit at the corresponding bit position (bit-2 corresponds to order-2, bit-3 + * corresponds to order-3, etc). Order-0 is never considered a hugepage order. + * + * Return: bitfield of orders allowed for hugepage in the vma. 0 if no hugepage + * orders are allowed. + */ +unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma, + unsigned long vm_flags, + unsigned long tva_flags, + unsigned long orders) +{ + /* Optimization to check if required orders are enabled early. */ + if ((tva_flags & TVA_ENFORCE_SYSFS) && vma_is_anonymous(vma)) { + unsigned long mask = READ_ONCE(huge_anon_orders_always); + + if (vm_flags & VM_HUGEPAGE) + mask |= READ_ONCE(huge_anon_orders_madvise); + if (hugepage_global_always() || + ((vm_flags & VM_HUGEPAGE) && hugepage_global_enabled())) + mask |= READ_ONCE(huge_anon_orders_inherit); + + orders &= mask; + if (!orders) + return 0; + } + + return __thp_vma_allowable_orders(vma, vm_flags, tva_flags, orders); +} + static bool get_huge_zero_page(void) { struct folio *zero_folio; diff --git a/mm/internal.h b/mm/internal.h index e9695baa5922..462d85c2ba7b 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1625,5 +1625,19 @@ static inline bool reclaim_pt_is_enabled(unsigned long start, unsigned long end, } #endif /* CONFIG_PT_RECLAIM */ +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +static inline bool hugepage_global_enabled(void) +{ + return transparent_hugepage_flags & + ((1<