From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEB87C369D9 for ; Wed, 30 Apr 2025 14:49:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F3A626B009F; Wed, 30 Apr 2025 10:49:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EEA146B00A5; Wed, 30 Apr 2025 10:49:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB20C6B00C8; Wed, 30 Apr 2025 10:49:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B98B66B009F for ; Wed, 30 Apr 2025 10:49:43 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B4C6714010F for ; Wed, 30 Apr 2025 14:49:44 +0000 (UTC) X-FDA: 83390994288.07.2277119 Received: from mail-qk1-f171.google.com (mail-qk1-f171.google.com [209.85.222.171]) by imf24.hostedemail.com (Postfix) with ESMTP id D886118000F for ; Wed, 30 Apr 2025 14:49:42 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dXuyw6Qz; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.222.171 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746024582; a=rsa-sha256; cv=none; b=CrDeQuNd/o4AbQU7k/BJaiklldehjtaIVbuQrDO+NRPYPhI3ogg+Anj2HIF6kpVuGnlP7N jkx4vi+TW6qLonDe7lLXHyF0XsjOcT7Jcp2SoSYWu5X/NeHk36zFOKPFQvI6LWZMfcJ5jD qNA4IOSGmbtW7pSpLbBHyJg8IHaWUSg= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dXuyw6Qz; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.222.171 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746024582; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cU4NKfHxJMeuIlQdXd/A4s7t9wkDKo3v5r7gsWNIVic=; b=naNBRgwIIyFOm1c+M3Yn7BPvIgNH+AQHqIIm938xsx6rCViTfkp4wPq/DaIp/iHJyz3oxI hpK2r0touLQhaqMsg/L/0R5Oxmcgb0GAmNazmvTT80l7/tI3gxfz1xdCH24ACI+1DNpUSd qfLjPajGoF0b6HT2RXdqZ30dp3Jov9Q= Received: by mail-qk1-f171.google.com with SMTP id af79cd13be357-7c5ba363f1aso1126377985a.0 for ; Wed, 30 Apr 2025 07:49:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746024582; x=1746629382; darn=kvack.org; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=cU4NKfHxJMeuIlQdXd/A4s7t9wkDKo3v5r7gsWNIVic=; b=dXuyw6Qzu63JR0hPdfB+VkMulWwzRREdt4OcovyrIocPVPQX7YAC/JXGH2iEUG5Sqk uxIVkAlH9RRf/NzcQrf54yjw5CPn1PgrJ5ag2woFSp44rTWBi5FCwzzxny+ug82yEZKk YW9DNPnaqOTyZEuwflPAeH8l4s2wr6bKVCA/+eIrFT/89pz6Mz7X2ANBu1islPob6r0j u9dlCjQQ+eWNr/qPS10aUkYi2QFXXHPewbhWma7ZM/uXlLiGUJHSC0U/65uBgIfn/lrc M2eVgyxNDCbCdK4ptbM33nynppxE/hz02uHiaW2JE3mnfeEpl8oaKexbkIK3gKAIS3MG P+ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746024582; x=1746629382; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cU4NKfHxJMeuIlQdXd/A4s7t9wkDKo3v5r7gsWNIVic=; b=p0ULroecY4jROxkJK/t+RtJJDhhVulh5yjAiydsd1RFYjvbEg4yrYNggnGgAGVs4UF a1GT6UXY8AYPCRK3MYAIioJLfF04xUxi2GyaHbyNqte3o4sSs0cT10MeiWFLosTsuv1M 6APqZXln4W618dLXjbOoPo+y2KEjDuASCUf17Ev+ZQBeoqCssKSEu4Uc2TcyKFNM1fzA j24tuuANRJ04C073NqLcMVTssHZj6DGCxLXcEoWN+D7rCq/OVCUJnGyaJ8DIKjxh221B IrDzScZ0WEvAa/Mh8ON0vdcdUJtkKDIDrUCkocGDD/70o5IOOlhEC/hkzUCvV5Sc/V0r T/uA== X-Forwarded-Encrypted: i=1; AJvYcCX65AZ6F8AjdaCogGNkLl4UP96gh2/AZfktejnQ2Fi4oZ7AlZVSuZRxx/LkDMmiBuStd/o+Acantg==@kvack.org X-Gm-Message-State: AOJu0YwF5dEi6uw+fhR9jga1JbXMjI8N66QthMK32muTYYbi4w5rFvvm i9Lb847t4WgqsafM2TfRUKSXTbakLknXCWfYlm7pP83xjxqJATD1IEONclTaD5JcrdE8I7kwP7y /I70AQUmp71IXkilvkJz4BwS+4Co= X-Gm-Gg: ASbGncsQ/wkQv4bVu8Jo9fSvG54o9fjyYOEgxnL+Y+ODd2tv8j6E9YdNETOxdSe+/EH 9w16wHIVVmPSX/Ct9tIdTwtTIZfx9oTdcMs8V3ZePJX30Ejs0ug/f8eyDb97m0RQYiySURjEEuZ MYfAogCKX4CISvVO0t9CYX7+o= X-Google-Smtp-Source: AGHT+IEFfOtXbhuo2KYEJrXOXA67a49pK/Py7Js2hhpEacw1tYE3Sy9iX9hFS4Mf3q0iInSN4YAnZM4MrrvTqazyL4w= X-Received: by 2002:a05:6214:2584:b0:6e8:fb92:dffa with SMTP id 6a1803df08f44-6f4fe081236mr42855706d6.25.1746024581871; Wed, 30 Apr 2025 07:49:41 -0700 (PDT) MIME-Version: 1.0 References: <20250429024139.34365-1-laoar.shao@gmail.com> In-Reply-To: From: Yafang Shao Date: Wed, 30 Apr 2025 22:49:05 +0800 X-Gm-Features: ATxdqUGSU4uUxIRl1XfcIDPquyr7Fb_J78ZWdDle-9uGuj6DAH62-ubcyPORadA Message-ID: Subject: Re: [RFC PATCH 0/4] mm, bpf: BPF based THP adjustment To: "Liam R. Howlett" , Yafang Shao , Zi Yan , akpm@linux-foundation.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, David Hildenbrand , Baolin Wang , Lorenzo Stoakes , Nico Pache , Ryan Roberts , Dev Jain , bpf@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: D886118000F X-Stat-Signature: 1m19wjtmwiadf7rsuhedab5eod6mqit9 X-Rspam-User: X-HE-Tag: 1746024582-374960 X-HE-Meta: U2FsdGVkX1/I7vRvCLcu7njBcGI68w6UPgR/Zt4bF2liYCzxgEQVzWYks2FutGM4kMOVuxNt77HQTA/M1zSq1op+ZrtUwCttTGCvd8A8F3QXdD1FbOCfeE/7PKvtmyGUVMx8PSg+kguq3y0vA7rMGvtrlgYCOvITel2Wg1GqOFyX8O8uF3/K1N5mKdbXjQqNQFNeUioKKh2C5GwIbQ/U+UVz5wrBU+FCpbCj5knlmXZmxrDJKt0mSEyjlKAhqhHqUvGiMOx9eki+Ws3LI+5Jf/4aYsA1REO/steAF4yxpEmR/1HijEZOf4cIYI8DuiUVNo0KR8zdOlZ9RSY5PBYlvAd4oZGk6wVsg7nVxvyBJ3Bb8YzLrDNOlDqlDRkvk+6hyMNpnlnljYCk5FSsD2JKzi2IRjKfO8sCon3Ede1Kp5zC1qOpHT7TZCqauXAKEPosgSqlD7i4eR4+oDPT+i1jViOYh0CbkgA7ZY9tsRG36WVgJmLnae/8eEgJDA7VhOx5r1IfgxjZQ7jDDuCnKXoCyMB32m4/KXLXS8GliggHXMc82Ygq2/IxAWVHpiM1r8TbO3aSettaAU2ML4C6Az8AZSrlFNC8HN9STNv+hPdqaRuz0DTNgGU3ItsORqgFS+YBvzhsiD5PiTsDIobqbJSUq5zGH0dwwtN5+bg9jEspJldDi95i8qDfZL2Lk7QqZl7yGZt0U5Yi3kyGvu4UpIKz2KbPWw2vt6nUGVPMeKohZcT7sAiwTwaXo21eIfijltzIVZYmJLvwb83jY6eJaxQ6hYM/QNkUwxH6/xVQD6MHy6sYEQ79fqJyBrjKF9aLRqpvb2qDLFqnL1aRXYBfgEYB+vGDuyvi2XIXKj6n6OaQgdNmeSlZ6//nMMUHadFqZYYU/KCnHfwtm1nFGcImYYmuPxCkhKJJwl8Q1wR+uUPIOf3p1qpAvvVcW16R+6Yg4ebtoBOvaQDcr0sMrJ2Yt/l EiDZK06l usO9gxjSVI6HVCk/MKVqi9S7O30Oq6jjh4MH2lZGq0DN6bmxoV5Lfc+pzDWQWH8aRaFHL35979QU9qJsobfzrpmjSOvLVk8MXs/8tiKMoqzbJ3/zNfFOI7IN3HmeZp62NoM9exs06BYT++wiZtmesa2LXgJeALBiL48lsVXc2DLyx4T1bhnQGEfkQexlsERkKJh+c6Z6mIzcSX65bgrzOpD8dI6NCFKQ1LeZNT2L43qSJisDZiJJF2Z/I1gAhmgcsRFo3B8kohhuvAaGrHLQrxCIFIJ02PGz6T9E3Vyh3GJ8dGte4VvrKeKXfzjGDg4rHaSTbDE7+vr9dAFuWB9elqRzJwdIgng/76R0msDes74e5QUNtSHyh2Dtf0qnm39YR+x7e66LEu+XtfephdknDzaJ5+kUlkXeiN+e65jY0EQE3zt11wp7fm2BjwlBs/q/MgTboKNuElIjDcQdQYHviVRVvsJ6TFwmeUNAoElz2tEtHuZ9TYjLfDfnjAr3gdpauPey5sCdqAN4eXlxMZe0mzZZah1RNC74pA78TNiwjIorbSyxOyO77JR6OE68Fz77jGi6acuATd1v7EVzKitb1XmEsU3paK1X/1LL49zyc7VMoN7zSHl396Wjq8g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Apr 30, 2025 at 10:40=E2=80=AFPM Liam R. Howlett wrote: > > * Yafang Shao [250429 22:34]: > > On Tue, Apr 29, 2025 at 11:09=E2=80=AFPM Zi Yan wrote: > > > > > > Hi Yafang, > > > > > > We recently added a new THP entry in MAINTAINERS file[1], do you mind= ccing > > > people there in your next version? (I added them here) > > > > > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/tree/= MAINTAINERS?h=3Dmm-everything#n15589 > > > > Thanks for your reminder. > > I will add the maintainers and reviewers in the next version. > > > > > > > > On Mon Apr 28, 2025 at 10:41 PM EDT, Yafang Shao wrote: > > > > In our container environment, we aim to enable THP selectively=E2= =80=94allowing > > > > specific services to use it while restricting others. This approach= is > > > > driven by the following considerations: > > > > > > > > 1. Memory Fragmentation > > > > THP can lead to increased memory fragmentation, so we want to li= mit its > > > > use across services. > > > > 2. Performance Impact > > > > Some services see no benefit from THP, making its usage unnecess= ary. > > > > 3. Performance Gains > > > > Certain workloads, such as machine learning services, experience > > > > significant performance improvements with THP, so we enable it f= or them > > > > specifically. > > > > > > > > Since multiple services run on a single host in a containerized env= ironment, > > > > enabling THP globally is not ideal. Previously, we set THP to madvi= se, > > > > allowing selected services to opt in via MADV_HUGEPAGE. However, th= is > > > > approach had limitation: > > > > > > > > - Some services inadvertently used madvise(MADV_HUGEPAGE) through > > > > third-party libraries, bypassing our restrictions. > > > > > > Basically, you want more precise control of THP enablement and the > > > ability of overriding madvise() from userspace. > > > > > > In terms of overriding madvise(), do you have any concrete example of > > > these third-party libraries? madvise() users are supposed to know wha= t > > > they are doing, so I wonder why they are causing trouble in your > > > environment. > > > > To my knowledge, jemalloc [0] supports THP. > > Applications using jemalloc typically rely on its default > > configurations rather than explicitly enabling or disabling THP. If > > the system is configured with THP=3Dmadvise, these applications may > > automatically leverage THP where appropriate > > Isn't jemalloc THP aware and can be configured to always, never, or > "default to the system setting" use THP for both metadata and > allocations? It seems like this is an example of a thrid party library > that knows what it is doing in regards to THP. [1] Thanks for your explanation. > > If jemalloc is not following its own settings then it is an issue in > jemalloc and not a reason for a kernel change. We don=E2=80=99t change the kernel to accommodate specific userspace settings=E2=80=94we change it only when it benefits users more broadly. By the way, this patchset isn=E2=80=99t intended to address that issue. If it=E2=80=99s causing confusion about the problem this patchset is trying to solve, I=E2=80=99ll remove that part from the commit log in the next versio= n. > > If you are relying on the default configuration of jemalloc and it > doesn't work as you expect, then maybe try the thp settings? > > > > > [0]. https://github.com/jemalloc/jemalloc > > ... > > Thanks, > Liam > > [1]. https://jemalloc.net/jemalloc.3.html --=20 Regards Yafang