From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 602D9C5B552 for ; Tue, 10 Jun 2025 15:46:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF4316B0088; Tue, 10 Jun 2025 11:46:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ACCA56B0089; Tue, 10 Jun 2025 11:46:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0A3E6B008A; Tue, 10 Jun 2025 11:46:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 84BF16B0088 for ; Tue, 10 Jun 2025 11:46:54 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id F2DFFC0823 for ; Tue, 10 Jun 2025 15:46:53 +0000 (UTC) X-FDA: 83539919106.25.3F9E1EC Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf04.hostedemail.com (Postfix) with ESMTP id 1665440015 for ; Tue, 10 Jun 2025 15:46:50 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=OC9KFQXX; dmarc=none; spf=none (imf04.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1749570411; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+joD00bwCj+l+Oro6kgeolprgnWfx+uVOp6jPA1PH9M=; b=mWunn8sOY9pBmPjlkIhNT5UqmluHIE96gPPVLLENGHZMbcK0vEog5rhFoQiHPyvxhf3mk7 l8ploCqfnmmy3FbYoNbL3IM/KjfG8ANK43Rm6aY1dbfYFkXuaJNM3IV1ll5q49zE7mc2Hz H99msAJDwhraDi0b2v5Tb57qZYos7cQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1749570411; a=rsa-sha256; cv=none; b=XZrwLAUNkTLy1/yeIA3MHatK0XKfHSNGMT+8G5DEVSEYslfpRrXFSaL542EtGdwE4PaXfj /d8uPOxZpRbXCrsEKF4bgthKVAWaIzyajazs+BXbxsNFyIiaFf0FgsLCzVYSFtt9l973YG iZ7dHxRENiyczz8t1sF0Xc92x+6DQKk= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=OC9KFQXX; dmarc=none; spf=none (imf04.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=+joD00bwCj+l+Oro6kgeolprgnWfx+uVOp6jPA1PH9M=; b=OC9KFQXXhGP3G9vcpHTSWUEnJN xvJ9rPmBlTUaX2pv1VBNgqiGAQC/s8olrOeWhhKTniu6hFxrcqiBqwR6/i0r5+99q6dRCcjvssmRb E7jwbz1o5cRZ8CmbLGipZ9aSupvXrooUtfnS3Z3/994hU8Au6pjQMM2emjU8nR0u5rh1oNGnzhw4i ZyfmWxByYt9/M2dQSim/j+o/zvwvbRY2/xsJeY+t25sJJzDI4hHmP9P9GUzN1NxWZClaO0CqpGOIk WORdUjnnw5G10WPJmgo+9LqoGQm6ZphxuYDiOCK8L1eg2c6/d13jSjY57iT49jeifhKer+f4xnFw8 zsM5PF0Q==; Received: from willy by casper.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1uP1BS-00000009ZrU-2Akh; Tue, 10 Jun 2025 15:46:42 +0000 Date: Tue, 10 Jun 2025 16:46:42 +0100 From: Matthew Wilcox To: Usama Arif Cc: Lorenzo Stoakes , David Hildenbrand , Andrew Morton , Shakeel Butt , "Liam R . Howlett" , Vlastimil Babka , Jann Horn , Arnd Bergmann , Christian Brauner , SeongJae Park , Mike Rapoport , Johannes Weiner , Barry Song <21cnbao@gmail.com>, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, linux-api@vger.kernel.org, Pedro Falcato Subject: Re: [DISCUSSION] proposed mctl() API Message-ID: References: <85778a76-7dc8-4ea8-8827-acb45f74ee05@lucifer.local> <2fd7f80c-2b13-4478-900a-d65547586db3@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2fd7f80c-2b13-4478-900a-d65547586db3@gmail.com> X-Stat-Signature: r8fbpwigyo3tctrd1r6zsw7ae7w7pqmp X-Rspamd-Queue-Id: 1665440015 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1749570410-965528 X-HE-Meta: U2FsdGVkX1/jijDETawcOTUnCBt4D5ABYCDUni1aa10iTmwZHYY52Rapk9d4BED+0brWy4ufCZBYPm73xmapB6tlpQ39unRZ3bPhQXrIzTIDxjbCYChYTkqggTPY3180YS9MGJ34EiV/pmIgQnunXh48Gq9qtdPIa+eTJHq7/RosYFMW+uDrKkpt+I1Mk1/dyQzbNd6LkiQk53hcmuvz8oOjE+9D8Gccb5Egl3LGjcVh16hZS3z17MrFnJgdBMbq6gh9eGOO2xCUssfVpNLkgnkLh+uTJ2rrvw23ux+01DpnClU+xcVzzvY9XUdzH0FLLPclceXEQEWXW46PFm6B3nGGgjU9abhOIo/d3+qLT5O9d2bmAcBQGW4BkoG6c9JXT4Y8Rnb4vcp856QSr5UWgmvv5H8ASOptSWECppJ1IctIPyEJpeP9angD04rNxTnE64+/ify5sJvWjHoRn7iYPdzT+ydbi5ZHp+O/qrj/RioRCB1mbjBXA4vuP3v0enxB06llTe4slRxHV7GtWBbva3FWXvDP1NHP+GMgkjpHZ9CnzVw1nQlDlSiYFetHx5vv1OdoAkFVT7Ud3SB/A3Cy0uVvQZ4GkkynCxm7R9wwD7R+2nbWsloQ/KFuw5x5wBC5/9DB0qSrLCP/dgcN3FwlaIBXovq+TeJl5MWjRiK8l6MQT0/iiJWdjDCuU6wkjbyC6zem1CkZG+xKpc2hEtmc1SSpahw+DnFVpM8mBR6y/zazsAuv6oSn2IJb26dGKd7NOHsLfFyv1lBX0lHKjZHJgHNVSvzZLYSUD5yqbV2oNa3VfVeNaSSvPj6QmXRtvb2JOfAizigaiceDumvC503l6NCdICqDmuo18VXaHoOuA9FjWdulURoACBTKxnYEFkXImdv050Jp6tMstYn9vUTNqn1fMzSO/A69bz/SN+OLdsRvC9QOcu+woZXLPtrFydL8HopEwPpV0jiDEgpGI+L 8fJa+ryS 5GAXTSylCmwSysUIzUZREI1HnPwppa0N8yXAI8x1ECHEAwqtQJS30tH74fWUnlo1QnlvDbjTBR7DJn0UJzIazynBcrHgh4B84Q254etyKIzDTXva5EJYFUgep6XobDPyuv1wuwjlIoRKUfrKhyOHtatXuFyiaLNu5Kf6paDeOwae6BT/lwa8D6thbG2jZTyf3J0VDT0lqk5Eh1qayqS1iPSBiTpI0+O2YN71hddAU/FK1iA0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jun 10, 2025 at 04:30:43PM +0100, Usama Arif wrote: > If we have 2 workloads on the same server, For e.g. one is database where THPs > just dont do well, but the other one is AI where THPs do really well. How > will the kernel monitor that the database workload is performing worse > and the AI one isnt? It can monitor the allocation/access patterns and see who's getting the benefit. The two workloads are in competition for memory, and we can tell which pages are hot and which cold. And I don't believe it's a binary anyway. I bet there are some allocations where the database benefits from having THPs (I mean, I know a database which invented the entire hugetlbfs subsystem so it could use PMD entries and avoid one layer of TLB misses!) > I added THP shrinker to hopefully try and do this automatically, and it does > really help. But unfortunately it is not a complete solution. > There are severely memory bound workloads where even a tiny increase > in memory will lead to an OOM. And if you colocate the container thats running > that workload with one in which we will benefit with THPs, we unfortunately > can't just rely on the system doing the right thing. Then maybe THP aren't for you. If your workloads are this sensitive, perhaps you should be using a mechanism which gives you complete control like hugetlbfs.