From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21114C5478C for ; Thu, 29 Feb 2024 02:04:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 584AD6B0099; Wed, 28 Feb 2024 21:04:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 536A66B009A; Wed, 28 Feb 2024 21:04:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3DD866B009B; Wed, 28 Feb 2024 21:04:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 282A76B0099 for ; Wed, 28 Feb 2024 21:04:20 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 9C9E5A1203 for ; Thu, 29 Feb 2024 02:04:19 +0000 (UTC) X-FDA: 81843196638.08.B2A25E3 Received: from dragonfly.birch.relay.mailchannels.net (dragonfly.birch.relay.mailchannels.net [23.83.209.51]) by imf23.hostedemail.com (Postfix) with ESMTP id 21667140004 for ; Thu, 29 Feb 2024 02:04:16 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=stgolabs.net header.s=dreamhost header.b=COYrD5wy; dmarc=none; spf=pass (imf23.hostedemail.com: domain of dave@stgolabs.net designates 23.83.209.51 as permitted sender) smtp.mailfrom=dave@stgolabs.net; arc=pass ("mailchannels.net:s=arc-2022:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709172257; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=E7DKESpcaV6chmUTi40ZFu7Xg1phWtBp7CMMFQ21ZMg=; b=8HXpLnLjQE7hQcUr/wP9mVQ5lquuoTEUROwFnpNbkC7c+YBKlTYp/4G0pbRWc1mvLCnIZu 5iqAet+Zylt6QSY4OOZ3LfHFtYhQclH8cFxVxpaxGUe/ocrR8eN1v67mqAyKc3W8de33LL Us7v2c0CjfOQZ5m6wjUuzDuDKdl4q3A= ARC-Authentication-Results: i=2; imf23.hostedemail.com; dkim=pass header.d=stgolabs.net header.s=dreamhost header.b=COYrD5wy; dmarc=none; spf=pass (imf23.hostedemail.com: domain of dave@stgolabs.net designates 23.83.209.51 as permitted sender) smtp.mailfrom=dave@stgolabs.net; arc=pass ("mailchannels.net:s=arc-2022:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1709172257; a=rsa-sha256; cv=pass; b=sPoEOOGMC/Zowk1kHJnO7qtGdxHbkvU0UGWezEUyye2LUMmNDHYJd/vvIC764TBqamEhQb +h4eHjsAK1k8u5YTXDdDFA3LznT98mygDB3p+sojpS6B0+x1soc1vAEgUNzvmQvdOTsusi /x0FRk5I09/OzsaSKwXRqaQZcgVY110= X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 94BCA900E8A; Thu, 29 Feb 2024 02:04:15 +0000 (UTC) Received: from pdx1-sub0-mail-a239.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id E7C6A9033D3; Thu, 29 Feb 2024 02:04:14 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1709172255; a=rsa-sha256; cv=none; b=8gldHBrNCZXKVID9xzgk8yfofQWqAbN6bt51BxqfNx87FY3Xwxj5HaxX4EOwV+2EDKoAq0 DROV48Jid5WSLtiHh1pmN9GTy6P6Pt1tgbusVorNOnAq8g3zNtX7H9F5gK/29mi08tQZI1 TEJR0DJGcYjwG/UGsMQZvhd1yWzJ3J4IhDqTlJ/EPLXZEve8h/AGQ1ezBvE9jSY4qa6UYi XQQvk8kKFBq1xKojrBbUMs7oZug/ygE/UjmExticEU577z2qC9/fYa05TDowng6MAm3+UB AcRc1CabMU4AeNjL5wxW3o72KDKMISbE5Hst4qlXuCo9tv/uADcIfbRW2cOIAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1709172255; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=E7DKESpcaV6chmUTi40ZFu7Xg1phWtBp7CMMFQ21ZMg=; b=EMkKNQNM/DwSuPP4Su4F4fmRPr/MBD+I2snQTz0XBlWaJdQsArDdlXj0p4DMkSl7bC9z7S m2z0PUEZlQED7GE9Ryr7p3/fzYvuup5jRPifMnEvp2HINvhgIwgHDI+wzbfub/LZK9Hh/J ZOiTeRqoCdtVoP+qLgOoNg7f7jsqmsuOqRz4eTYm5W0GlCHQRW8UISdlK8JI46GmGBPYf1 qbcdlg2mG36XqtFdT9kQl6NQnWK+Wy26351neVb1cnG2ARmKYIV9Ua+rDLo4cKEmz+GJKy Dvf8YziUGXYYBO22UEWHZKoOAzFouF2D/P1e3W5P30PAouxppQgYvAvnSluMdw== ARC-Authentication-Results: i=1; rspamd-7f9dd9fb96-tcjk8; auth=pass smtp.auth=dreamhost smtp.mailfrom=dave@stgolabs.net X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|dave@stgolabs.net X-MailChannels-Auth-Id: dreamhost X-Abiding-Interest: 17d420dd35083f41_1709172255374_1846128221 X-MC-Loop-Signature: 1709172255374:3378886407 X-MC-Ingress-Time: 1709172255374 Received: from pdx1-sub0-mail-a239.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.109.8.104 (trex/6.9.2); Thu, 29 Feb 2024 02:04:15 +0000 Received: from offworld (ip72-199-50-187.sd.sd.cox.net [72.199.50.187]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: dave@stgolabs.net) by pdx1-sub0-mail-a239.dreamhost.com (Postfix) with ESMTPSA id 4TlZK95D54z17; Wed, 28 Feb 2024 18:04:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stgolabs.net; s=dreamhost; t=1709172254; bh=E7DKESpcaV6chmUTi40ZFu7Xg1phWtBp7CMMFQ21ZMg=; h=Date:From:To:Cc:Subject:Content-Type; b=COYrD5wy7g52RX9WDQTcsPYc9LWrBFWV2Fv4l+ElmWovZOPYyNGVWjK5kOaXK8IPW Re6uT0iP92sOw6N344seNQaJRmrGkoQBaEBVKiPbdplY3Ew/oJxsclz+bNgCNi9EJ8 W0D4kIo71sY3C8pvolI+6C5d/CdQxczd6ymO8tRvYcZTdK4ngI+HccFnoC6ljEl2xk 5DhOGs0RCJ6903iJvGYF+6U7fEvJEbM3MrXHozNzaXOctr+swdnxPpyaD4+Y1oiIFF K2cpJvSpdeX9XL0dNXI3lTAdLUx+avzHWHMz+IqZKvDkYY20wMMpsKL3qbS4xKwLDf JaasSeXjBe1Uw== Date: Wed, 28 Feb 2024 18:04:10 -0800 From: Davidlohr Bueso To: David Rientjes Cc: John Hubbard , Zi Yan , Bharata B Rao , Dave Jiang , "Aneesh Kumar K.V" , "Huang, Ying" , Alistair Popple , Christoph Lameter , Andrew Morton , Linus Torvalds , Dave Hansen , Mel Gorman , Jon Grimm , Gregory Price , Brian Morris , Wei Xu , Johannes Weiner , linux-mm@kvack.org, Adam Manzanares Subject: Re: [RFC] Memory tiering kernel alignment Message-ID: Mail-Followup-To: David Rientjes , John Hubbard , Zi Yan , Bharata B Rao , Dave Jiang , "Aneesh Kumar K.V" , "Huang, Ying" , Alistair Popple , Christoph Lameter , Andrew Morton , Linus Torvalds , Dave Hansen , Mel Gorman , Jon Grimm , Gregory Price , Brian Morris , Wei Xu , Johannes Weiner , linux-mm@kvack.org, Adam Manzanares References: <75f21150-1e12-4f4b-e578-e170e4fea18b@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <75f21150-1e12-4f4b-e578-e170e4fea18b@google.com> User-Agent: NeoMutt/20231221 X-Rspamd-Queue-Id: 21667140004 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: g4mxih5y169oer9qhrxorwgk7jgcbwew X-HE-Tag: 1709172256-321770 X-HE-Meta: U2FsdGVkX1+cWssAiZRvg3wH8bOPwKRVwUxGlyitObGYXbfP9PCmAGMHqVYTcCorBiwb0LjtN40mjn4QtodBxdSpU+pvdFrw9BWHikFKACs/6nHFc3wRAOpsmHdcCsaxXKaRwRWl5AhVbPD687HEuJ8HZgGfR6Q1mmhCBqpzhPHD3ITDbxz2PTTosh1mOwwtXcdc0sD2x1TOY8zv8V1i+z2rrVOh/4XV1MSZtHL7gfxeQZs0g+2pNB0J0kLk25Ph3b9ReabfPGkFcFu+d/wqCU0gs2Udaa6IkimrhCyI0q37vlRq1TWvgvwMYoLrOogRmAvI5ZaSz2pqTMGZPnQyk1yhEq9puynuyYlEjCsMjWU5HwtJTXPOv6yiOtcF5I6kgeqQEg0p98wjnalMF5CIHdvYauV7nwAQ+vBcWvkY4sTdb6yMFXib9Jk4R12zKIxMD/zZzcOdDt+J0EOjTXfQayBxwqALnoqqI8Tpk3kyjVglLl4JlPQHX1W0M6NHx4czdKsLyd8BdWcUlDAy13nWDr1dBEhWv3dLGZ8CxbVLpX6gRblfhEVvC44NcIpp45IqrhvSZDcrnzLWFsao0Ai3Xcd7LZsAUmxKUJZ7mU6yTa8V51RLAPu2an3ZHIzoaSqphBtwxJNxJi8CIupJcL+Ks1F3xWgGB8FjW7/0RQGbzVi7h5kvGmpGT+4CZp68s7OspJM1XJ4h4itTQi/BsIGCWRkJn6N0Yg0hcee5WH6/zNZhykL4WMEM/q2fE0OhGG9KmtPM7fq67Xv/lFJaZvPaJ3lQlK4nnLfQZT0il8GlzQNbwckD9j4aq+NnjHkNI3dgwUKQWOggM4luIlQezh4HSiDl/K5IK6muX+xiXcbukn+uq78wEhaATXXWbana/xHAsTDf3o5/rQgVwvLJVo3LBS6TIsEgfbx2gftN5pch4ofv48kx5RHUF/llh2/fBRf8JoRcAWKwE7+ckFagH2w 0eAY4Ede cQbbJVf046PZZHXv3jVzje9LJw43ldbH3oFPH5MgrgV0/llnPaHzk2pHej0KRhOpC6rPFtxySt4aWiRUYVyo8h/xkVqOgzMtu4elMoUr2Ev3BtW+Psp6X6bshVGgqTViAyW7W8rEGEWLpCiZc+7Rj3FikDjLKZuhURT3K/61gDVC4x0nHRLgYDdJ0nHTNahtImUx+2CuYZ7IHKKwfu2fCjiEB/mun89VdH8yVu+NCfiZ8tK6HARZyuXI7JNo0HknugHW4cF0uItk3CxsquLAHxpb+tNS/6w94po7Qn1adyQP2oXHWAoqXlVjRjXz5QxCMdaDLmDvQJ7jfvmg6WHccW3uUVlW14tjk013gSdfDG9x1mSXvS+VvMFm+By4UCeW2LmWlPTHx+kcooah5CRmze2vr+PG4s5TFh7wL+WY8vxBIJpc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 25 Jan 2024, David Rientjes wrote: >Some recent discussions have proven that there is widespread interest in >some very foundational topics for this technology such as: > > - Decoupling CPU balancing from memory balancing (or obsoleting CPU > balancing entirely) > > + John Hubbard notes this would be useful for GPUs: > > a) GPUs have their own processors that are invisible to the kernel's > NUMA "which tasks are active on which NUMA nodes" calculations, > and > > b) Similar to where CXL is generally going, we have already built > fully memory-coherent hardware, which include memory-only NUMA > nodes. > > - In-kernel hot memory abstraction, informed by hardware hinting drivers > (incl some architectures like Power10), usable as a NUMA Balancing > backend for promotion and other areas of the kernel like transparent > hugepage utilization Regarding the hardware counters, can/will CPU vendors provide something better for what is currently there for PEBS/IBS - which needs a lot of stat crunching to make it useful for hot page detection. imo if any sort of hw assistance is going to be used, it better be *big* win vs anything only in software - muddy numbers aren't worth the hassle. Power10 accounts for time decay, and therefore would be better suited. iirc there was some mention to possibly model after something along these lines. Thanks, Davidlohr