From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F207C433FE for ; Tue, 3 May 2022 17:47:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 55E076B00AB; Tue, 3 May 2022 13:47:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 50C046B00AF; Tue, 3 May 2022 13:47:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3ABD26B00B0; Tue, 3 May 2022 13:47:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id 28FFA6B00AB for ; Tue, 3 May 2022 13:47:17 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id EDDA21209C2 for ; Tue, 3 May 2022 17:47:16 +0000 (UTC) X-FDA: 79425163272.28.2021ED5 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf23.hostedemail.com (Postfix) with ESMTP id C2F49140082 for ; Tue, 3 May 2022 17:47:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1651600036; x=1683136036; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=sxiHAHKs3kGHAX7nEqE25i1M6VS4EbzVzl9niPw4Uv4=; b=L10LkxKZCjaDgK/Kx90zITzsA+bmEOM+LXDoR90ezzkm2WtczB6DAIVr HJWW9rIV9o1GtRpiYx5uyzUM+BUSVpADauQ3q5JjXadtauA77PU4V1P7P E3fksU2KXtkva2GoZZSn5nD1fLdmSrLyzGwVOh7a+OLWdWOQ+qhU3nx4X +hM8m5ZA8V8935UtdZIYdPdKYYw3KD4r/2QgmznAZ1J0yrDgAQDyxtsY7 bosQ4bVi3XrGAH+jOLUfn54vIGH9+7Tg2+mT6y3FG/gBeMkSYuB0vI/1b hN/IJzyn2MjKvu+uziN1pP851kFew79F083OU9MyBr+ViogCZaUYtC9Ig Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10336"; a="249521851" X-IronPort-AV: E=Sophos;i="5.91,195,1647327600"; d="scan'208";a="249521851" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 May 2022 10:47:02 -0700 X-IronPort-AV: E=Sophos;i="5.91,195,1647327600"; d="scan'208";a="536435108" Received: from prdidome-mobl.amr.corp.intel.com (HELO [10.212.51.158]) ([10.212.51.158]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 May 2022 10:47:01 -0700 Message-ID: Date: Tue, 3 May 2022 10:47:20 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: RFC: Memory Tiering Kernel Interfaces Content-Language: en-US To: Alistair Popple , Davidlohr Bueso Cc: Wei Xu , Andrew Morton , Dave Hansen , Huang Ying , Dan Williams , Yang Shi , Linux MM , Greg Thelen , "Aneesh Kumar K.V" , Jagdish Gediya , Linux Kernel Mailing List , Michal Hocko , Baolin Wang , Brice Goglin , Feng Tang , Jonathan.Cameron@huawei.com References: <20220501175813.tvytoosygtqlh3nn@offworld> <87o80eh65f.fsf@nvdebian.thelocal> From: Dave Hansen In-Reply-To: <87o80eh65f.fsf@nvdebian.thelocal> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: C2F49140082 X-Stat-Signature: 3ngrgcri9rue1b5za1u8j6ifqosqd5de Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=L10LkxKZ; spf=none (imf23.hostedemail.com: domain of dave.hansen@intel.com has no SPF policy when checking 134.134.136.126) smtp.mailfrom=dave.hansen@intel.com; dmarc=pass (policy=none) header.from=intel.com X-HE-Tag: 1651600026-815089 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 5/3/22 10:14, Alistair Popple wrote: > I would certainly be interested in figuring out how HW could provide some sort > of heatmap to identify which pages are hot and which processing unit is using > them. Currently for these systems users have to manually assign memory policy to > get any reasonable performance, both to disable NUMA balancing and make sure > memory is allocated on the right node. Autonuma-induced page faults are a total non-starter for lots of workloads, even ignoring GPUs. Basically anyone who is latency sensitive stays far, far away from autonuma. As for improving on page faults for data collection... *Can* hardware provide this information? Definitely. Have hardware vendors been motivated enough to add hardware to do this? Nope, not yet. Do you know anyone that works for any hardware companies? ;) Seriously, though. Folks at Intel _are_ thinking about this problem. I'm hoping we have hardware some day to help lend a hand. The more hardware vendors that do this, the more likely it is that we'll have good kernel code to consume data from the hardware.