From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00714C54E49 for ; Fri, 8 Mar 2024 00:49:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 420D16B02E7; Thu, 7 Mar 2024 19:49:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D1106B02E8; Thu, 7 Mar 2024 19:49:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24AE06B02E9; Thu, 7 Mar 2024 19:49:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 139D86B02E7 for ; Thu, 7 Mar 2024 19:49:40 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 95EF31C14EA for ; Fri, 8 Mar 2024 00:49:39 +0000 (UTC) X-FDA: 81872038878.28.9489B48 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) by imf01.hostedemail.com (Postfix) with ESMTP id A2FA440007 for ; Fri, 8 Mar 2024 00:49:35 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=IR3ri9EM; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}"); spf=pass (imf01.hostedemail.com: domain of dan.j.williams@intel.com designates 192.198.163.18 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709858976; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=11MxcUOCtP911/rb5zM/pP5KO2bWBqcCAwraUc8GIVU=; b=j25dtrBdrt4OTWNjw/5bpn7zMuiU2oOLNpe8oh4YzR+JomZ1EQLvHa9351G7e5GIJOhWs/ 9RcukvyDIkFySJpqnoSNMJsCy63W/D1BihzlTc5Jz5ROPy6wOM4I2YU+poqmqNNcltLXCg l7/2KNkCE0zPCEP/9L1O9AkICy0smJQ= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1709858976; a=rsa-sha256; cv=fail; b=FvINdKxvkQRe9CsYTbjbOibRBsCuA+3+sA8UG9N42en+qKHjLdOh6xG7d40wTDAeWTl8Nh hmomnXt9Z0uJWBjBRJjdNDTRa1CosssHvSb4NTWuf57vqM/buOrUjc6/EliyIKG1Lu/1Ky zgG/buLeWMe2LUVyEMT48qe9oBxFe+k= ARC-Authentication-Results: i=2; imf01.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=IR3ri9EM; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}"); spf=pass (imf01.hostedemail.com: domain of dan.j.williams@intel.com designates 192.198.163.18 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1709858976; x=1741394976; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=y6mD6WnYeDrjUcQ9skwEljpABm408ZoRA8rQwE/BaQQ=; b=IR3ri9EM9plyBmgXcK5UIroATdgMcgcYd6Xq64n/sceXDDQB/ruLdyGI B3SRDYparluuLBjj+563NKFBiz1yT+sISYpKYhwj1PJ9xaI3A1lJwq74e IX9pDwUJssfbFLTPgcaM1whNRLpLg4p6/2EU+d4TRtByUVkYnRuqB5Eab AdKrEN05heT7Lor9Ox5eVEyK4GiRyuwrDpaeW5I35FFnVfD7MqXpi9Tu4 /qNxHy+BxXDJV/w/VVK7qBkvCQExHhnnpUvqrku36MyEdtapUtp1ycZ4T ki1DR3QlEo/4BTaUFDY2/kT0WKxUFuX9phG7XOcM4PsI8u8fCYselBSxd w==; X-IronPort-AV: E=McAfee;i="6600,9927,11006"; a="4408701" X-IronPort-AV: E=Sophos;i="6.07,107,1708416000"; d="scan'208";a="4408701" Received: from orviesa010.jf.intel.com ([10.64.159.150]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Mar 2024 16:49:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,108,1708416000"; d="scan'208";a="10273239" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by orviesa010.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 07 Mar 2024 16:49:34 -0800 Received: from fmsmsx603.amr.corp.intel.com (10.18.126.83) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 7 Mar 2024 16:49:33 -0800 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Thu, 7 Mar 2024 16:49:33 -0800 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.168) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Thu, 7 Mar 2024 16:49:32 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kpvGDtBDA3/CRxeC9AAkMunroFgNN4rmazgghSAvJ1wi6qbXDdhC3rsKtT4hQypDf+JmwH/aiJ9EiYlmWWkcenMBFXPOVfXYQHjxY2Y87Aa0txjqnnl5Slc7A6ujquxukDCyZ8x1l1Ae43u7PZt37JoxUgWu9D4S7EQDGpkkVkDtBFe1YoBnRXukTQCwkP6Nth9Xp65MbMzDl2IBmr1Eu1GD4sCEttmaNnpAu5NvpkGm7y2QtU0nBoQtmUtPhVAW6IFrZMqPni+3VgHWY5FKn09VpCq9PV2z4gM7qToqHUqSYTgwCxmlGJd2OngNs4Nkk1RQ8FCCrHC44/9irl+WLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=q90tiaL/D+vLn0KGcVqSQSPo62MR/ZYcALXIua2ukI8=; b=FtU8pbJ6ig+Qk6YfvtPcYry7toPQwv7V86+wZOOTJIbR15p2YQNP1/gxv3pMLwiC05Cv27kIm2v0GqSpX6DwlMjkO83YkJ5GjDfS5nWqlhAbmszcSv+zcyI69L+ghCu9itZsr1+74xupucINsODaH/kwELSJkXzcmMu9zusGByzcq8pNs7Bam8TG7y+WHex1oyXWTzfqpzGHhKmsvr6fQ9KNf2VjRX2W8vf0OQjRhTqHVeieU1keA/BfsuiC9fr4xzQ+4B9TuIQQWCmYGPNTPfqT2UAyz1Th8+315a044PdrmZw9eiuDEQqgKwbtRRGj32Edp86x2iFJin0s/hLu3Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from PH8PR11MB8107.namprd11.prod.outlook.com (2603:10b6:510:256::6) by DS7PR11MB6064.namprd11.prod.outlook.com (2603:10b6:8:77::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7362.24; Fri, 8 Mar 2024 00:49:29 +0000 Received: from PH8PR11MB8107.namprd11.prod.outlook.com ([fe80::82fd:75df:40d7:ed71]) by PH8PR11MB8107.namprd11.prod.outlook.com ([fe80::82fd:75df:40d7:ed71%4]) with mapi id 15.20.7362.019; Fri, 8 Mar 2024 00:49:29 +0000 Date: Thu, 7 Mar 2024 16:49:27 -0800 From: Dan Williams To: Jane Chu , Dan Williams , , CC: Linux-MM , Joao Martins Subject: Re: Barlopass nvdimm as MemoryMode question Message-ID: <65ea60971700e_1ecd29472@dwillia2-xfh.jf.intel.com.notmuch> References: <65ea2c1a441d_12713294d9@dwillia2-mobl3.amr.corp.intel.com.notmuch> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-ClientProxiedBy: MW2PR16CA0009.namprd16.prod.outlook.com (2603:10b6:907::22) To PH8PR11MB8107.namprd11.prod.outlook.com (2603:10b6:510:256::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH8PR11MB8107:EE_|DS7PR11MB6064:EE_ X-MS-Office365-Filtering-Correlation-Id: 19e79aa1-cb8e-4b3a-a626-08dc3f099c66 X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4xfzK7sgE17qPYVUPq1OzE0R/JUJ2aeJ49V49KXEygydbL9ECOEoT+WQOqLXp5u4RaHUyruFARc3R+pWowKY7baWf+xciSzQXFrmGXYX4a75YjCqolzQhPU+lceJdZcHxm7YhC6zeKSaVVIRsxt/dUX2uW0jbR28Nw0pHBilNsfPuwmTPWUm1K6LhfKmQOJIXz4s/txCTke5cv9mRv3oHmUetBwNL6H6wnbbKgCAvuXhmMHMZw4NxoTSJrBUGKKBfI8EKD98GOH5xFGEICf62/jEtFmuv6ByE1JuksVMWP2fVECu2azXwf5z6Q5cCPel94fzDPXTweXxk9NbSo/wLHtxIuRWO5Kg+HwBuMuMBV0CCLOs1MfMYGE264wClf+v1btBNZGk+TI+SpHr63vw39rTOlXnNla9ih2EIoBZmUt2sdrdmLFK4IT9WX/iaosCwZwde+MLpKZWVSOe67/FKGCLtnyMjw8dJuETFOcW1KJjxqdsKzBkBtjEefzwBBfSjdwTiJRby8dnOtVAhiwcSacq7boe+KfoMx2RgeWM6WTHF7q6Z+/Gjvhul9SM5Rqok7ZbBuEgFe/55OHbqWK5tRvd+FfcgwIaIpURPPadMdI= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH8PR11MB8107.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?pNlBEu/8wYnU8EovE2cEFuQDDT5VJjSczZCaLN12PawplI3vZiCCiz8xiZ?= =?iso-8859-1?Q?AZmgnKgA1h09KOFFdc7VS4zHIaqx/Mdd7GjdkjS/9W+NBBoPMDhRwyEBRk?= =?iso-8859-1?Q?cLWv39bzdF2kSw/zjFr94DR4jx2WDMcL+Umqz+XOOmQgKJ7tDZQ2/Q2/uo?= =?iso-8859-1?Q?HLElEFzg3rHr9DblTlYPXMG7p7kU+szJtnO2sCBKKEfouUAnl1il/ro+9e?= =?iso-8859-1?Q?/hpBD1T8cYX2898RctuvyUb9pnYOL2mofQ0CP5hovr8UBy/f/HrbN0STAu?= =?iso-8859-1?Q?7/6GVrzg8NnglKz3sRZEiCQNyC08Gb/TSe8GQ1eLO9iSteLPYP1MsbWomo?= =?iso-8859-1?Q?Qvhoppp+5nNmgbmazdCKLnYSn0DqWKrICXycxz5PsznwVBfbgQXQqwUQ7h?= =?iso-8859-1?Q?489cTjbROL0EBmHBYAgDb5zmco8pizfFPbmdZZtvh/0Pde4ie/3/YgC6md?= =?iso-8859-1?Q?A7KRyEX8y6s5d/tu+WLokIIo7omL1gGo5GVuf06LCIH4ExQpGHgC33Qj8A?= =?iso-8859-1?Q?AsRqHF0li8otr4wa+rjBPUKKcB3baYFAKauU0iDBfBCU5i1YGWwcgZ2VJx?= =?iso-8859-1?Q?G1PPWqgM1ekTRapjErW+H2NZVI8lATP6km1HyfAiX6ZZCUAZtHFq9NernY?= =?iso-8859-1?Q?LFfDLGdryqgraabh/i7V9omAXpQcbQNxTOibc1kX6PKR0/5aEaM3R5flpK?= =?iso-8859-1?Q?WPuU1Eeu/Jdxd/IanShv45H4U3LRNvhAmewh8Fvm98vETDjEqq3izrZ+8x?= =?iso-8859-1?Q?1BthOzLxBDNQaVtINbD2AvKIXf91cft2dF6Zvgo5aYOeVhCcFL+KCuoOVp?= =?iso-8859-1?Q?LvKBSM9LpANYusNMCJiZQLHOv68/QRScbFVKqjh/IdO1/lQZyL+q9APPds?= =?iso-8859-1?Q?1szGsfKiO916ah4UQGrm7iG0RDeFeZHnWeldg5Ab+up6g8jjbUDWH7J+/r?= =?iso-8859-1?Q?rnsjzEnl/XkptLzfnTsPUMT5tAmchJPF2JtNDdirDEItt3s5KsY8v/47xI?= =?iso-8859-1?Q?wpOcvP+F7F2qc4Xa42JiBHMOfw0aHkvuU6Qn5GfheJuZ87D1JLYFGM3z5h?= =?iso-8859-1?Q?NLVUXcQh+7vxd+ggzBEzVsbMmxeqinyaKWbXiyGNExF/uC0+VCqe4rRQ9B?= =?iso-8859-1?Q?DRayz1jt/ymmEer05J6hNtwRDcvJl6zw7rxtiz3eqAX2JQSoD72fuRbBBO?= =?iso-8859-1?Q?MSHmeaak6CpqdPuy+VxRsHmsSs103CP33+UiZZsbjC/C1OqmOp3WPA0PGd?= =?iso-8859-1?Q?Idi6gdlWuBFN9ZzNl4DANICzqu0OPOCpF623V4JSqoE7hHs4KK1EXSKgCD?= =?iso-8859-1?Q?OE06lVVDbPV+DTaRhMIIY5BoZnypooSwqXfEvm8Il3CbVu4SqHcMuQzCaG?= =?iso-8859-1?Q?/7oDj9YV87Xrsn1YOvdXDw1dRK1LcGyw6EKaF6RE9MnYclDBhVs0UC9rZt?= =?iso-8859-1?Q?JSsDDDEsANHWnR14WDC0JZQHIF0r3EqFX8g6/eqroOMPqw/HWWOy7Z2l8r?= =?iso-8859-1?Q?JuhHahCRUd5QxLTHBxhSM+i7psNVWy5/jYjn1U3vgcAwcSdjqDkb0XqMEu?= =?iso-8859-1?Q?sHWXWWXNnqo/mBNiDoM9vdZ/dmWIMhL5cD6L7vHAYQrj6hZ2wsDZcYqwer?= =?iso-8859-1?Q?qYacEXbOuPid94gtPGPGDwnlxgYPwMhFbzA9lKJS6Avb9fgwedNdewUw?= =?iso-8859-1?Q?=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 19e79aa1-cb8e-4b3a-a626-08dc3f099c66 X-MS-Exchange-CrossTenant-AuthSource: PH8PR11MB8107.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Mar 2024 00:49:29.3136 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0bhN9FIDihG1ah9vYDYOmaMnj0MUm8LR8igsH6Oal4PidURyDddWxSb8zUSCDWw8QIAGGffckW2pbCLGbACd9wi9qPxc4EwUBPZE3zf4oFw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR11MB6064 X-OriginatorOrg: intel.com X-Stat-Signature: eco7grajbyjqtihtuuaeg9odfcmmp5xb X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: A2FA440007 X-Rspam-User: X-HE-Tag: 1709858975-397913 X-HE-Meta: U2FsdGVkX1+sd7xnGSRURC0xI7thx2r0fA1P6Hx0KYgWfOaLVs91Wsy9+0c+MzW3fuI3us7P/hCsds4KwHBSOGT7eu3LQXiGeKAgh5sas3CvtJdTeuqWAf5/dW8g8kPXbwBSKdlUI4w9N2W3aJj8Y2y6kf+cnHZfTHFFvHWgl4zKireU5o7IpRz0VkUTe3uplqZz1CKb2LoFGmsqBrR8tixrSgPhEOb5CezPRpCO5AikSDSvwNnMkcIfx3eIMuzlJzYp9H9axLIVystxaV26Yd6nL9AcJAhFpsrAkC13/977QZca51IZ6iCoqkV/OCTDgiJ2awei4y1iq6IUtvGt5/wOhuzw4OCiaaPhC2s6k1Af7wUrBVUyPY6dNBL02W+BFNQjIW0qfnR+EdLKr/8hRkkNzwrfs6S/h7Hzx5aB6/HARLBGyoMv7PhaLTc+wXsJxc81Y3Dat8wRk4P6LapndzQ8kKY7OkW9NNC8gXVd+mxQHvJpMwQz3RpC/nASRIad55pVCL0tYDIkQttu9K77OR2e2vpea+kukZQcI7FzqcPvQbt8zh2sL57Vm391RSn8+oS4sikxGcpJT4jf7bkebQKoP5xVMbWtcLArh0jWSseMM+bLLG69rh6/PflEga2DBdi7cHz6lxWAig5TCZyewtABzf+xPpYKZ3siad56SPWIZgq1jnbIMC50iVCTpJ2op+bs/CPoOdhVI10SJ1dsJaSHEHxLrmByQI5VpwDsWXlIzqe0miYi5tOjWaOq1Mu/ok+6d6Gspp08cnL+3YTsMM79mm3d64Nwfu7rrFkD8ixAoxwwcjzddOSHZX49ar6WhLybpam36ovL9Lz/xBGtIDZLumD+MSk85tXUb6363YdUPe8KGwKiWN/O7F8ft+S745pdMsBQrtM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Jane Chu wrote: > Add Joao. > > On 3/7/2024 1:05 PM, Dan Williams wrote: > > > Jane Chu wrote: > >> Hi, Dan and Vishal, > >> > >> What kind of NUMAness is visible to the kernel w.r.t. SysRAM region > >> backed by Barlopass nvdimms configured in MemoryMode by impctl ? > > As always, the NUMA description, is a property of the platform not the > > media type / DIMM. The ACPI HMAT desrcibes the details of a > > memory-side-caches. See "5.2.27.2 Memory Side Cache Overview" in ACPI > > 6.4. > > Thanks!  So, compare to dax_kmem which assign a numa node to a newly > converted pmem/SysRAM region,  ...to be clear, dax_kmem is not creating a new NUMA node, it is just potentially onlining a proximity domain that was fully described by ACPI SRAT but offline. > w.r.t. pmem in MemoryMode, is there any clue that kernel exposes(or > could expose) to userland about the extra latency such that userland > may treat these memory regions differently? Userland should be able to interrogate the memory_side_cache/ property in NUMA sysfs: https://docs.kernel.org/admin-guide/mm/numaperf.html?#numa-cache Otherwise I believe SRAT and SLIT for that node only reflect the performance of the DDR fronting the PMEM. So if you have a DDR node and DDR+PMEM cache node, they may look the same from the ACPI SLIT perspective, but the ACPI HMAT contains the details of the backing memory. The Linux NUMA performance sysfs interface gets populated by ACPI HMAT.