From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6161C43334 for ; Mon, 13 Jun 2022 07:08:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 451256B0254; Mon, 13 Jun 2022 03:08:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D9B16B0256; Mon, 13 Jun 2022 03:08:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 27ABE6B0257; Mon, 13 Jun 2022 03:08:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1397E6B0254 for ; Mon, 13 Jun 2022 03:08:29 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id D967560987 for ; Mon, 13 Jun 2022 07:08:28 +0000 (UTC) X-FDA: 79572334296.20.130874D Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by imf28.hostedemail.com (Postfix) with ESMTP id C6E26C00A3 for ; Mon, 13 Jun 2022 07:08:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655104107; x=1686640107; h=message-id:subject:from:to:cc:date:in-reply-to: references:mime-version:content-transfer-encoding; bh=4ZL5x0SSS5qZVmA+JgyUkPjLyNtKpHowUfKdZup163A=; b=SgRq6KA/vGTRUbwFYo4GKruaYNfkuKIKhyGwTP2Gf1LZvd5ulMNZKp3X SYIuskyWFpOCq4B+fn7eTKwMpSlZ6wTAmE704DGMZGtK9tY0iGL0Ozvu8 jtGKAhPHobW+ALzG2xcyUmMp/28k33ZpaLSI7CdLlZhKhxMtrXAJsIjmX zIbA4wliHHSKAQobJgUkrridCOnwkRUGFDyk5voJ+TBJDX0+oQNL++iUc NRm4ieBZoYgoa5ftYYBdpJWJlPUqG9LPNVhTOJxab2kJXrjKlV0qBAq6L 67KPTJ/aJu6wkEH9TAafMlXJII2/eeWy6czKo8rk6pZktlIgP144UKrJv A==; X-IronPort-AV: E=McAfee;i="6400,9594,10376"; a="278227176" X-IronPort-AV: E=Sophos;i="5.91,296,1647327600"; d="scan'208";a="278227176" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jun 2022 00:08:04 -0700 X-IronPort-AV: E=Sophos;i="5.91,296,1647327600"; d="scan'208";a="651263450" Received: from xinyangc-mobl.ccr.corp.intel.com ([10.254.214.65]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jun 2022 00:07:59 -0700 Message-ID: Subject: Re: [PATCH v6 09/13] mm/demotion: Add pg_data_t member to track node memory tier details From: Ying Huang To: "Aneesh Kumar K.V" , linux-mm@kvack.org, akpm@linux-foundation.org Cc: Wei Xu , Greg Thelen , Yang Shi , Davidlohr Bueso , Tim C Chen , Brice Goglin , Michal Hocko , Linux Kernel Mailing List , Hesham Almatary , Dave Hansen , Jonathan Cameron , Alistair Popple , Dan Williams , Feng Tang , Jagdish Gediya , Baolin Wang , David Rientjes Date: Mon, 13 Jun 2022 15:07:57 +0800 In-Reply-To: <20220610135229.182859-10-aneesh.kumar@linux.ibm.com> References: <20220610135229.182859-1-aneesh.kumar@linux.ibm.com> <20220610135229.182859-10-aneesh.kumar@linux.ibm.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655104108; a=rsa-sha256; cv=none; b=8celEDVfvPk1UBCPZyAaFykCaj7SNN8Vp0lis6vCEyy2W5mcZWYYJ3WrqLc6g0wUgprwg6 0TSCw7eUF1gXtXMal4zc496pbvpmFOxYLEItuzqMVKBz47DctWf9oaR2xTdqUuPL8dht9F 2O4Syamu8G2RSn6fC0XOegh/96CkZfs= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="SgRq6KA/"; spf=none (imf28.hostedemail.com: domain of ying.huang@intel.com has no SPF policy when checking 192.55.52.115) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655104108; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kHTKPRyvqj5750oduopTRRwxZ/fyFzffKUCouJbXbUc=; b=U9TIoYDp19kq43QCzXk/VIX78wPQ6nKrxqxl27U4p+DYxySUgPUpcwmd6RokRynRH7pzMg e0ZOR5/6AndSpalTFtKbOpeIAsKtAtVG3Ny/u6yk1T3ewXOSl+/mI8QJc3LTTyZNMqkam6 DtPd1IJW6IkGf8ao64r1hTPC0ibzkaE= X-Rspamd-Queue-Id: C6E26C00A3 Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="SgRq6KA/"; spf=none (imf28.hostedemail.com: domain of ying.huang@intel.com has no SPF policy when checking 192.55.52.115) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: c911nsdxtb7aouz5ypxpbut8ju4rpcpo X-HE-Tag: 1655104107-836407 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, 2022-06-10 at 19:22 +0530, Aneesh Kumar K.V wrote: > Also update different helpes to use NODE_DATA()->memtier. Since > node specific memtier can change based on the reassignment of > NUMA node to a different memory tiers, accessing NODE_DATA()->memtier > needs to under an rcu read lock of memory_tier_lock. > > Signed-off-by: Aneesh Kumar K.V > --- >  include/linux/memory-tiers.h | 14 +++++ >  include/linux/mmzone.h | 3 ++ >  mm/memory-tiers.c | 102 ++++++++++++++++++++++++++--------- >  3 files changed, 94 insertions(+), 25 deletions(-) > > diff --git a/include/linux/memory-tiers.h b/include/linux/memory-tiers.h > index 52896f5970b7..53f3e4c7cba8 100644 > --- a/include/linux/memory-tiers.h > +++ b/include/linux/memory-tiers.h > @@ -6,6 +6,9 @@ >   > >  #ifdef CONFIG_TIERED_MEMORY >   > > +#include > +#include > + >  #define MEMORY_TIER_HBM_GPU 0 >  #define MEMORY_TIER_DRAM 1 >  #define MEMORY_TIER_PMEM 2 > @@ -18,13 +21,24 @@ >  #define MAX_STATIC_MEMORY_TIERS 3 >  #define MAX_MEMORY_TIERS (MAX_STATIC_MEMORY_TIERS + 2) >   > > +struct memory_tier { > + struct list_head list; > + struct device dev; > + nodemask_t nodelist; > + int rank; > +}; > + I suggest to use two data structure, struct memory_tier { struct list_head list; nodemask_t nodelist; int rank; }; struct memory_tier_dev { struct list_head list; struct device dev; struct memory_tier *tier; }; Then we can put struct memory_tier here and still hide struct memory_tier_dev in memory_tiers.c. In this way, we don't need to force all struct memory_tier users to compile the entire driver core headers. And we can separate the user space interface implementation from the other part of the kernel. >  extern bool numa_demotion_enabled; >  int node_create_and_set_memory_tier(int node, int tier); >  int next_demotion_node(int node); >  int node_set_memory_tier(int node, int tier); >  int node_get_memory_tier_id(int node); >  int node_reset_memory_tier(int node, int tier); > +struct memory_tier *node_get_memory_tier(int node); > +void node_put_memory_tier(struct memory_tier *memtier); I don't find caller of these 2 functions in series. Can we remove these functions? Best Regards, Huang, Ying [snip]