From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF19EC4167B for ; Tue, 31 Oct 2023 00:38:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8F0736B00D0; Mon, 30 Oct 2023 20:38:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 828F76B00D2; Mon, 30 Oct 2023 20:38:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 67D6E6B00D5; Mon, 30 Oct 2023 20:38:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 4F5266B00D0 for ; Mon, 30 Oct 2023 20:38:30 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2A0A5160735 for ; Tue, 31 Oct 2023 00:38:30 +0000 (UTC) X-FDA: 81403895580.21.EA62A63 Received: from mail-yb1-f195.google.com (mail-yb1-f195.google.com [209.85.219.195]) by imf21.hostedemail.com (Postfix) with ESMTP id 591F31C0009 for ; Tue, 31 Oct 2023 00:38:28 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=je3ZuCC6; spf=pass (imf21.hostedemail.com: domain of gourry.memverge@gmail.com designates 209.85.219.195 as permitted sender) smtp.mailfrom=gourry.memverge@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698712708; a=rsa-sha256; cv=none; b=iD+okkqifVzV8JLnsiK/5O5rcPpZcumbrFaG8LNywtvTEt0Vmm7r2VHq4fR96xb8qMlzFi oYCHxGmFQPzxzl8DQEeFeBbz+n4R2gFoRWmTulCOIhib58CZIBTl+nTaPbJfoobtwEzNKN uL76iIMYpJQZeFN8TBiYoWO3ZWWqwYM= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=je3ZuCC6; spf=pass (imf21.hostedemail.com: domain of gourry.memverge@gmail.com designates 209.85.219.195 as permitted sender) smtp.mailfrom=gourry.memverge@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698712708; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wqzp8UEwRJTgLaeQpU+LsOI00hLnye5kfN/uXwQ/cgQ=; b=ea2UWDfyHmk+PWpGSmH+uLDbsQ2HpWBGzN1Ec8VNzJKIMFHpw9cXUMYii4tB6N5pwUThUl 3eDic1UFtqFY9UPu+v9801ZKvrFL6+RaoSN9ncC1vP53MKUesmfLh5kCSRGKKPSc+tblCb mz8oUGMPUS+vUGlnGKKFpINinguhVlU= Received: by mail-yb1-f195.google.com with SMTP id 3f1490d57ef6-da37522a363so299220276.0 for ; Mon, 30 Oct 2023 17:38:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698712707; x=1699317507; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wqzp8UEwRJTgLaeQpU+LsOI00hLnye5kfN/uXwQ/cgQ=; b=je3ZuCC6yD97HITkOouZ/dbz1HBuSF3hs08yjFs7hjrR5LOLlBiVlxQvZoUaP+icBM xATTdv1fNCenCHnmkOOTcPigv/zUDna9iYIOTOrgEeALaI6UYPQCgJJN994pjvL6xhq+ Fx/kIKmFOBd4BjUorPnJGbI2/XI1SQ2IJZTzVKANqbCJZjJz9bgi4c4+29hcVWjY1HzD jGGsJITz1JrRQ+safX1EtoVEYjieODjkTl8ZWHfO3ohr/qQK204n1PxiW429s5U/2ocW rW4BYHG8nVdxUZ8sFnPPzz7ZmJq1evJr0M5F6oSLbcZUF/1oKEelGoUbkp/uOAIFLxce q0sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698712707; x=1699317507; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wqzp8UEwRJTgLaeQpU+LsOI00hLnye5kfN/uXwQ/cgQ=; b=lbWYJ56cFFnmPNdfRS46cyAc6VEBB1e/hYM6W03KqenHiuoPs5SGpry2jmylzXnJHb iDDvgHIczFpM8ypdWbS+G7+QS595KmeZlzRqHeBs5psqu0jvjnpvb2LG/bIaaVtOl+WP 0RdjVM7uVIhMlveAoz/I6yNDfn+PQAlyin0MvE1ihOx3R5so8wLZOe6Y5i7Qj4ddPAuj dXPt5geu63hg44I+Qnal4WAj+CtYFxbaBONDfKDVJq36EVQM75kDgX8AVqfkhdi12/jk 5oBVlK6QbCyPQmizE5UMLQN18aFVgzwro5e4C1NLJ8I7bVisnI0lU8GOBZ/bnjoId7Mp EaaA== X-Gm-Message-State: AOJu0YxaI4mtRj7VFFYC/1a4InrSCHTUzAh0TxclNjalgp5f0sxnjtqG DqnLMGAYAM1qxEEphtqf6nstxTE2Pwed X-Google-Smtp-Source: AGHT+IFKcy1sU3dFigUiCwURyOPeE7JfdrOwJH3YBwiTqPhB7WcZjjcQyS/NuIc9Ms5Q8SWg+UlvIA== X-Received: by 2002:a25:d1c5:0:b0:d9a:401d:f5da with SMTP id i188-20020a25d1c5000000b00d9a401df5damr11555099ybg.51.1698712707500; Mon, 30 Oct 2023 17:38:27 -0700 (PDT) Received: from fedora.mshome.net (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id b19-20020a25ae93000000b00da086d6921fsm182750ybj.50.2023.10.30.17.38.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Oct 2023 17:38:27 -0700 (PDT) From: Gregory Price X-Google-Original-From: Gregory Price To: linux-kernel@vger.kernel.org Cc: linux-cxl@vger.kernel.org, linux-mm@kvack.org, ying.huang@intel.com, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, weixugc@google.com, apopple@nvidia.com, hannes@cmpxchg.org, tim.c.chen@intel.com, dave.hansen@intel.com, mhocko@kernel.org, shy828301@gmail.com, gregkh@linuxfoundation.org, rafael@kernel.org, Gregory Price , Ravi Shankar Subject: [RFC PATCH v3 3/4] node: add interleave weights to node accessor Date: Mon, 30 Oct 2023 20:38:09 -0400 Message-Id: <20231031003810.4532-4-gregory.price@memverge.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20231031003810.4532-1-gregory.price@memverge.com> References: <20231031003810.4532-1-gregory.price@memverge.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 591F31C0009 X-Stat-Signature: ip8etrx6z7sxj4ibah949stodwruhrdw X-Rspam-User: X-HE-Tag: 1698712708-143315 X-HE-Meta: U2FsdGVkX1++5BJyPqJb6grbiapIImk3zqZNbsmG6pG5wbPiX44uXdbT/xWynTNV3d54huGCQCuxXZ8p43PMND0ENjQDQNHsRG8mRLhkut4uDoj/WQuu0e0XnFhFGmvBWxC7LO1lYK8NcSAo/vfDJzEeqyNG9h62gONigjum2hPbXb2BrxJzHMocr+1dR0hA2UlY+Pq4SbLZVZbNYmFvfJI//0RKmjQ/tcquxY/mudhUczsXuDyzHMKYaFg3DWOQ0Tvg5klX5defRChsgX2t5VzJBPYkG3ASzU0xI11bqdL9cU3SLL+Sz3f/L0dGsVcaUphyr1gnZ4Trzj8m9pY12yp3h2vWsyOWq1IATgltR6WRtQXVVb61C+miiC5FyyO39P6vpeoJu37kGW5BW0SH2+g7LHG3kgTdtkgh4uMvxZ5U6Yn1+XxtMeYAFBephjiVsSfcVsw2Gir+T9XCqb8r30REDwPB5MBbF/84TaZh4LYY3fChpLfW7iTLnxhGK0FPMkQ+qDo/tcBfbwW9Vyt4W5Fv1XCEMz5NMUxDskPY+MI5aEEikcaeQ85v4P2CERnyRPqQ2CYgXXbrENS1chDCUBbHU664GhCBs0H6r3iypS1grDRXhQgclECCgbb3URlgsGnIVCDwnuoc5v80rQ0rHoOf6RKZ6tHAeEmmLKliBRjDlk8Rrv0NjbbV6EW9cpNMbT6+jse6YqWUy5nLlwyqes1Q4PbhOR81wM2CrbKVo+rTEUoIKGR3qXHGfvU53ltAuQVoPCWZTe8kDWmA4fDIgff8FDjzA/Kp2t3AO35BEhXLFpAbUigl0ZN3a/5TTjooR5466u5HEqs8FGl3zmPwfmiZVd6wFZ5qxejZWL3p7jcJSLWl7LvF7EmCMRN/0acviOHqYv6G0I5jYojhgpMzqt25L6AeZsysgwWytZIBun/GkRoD7DZ1th6+GlIttOIvaboPnF7K51NpPzOyJLC ctqGlYRg 3bsR8eACVkGdylrigUOuK8t/gUhoOtJzqAigceN4NEqxRKSt9bnAegXMvYn6X1LKDrrpvjGLu8JfvzDD5U6+IiWx33/9Yd91luQyglkC6iOP37m6pEmLGaAynYte8X1NEay7j8MKKV/Gu0149rOV45RIdmdkzO9uNcT6qSI5MIx/KSoMGWbrQoILrvgqzVQtVYi7DF/gUCNeH7X54ZlAD4vo/yWkpULnshE0S4TEbhpelYSul+ZJplWwTwjh+NeG7gm0wbs1VK8IIe+j8wv8bWoYU8+npbiNwrHLvNIOesdbzQcNz9l/fzyxOUwNmtS8OHWMjPo26aXXWdXHHRc6gxHAXntMm+NCv811jqOw9Za8Tti6diAN9NKkI/SkaBuRIkRsffNBhuBsCoXisJWAFveu7dLLIe12dC3jtD4cvgiMO2rukn62s1EgUPZDcQuvZ2xXy+4/vRhlTIo+H4sOvgBmUX4xK/jqdpw2odx5Yl5tseJoBPzggMB0GzJeWjT60D+HZq1d8rr0LBfphk7B96jB6C01L6YhltUQniwUQPj8W8x+8WczPnxzZ/IiCJRKS/d8xCjm9wYw8UlJ3L3ZrVmnyQ0/75sC+hByYupktthWO8Pg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a configurable interleave weight to the node for each possible accessor. The intent of this weight is to enable set_mempolicy() to to distribute memory across nodes based on the accessor and the effective bandwidth available. The goal is to maximize the effective use of available bandwidth. The default weight is 1 for all nodes, which will mimic the current interleave (basic round-robin). Signed-off-by: Gregory Price Suggested-by: Ying Huang Suggested-by: Ravi Shankar --- drivers/base/node.c | 95 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/node.h | 17 ++++++++ 2 files changed, 112 insertions(+) diff --git a/drivers/base/node.c b/drivers/base/node.c index b09c9c8e6830..29bb3874a885 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -83,9 +83,84 @@ struct node_access_nodes { #ifdef CONFIG_HMEM_REPORTING struct node_hmem_attrs hmem_attrs; #endif + unsigned char il_weight; }; #define to_access_nodes(dev) container_of(dev, struct node_access_nodes, dev) +#define MAX_NODE_INTERLEAVE_WEIGHT 100 +static ssize_t il_weight_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + return sysfs_emit(buf, "%u\n", + to_access_nodes(dev)->il_weight); +} + +static ssize_t il_weight_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + unsigned char weight; + int ret; + + ret = kstrtou8(buf, 0, &weight); + if (ret) + return ret; + + if (!weight || weight > MAX_NODE_INTERLEAVE_WEIGHT) + return -EINVAL; + + to_access_nodes(dev)->il_weight = weight; + return len; +} +DEVICE_ATTR_RW(il_weight); + +unsigned char node_get_il_weight(unsigned int nid, unsigned int access_nid) +{ + struct node *node; + struct node_access_nodes *c; + unsigned char weight = 1; + + node = node_devices[nid]; + if (!node) + return weight; + + list_for_each_entry(c, &node->access_list, list_node) { + if (c->access != access_nid) + continue; + weight = c->il_weight; + break; + } + return weight; +} + +unsigned int nodes_get_il_weights(unsigned int access_nid, nodemask_t *nodes, + unsigned char *weights) +{ + unsigned int nid; + struct node *node; + struct node_access_nodes *c; + unsigned int ttl_weight = 0; + unsigned char weight = 1; + + for_each_node_mask(nid, *nodes) { + weight = 1; + node = node_devices[nid]; + if (!node) + goto next_node; + list_for_each_entry(c, &node->access_list, list_node) { + if (c->access != access_nid) + continue; + weight = c->il_weight; + break; + } +next_node: + weights[nid] = weight; + ttl_weight += weight; + } + return ttl_weight; +} + static struct attribute *node_init_access_node_attrs[] = { NULL, }; @@ -116,6 +191,7 @@ static void node_remove_accesses(struct node *node) list_for_each_entry_safe(c, cnext, &node->access_list, list_node) { list_del(&c->list_node); + device_remove_file(&c->dev, &dev_attr_il_weight); device_unregister(&c->dev); } } @@ -140,6 +216,7 @@ static struct node_access_nodes *node_init_node_access(struct node *node, return NULL; access_node->access = access; + access_node->il_weight = 1; dev = &access_node->dev; dev->parent = &node->dev; dev->release = node_access_release; @@ -150,6 +227,9 @@ static struct node_access_nodes *node_init_node_access(struct node *node, if (device_register(dev)) goto free_name; + if (device_create_file(dev, &dev_attr_il_weight)) + dev_warn(dev, "failed to add il_weight attribute\n"); + pm_runtime_no_callbacks(dev); list_add_tail(&access_node->list_node, &node->access_list); return access_node; @@ -363,6 +443,21 @@ static void node_init_caches(unsigned int nid) #else static void node_init_caches(unsigned int nid) { } static void node_remove_caches(struct node *node) { } + +unsigned char node_get_il_weight(unsigned int nid, unsigned int access_nid) +{ + return 1; +} + +unsigned int nodes_get_il_weights(unsigned int access_nid, nodemask_t *nodes, + unsigned char *weights) +{ + unsigned int nid; + + for_each_node_mask(nid, *nodes) + weights[nid] = 1; + return nodes_weight(nodes); +} #endif #define K(x) ((x) << (PAGE_SHIFT - 10)) diff --git a/include/linux/node.h b/include/linux/node.h index 427a5975cf40..3c7a6dd2d954 100644 --- a/include/linux/node.h +++ b/include/linux/node.h @@ -138,6 +138,12 @@ extern void unregister_memory_block_under_nodes(struct memory_block *mem_blk); extern int register_memory_node_under_compute_node(unsigned int mem_nid, unsigned int cpu_nid, unsigned access); + +extern unsigned char node_get_il_weight(unsigned int nid, + unsigned int access_nid); +extern unsigned int nodes_get_il_weights(unsigned int access_nid, + nodemask_t *nodes, + unsigned char *weights); #else static inline void node_dev_init(void) { @@ -165,6 +171,17 @@ static inline int unregister_cpu_under_node(unsigned int cpu, unsigned int nid) static inline void unregister_memory_block_under_nodes(struct memory_block *mem_blk) { } +static inline unsigned char node_get_il_weight(unsigned int nid, + unsigned int access_nid) +{ + return 0; +} +static inline unsigned int nodes_get_il_weights(unsigned int access_nid, + nodemask_t *nodes, + unsigned char *weights) +{ + return 0; +} #endif #define to_node(device) container_of(device, struct node, dev) -- 2.39.1