From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E4C8CDB484 for ; Wed, 11 Oct 2023 20:44:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5632D8D00D2; Wed, 11 Oct 2023 16:44:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 510DE8D0002; Wed, 11 Oct 2023 16:44:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33DAF8D00D2; Wed, 11 Oct 2023 16:44:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2410E8D0002 for ; Wed, 11 Oct 2023 16:44:07 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id EB0181CAAD1 for ; Wed, 11 Oct 2023 20:44:06 +0000 (UTC) X-FDA: 81334357692.24.013A125 Received: from mail-yw1-f194.google.com (mail-yw1-f194.google.com [209.85.128.194]) by imf08.hostedemail.com (Postfix) with ESMTP id 168F1160021 for ; Wed, 11 Oct 2023 20:44:04 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=i+R++jaP; spf=pass (imf08.hostedemail.com: domain of gourry.memverge@gmail.com designates 209.85.128.194 as permitted sender) smtp.mailfrom=gourry.memverge@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697057045; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/AKsmkTQtU/rhqiAGHEzho0MRurJuhinBZFCGxQETC4=; b=ynHiJ41UBgtEHTZUIQptpk1buQF/Nx7k3ma7kfZe8uqn5hsT9mqYagra9w3PLHiK+6Akij Ku3Cs/7mLarnapmv62gV+YLFJq0hYn2GIMDKd8mMrPqkw/DiIqmtbxurLPScRGOhRDJEJ6 MoreFB184Bl/Fotn0QyopHvO8aDqGWg= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=i+R++jaP; spf=pass (imf08.hostedemail.com: domain of gourry.memverge@gmail.com designates 209.85.128.194 as permitted sender) smtp.mailfrom=gourry.memverge@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697057045; a=rsa-sha256; cv=none; b=zFeypuUqUYXjo/KW5QACq3VEpzgOAQDRBNxYB26G14pyg9WsN7g+X+w7eS25+Mtb7W95ZG mwcSH/YcP7Kpp9OYwir+eQ8JONlVmq8YiJLPhd1DRJ0fa1TkXe/uOL6MoZM4koEGaTqVWr 9FFfQ9j34t1pQrDzqt2khUTzYKhntlI= Received: by mail-yw1-f194.google.com with SMTP id 00721157ae682-5a4c073cc06so12333757b3.1 for ; Wed, 11 Oct 2023 13:44:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697057044; x=1697661844; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/AKsmkTQtU/rhqiAGHEzho0MRurJuhinBZFCGxQETC4=; b=i+R++jaPdbmu4stHg2SLGVIZqDsL+Tq2uuuSGzCq67JC6Yj++ygYgFyiCo9glCHwSN peo5gjpoOv3JIlnneIJixirPvTPzXn+4VXkI7nC8QHysrTq01HgNefVOJEJ5bLpbbK4G qQLrRgDF5azGzRyFn5VvKWEkv3vkUrp5QX2IqAlHuo62yHD5TLUvfghm1F6NrHI/Tp42 ykthZWmIxqZdWtLt9YnhdwCRwIBcBzM6+TrRwCXOxVOphu3Ma5zXbLMB4o49chjTZNxN vLYQEQR5YSskQ6F6MeUy6Dl4M3Lo2NMxHzC8U+20nswMvG1dgWcOAfOaA0txIm6uUjHR ctlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697057044; x=1697661844; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/AKsmkTQtU/rhqiAGHEzho0MRurJuhinBZFCGxQETC4=; b=oEzAtCFuMIOffJKzC/21ETgIN9Qv6vzWP374ar+QnwYzkFojnzqge5J8o1geQcm60x DCQ/TjeSpczbOP4o0zGnUgjPY1gMGbvyHxzcngEyrkdAD0DzYUfTocSYJ6dFc0cEqRRD BFbX8B9IqL0U2BDTvOp6UiWPXLpzRx92kWk9MG9bSxRsYMXstzGA0vQqBXcr1Liyo/OW 9Z1hTCLYQ1JKLCAPycXK5PSBYCbnK1hVFklvUjv7EwP5rW+z2c+y91wZBRRC5DZ8+/mw Nz31Nqs6xmV3uZxQm4psL799dZG8d3+lGHRmE5XWm06ctcD6ehOu5diJzkMCmfDpyN0y UJvw== X-Gm-Message-State: AOJu0YyOohlJcfKQ2bF5yUHbinWFzpNhPYrWNL4UQ6bvXixK9wCyBpiS 4nFr6Yu0Li3KBlA89aPbSJwziGon7IXuvN8= X-Google-Smtp-Source: AGHT+IHk5yLZJeaUXqy1B+rDus9A8q+EqrUxM9q2jVm64kJSM7T3hF2zy2+/xYWfSyGz2fhCfP8TIg== X-Received: by 2002:a81:528b:0:b0:5a5:575:e944 with SMTP id g133-20020a81528b000000b005a50575e944mr11929290ywb.4.1697057044087; Wed, 11 Oct 2023 13:44:04 -0700 (PDT) Received: from fedora.mshome.net (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id q2-20020a819902000000b0059bc0d766f8sm1844588ywg.34.2023.10.11.13.44.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 13:44:03 -0700 (PDT) From: Gregory Price X-Google-Original-From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, linux-cxl@vger.kernel.org, akpm@linux-foundation.org, sthanneeru@micron.com, ying.huang@intel.com, gregory.price@memverge.com Subject: [RFC PATCH v2 1/3] mm/memory-tiers: change mutex to rw semaphore Date: Mon, 9 Oct 2023 16:42:57 -0400 Message-Id: <20231009204259.875232-2-gregory.price@memverge.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20231009204259.875232-1-gregory.price@memverge.com> References: <20231009204259.875232-1-gregory.price@memverge.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 168F1160021 X-Rspam-User: X-Stat-Signature: x4k8h6yyiz885uytt5w84k6tonfpuit5 X-Rspamd-Server: rspam01 X-HE-Tag: 1697057044-592801 X-HE-Meta: U2FsdGVkX19mcLyXR0auR2jiO7KP4Iy2aHaBhjbBoX2j7lCr3RISZs5U0tgh4VuVrT/6oBn37kpe2lF20eYBtZZcm0KP7XgrjSai8DOZuy6dyfeDkmjI3jeGkjoDuxRMqGRFSCsP6MYms+N32gAVvi4tXUXllHTh4USgrXVfaSibOoykm+Ob722nucSbFGyvohAE+vJrg0PtL7Aerz53Kw2f4ncAS/f0rE7ObvqPc+Y+NYkaiFnH6MlA+XW0T47RMAJSl/QoOqUcNkJwE637xHfRHfAAe3rXY8+HHjIJijfjp7Jn8mUAb1hB42kzMWszQbONDmW5HDIlUPqPNYyQf4X/Fe1cfxrXBra8/AP0RSSWovbP837Ahxe0s5nSkFE3ZUHh8Ra8DvefEelyFg128scNYR+rX7rVCegkgMmUTmIgLuhRR/vgr25jB5lehqZPZvxs9Og63Couzb59NoHGA56lQ6yTpG3RYKeO1h4gcevll2Kcx8B6q+qInXUPUCEJ10TPqyyKCVITR2T6JZD/BF2vgArdKe2U3iCWCAPKsCpGmFS6UNSwzWRJcbYyFeg1r1Yr9Sk46In0LyDJosAW/r+ehgxZ3jrNR0D+I1oWCELilL73i4OYz6O6Mm/Hvh+vrjgkp/mpy7sDvWLPkH4ZKpbjNA9wKNzYBiUZfow82/ZH2GiMDy8VBvlSRFKk84hryYSe8FEP6GLa+5zcyUuEKw3l/75NuEFQzOLNIqb8Qtqq3E6fAXitEuNP5tKTbM9gj/1X2M9YwrHJI4HG30RgOMJmi/qzTk3o7SlCWExt4Ru3NVQ9zf0+eDuiL2IHWyfSW1BTCp1rDAUdgDXkTBLhdYwoSk83oxErXHqCzRr90S2OTcdPa+stoRewdL1DKfaImhuxe0rQa/GgB4EZbOMYb67VIECJUOHGL4AmP0KwzYSSZKmsdLNWy14NhCjB1FbSccHH2NgdAHAZ9bNkLVF mO1YiwnS WGZeQUA05upPsY1rUdwdk+FZbT7a/yrXhaxMdxkKmaNXwNzyA6rA1AZ0nbwlFG4xdr7q2noL9XZHflWjLykw0Uos2WNpvDHUpCshZNqIIU0VqesobFT/JLXp5++4dZiPahgmYiA5bWxyt1lrR95shTpCPQhP+ww8dD/cxlvdWo1HSEg9w/aGaTJxA6EDbIMTqtwtqG3cWWYb6B06CPvIEIO4oU9Ei3v/4Yhrx/NmI5thELActBHFyD7issMW22M+bwQWSXETz1BvyKYvOZwQSPss4WtJqwpTv4/3ka9ycm7FQq2X4ZmdOa7I9kQtCb8oWb+XwIjh+zmXcEe/NPpA4/hpFB5UGqwruJD8zyFwmBw7th0UqPYqCsKDwvCHwWEjqGWCvslbk6dVoSwcWyRW5/Ng0L8oc1CgEOAotEwd62VssrsmDQCwYOc57DTR6RewG7KCO4y+19ixfgb01md/JOjGoleA5nIuuqmnmEJdenCBjmKc0lnJyX8rqLNkAvXWUdjw/8TS/+ymqnFToXCPOOGzy1PAl/nEM3qXiQa3E3D8nP3qEp/+DMLQiNw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Tiers will have externally readable information, such as weights, which may change at runtime. This information is expected to be used by task threads during memory allocation so it cannot be protected by hard mutual exclusion. To support this, change the tiering mutex to a rw semaphore. Signed-off-by: Gregory Price --- mm/memory-tiers.c | 39 ++++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index 37a4f59d9585..0a3241a2cadc 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "internal.h" @@ -33,7 +34,7 @@ struct node_memory_type_map { int map_count; }; -static DEFINE_MUTEX(memory_tier_lock); +static DECLARE_RWSEM(memory_tier_sem); static LIST_HEAD(memory_tiers); static struct node_memory_type_map node_memory_types[MAX_NUMNODES]; static struct memory_dev_type *default_dram_type; @@ -137,10 +138,10 @@ static ssize_t nodelist_show(struct device *dev, int ret; nodemask_t nmask; - mutex_lock(&memory_tier_lock); + down_read(&memory_tier_sem); nmask = get_memtier_nodemask(to_memory_tier(dev)); ret = sysfs_emit(buf, "%*pbl\n", nodemask_pr_args(&nmask)); - mutex_unlock(&memory_tier_lock); + up_read(&memory_tier_sem); return ret; } static DEVICE_ATTR_RO(nodelist); @@ -167,7 +168,7 @@ static struct memory_tier *find_create_memory_tier(struct memory_dev_type *memty int adistance = memtype->adistance; unsigned int memtier_adistance_chunk_size = MEMTIER_CHUNK_SIZE; - lockdep_assert_held_once(&memory_tier_lock); + lockdep_assert_held_write(&memory_tier_sem); adistance = round_down(adistance, memtier_adistance_chunk_size); /* @@ -230,12 +231,12 @@ static struct memory_tier *__node_get_memory_tier(int node) if (!pgdat) return NULL; /* - * Since we hold memory_tier_lock, we can avoid + * Since we hold memory_tier_sem, we can avoid * RCU read locks when accessing the details. No * parallel updates are possible here. */ return rcu_dereference_check(pgdat->memtier, - lockdep_is_held(&memory_tier_lock)); + lockdep_is_held(&memory_tier_sem)); } #ifdef CONFIG_MIGRATION @@ -335,7 +336,7 @@ static void disable_all_demotion_targets(void) for_each_node_state(node, N_MEMORY) { node_demotion[node].preferred = NODE_MASK_NONE; /* - * We are holding memory_tier_lock, it is safe + * We are holding memory_tier_sem, it is safe * to access pgda->memtier. */ memtier = __node_get_memory_tier(node); @@ -364,7 +365,7 @@ static void establish_demotion_targets(void) int distance, best_distance; nodemask_t tier_nodes, lower_tier; - lockdep_assert_held_once(&memory_tier_lock); + lockdep_assert_held_write(&memory_tier_sem); if (!node_demotion) return; @@ -479,7 +480,7 @@ static struct memory_tier *set_node_memory_tier(int node) pg_data_t *pgdat = NODE_DATA(node); - lockdep_assert_held_once(&memory_tier_lock); + lockdep_assert_held_write(&memory_tier_sem); if (!node_state(node, N_MEMORY)) return ERR_PTR(-EINVAL); @@ -569,15 +570,15 @@ EXPORT_SYMBOL_GPL(put_memory_type); void init_node_memory_type(int node, struct memory_dev_type *memtype) { - mutex_lock(&memory_tier_lock); + down_write(&memory_tier_sem); __init_node_memory_type(node, memtype); - mutex_unlock(&memory_tier_lock); + up_write(&memory_tier_sem); } EXPORT_SYMBOL_GPL(init_node_memory_type); void clear_node_memory_type(int node, struct memory_dev_type *memtype) { - mutex_lock(&memory_tier_lock); + down_write(&memory_tier_sem); if (node_memory_types[node].memtype == memtype) node_memory_types[node].map_count--; /* @@ -588,7 +589,7 @@ void clear_node_memory_type(int node, struct memory_dev_type *memtype) node_memory_types[node].memtype = NULL; put_memory_type(memtype); } - mutex_unlock(&memory_tier_lock); + up_write(&memory_tier_sem); } EXPORT_SYMBOL_GPL(clear_node_memory_type); @@ -607,17 +608,17 @@ static int __meminit memtier_hotplug_callback(struct notifier_block *self, switch (action) { case MEM_OFFLINE: - mutex_lock(&memory_tier_lock); + down_write(&memory_tier_sem); if (clear_node_memory_tier(arg->status_change_nid)) establish_demotion_targets(); - mutex_unlock(&memory_tier_lock); + up_write(&memory_tier_sem); break; case MEM_ONLINE: - mutex_lock(&memory_tier_lock); + down_write(&memory_tier_sem); memtier = set_node_memory_tier(arg->status_change_nid); if (!IS_ERR(memtier)) establish_demotion_targets(); - mutex_unlock(&memory_tier_lock); + up_write(&memory_tier_sem); break; } @@ -638,7 +639,7 @@ static int __init memory_tier_init(void) GFP_KERNEL); WARN_ON(!node_demotion); #endif - mutex_lock(&memory_tier_lock); + down_write(&memory_tier_sem); /* * For now we can have 4 faster memory tiers with smaller adistance * than default DRAM tier. @@ -661,7 +662,7 @@ static int __init memory_tier_init(void) break; } establish_demotion_targets(); - mutex_unlock(&memory_tier_lock); + up_write(&memory_tier_sem); hotplug_memory_notifier(memtier_hotplug_callback, MEMTIER_HOTPLUG_PRI); return 0; -- 2.39.1