From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BB3AE7717F for ; Fri, 13 Dec 2024 20:03:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F5176B0082; Fri, 13 Dec 2024 15:03:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A6A76B0083; Fri, 13 Dec 2024 15:03:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 046FD6B0088; Fri, 13 Dec 2024 15:03:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DDFE76B0082 for ; Fri, 13 Dec 2024 15:03:21 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 60ABC8117D for ; Fri, 13 Dec 2024 20:03:21 +0000 (UTC) X-FDA: 82891009908.28.8A8B25D Received: from mail-oa1-f53.google.com (mail-oa1-f53.google.com [209.85.160.53]) by imf23.hostedemail.com (Postfix) with ESMTP id 40A08140020 for ; Fri, 13 Dec 2024 20:03:02 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=DepGqAOX; spf=pass (imf23.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.160.53 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734120187; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TSxjdSUDfHmzmkE1Ayo7U0AvrT2yhA3Ltr1QPBiYLK0=; b=W2bq8XbK5CbnxaESPz32YH55pH4kUj+JSCSGXAUSdNXh2uF19ZkKQs7GAyY/BtPfA4Iahr 0ccMP+r07pPXqRDp6X+kKielp/rTgGyRmi6Uvvl2QGvLREs39Ujv+5TWyMbxUwpCSPzAyb YEKUAFqKenjrMl1fEzEoYAzFnbAPZ98= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=DepGqAOX; spf=pass (imf23.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.160.53 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734120187; a=rsa-sha256; cv=none; b=U0fLJfUy8bfNqr452xmzg90p8sx0amrHHbksw28fV/UiG+zxzokXhdfLQcsI3svkqyk70/ frkT9NKUu3q7rddztDdkUnfcvAOvwYF5USBCWVLZGDAmxlbhyDnMEqLI2J28Iy7aKIRPIr fDDBhxbQIfK9hskC5RoNTjCoMpeJvTQ= Received: by mail-oa1-f53.google.com with SMTP id 586e51a60fabf-29737adb604so803115fac.1 for ; Fri, 13 Dec 2024 12:03:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734120198; x=1734724998; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TSxjdSUDfHmzmkE1Ayo7U0AvrT2yhA3Ltr1QPBiYLK0=; b=DepGqAOX1u1fRYjmO9uKmg6ceOI9uQm/isLnJmDBkKLy0yCE5WA+ksNsqBSoHijWIX oK3qeS8VIWxr1e5JNb3+olhGq13J4NmudoJ1jvjptdNHsfAn7YJkAFPzZUnBSxAJ2ix0 AZRCHZghwOsOGwCaJLxbFjIKyfebkZptoVZ1m65DkERSNk93CaKeBcyxNnSXjykoCiuG z1LMU0teL2RZpqqJt/o9KA2H8txlJ1vdPDL8axFOUhsvJaPfE/w4VFovpeUNIpf6Tl// Wr0VPbmF70eKcFUJtbIukshbkLBksiMMdsGK9sSRqjiwq7AeD4nGzAlHwYUCJUJnzS8d QnwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734120198; x=1734724998; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TSxjdSUDfHmzmkE1Ayo7U0AvrT2yhA3Ltr1QPBiYLK0=; b=Efu2L4sl9Ksrb3g3kMxFExKKP/978F86cdUMUkCAnvnHJLKpOkU6Mb/in0+9Bf6VMk +HQs7Zckzrudxfn6XU0e0U53P4CEWPLinEAI8UjSPgodNM3Y/LK4INJKlIV4DbKel4PU z2qy6Qjfsp8LCzWC75t82P+yX+pBGQHAx0ZlyJOHJpQGH1Ce0A084dmiOrcTwe9crx/H YDW5LDhcaZvisawbGFWfJvnYE2mS2ahjXzMR2o+aTpPy2mBk5n3G0x9GAzTHB22IwaZR OesoWRMG7W6ifDzZMrB/JQ9Mt/JDUGzqK6duz/xii3vNJ1Txg/JYMfLQxSLzuI+z+SMH YP1A== X-Forwarded-Encrypted: i=1; AJvYcCWd6sCNio0d6+XOvaF5QVPRxaFYqd8JGJdOioGoQOXSqxqjeV1a9UGdlyg7X13kB/+B8bBUFmR85w==@kvack.org X-Gm-Message-State: AOJu0YzW7vbDGJ6B5XZzBoaanr4ESloRb4FKQ+GAfRrC2Q/g5XmSa6+F KpI/CzsEjVurZ2SumQYNdCJSFtH/4f6+Q65zGW5FMJ5Daqyg2y0870Mcv2S9 X-Gm-Gg: ASbGncteIW5yqsZFHprF/l8bvC2uwNlUG+75GUMzJGoxvnfGt8ALZbALaQkGngM3jMh dkj4AOkeYaIm9FQFPZiaTzco6fDiYqX2HBJBlw8TBjV2HI+hhbPtobRsGiAo5xgVf3IK53eQheN VgUXHeYjUpxseWpz55PNvRrhXg/ZNkyDJ03CjaoLqFSwCVtMBqFoeX8nJBL22GWQhMkDadp2vdC Imp/mv052fx0BCJ3nMZ8Tb8RzP8GiR81XSTq/NEJi3ATcfb5RtAPoPMgBkiPfNwYQ0nCFeMQi30 9EfqBPLbibZUyygI X-Google-Smtp-Source: AGHT+IHS/4vG0XdSp0prcXAlpEvsdSuF07y6AmAofYdTggMqlAhkijuk0JcmF7UBkvKs9T+bdO/kMg== X-Received: by 2002:a05:6902:a87:b0:e38:1b17:8980 with SMTP id 3f1490d57ef6-e43492f57famr3483416276.4.1734119878971; Fri, 13 Dec 2024 11:57:58 -0800 (PST) Received: from localhost (fwdproxy-nha-003.fbsv.net. [2a03:2880:25ff:3::face:b00c]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e47024dc3f2sm57344276.5.2024.12.13.11.57.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 11:57:58 -0800 (PST) From: Joshua Hahn To: hyeonggon.yoo@sk.com Cc: "gourry@gourry.net" , kernel_team@skhynix.com, "rafael@kernel.org" , "lenb@kernel.org" , "gregkh@linuxfoundation.org" , "akpm@linux-foundation.org" , =?UTF-8?B?6rmA7ZmN6recKEtJTSBIT05HR1lVKSBTeXN0ZW0gU1c=?= , "ying.huang@linux.alibaba.com" , =?UTF-8?B?6rmA65296riwKEtJTSBSQUtJRSkgU3lzdGVtIFNX?= , "dan.j.williams@intel.com" , "Jonathan.Cameron@huawei.com" , "dave.jiang@intel.com" , "horen.chuang@linux.dev" , "hannes@cmpxchg.org" , "linux-kernel@vger.kernel.org" , "linux-acpi@vger.kernel.org" , "linux-mm@kvack.org" , "kernel-team@meta.com" Subject: Re: [External Mail] [RFC PATCH] mm/mempolicy: Weighted interleave auto-tuning Date: Fri, 13 Dec 2024 11:57:31 -0800 Message-ID: <20241213195754.2676135-1-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <4ddfa283-eb64-4032-880b-c19b07e407e1@sk.com> References: <4ddfa283-eb64-4032-880b-c19b07e407e1@sk.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 40A08140020 X-Stat-Signature: rkmj3pdc5o55gcfq4ib8bwy1muqbmyec X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1734120182-281768 X-HE-Meta: U2FsdGVkX190axvHynJ8wekvm7Duzziq/lPvIhx5QyylY/Z1QBZPk6Q/wSrHcqXgswpiWGrWkAHu73ATZYV/6qWQ8CKpqfU5Sam0naz16+RKXidH9ASmT6Q4LHEyEA46qIZryvmwuXKDbTlvLb0GZDCFTMCmQroHBvioFx4daaUHEIo9ucb52z46RuezN6hh6G2juea+Rrb8dSvJSt7mTEtrfbga+G0bDo6TJ4pgF14WrrdHvujMXSi9k5JJ2eEuftHo0jc0RKNJ18r0oLHrbxis7b0pqDoxlQ0Ycj9/ixGKGKJCHIQgK74JacfHD9VhepcywSG051xQMDbxYgg8i+aiFGq014wTlE9rfRIpwBYns6c/HL+asQjkyXZYPwpsEiAKDCHV545MY8lsGNl8koNhKwYl1w+K76i1uHTcnL9K0ioy9GS5Jiy4xo2L0wsNYHCWr/BWtj/2UAbcvZ8Mb9qzdHfM2Mghvr/XYzJ8X6afEQsxZEsnm5hQdH/dJM0bEdh2daMbJSDiISOOeUw6pSx8j8kHkg7yne5f74MhyqdeBfn2GBl7kei9Qn3T3A1axTeTgviaN4mp8g1qJj+drOEr1FiS6F5AQUHVSNIU87R87qS/79FZQh1JV9z6qJvplu4Oit8chGSDPYgu3y/PqrbnESmAe6jRByhHC2rCcFgDyhfmPskVoNI+gMIqxDcLrMIgxxg7hnFqnnt1om3fN60Xp5oir71KhDejLqjIpZHRnREExtW4crMIuZbN6T5TW/9QOdFx1jLRds4KHJMZo0F3JBxsBSIhw+cBYfCICaXdPd+ga5x6NxuZfz3y/wrWN4q84pdCuSEiLPXlpjrV/OYy/42csHavcgOMpfMB7wradUW5PQQz7acDTIjth9wZ/Kh0zCE1h3jZH3n6ljxxx+Pj4KAZKXbwH7dkXdV9m6sKrLI9AVpsIVb3Qq36CnXf5v6sKIo7SpK7xQOp7rL YvhySy+T I+Yu2EvLbblTdXjxjuxASqbHeKr1m9sHPv9F1sfzDGWdFQgrN0nrdvsPhIlpa2SRErbNC05o8dnI254/d1Ti69ppScvfuo39t2VzgP4kKig1ZkLwHEVusBgo8zBm2p26dm+CjX8Ftwbjbj4DbXz9/RuJFcyNUJQgGDZgHhAQWe3isYsQJHqT27SEoiFofu7eS0WpuzZOPb+8U56xyChM0NSN1KhHMhpjSee2duz5GWRl7hQJNcb/hjghP0uUXfW4eN8brEQ0UoiKw5qn4WTo9k1kM5uuFl11HidTapV9bqnrLYQE9TZOZuplInNWc+6Z+FP0DPzlAuwyXF7z4gWkEpunEBHqb8vgxjuXbnxXIb3SFzVoRnAXL6D7fXMJf2DshqLVbNvO7+bOVFtT9S57n8zyD5Pu1je8DmXKg8WZoMQI9/Qomav4hwexEqMO7FeeILsApC/qq3ykBLsGlq5DSRz7OIiWjRl2Jrg3EHJPFsptFt33dK42h97l4cOLD1b09ncWnDHSXSF0A6qKCaQnM1Ri4aw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.050941, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, 13 Dec 2024 15:19:20 +0900 Hyeonggon Yoo wrote: > On 2024-12-11 06:54 AM, Joshua Hahn wrote: > > This patch introduces an auto-configuration for the interleave weights > > that aims to balance the two goals of setting node weights to be > > proportional to their bandwidths and keeping the weight values low. > > This balance is controlled by a value max_node_weight, which defines the > > maximum weight a single node can take. > > Hi Joshua, > > I am wondering how this is going to work for host memory + CXL memory > interleaving. I guess by "the ACPI table" you mean the ACPI HMAT or CXL > CDAT, both of which does not provide the bandwidth of host memory. > If this feature does not consider the bandwidth of host memory, manual > configuration will be inevitable anyway. Hi Hyeonggon, Thank you for reviewing my patch! As Gregory showed in his reply, I think it would be possible to get host bandwidth information using the ACPI HMAT. [-----8<-----] > > +What: /sys/kernel/mm/mempolicy/weighted_interleave/max_node_weight > > +Date: December 2024 > > +Contact: Linux memory management mailing list > > +Description: Weight limiting / scaling interface > > + > > + The maximum interleave weight for a memory node. When it is > > + updated, any previous changes to interleave weights (i.e. via > > + the nodeN sysfs interfaces) are ignored, and new weights are > > + calculated using ACPI-reported bandwidths and scaled. > > + > > At first this paragraph sounded like "previously stored weights are > discarded after setting max_node_weight", but I think you mean > "User can override the default values, but defaults values are > calculated regardless of the values set by the user". Right? In the implementation, the first way you interpreted is the correct description. That is, if a user manually changes a ndoe weight, then updates the max_node_weight, the previous manual change will be overwritten by the newly scaled values. Does this behavior make sense? Perhaps it makes sense to ignore user-changed values when doing the re-scaling if a user decides to change the max_node_weight value themselves. Regardless of what implementation makes sense, I can re-write the description so that there is no ambiguity when it comes to the expected behavior of the code. Thank you for pointing this out! > [...snip...] > > > +int mempolicy_set_node_perf(unsigned int node, struct access_coordinate *coords) > > +{ > > + unsigned long *old_bw, *new_bw; > > + unsigned long bw_val; > > + u8 *old_iw, *new_iw; > > + > > + /* > > + * Bandwidths above this limit causes rounding errors when reducing > > + * weights. This value is ~16 exabytes, which is unreasonable anyways. > > + */ > > + bw_val = min(coords->read_bandwidth, coords->write_bandwidth); > > + if (bw_val > (U64_MAX / 10)) > > + return -EINVAL; > > + > > + new_bw = kcalloc(nr_node_ids, sizeof(unsigned long), GFP_KERNEL); > > + if (!new_bw) > > + return -ENOMEM; > > + > > + new_iw = kzalloc(nr_node_ids, GFP_KERNEL); > > I think kcalloc(nr_node_ids, sizeof(u8), GFP_KERNEL); will be more readable. I see, thank you for your input. I will make this change in a v2. > > @@ -2012,11 +2105,12 @@ static unsigned int weighted_interleave_nid(struct mempolicy *pol, pgoff_t ilx) > > > > rcu_read_lock(); > > table = rcu_dereference(iw_table); > > + defaults = rcu_dereference(iw_table); > > Probably you intended rcu_dereference(default_iw_table)? Yes -- thank you for the catch. I will also make this change. > > static struct iw_node_attr **node_attrs; > > +static struct kobj_attribute *max_nw_attr; > > Where is max_nw_attr initialized? Oh thank you for this catch! You are correct, max_nw_attr is never initalized. Actually, there is a typo in which I never use max_nw_attr, I accidentally rename a different sysfs interface to act as the intended max_nw_attr. I will make this change as well and post a v2. > Best, > Hyeonggon Thank you for your input, I will make the changes that you mentioned regardnig readability & typos. I hope to hear from you regarding the thoughts on the behavior of re-scaling all node weights when users update max_node_weight, and whether that should overwrite manually set node weights. Have a great day! Joshua