From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F8BBC0218B for ; Tue, 21 Jan 2025 21:24:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 33767280002; Tue, 21 Jan 2025 16:24:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C012280001; Tue, 21 Jan 2025 16:24:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 139D8280002; Tue, 21 Jan 2025 16:24:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E5719280001 for ; Tue, 21 Jan 2025 16:24:36 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 765B4160FB9 for ; Tue, 21 Jan 2025 21:24:36 +0000 (UTC) X-FDA: 83032738152.27.AD3EEB6 Received: from mail-yb1-f178.google.com (mail-yb1-f178.google.com [209.85.219.178]) by imf24.hostedemail.com (Postfix) with ESMTP id A06AD180005 for ; Tue, 21 Jan 2025 21:24:34 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Uk12KBCe; spf=pass (imf24.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.219.178 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737494674; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=D875B0kpOelYr3S2d/i+EM+Vu91KG28kMKlzUneXEt4=; b=GZdeI2WWYhP8lZlumB9b1MQRfXwx0L633V8iASpbZOsdP4+cEhcYHSEffmQ2zIXVdLVTem HvqKhRbxkhY8IzJK43gWL+vg4p7BUY8QvDlodn6M+r+MVdTTFmWS/5cp/THKmdiWDDa6tU djXJ0wev2UpuvM2y0FLJRD7EoozDq+8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737494674; a=rsa-sha256; cv=none; b=HWoppNLSSbyFUn2yQI8LIN7RTcTtghgPNZL6X4jOJ/Kr34vXR5SDOYQfTMUKZK+cZbXGm2 M65YhIl6hPi548E254sWdPu45uDOmk5hmaAHW6gJN6vL6oM5VxGfi4vB5R49s3nkSgUfBU VUMb/0BzK1hmg9+hunizZaXq6Eg2hVg= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Uk12KBCe; spf=pass (imf24.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.219.178 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-yb1-f178.google.com with SMTP id 3f1490d57ef6-e580d6211c8so1370333276.1 for ; Tue, 21 Jan 2025 13:24:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1737494673; x=1738099473; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D875B0kpOelYr3S2d/i+EM+Vu91KG28kMKlzUneXEt4=; b=Uk12KBCeHyJ5VvFMkfYOpdTn1nPfZ9DPDbLdhGIDjgqG1/yBtaZtI6DVxanWEK3VyO 8+IqixSzr3MTdlsUU3p63w7Xlg71HiJa8bK8N4L8RZhU0GQUc/ZiwhG270GzM4teMG5Z KEgoYXrvtGhz0eG8hm/yda/v086iAxL3zvwUA8U4e9+JjI6cOSBqYGV5gCxfo2En9JAN 4m/+eFVT4921fycUTQ8/Ksd3nIyhZmYEeIT6Y0URjqZ4GPraXwcvjxb176jlg0rrzoRr Vixqv+TSovgOZCrlwhY0p+jNj3jAN/KvR1iFltWyyDUwWqDbuHr56wG74maQHtIPWjJx SECQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737494673; x=1738099473; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D875B0kpOelYr3S2d/i+EM+Vu91KG28kMKlzUneXEt4=; b=C9yEsqhSHqklfd2uQseTSsc7QssozeUKh6yWgWB6SLDdIBdFoKCZIixqGuE7HcMnUW zGZyOtt9/uqWOHNpEA2AIsNmDvVWkyBJjeMm8euv2JNMRn0O+pukboGWEyqDZhncH14E /l0ibuaX9Mn4iTWLpuW4+RVxdEeI1WgPtoWYzOoyKlf/Xjrnp4IRpmX9i2u4joN3tLST zNEoSC9zOUQMw4o+1MEcKqcCA7UVMVD+mclzvSJdqEj+DD4DawKw6TigC7crE9sIzrdq q0WiVLSZrGVRJzA9RmCJ7e1oMn/o9cr9WhoGCm3SlBNDEPFw6krdrltm834SX/h1tqXb M0Mw== X-Forwarded-Encrypted: i=1; AJvYcCVLCoNo1uY9LhuGpiHWG5x1JI7RqTBFKEMrNp+M3q+fQXhXrTAWMZEszljUD4/nv1Toi62QYEo/mQ==@kvack.org X-Gm-Message-State: AOJu0Ywv0Yqm+2qxWsvyMKP5qO9xWuZLwbeQxIGAWHGUQUDxVmghkdq6 ooK2x3Q5NAZpkWZqmhNZqvTGHCbSKSY+B4lBOeZe2m+2HCkzp1D4 X-Gm-Gg: ASbGncv4E00U6S8FAvFUmcf3vuf/++0qcI3ahX98WQWUp2QRzpea2IIEmhT745921Zu LWdxlF02FV5qJM/OFi6xWJUEliOwQMkc2hP7nyWdlsBauMUv+Mnrg/dQp7vWfK8eN9BXp88sAfB JRjc/hG60qaDuR/hBBHjEga5Dp3UV2dLk0wTZ/D4WU7PER5e1qPevBerlHuJxfUoO+EjrmOxqSj zYJ7IkbG5hnPEjrc5oj39NQK5WqJLjpSTpJ7uNt6rrylo7BPwCFukH1X1JuNzUf8nWM X-Google-Smtp-Source: AGHT+IExV6nu3DeIm1Zy5TrgewQmR6yEOGT+zm24JFSRqTPSmvKk0BsuLbZ0KG25SLFTZYUT9pxefw== X-Received: by 2002:a05:690c:7084:b0:6ef:7f34:fe1f with SMTP id 00721157ae682-6f6eb677dbfmr152268897b3.13.1737494673563; Tue, 21 Jan 2025 13:24:33 -0800 (PST) Received: from localhost ([2a03:2880:25ff:4::]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f734de43basm1769637b3.52.2025.01.21.13.24.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jan 2025 13:24:32 -0800 (PST) From: Joshua Hahn To: Hyeonggon Yoo Cc: "gourry@gourry.net" , "ying.huang@linux.alibaba.com" , kernel_team@skhynix.com, 42.hyeyoo@gmail.com, "rafael@kernel.org" , "lenb@kernel.org" , "gregkh@linuxfoundation.org" , "akpm@linux-foundation.org" , =?UTF-8?q?=EA=B9=80=ED=99=8D=EA=B7=9C=20System=20SW?= , =?UTF-8?q?=EA=B9=80=EB=9D=BD=EA=B8=B0=20System=20SW?= , "dan.j.williams@intel.com" , "Jonathan.Cameron@huawei.com" , "dave.jiang@intel.com" , "horen.chuang@linux.dev" , "hannes@cmpxchg.org" , "linux-kernel@vger.kernel.org" , "linux-acpi@vger.kernel.org" , "linux-mm@kvack.org" , "kernel-team@meta.com" Subject: Re: [PATCH v3] Weighted interleave auto-tuning Date: Tue, 21 Jan 2025 13:24:28 -0800 Message-ID: <20250121212430.3902148-1-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <4ab7bd83-1191-434c-8b5f-75e71ceb26c1@sk.com> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Stat-Signature: pih9e4t4kgydhrgcm93wm45h9os4shks X-Rspam-User: X-Rspamd-Queue-Id: A06AD180005 X-Rspamd-Server: rspam03 X-HE-Tag: 1737494674-14565 X-HE-Meta: U2FsdGVkX18qvh980jvCQYDVH2q7kU3735y445FXWjRyTlbjc3jEosrhsKPXER9XhDNX+ILE0JbGgq0L0h99ciRp44+z4SLH3+19yw8T/ah+lGTurri4l41JBczXZgOycARtIiXd8y5fJe90Fx1GtA+nWhC8gAxluqPYmNqHM+2kDdXGfJerdoDd4+oNfecuaWrGdqSB8D9EIq/c/hvgrgZ9U5AICO8aCYiD3TxdzlWg4dVUrWdkDKmjjjE66qin2594vLU04Lvds3zmC57j0yH1K2pNk+c582G5tNRyqHBO7kVX4PKJgs45ev07+1V39LNUYd29tgkeLQJVjY90/pRjwJGEKrJUtF/EKBJQ28UQDVeg7Qle7Jh+hFPUISS5Neti+pgCCGoIELhIUO/veYWf7ZZX/LBk0iZoD6IFfApoHASemf/k8umAvILCs0s0qqpbWixyCbU22fwNbbTatfCMwMUolLXOQYFeSHFX3IvC6IjylN+/0f3LqQnrNOwXG2Xy7L2PkuXNKdM+Tmw21LkarGRje6Jj/YlUqdpTqgLaoSbEtLZDdCe2AsRfMRvWOb4po/+INBWX8nBxpp/Td12aiMXvb86iuX/aUCtLDBhaVbeYOrCJRZgH9HQ2FcBDkSQwWy1Eh7ybd/KRQKEIObD7MolN0RggbtlG4D0ABEuE9edsBjeCcAcwVLqjK3otGFADg5Kb+OLn5hFAYuOpdQRBrXYRrY4+mnfOu98gRj4vOJll2nW0enPuhWwLhl/xfNhEQ2/lHwYTSAARSNGEzbHkecpWR3DMilMSoxf3ejAWE3Zsu6+JOxhRSNiSrvNRbb+/bJBRcWZpNw5V3WcyKBUEq/Siwqv8Fd65kEucFCOHp3v/eKQDOQN29Zgju9OOpzj8A+iV9yc+6jZ0a7+MuNq244RrQIaoZUjYgfUyhFpH8+I7x8eDH3cT+hdLDaOvSPCRKqNPq/TD6IiCNaQ LcuFv3y8 ApeyWbFcHHHebD/YjT3BzrUzr26RMk7NXH+0SP0pqBOjO4MP0oMzWuLooP+KQRtMxdoducoFPmhYA3vPlB1wEmP5iVGvVOH1SvUIEAqE2w4aPm2sdUuqWY8vlAezAAEVYQkr9LXgPejhQWlAZesbORO9B0h5akvC5P9DrNEWv0NV0+FRKAt20s/18/dp32GuN0EthWLTOqYUN4saKX415ofS2FXWwNsD1oGXpmdfYw7pYcE+oCS/fzz5qAwXlXqaJanXi2aM5xOe1HtDLTOVj8Jeq64X+1WezYOObW3frZ7MZfdrZ3yX18X6mw5VNwwxWiR9V3brlJXDRy5ahlbu94yLelRqGMIRxDOlkRZVjdD+LdLhm0NCaxS8DG4jfz87Tu6w/GnlbPcPv6GjSqhxNvDL4GbDMgXNofmxkl24eJdnt/ER2FNRhg81dXGNN+10hetryBjFiBOQd62makw+7RIcoFZR0hVyrYCbjmgzV/u/vnAge+PMShc0vLMm97mzPKqQJx2zpmJQKv0cvdD3Zf8P0giXnbxUVGkC0aZ69IeRfBUCwEMFltVOhorHcolVInqXPuIuEtHvp07uWDT+eMEfKd+g1hIMv95qxORg6Kvce8Xy1bRBlTCHY3SIJ5S428C/tk/LqltyKEz4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Hyeonggon, thank you for the review! [...snip...] > Hi Joshua, thanks for the update! > It actually is what I was intended in the manual / auto mode description. > > I don't have a strong opinion on the weight of the hot-plugged NUMA node > in manual mode, as it's not ideal whatever weight we choose and the user > need to update the weight after hot-plug events anyway. I'm glad that I was able to correctly interpret the framework you laid out in the previous conversations. And yes -- I agree, I think no matter what value I choose, it will always be sub-optimal for some definition of optimality. I simply chose 1 because it is now the new smallest weight possible, since 0 no longer works. > Some comments inlined below: > > > diff --git a/Documentation/ABI/testing/sysfs-kernel-mm-mempolicy-weighted-interleave b/Documentation/ABI/testing/sysfs-kernel-mm-mempolicy-weighted-interleave > > index 0b7972de04e9..d30dc29c53ff 100644 > > --- a/Documentation/ABI/testing/sysfs-kernel-mm-mempolicy-weighted-interleave > > +++ b/Documentation/ABI/testing/sysfs-kernel-mm-mempolicy-weighted-interleave > > @@ -20,6 +20,30 @@ Description: Weight configuration interface for nodeN > > Minimum weight: 1 > > Maximum weight: 255 > > > > - Writing an empty string or `0` will reset the weight to the > > - system default. The system default may be set by the kernel > > - or drivers at boot or during hotplug events. > > + Writing invalid values (i.e. any values not in [1,255], > > + empty string, ...) will return -EINVAL. > > + > > +What: /sys/kernel/mm/mempolicy/weighted_interleave/mode > > +Date: January 2025 > > +Contact: Linux memory management mailing list > > +Description: Auto-weighting configuration interface > > + > > + Configuration modes for weighted interleave. Can take one of > > + two options: "manual" and "auto". Default is "auto". > > + > > + In auto mode, all node weights are re-calculated and overwritten > > + (visible via the nodeN interfaces) whenever new bandwidth data > > + is made available either during boot or hotplug events. > > + > > + In manual mode, node weights can only be updated by the user. > > + If a node is hotplugged while the user is in manual mode, > > + the node will have a default weight of 1. > > + > > + Modes can be changed by writing either "auto" or "manual" to the > > + interface. All other strings will be ignored, and -EINVAL will > > + be returned. If "auto" is written to the interface but the > > + recalculation / updates fail at any point (-ENOMEM or -ENODEV) > > + then the mode will remain in manual mode. > > + > > + Writing a new weight to a node directly via the nodeN interface > > + will also automatically update the system to manual mode. > > I think the last paragraph should also be included in the nodeX parameter. I agree, I will definitely add this in the next version! > > @@ -2450,16 +2548,8 @@ static unsigned long alloc_pages_bulk_array_weighted_interleave(gfp_t gfp, > > if (!weights) > > return total_allocated; > > > > - rcu_read_lock(); > > - table = rcu_dereference(iw_table); > > - if (table) > > - memcpy(weights, table, nr_node_ids); > > - rcu_read_unlock(); > > - > > - /* calculate total, detect system default usage */ > > for_each_node_mask(node, nodes) { > > - if (!weights[node]) > > - weights[node] = 1; > > + weights[node] = get_il_weight(node); > > weight_total += weights[node]; > > } > > Uh-hum... > Looks like it now allows copying weights from different versions of iw_tables? This is a good point, this is actually an artifact from a previous iteration where get_il_weight was needed to handle the weight being 0, but since we no longer allow 0 as a value, it makes more sense to just take a snapshot under a single rcu lock. Thank you for the catch! I will also go over the other places this is used and just make sure the locking behavior is as intended. > Otherwise this patch looks good to me. > > Best, > Hyeonggon Thanks again Hyeonggon, I'll send out a v4 with the changes you mentioned! Have a great day!! Joshua