From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB350C0219E for ; Sat, 8 Feb 2025 05:06:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4988E6B0098; Sat, 8 Feb 2025 00:06:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 41FC76B0099; Sat, 8 Feb 2025 00:06:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 29A826B009A; Sat, 8 Feb 2025 00:06:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F3C846B0098 for ; Sat, 8 Feb 2025 00:06:12 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 85EC6121812 for ; Sat, 8 Feb 2025 05:06:12 +0000 (UTC) X-FDA: 83095590984.29.7276AEB Received: from mail-yb1-f182.google.com (mail-yb1-f182.google.com [209.85.219.182]) by imf04.hostedemail.com (Postfix) with ESMTP id B292440006 for ; Sat, 8 Feb 2025 05:06:10 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=XMUDB31D; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf04.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.219.182 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738991170; a=rsa-sha256; cv=none; b=7cJq+4ZpGwPvJKtfYOOEhv2l6RXbFeRMLZFx5MlrpNwnGh0Qq8MemluATAuuhrlqfKi/9v UyVpV8dmZw+WueH1LIGC36VkbsK9lcKlziP68vCOGQqvBuYxGeGPHAvgNSQJi4GNpy6oUp o/lWZmkhsWpeL2kSFknSuy0QiEEi4LI= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=XMUDB31D; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf04.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.219.182 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738991170; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QRG2Qi00mFeU6EAR49fnOvSup+6EjAplRe9HZkIAyc8=; b=27/Wz8QJkvwNLsoE+2n6xZROtkVhTY3WoyjY9HjZS3UOMg4uzp9ZFI2sTIacHfcM8HXEEr F93ui85CPKoEmtW2dTtDFi/RZmXln/rsNBJxc6qbwCtXOJfITHkQjHMcLQE65WHcteELHL Yx3abmig111DxztbWqit3ABlkh7L6kQ= Received: by mail-yb1-f182.google.com with SMTP id 3f1490d57ef6-e3c8ae3a3b2so2200684276.0 for ; Fri, 07 Feb 2025 21:06:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738991170; x=1739595970; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QRG2Qi00mFeU6EAR49fnOvSup+6EjAplRe9HZkIAyc8=; b=XMUDB31DamTnfjrNC71+Qv2dcPm2LPxp6NxRA74OAz1t1Xd2/q4PCACktNVYmzXXzk ByBqcYOMSrE3mW5Y1HyR5cgYwd+H78Tq60I5IWfoMJf8roiGigo6uB32GZRlvxJzOHvB iubVSJdKNn2eBISYsji/aZZ7n7a5VZ+/ZKPrLu+H9X2dYpVqAFHG8lIKNjpUhMenpP6G NsgNqI67FFihUzsQfHEB4TCw7Ir1MY65fRWxJbf12KdV3eSNq72qKiU7TY3sBCbNxC6k NqROAYPksk9qZmkAwsUuFw2sqywTVlt7mY0F2mUIqm7Tk5Go1HWir5fzqi0tCMIv0P2O QWvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738991170; x=1739595970; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QRG2Qi00mFeU6EAR49fnOvSup+6EjAplRe9HZkIAyc8=; b=Nhq/poUve9EshRcu7yAMLu18IWjkxbslqfR1wS/ywuaUXKdAj2W9cStw20ZCaTC+Ur 6tUOkAtbV9kWZmBjMOO0m+IBZnPLdI5JnKoPHYlHD1N2Pwx3AabnBEtro10c7MZ+7FE5 jnicTpDhDia1+/tNB+lEfu7pvuYpw7BTyXRIknMbxmYzRsGSDOhRE0bCIWEu2pmCIxHf JnO/DEk8vaaXMLPjcNCE+gqLZlrGyC4SCF+ws6oYq8IAFemcAtLN6rY3eODuAzy1aqm3 jwaWUU2s6uVcMqyFqXLcaNFvFO/I+TZbT7GZZv8Gjqo86zDYeEjZYySWIT8PyOZJqE8p 3uWA== X-Forwarded-Encrypted: i=1; AJvYcCUoFxEtUOHe0jju0z4rlHltE6PHqgUTtvFl+b704FvAuMJNrqfD/A48T0/2mPLXKpOirk2IkhgXcg==@kvack.org X-Gm-Message-State: AOJu0YyxyiTh/bQEN8bmJpcEgnszqkDV+WLv/1f+FyAt4BCCkWXF9M+h c7XBDrfSeG9ykLRvCgzcnqnWagbF9/29fZuS7RuEfLzJ47P1Bu+c X-Gm-Gg: ASbGncsxdyKJ0oTwAW4XQFEEaJdBEnY6nrluCDMz49txivr9yMSvuxdmlvuSJzvhR62 5BSGJcVvlG35Bks42SpzzJtHMYwwxD7+5XHPiWePoBhu2xnWRg2eb8/TOWEC3vj6nZsXDQ3UnaJ YDjIZ3piN1lA/8QWdrspVISHABcWpt+quUJK6sdRZOD9dJLo2BDvzTIV0CP2159xOrFTIOdC9mh IoBC+tNOSUp+M91h2RgK7YVQKO2NfXuv2kSqBli9kG4+bI4XPuw3e1OihpDawNo2/nprcc9YU9I zWXxd/wMCdIA7g== X-Google-Smtp-Source: AGHT+IGhnserk8LdwS6kdvXs+W5KoO3op4C5IpMkH1t/iBSftmI0GHRkpf3xyKvbGW6NMgWl0qNziA== X-Received: by 2002:a05:6902:1586:b0:e5b:1dda:8f6e with SMTP id 3f1490d57ef6-e5b4626c608mr5851213276.39.1738991169649; Fri, 07 Feb 2025 21:06:09 -0800 (PST) Received: from localhost ([2a03:2880:25ff:7::]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e5b3a4898dcsm1189184276.47.2025.02.07.21.06.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Feb 2025 21:06:08 -0800 (PST) From: Joshua Hahn To: Andrew Morton Cc: gourry@gourry.net, hyeonggon.yoo@sk.com, ying.huang@linux.alibaba.com, honggyu.kim@sk.com, rafael@kernel.org, lenb@kernel.org, gregkh@linuxfoundation.org, rakie.kim@sk.com, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, dave.jiang@intel.com, horen.chuang@linux.dev, hannes@cmpxchg.org, linux-kernel@vger.org, linux-acpi@vger.kernel.org, linux-mm@kvack.org, kernel-team@meta.com Subject: Re: [PATCH v5] mm/mempolicy: Weighted Interleave Auto-tuning Date: Fri, 7 Feb 2025 21:06:04 -0800 Message-ID: <20250208050606.800498-1-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250207182009.7fc099c3074ad00338f25e0a@linux-foundation.org> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Queue-Id: B292440006 X-Rspamd-Server: rspam12 X-Stat-Signature: db8hc6hobf8s9nse63pjyg8nq4sykww1 X-HE-Tag: 1738991170-182046 X-HE-Meta: U2FsdGVkX19UTy8LBLl2JznxYekTvbiTHaXaHhmeLlUmsSI+qKqDYLzbDzgXTGNa6peDX74uPUaTllFSswId0xVWqgRPIUn8j4q7FZcmBE8gAp59gKDmfxKksiEO468hG/rUjZfPod3XBM30WqI+tAsmESSh8mGjHzQEt11crwncszhUu3EKP/WcKma5I347ML8IigRKUQpxzUfE1Ulz0TU19Mqpjzy1iJNDwuxkmgJt90XhMsm18BPoIlozZE6xN3SiZM41mJJ9/EqNh04cDExQWowkInbxUEOAfz7C5xsN0J0RKXLDj734it6Rx2/te7DtDkzUTq4+/jXPy1ww9+KhVSpXP8QPsGttww9xPC/LjxEIsnDgXGMZUg6q6A7yTxnhKCWtcW+K/5VGAwQpQDAdsHkpkHZJDE7i4Gh+0GX+zXjtkTljpcV9CvjhOrGtYCY63G28lFUFQ34aX8IEg3D38h4dgJyocTAOtxh1vNEbTVw67BvZuJFIcGVqPja7c0hYaSkSQthtkUZhYcA72GD1auRkitZlyPQXM/TxiT/Pl8OrK6Y8OLBa6Q9vN4aPm2GSsn6M9dvtYFpdmaVVqlsci9Zhu5vlGMFAc+S2Cdxq4oUbiLEuEtThbCiUcKmd86iI7zl9BUjlGmeVE57ZzzgojQcyZ+7E5k/EQLMFrXF6gIr3tRRbTfiFhvI3H8KYD7C4Uo2sN6izSKCMJaJPFIYz6fO33gGaWl4HKCKFnY/yHd22KRTeAV4yySN4UnBkrSRh8qFbOWUoZSVtV8usCUKE0IXIVHSaphSoiP/PEiv1IGQwVUttZY+K0NPgZ71zoWEew+pgVaioYsi7TpTMXd7S3TWP2ulyNKUCgf87hz6mPNM5ftzRDpIDe1XiV5tIn7jImYumJZZ1XXmSLbqVfMTgOfmSi7tjTAfRTTkqK3H/bMo+qPPCXI7DxA3yaLCIlIHwNzSUOpb5lw5oj36 dVR56KJ5 Cgmy8AL23pMru1dMqeCVnltnPG48UkrbFQMGRmaAYcGyC5nWWZk5lWb34tnsQUMJY5+zkigYCn8eEdeAzQdjPPVHhSePXQXTDn+09fdtRrLfWHbyoCgStirLzFZ/NLG/aPTOkG/xrLAzAp9Ap9wNlGiPKQETCk8TnpSAuU71bR2bBnYwlvhIQcnk7Cc3dhCdjWrHfadcUMger0GrQy5VFOt0YOkrdwuAmcJRjwkRj7SB4IgZmaj0aqpA+ovzZAXnUo0oAF6gVxm4VV0eBxQ9bEGXZ+pxi7inX0GtWNj4WtH6koZGWMUhmfPemtTh41/U19wm0QRSafF5JNLhng+MAi3wmCne/6eFdYE7KWJwsdCma3x/mFv8dp8AGCD5JoFQOQ2vyC3ECjl8ZbM22XhsPTcNqSFBkk+vRbw0oU2AqxXldJSynXc9nG1fUthF6uKLlrWZGRR9bMLD6iU/SB7cX2AqglZf28J9vhxpcWkH0/m+1KSBRXrqxcTI0ifuQFors4v9Wn4s2uPBigrE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000046, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, 7 Feb 2025 18:20:09 -0800 Andrew Morton wrote: > On Fri, 7 Feb 2025 12:13:35 -0800 Joshua Hahn wrote: > > > This patch introduces an auto-configuration mode for the interleave > > weights that aims to balance the two goals of setting node weights to be > > proportional to their bandwidths and keeping the weight values low. > > In order to perform the weight re-scaling, we use an internal > > "weightiness" value (fixed to 32) that defines interleave aggression. > > Question please. How does one determine whether a particular > configuration is working well? To determine whether > manual-configuration-A is better than manual-configuration-B is better > than auto-configuration? > > Leading to... how do we know that this patch makes the kernel better? Hello Andrew, Thank you for your interest in this patch! To answer your 1st question: I think that users can do some experimentation with the specific workloads they expect to be running with. In particular, since the weights that provide the best results are workload-specific, it might make sense to compare the results across a variety of workloads that the users might be expecting and comparing what settings provide the least amount of throttling. With that said, this patch introduces defaults that will hopefully help those who are either unable or uninterested in setting weights themselves. For users who already have already been using weighted interleave and know what specific weights they should use, the auto settings might not give as much impact as someone who is unsure what the best weights are (and would rather defer the decision-making to the system). As for measuring the accuracy of the default weights generated: The auto mode works by taking nodes' bandwidth data and trying to use small numbers (between 1 and 255) to approximate those bandwidth values. For instance, [19000, 4000, 7000] might be converted to something like [4:1:2], since of course we don't want to be allocating from the second node only after 19000 pages have already been allocated from the first. But simultaneously... 4:1:2 is not the same ratio as 19000:4000:7000. So there is a tradeoff between trying to get accurate weight values, while keeping them small as to not have unbalanced distributions. This is where we chose the value of 32 to be the magic "weightiness" value. Gregory and I spent quite some time modeling this behavior, trying different reduction algorithms and weightiness to see what could give us the most accurate bandwidth data while using the most reasonably small numbers possible, and ended up with 32. (Earlier versions of this patch also exposed the weightiness parameter as a sysfs knob, but it was removed for simplicity's sake.) We've gotten some nice results (under reasonable conditions) after running exhaustive tests for a wide array of bandwidth configurations, which is why we were confident with selecting 32 as the default value. As for the 2nd question and how this patch makes the kernel better : -) Like I mentioned above, this patch might not have a large impact to those already using weighted interleave to see performance gains and know what weights work the best. However, we believe there are users out there who (1) have nodes with varying bandwidths (CXL), (2) have workloads that are bandwidth-bound, and (3) would like to take advantage of weighted interleave but do not have the capacity or are not willing to manually change the weights. For these folks, having defaults that make sense (as opposed to the previous defaults in weighted interleave, which would make it functionally the same as unweighted interleave) can provide more options and performance gains to those who wish to opt-in. I apologize for the long explanation, but I hope that this answers your question. Please let me know if there is anything else I can do! Thank you again for your interest. I hope you have a great day! Joshua