From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FB2EC282DE for ; Thu, 13 Mar 2025 15:57:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9E3CF280003; Thu, 13 Mar 2025 11:57:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 99568280002; Thu, 13 Mar 2025 11:57:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8340B280003; Thu, 13 Mar 2025 11:57:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6623D280002 for ; Thu, 13 Mar 2025 11:57:12 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 66F68AD679 for ; Thu, 13 Mar 2025 15:57:12 +0000 (UTC) X-FDA: 83216981904.02.C4C1451 Received: from mail-yw1-f169.google.com (mail-yw1-f169.google.com [209.85.128.169]) by imf11.hostedemail.com (Postfix) with ESMTP id DD94A40021 for ; Thu, 13 Mar 2025 15:57:09 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ZeCD5bfO; spf=pass (imf11.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.128.169 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741881430; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bBmyDKUpiommKIGSKMzZOKB9b9qGKMc0bJ8kfU2qbbg=; b=gKYbtcwAhD+daFZn8PwT0+6n6zphykfSNRSm+piDMaIMBt6y/BADAS+3J2vKskMAJvGPhH B7jxETVUQbauzfNaWFU/a+uoogpuQowocPkWNctKF4x61xqBmdWetieL6oi3COWVLEhw7e PTmeaozBHkY+OAzSNY3XXAoFcrGg2aI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741881430; a=rsa-sha256; cv=none; b=H5rdfANFKVnfyt0sdDC90mRXF7TuDhMnsxDUK5GOLPE8yxHLV3AGgU3kSk4zv9lb6Q87TB rdEEGWH8ChAGu0O2myB3F6vIooatXXCJuKnNFdYeU+u57FXZg1RQeEi9SVZnuaRYBXkVQ7 6W3Zw2O+CGdfdF4a/j62Y8ckk7ieyis= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ZeCD5bfO; spf=pass (imf11.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.128.169 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-yw1-f169.google.com with SMTP id 00721157ae682-6feafc707d3so10455937b3.1 for ; Thu, 13 Mar 2025 08:57:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1741881428; x=1742486228; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bBmyDKUpiommKIGSKMzZOKB9b9qGKMc0bJ8kfU2qbbg=; b=ZeCD5bfOWf/rhogBBG+WYEZvTSG50Z3bPbXbypIm6kCx5t4wwYPfrroJhaOHgcUTSR tdKahVrqyf+qBY6ON8K4xSGznOECxSWLoigzARXdDVdk9VpBUDFDoAnmRCtaUE9cpVbr B5uVovcoHLvFeDVHPfrqlpsoltfspLLrGoP5razY6NcqbtgotFFyYV43+R+WqdaiXR4T TIdpplVwNwbot/Ekn/vtp/3NkELicZ5nH1yq2i4xX0ypOYlzDVKdu2tS/9FCh1fsBi/5 eG5j4rKNNM+ta2pUrp1fR+6NBed1UUjCRMFVXNv8HGIko/hCcE9dKdC+bZAai156ZT/p mFjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741881428; x=1742486228; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bBmyDKUpiommKIGSKMzZOKB9b9qGKMc0bJ8kfU2qbbg=; b=jFWWgL9K/tpp2ttgYYPoFY3C28q8T2ve4TupQu9M4TTQZ2d9MjCRUv97YjbwXfbzmB cbTXl+KBbu6dJholx1XXkjkrhYqTvAXESwKMnFhSwBZRt9RMUWgC/I++uNzQR7dUlNey M3H0ExKrA+CrVjcX2CS6q74K0Qqi95ggcoT8b0VapExwQwNuQg6SUeMG6dwDUZuX/AUk Wv7b7EIOgjQy6SPWrsbDSDBazplHt/PMrCljzO/kDsmxW9oBZv55OxqbmGmgTih1gnLp dBzFeliseFOE/KFNAANVqe170ub1VuR13ll9RVpIbrm3iALC0pevuT1AZWemech33IGp HjCQ== X-Forwarded-Encrypted: i=1; AJvYcCXDxTipTin7ntM5fXEultbFnToZYhdHty7Su+yfeH02sIuS9mSBAOdGjMOrx4imAaCs7NERoYL3sw==@kvack.org X-Gm-Message-State: AOJu0YycCcGKA0akGRZCqUrVG83QW/twnM0QpXqcKUjs7ejyXF6hHhXA jwayDp6T46wvS1/S5yNFyj5ZM92cxUfkXWs94KD5lZF7KAKkmAfm X-Gm-Gg: ASbGncsu+v8DdQedSoGlzbBunOJC9jLsAPZJQfqB4DMHxCkCwIFq224BJQIfLDUYw9k WvSMhD9ITaO5r1Jn7Gxm5edJEWATnctG2US+uN0lOBxiQDlYS0zXkoDzFPb50Nz+8J3bz4AddLX j7p94Das3iq/ffugUonXwaUz2EOVYwLboTeY21+OP4su5zGGasqppGjrxjpujqANE2DOOSgVDt3 RubDMROPtWn4UuYYhD7uZCskYHvykMLSYPUBaip39ooWpxSzuQtysUe7ZjnE9bspA6B5M1m8kNI +9p7wlZXFA/tiz21acwWXWJpaSpO7Pc2PhbnRuXRKQU= X-Google-Smtp-Source: AGHT+IFxyQ9wNWdzuK7lTh0XEvFD/n9auheeFjnwpeBULqjgsHi1P64eAWAsA7fFVJQvuALPUUaprQ== X-Received: by 2002:a05:690c:c92:b0:6fb:a4e6:7d52 with SMTP id 00721157ae682-6ff092928b5mr181604867b3.35.1741881428160; Thu, 13 Mar 2025 08:57:08 -0700 (PDT) Received: from localhost ([2a03:2880:25ff:5::]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6ff32874fc2sm4065987b3.45.2025.03.13.08.57.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Mar 2025 08:57:07 -0700 (PDT) From: Joshua Hahn To: Joshua Hahn Cc: lsf-pc@lists.linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, gourry@gourry.net, ying.huang@linux.alibaba.com, hyeonggon.yoo@sk.com, honggyu.kim@sk.com, kernel-team@meta.com Subject: Re: [LSF/MM/BPF TOPIC] Weighted interleave auto-tuning Date: Thu, 13 Mar 2025 08:57:04 -0700 Message-ID: <20250313155705.1943522-1-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250109185048.28587-1-joshua.hahnjy@gmail.com> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: DD94A40021 X-Stat-Signature: ep3gn9ke4qbq7fww9ubutzcnqmozbzo7 X-HE-Tag: 1741881429-368832 X-HE-Meta: U2FsdGVkX19vngkVOTtZLywRR0BXQlyXw0e1/4t7SX6BgdU5Yb7oxdFZGNuKc2BBq5M8bfdQq/0h3CG/T/uHXieNeBNz2aALsvYjtUdtZoRDbp2Pc0kUz5aawYWbRxwrx+ygppvNXaKi3V1wSuIqN3bJy4y1m2HGQYipwCclWmBwmWADlADmfdv9k3Z39uUWiAB8zJ0ImBhT6p0wHmnyJkCRt81hR3VR38h1gerThnaFTbWdgMVhVL+kNcw2L7Z+0umTDqT9NXL28c+W1ELkYxEyUMpVvsEpL1G4miZ1HjUTJToifxpGJp9iT22vCm3QYK5jzPcf34U35Fd/JePm0fsNdKsPs8Jvu91V5W6sEucYmFMpKPSR2/n4zLmtPy8TmwLmyt4VaBRMYC9Wt/CRmniwp0wPJwOSb5qhU4qlU0j3zlcXmj1L7gAld01FoL1GrMj0wvBtkezab1+arWVVr/n521CILCoej1nKKk+XpPwEY3pPFmBmdGBchNXPmv74mP9JeULzHLEbX92AKHxr8bwzF7w9at6HdWel89+ywR8zGanW3AuFPQ8QWg5XWbGdkkC0qbZjtcsLDuRp1s88/d9XfQfbjFvjHXb6EsWFZeigt/zQsMMmRV5mhg9JGZ9YSFMqEvisU+GtEthOiYlb51np4ErjodocrcZZMLoAR5WXZhL5CC7dXvTPyAa8nxbw62dmZjWNfpeOWNKGmPCtqAfFaWwJQENRvVY8Ms/EmzPEvyGRfdbT1lZXu6OszPfFce7E7Njxa10Sk1fh7VjfZMXK1rSkBiiw5RJd+6qw4w3tI8nKXWUQOMD/l930vkVRlMez073Y3YPbskCqmVdH+1pljCRbrS1fDoKO5moMNMX3gGlYp/AYy/VLVLWjWWOGKlyFRcLjAO99Aa/W2wyuO9td/LoMD8J7ZCwozn8aE22XCUEF4HVC72zVV5/YDVe9umM7ix/kg0ek+h4PhKy ihFLVDWI jB4rLz3QvimrN54THw2qZSlVzY6IMn7ekKHVLWCfwiidS/NxGiPtvAofw6lONSapevn/O1qMBWqE0P5J0LCAJJrc/CNs3atXv93cRxgBXfsXYwaEoCyM+z39cXufnFknw1av2ZhFhJOZKjvqNMhEKPEBimhNKWPG3qPmNRz7FOc1uor7iX5pCH6t6shQPfbmY6lNJjFJZL9psZuhgy6dOGvi8x874XormA7TjpN5Ou4EldMjpDYJu+e9OkJj7GbyYAkxmomrpQxayF9ha2qlJTCh5WBDzBUTDpJnSk0XCxyz3+fzbdtg8RiqzzEHdYG0oVPeq3HP12uSBmLaDOTfu3OXykZa/CtwZf9wdMWNBHGMUP0vtfBp/C7PqrUqreRk+1s4zuSZlrFERfYuI0jAgfgBBdO/PHfx3fZIkGoZJ8q9QIo86S0KI6PTCuTLK3XWYgTPSakPAjkuKpRLM+vWy2Xr0qBKeYx2eumnC3clpzigRvMLibAi8JeNEfztnEtj0Ppjq/nfcA/CW0H2j3SyUN4LYhfuSFHbd92WzaKbmPwsnDbDkI2lSTqbDC0FpWpRPBI6L3P9io2l4ixS0E/FqwnqZODgPDU70vADu X-Bogosity: Ham, tests=bogofilter, spamicity=0.003253, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 9 Jan 2025 13:50:48 -0500 Joshua Hahn wrote: > Hello everyone, I hope everyone has had a great start to 2025! > > Recently, I have been working on a patch series [1] with > Gregory Price that provides new default interleave > weights, along with dynamic re-weighting on hotplug events and a series > of UAPIs that allow users to configure how they want the defaults to behave. > > In introducing these new defaults, discussions have opened up in the > community regarding how best to create a UAPI that can provide > coherent and transparent interactions for the user. In particular, consider > this scenario: when a hotplug event happens and a node comes online > with new bandwidth information (and therefore changing the bandwidth > distributions across the system), should user-set weights be overwritten > to reflect the new distributions? If so, how can we justify overwriting > user-set values in a sysfs interface? If not, how will users manually > adjust the node weights to the optimal weight? > > I would like to revisit some of the design choices made for this patch, > including how the defaults were derived, and open the conversation to > hear what the community believes is a reasonable way to allow users to > tune weighted interleave weights. More broadly, I hope to get gather > community insight on how they use weighted interleave, and do my best to > reflect those workflows in the patch. Weighted interleave has since moved onto v7 [1], and a v8 is currently being drafted. Through feedback from reviewers, we have landed on a coherent UAPI that gives users two options: auto mode, which leaves all weight calculation decisions to the system, and manual mode, which leaves weighted interleave the same as it is without the patch. Given that the patch's functionality is mostly concrete and that the questions I hoped to raise during this slot were answered via patch feedback, I hope to ask another question during the talk: Should the system dynamically change what metrics it uses to weight the nodes, based on what bottlenecks the system is currently facing? In the patch, min(read_bandwidth, write_bandwidth) is used as the heuristic to determine what a node's weight should be. However, what if the system is not bottlenecked by bandwidth, but by latency? A system could also be bottlenecked by read bandwidth, but not by write bandwidth. Consider a scenario where a system has many memory nodes with varying latencies and bandwidths. When the system is not bottlenecked by bandwidth, it might prefer to allocate memory from nodes with lower latency. Once the system starts feeling pressured by bandwidth, the weights for high bandwidth (but also high latency) nodes would slowly increase to alleviate pressure from the system. Once the system is back in a manageable state, weights for low latency nodes would start increasing again. Users would not have to be aware of any of this -- they would just see the system take control of the weight changes as the system's needs continue to change. This proposal also has some concerns that need to be addressed: - How reactive should the system be, and how aggressively should it tune the weights? We don't want the system to overreact to short spikes in pressure. - Does dynamic weight adjusting lead to pages being "misplaced"? Should those "misplaced" pages be migrated? (probably not) - Does this need to be in the kernel? A userspace daemon that monitors kernel metrics has the ability to make the changes (via the nodeN interfaces). Thoughts & comments are appreciated! Thank you, and have a great day! Joshua [1] https://lore.kernel.org/all/20250305200506.2529583-1-joshua.hahnjy@gmail.com/ Sent using hkml (https://github.com/sjp38/hackermail)