From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1396C4332F for ; Thu, 2 Nov 2023 09:30:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5278F8D0084; Thu, 2 Nov 2023 05:30:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B08C8D0026; Thu, 2 Nov 2023 05:30:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 351388D0084; Thu, 2 Nov 2023 05:30:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1FB4C8D0026 for ; Thu, 2 Nov 2023 05:30:52 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id DFB57160DA1 for ; Thu, 2 Nov 2023 09:30:51 +0000 (UTC) X-FDA: 81412494702.10.AF24C51 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf05.hostedemail.com (Postfix) with ESMTP id CE0AF10001A for ; Thu, 2 Nov 2023 09:30:49 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=Dym5O29y; spf=pass (imf05.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698917450; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pcw+LbWUfSvxfYB/h2cYbMJhx/7SmPVWKYqXZJ8Pv9I=; b=ui5671QLYkH5LGVzxIaIdMhehL/cbQpr2XWodcL7NKUUhLdxMOvRMiWVpiRmsJ6kETE8HY i6vNPXjQ03mR2henBjgKL/WY9ksaEIfViVaFa5RVtWkAPFLsWhctpJErNS6NKudNZkH46y wnA1tgU0o0HL9dU4E05nuzPqlWrV66Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698917450; a=rsa-sha256; cv=none; b=rHCkLoTbmp+r5HFCXl30n3erv2GGcR0phhUaOBu5Klm/V/pOgwM4MNl9Ejvcp7l+Txxbgf WLHDFaZXoSL3XdhH021ayMjNwk6eriw/xEHg+HnyanrUEurBQeuP/ZCM7TUNPxOApoyxcF U7KIExouXn8Pz4+uFVy/n4z0mzfSu94= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=Dym5O29y; spf=pass (imf05.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 0531021A0C; Thu, 2 Nov 2023 09:30:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1698917448; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=pcw+LbWUfSvxfYB/h2cYbMJhx/7SmPVWKYqXZJ8Pv9I=; b=Dym5O29yyxXSjikUeyZ4MonWJ8zuFBbC0jJXdcLz/yKg3Gsf7Nl2fmYrR6RPXDy7hwu98N pISuW0g3YutuhubxJnAcbXXSN7q4kDEjywDsluYpt9moR+m8xUo9djqpMnmko+tdBNXN7F VDE3IgXVtacgzwmhX29LTAvEZsPh6tg= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id BA56C138EC; Thu, 2 Nov 2023 09:30:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id TCV1KkdsQ2U/awAAMHmgww (envelope-from ); Thu, 02 Nov 2023 09:30:47 +0000 Date: Thu, 2 Nov 2023 10:30:47 +0100 From: Michal Hocko To: "Huang, Ying" Cc: Johannes Weiner , Gregory Price , linux-kernel@vger.kernel.org, linux-cxl@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, weixugc@google.com, apopple@nvidia.com, tim.c.chen@intel.com, dave.hansen@intel.com, shy828301@gmail.com, gregkh@linuxfoundation.org, rafael@kernel.org, Gregory Price Subject: Re: [RFC PATCH v3 0/4] Node Weights and Weighted Interleave Message-ID: References: <20231031003810.4532-1-gregory.price@memverge.com> <20231031152142.GA3029315@cmpxchg.org> <20231031162216.GB3029315@cmpxchg.org> <3ilajsu7rlatugtmp63r6ussfdhqoxokj2vgmx3ki3zmx7f5po@i64b27upx5qx> <87edh81xqa.fsf@yhuang6-desk2.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87edh81xqa.fsf@yhuang6-desk2.ccr.corp.intel.com> X-Stat-Signature: ryh4ie151j9magz1q5rdnd3t185e6nga X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: CE0AF10001A X-Rspam-User: X-HE-Tag: 1698917449-601626 X-HE-Meta: U2FsdGVkX18L4cd8S2zFs1NGdOTsrn1ClYM3l99WIxLC5iK8fYpp8CoTkCjVTr570ivRXU9U+w2QcBChTNiQijdofZ6wMnEArbK/H8CqX2raRulzr6bb3ixQL1rWAhlpGrfmKmFsLku3gzsnKIdVxqNvq1DTY9Wm+p6Tjsw0vh6rwN46+cLNxDArLL4wv43gy9aCe42nEP7RXVxcjoI+FEAQg9DJYn217z8WqmkMNecB+9doHBcgK2utbTa2M1oinEWPWipVnWf8sbHUvG0WNSKmjjdbK/A0HwYNAYDsal4B56i1R0y3w4MUnO0GKH4Ay9qCk2e/12qZIKXrYjmR2GhA0UqlTZ99XtlXUXzcvCyYyon4qzgIbXOnzAU7sGDQ1q+SHcOrkSMdUN2rdRMf9M52VDn5+c0jOd2VWGGx2sBwA5St6GhwuGO4t7BXQcCKsNnE19Mdn7m9692IuZUzKmIwiZytEkF5aLx6v6RHS0BJrYeDxGDTB/CsvBQ0hQDahuCKeqQdT1dnjYdkJTO4nRSo/NyLB5XVFYnDIdulp22PH7ddMQY3wJ8GDBkgMAzoEH4VSO8lBfjLc6HxS9wUrxwcmbd2YSmdf1DFNpjs6K/Gnq2CkATwSY1xUkF08rc537IIIsKXfshbAbOyZUbFYh4X2KiMYNjdgcPycAruUJT5+xsAK9LPDUkbD0xvZNh3prhy8X/5jc3DLbEUZg0WAWQlUXdElDZGBPdMNrb+WVIWqRfQzmTaCS5L8x6mTvuXr7nUXJw37i8BcgaPq1y1w+X7KoKrx9kNzHbLp4dUov1MDoaNOX+BgA0fNmNG/u2CJFuchEr7w5HD9tw/M1PHOzmq3t1TMKmlLRB3tYukj8rxYG6SV0AuliCZSm7Tp16BVf7Qnge4GJtcZZpMIiwhoBFGbsdK8h6ker40sInVJjmyPueaqdQxpAdvRdQggbYGBhWqEgHrjjaS5hsLqzt BGrAZBWi wjZP4C8CfNVjGVkyugkwE35csMPU5D4JK6fIp219LQVvFNe1Jokv/eq8pOwiI5Cp5hOmafwIqamn0ZtzpVG0QB8rr0PZVS33dzIWYNRQS7WPsvQAM8yhb44slTQbWSVYH+Zz3k+67IDTxLDKDvWOd0WrtCTgtuBsu/72JLedHAPXSMQ2FL0QiSMJmtfxDgo+wC8qlvymU6tT7+qJZTbje/q193D3u3UXy1IyHdJVzsxXGzFXe0cPJXdE7BqjBS+8402nPDMMFbz8lKi8wFUqtrr00arV6mOdqWbXnFTz2t+AQYDmRQsKRJb0pWXihPKUpeDiGoP89VoC2hopTpGC1yiP7uS/H7tIFnYZqW14Ay9WFOZwQj/GHQRfWCa17UrtyVhBFF1fXSTMkRZIVKqxpZoxdGXO3L+XK9b1kulKzzVrDsIpZ1l8tcVjv2t5JSM3d/upF X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu 02-11-23 14:21:49, Huang, Ying wrote: > Michal Hocko writes: > > > On Tue 31-10-23 12:22:16, Johannes Weiner wrote: > >> On Tue, Oct 31, 2023 at 04:56:27PM +0100, Michal Hocko wrote: > > [...] > >> > Is there any specific reason for not having a new interleave interface > >> > which defines weights for the nodemask? Is this because the policy > >> > itself is very dynamic or is this more driven by simplicity of use? > >> > >> A downside of *requiring* weights to be paired with the mempolicy is > >> that it's then the application that would have to figure out the > >> weights dynamically, instead of having a static host configuration. A > >> policy of "I want to be spread for optimal bus bandwidth" translates > >> between different hardware configurations, but optimal weights will > >> vary depending on the type of machine a job runs on. > > > > I can imagine this could be achieved by numactl(8) so that the process > > management tool could set this up for the process on the start up. Sure > > it wouldn't be very dynamic after then and that is why I was asking > > about how dynamic the situation might be in practice. > > > >> That doesn't mean there couldn't be usecases for having weights as > >> policy as well in other scenarios, like you allude to above. It's just > >> so far such usecases haven't really materialized or spelled out > >> concretely. Maybe we just want both - a global default, and the > >> ability to override it locally. Could you elaborate on the 'get what > >> you pay for' usecase you mentioned? > > > > This is more or less just an idea that came first to my mind when > > hearing about bus bandwidth optimizations. I suspect that sooner or > > later we just learn about usecases where the optimization function > > maximizes not only bandwidth but also cost for that bandwidth. Consider > > a hosting system serving different workloads each paying different > > QoS. > > I don't think pure software solution can enforce the memory bandwidth > allocation. For that, we will need something like MBA (Memory Bandwidth > Allocation) as in the following URL, > > https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-memory-bandwidth-allocation.html > > At lease, something like MBM (Memory Bandwidth Monitoring) as in the > following URL will be needed. > > https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-memory-bandwidth-monitoring.html > > The interleave solution helps the cooperative workloads only. Enforcement is an orthogonal thing IMO. We are talking about a best effort interface. -- Michal Hocko SUSE Labs