From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1628AD597B7 for ; Tue, 12 Nov 2024 23:48:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 32B526B00A0; Tue, 12 Nov 2024 18:48:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2B3656B00A8; Tue, 12 Nov 2024 18:48:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 154156B00AB; Tue, 12 Nov 2024 18:48:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E80726B00A0 for ; Tue, 12 Nov 2024 18:48:01 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 9E3EE12059B for ; Tue, 12 Nov 2024 23:48:01 +0000 (UTC) X-FDA: 82779081210.09.D76F44A Received: from mail-qv1-f43.google.com (mail-qv1-f43.google.com [209.85.219.43]) by imf21.hostedemail.com (Postfix) with ESMTP id 99E6C1C000A for ; Tue, 12 Nov 2024 23:46:39 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=VYUqv3jk; spf=pass (imf21.hostedemail.com: domain of gourry@gourry.net designates 209.85.219.43 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731455136; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nOX9DSgNLbyzHMkGOIxDMiwsq1o4f6ejGQOhBxaCHpU=; b=mYfS2n3OSoMWP8Puim6Gp3zUdAtuoZwn9ZqKTMFuMFujKsFPc7elHAMVOb7led4udOYdvE +ciTCxoO1MkwJvcW15DEDuAw8YQ1V4bsae4u3iPpnvacWjKZbXirTodUjNhEc9dzQgEbrl iQEuTuXHr4vW+WLTXoiAjZeoo4Gq0Oo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731455136; a=rsa-sha256; cv=none; b=QBD+vEs4kwAfgGFtNDUajmOitleTy2V234GEOCrsaMsXp3hDS+JDsOZteYcdW3NICDXV1f skssG8SvL58ROY3/Ha9BPpvAlW7Sa8wo6m1LWS8lhYH9TRlLv/Pksf94bVZcRVv/+HmbEu lwmJZLAMqHa83lVL5KreBLLowX73vOM= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=VYUqv3jk; spf=pass (imf21.hostedemail.com: domain of gourry@gourry.net designates 209.85.219.43 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none Received: by mail-qv1-f43.google.com with SMTP id 6a1803df08f44-6cbcc2bd7fcso42462886d6.1 for ; Tue, 12 Nov 2024 15:47:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1731455278; x=1732060078; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=nOX9DSgNLbyzHMkGOIxDMiwsq1o4f6ejGQOhBxaCHpU=; b=VYUqv3jktJDp0r57i+uWrBRMwYTBv+4JF/rVDY5NMLTlaplbMmpWgcyOc3avKunLyl Fs1zU8LURoprIT4TtXrgyoI+t8NS4DIqZpo5CNRJWXbgQL7P3CzzGVZuB39K1e6GM6hu cvDf74xViYe7j/uZ/6G5JGn2zTHcCaIraOPf7AH+QjLrpSEoP11I4/o8VyBK9FvI4k8k REf1/Lst9IoEUkguLes4fHr7eiQzQ4+lXrcE321G4hEcnmRpABBdhSqd5P3cLF3YyCah EWwHPfKbdCwNI93nccmosbN0jzwum4dER8mopWrNfw+ix3IkDevxX2QFhpKLxsfr8wFy rDsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731455278; x=1732060078; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=nOX9DSgNLbyzHMkGOIxDMiwsq1o4f6ejGQOhBxaCHpU=; b=qzw+0tCN/NnIQ5h3A6qEg/n0UIN4nmEwXj1s49Rx4HEcLCPiyjOi5p1LplgP0UBgn+ c3gYKKDzav9l+vgCr0qgaCQVEaOJWUbqfWUnIuKdTPRc0zc9A0mrBxX+95EPv0XRDgMJ 37BgBbpcm/oO6y1PmG/UYb4HliZ84kyNdMcnBAW1RUAIWR3ePMZl1oYtSF35GZlB5Cp3 m2WQvAfgTnXuEPbp6vhbvG83yU0ZzlZ5sfCdIPflcODbAxdgT6/6/nGW6wtm00ESSqK2 hIPVal3zdldFwtgSJwPzN8k6KTxZ+47tan6hky9TVhnVjnOlXNp9hrFptHEBxZKE5irY dHdQ== X-Forwarded-Encrypted: i=1; AJvYcCUCF9kIcJSKSEbvzqrGIwlsXLp7WWs57PwZM2nIv1onx7bcDxNa1BKC1uqDhbfOWLiUF5oGh3Ki6g==@kvack.org X-Gm-Message-State: AOJu0YwrYlo68xhjiVtLJdc2mrF4nvNhu9cdIQYyBwNbqgz6hvrZ/HfQ jdWW4khR0b3Fkt2A7uRyA6nVPyUpgGtI5lM4bmeqzDZpjEjsfbNdarOUK5kYu/4= X-Google-Smtp-Source: AGHT+IESCbVOvkdTgqqmwdwGBZDyO5gMTGBHvzuur/6mM+yLjVaQptGL8/RUONrYdiW+pZ1cU+a0nw== X-Received: by 2002:a05:6214:5b87:b0:6cb:f16f:65d5 with SMTP id 6a1803df08f44-6d3dd039d0emr15720396d6.12.1731455278579; Tue, 12 Nov 2024 15:47:58 -0800 (PST) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6d39643b678sm77555456d6.97.2024.11.12.15.47.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 15:47:58 -0800 (PST) Date: Tue, 12 Nov 2024 18:47:35 -0500 From: Gregory Price To: Dan Williams Cc: x86@kernel.org, linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, linux-mm@kvack.org, linux-cxl@vger.kernel.org, kernel-team@meta.com, Jonathan.Cameron@huawei.com, rrichter@amd.com, Terry.Bowman@amd.com, dave.jiang@intel.com, ira.weiny@intel.com, alison.schofield@intel.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, rafael@kernel.org, lenb@kernel.org, david@redhat.com, osalvador@suse.de, gregkh@linuxfoundation.org, akpm@linux-foundation.org, rppt@kernel.org Subject: Re: [PATCH v6 3/3] acpi,srat: give memory block size advice based on CFMWS alignment Message-ID: References: <20241106155847.7985-1-gourry@gourry.net> <20241106155847.7985-4-gourry@gourry.net> <6733cba395c30_10bc6294df@dwillia2-xfh.jf.intel.com.notmuch> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6733cba395c30_10bc6294df@dwillia2-xfh.jf.intel.com.notmuch> X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 99E6C1C000A X-Stat-Signature: pz7keszf96mt9hoof4gqead5c6xxtnia X-HE-Tag: 1731455199-315347 X-HE-Meta: U2FsdGVkX1+aCcSz7M//JqvYz2xWIW7QM4GB/37gkLqx3b5zehM0+jNDCMrDjBB1dATfHSzVWHB2yPrbnJ2aFxRKg6InncHIG+6RER4g5Wn2OGcBsWuaJlNUCrS1TIgELyNxGTOkMMyfK285TXNILRgiVLG4LXDx5+HG/2IBDvFPfC9EUIO9zIj5Lk85TMH1hcooPexvnTtlcw2mZz5HdC+5rECWBmztrqnuPdHEnsej1fM7/EmpoKZsKQPfQLpZ78P22hEzy1f+geEVkIyQAm0d1xRTFoNvOFa0r3pIyfAm07ccIzXwI122Rp8RrjE97dToIaDYDMvI5zK5cZHFUak3ptOJfun7BAK62nsvz2+JJZTo46bCj44Ym0n36BPoY9i6Yz+9xrl8eJ2b7Q04lb96zT/ibFOX5BUQyf0kTr1c33njp3+csYKOMUAKXOzsY+inzsfajNwOYoo4rSUEpz/J41YGgst8XN9TfgxEdTmM9aGTL9TktEGYB6Sc561IoPB+XZfrikiPz3fqEvTeXvR29qktDjux5XdU8S6CQOZeVn2Wj+E1KnhZ+EQha6Ruh+Kk9vuQxu+7bYsdw0tGlNXn0a/QbXIGYuCi6UlFaG4soxsJelcQ8YJANynd4cScqjJP/OC6XMcZX+iHqUvJAIb96CB0tsMSrHdbHVlU5JCMo+kyYIAhLab5bnK3kA91c/yGSex9r7/SA2RMh//xjXiDYJL5vezAWGYBwdR8VUXhryfZHMG1w48BZ/QmS76x0KADZ2RFPggrLRmJ5A8QSrAnIIp566Alg7fFfYT5uo7SZVBXoI5oM/5vCG0lqy7oj+0vpYC0uWD1qwH2D+saMtBvHlp9fxrgSSG4gDjYtm9gFvE75QrbYSDDJ8tUGNyPGI1osOr7dOuR/h6uhTt/cuaiSOLffqcSIwY94hVosBEnaV7Kf84JT+wd23jErlH//OxZ0c202OVykcFb7yB SL/JWxuA GBUJApUdv//5BAPWJbOfYl8TrH7al+3fmrNaRCvV3rzIkZIgh9BQsABT5MbBS4WXci3ZhmAKpdUpWGwTopio4L/ivbTqvwZ9bHmoHnXJJt8ekAhEjkZ81h7P2lPPP/qYqqs7pDDIEZJp6yTbzDAOkasgntu/d3rSH9eCw5SLcRYjmA+tahBBQEZc3v0xRqOndG2Xi6bsKfp5/LEkw7LQv6fLP+bvRgWmLbjImMWBvbEIkBH3VVg1wbBJHDWtOnAbEJ+EShIEh5A9IEl444mYSy8nCs+xVnVxCrONBWNj5f6OycXsR5acGQAh/HpQID/WNvY1ttFLmOBs5lnZ0uIevaAXjkTWnIptapf+CPpSeDnWHaC2VGigW4j9mU5bXAzs8jaed7qxhGib7qeGO3ciQ87ZpMd3iqwDWRBX3qFPcEMz975Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Nov 12, 2024 at 01:41:55PM -0800, Dan Williams wrote: > Gregory Price wrote: > > Capacity is stranded when CFMWS regions are not aligned to block size. > > On x86, block size increases with capacity (2G blocks @ 64G capacity). > > > > Use CFMWS base/size to report memory block size alignment advice. > > > > Suggested-by: Dan Williams > > Signed-off-by: Gregory Price > > Acked-by: Mike Rapoport (Microsoft) > > Acked-by: David Hildenbrand > > --- > > drivers/acpi/numa/srat.c | 12 +++++++++++- > > 1 file changed, 11 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat.c > > index 44f91f2c6c5d..34b6993e7d6c 100644 > > --- a/drivers/acpi/numa/srat.c > > +++ b/drivers/acpi/numa/srat.c > > @@ -14,6 +14,7 @@ > > #include > > #include > > #include > > +#include > > #include > > #include > > #include > > @@ -338,13 +339,22 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, > > { > > struct acpi_cedt_cfmws *cfmws; > > int *fake_pxm = arg; > > - u64 start, end; > > + u64 start, end, align; > > int node; > > > > cfmws = (struct acpi_cedt_cfmws *)header; > > start = cfmws->base_hpa; > > end = cfmws->base_hpa + cfmws->window_size; > > > > + /* Align memblock size to CFMW regions if possible */ > > + align = 1UL << __ffs(start | end); > > + if (align >= SZ_256M) { > > + if (memory_block_advise_max_size(align) < 0) > > + pr_warn("CFMWS: memblock size advise failed\n"); > > Oh, this made me go back to look at what happens if CFMWS has multiple > alignment suggestions. Should not memory_block_advise_max_size() be > considering the max advice? > > if (memory_block_advised_size) { > ... > } else { > memory_block_advised_size = max(memory_block_advised_size, size); > } > > For example, if region0 is an x4 region and region1 is an x1 region then > the memory block size should be 1GB, not 256M. I.e. CFMWS alignment > follows CXL hardware decoder alignment of "256M * InterleaveWays". Max size to minimize capacity loss to due alignment truncation. If CFMW-0 is aligned at 1GB and CFMW-1 is aligned at 256MB, if you select 1GB then some portion of CFMW-1 will be unmappable. so you want min(memory_block_advised_size, size) to ensure the hotplug memblock size aligns to the *smallest* CFMW (or any other source) alignment. Unless I'm misunderstanding your feedback here. I'm not clear on why the interleave data is relevant here - that just tells us how decoders line up with the memory region described in the CFMW. The window still gets chopped up into N memblocks of memory_block_advised_size. ~Gregory