From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 393B2C282D1 for ; Thu, 6 Mar 2025 17:08:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 62444280005; Thu, 6 Mar 2025 12:08:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D46B280003; Thu, 6 Mar 2025 12:08:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 49D78280005; Thu, 6 Mar 2025 12:08:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2F7B2280003 for ; Thu, 6 Mar 2025 12:08:57 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4F9B0BA80D for ; Thu, 6 Mar 2025 17:08:57 +0000 (UTC) X-FDA: 83191761114.01.6C8FF5B Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by imf20.hostedemail.com (Postfix) with ESMTP id 2E89D1C001D for ; Thu, 6 Mar 2025 17:08:54 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b="UQ7/fgOw"; dmarc=none; spf=pass (imf20.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.172 as permitted sender) smtp.mailfrom=gourry@gourry.net ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741280935; a=rsa-sha256; cv=none; b=ixhU4OuxU8rrL8Jm0e8p8ayemtrirBUIoPM7mEehQz0olz3ghpSDIyMelKTp2iJSFK+/om KseWJdIN/7gfWekWO5LaeypPcl/97onccjFf7Tjo7nRXkuYJQF7T3KAVxN9B5hJhfRDeP1 sOnhLV4wIk2xFpnKCtaUDg7aeX0HTqw= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b="UQ7/fgOw"; dmarc=none; spf=pass (imf20.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.172 as permitted sender) smtp.mailfrom=gourry@gourry.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741280935; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1wNyXxcCEsmV3rMjIVb7Etw5fSJV7UEopsGd2j/hR1s=; b=ECEDBlwVd50Qh5wosb9sNMAYZU4TNhqKFO0wM8RSjNF7HES6Ni2Y/Rj0tcrNE79smg6I3g kU1kGte5gGkjhVROOoz/7q68hepgKyYD8LXq2Jm3zcLX+MtJ8D9WkdsXZfLdn8mdXlQuaD RyZ6oRS8oj7V77u8CiPdQYfpSXIFD1A= Received: by mail-qk1-f172.google.com with SMTP id af79cd13be357-7c3d591e50aso99594585a.0 for ; Thu, 06 Mar 2025 09:08:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1741280934; x=1741885734; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=1wNyXxcCEsmV3rMjIVb7Etw5fSJV7UEopsGd2j/hR1s=; b=UQ7/fgOwyEf2S38gcE+p/ZDhPeAaR+K6g3l6nXMF22xb7HfmmFjsuKaUg5oYZCisy+ vc4fIxBQHSbq+zRUSi+8QFIHsbIyOczA83U64nXHZRtbktEXupo95XD+vJxKTX8Yu69J qkyeZMIzKWdHiVuBSL9KL3D2flcGgyXdqAIgcR/hHAjQ6TLK+A4npREBTtMQjv1lWGOS YnW+c39VXX2Q4gi48aIqOPgzx4gZczfV871tHfgTZOm1AfOekTYHj/YiftRxXpDDY2Tw HGQGnwzplL/6iZH8xXVL145CVAK28D2waCpEvAEnQgXdCGpJMaPdg2cAoCx/B2TVIrXE q61g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741280934; x=1741885734; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1wNyXxcCEsmV3rMjIVb7Etw5fSJV7UEopsGd2j/hR1s=; b=naIBbFX39AHAhyKpddQQzTHdWQADUwFO2Cb+h4Mkt6B4Hs0LKjuMUElP04OSTHKMkq 0240v/k2n/PpNqEyy3Ospd5fxG0ah8pZmHKeJDVPow88JMljh6ipRLcHhw1wZJLUKhWa /xU/P2mbm8ndLl7gIqFS5S1+Vq50zbPMPnOqdh5RUh+ZKf+QPvs/c5BZT9TttBrft5jM wVtY5Kgu7MGuJ4AuY9DoJQFqXSnDtAFfdvcwOB/qpnE7JuCVpRh+7QWnTCWitNsi2MK/ eWe+1wwCuiDXJSeXmdnabvWmniUGLhCgQ+aNB/3PZjbefphBVfBJiIC6B4aPhN19Dc6R kMDA== X-Forwarded-Encrypted: i=1; AJvYcCVRkYrNTTkRTiszbsYyfYdmqow/dsvOMqndtmw1+Y5AKzi/osztMnL1KH8kFMigKGgGcz1ROwMKwg==@kvack.org X-Gm-Message-State: AOJu0YwkV6tbxsN2+PPfAlRLvwDs1jwtUfiJOIBh7OAZhmlGW8bY1o5G g2+XhWTt6nFI/FHJ79M9gmThBiVVfQxCBZdvuWCQ0oyLEhcGSFCP+cL5PiGkZvc= X-Gm-Gg: ASbGnctdWilYUHDI9n/OaZV+SGJIzRrHZkbhmouN62IT6Vy0oIT/3Qvjd9+8TpMYCD0 hmUGJl/2MgUhZJimJw1cbxYncDOKaeERvpIg6QMBaQTqgnP/lL2vHEmqUo99KCQ16ofJPmpeuZo BQVw7AeMk26Rhq/pKMXFtx1gInQxgyY2lxzN2F5vz3ZEFdFjh4RAsauJaSbUE68yX9nT2F2f954 fPaqgotwdT4E40KabaLJ0fSYJEVvNn9S8G/mo1pZQ4Rc20Iod0z90xuokf5Cps5Cncd7szoRkTX lcLQ6E9qxlNe03uoI7Wk+jow133FpywH5uNuXxsrvI6vLZ/OVRSgO3kxLO+wu5YrJEyNbdvrDUR AhcqwCps6C8ztia3RbyG3tKScH/s= X-Google-Smtp-Source: AGHT+IGzXdo+ayTPw4Ob64P9xw3wnYIZAQKLG2XeMD/1+CZlCiHLJrym6jweO9Vd+DG+fYhwPqgi2w== X-Received: by 2002:a05:620a:8806:b0:7c0:a357:fe62 with SMTP id af79cd13be357-7c3d8e65564mr986859185a.19.1741280932574; Thu, 06 Mar 2025 09:08:52 -0800 (PST) Received: from gourry-fedora-PF4VCD3F (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7c3e55116a5sm111483085a.106.2025.03.06.09.08.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 06 Mar 2025 09:08:51 -0800 (PST) Date: Thu, 6 Mar 2025 12:08:49 -0500 From: Gregory Price To: Yuquan Wang Cc: lsf-pc@lists.linux-foundation.org, linux-mm@kvack.org, linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [LSF/MM] CXL Boot to Bash - Section 0: ACPI and Linux Resources Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 2E89D1C001D X-Stat-Signature: ctmxkfg4b87eauteqegwtkyeg1smj3jx X-Rspam-User: X-HE-Tag: 1741280934-524470 X-HE-Meta: U2FsdGVkX18quTx7hXaCYWtBZle6JTZrg3t5/vPTXBAkFYaxdnJlGJI8AiRGn4WKQ9WCsc/WwAdinlERh1J5w1qQ862vGLYLKOjQM1J8vJoM3oGZlrXPLv9llNuoMmxaI/w1JjztZ86+dfZIY2eZu53SloZomKqlR5xn85Q4wycWD08rtglBT/ndzd1Dr73m6glOjkP5l+D/v6fB1CLzjiEdqFQ3MbIPTMiE5IJvfk80qOOp93jG2x2GU6KdMNtw7xBLeb7rnRRtEulj02EIJZi84RLUZU3NKsLW1KZ4vla21R1zdpBRo+Wm4yKxjG+tXT7b1Y9iiT5R98YuEv0ARH3lyPtGFdKw86kX9xyqGad/cSIh3HiZznz9/VjxHtbZ398HKCgQNfek2fPAfQmneWsUNUi1KSzRZ5yfsFgtRoGLK5NvsGX4cgYk1JhOkhilfuo4whD3YZ3TvZC0uVvOHrrkpTneiGR90mCWbAEoVYM7DMcdIXS8ag4+PSOxuHzLJcii2Ocgdu28VyV/vwMtBPxcH+TOEuPYzhE1gzqtTg/dOyzyZLv1/xiZxL7UqJ12PBIZlTtAmBp8VGho5CtGwruwX5ZANGdv04+BsnGHZRGVXWka+1jx83XDfOGyAGIeJei/edfakfTEuQ9nK3XXb8DTPqhMfn6oF6IprO2ohh/4W+x7IGAbDZ2VUt/C+ibEmfDjMN0xqwJfMW78KIcnZIOoWUYN5GiMmporDvbGG7wXp/03d5xuYYzxIcKjq833ohKSoD+iAkqF4OUwaQ5SlE7btZknrALyeyN/fJtGMWaU/tV7Y65Gq+lSRRkGNUJQN7c512JaOGErrr9WKjaJlfwJ1Kax5MJARqalhcm7+cZCJ6VJywrHgI2+oZbnXh3t5kLj3mM+jDGqxYfoAKoWbF1NqcNQGelr4385Sic++jCR5ucWAaTGgq9C/vwTSzfBO6fzfgXXbNM6TmB76hR Iz4gwpCI 84b0WAjoKqAkrC+TqWJJqUP7gJeNlHNVQm2Y76mYCOiI7WbVrsxkLsTO0TwAbdT67/gy7T0GtRoO3V9v0rUAsGu5SdXcB9Os8hvJ0XDbrID30IKH1uIQHeQDZB1t2Jh6I0sFUpBUcxb7rrOraVGiWHB2zHEDtP86bXTu06ftji4t35h8hjg0xD29VcNJrkSpeuhG0P8SzjIUDe5UpHtVzB5xrsLSrJV8O6i7HOg4F4PVboLEIc0WC+BtKSGvoxNAk9TTJwagY/djtv9Yw7IX0+ajYOiVRw5RkPwPnBNyS9PEz6bAhByk+eQEgkCmkgphnh3ZG5yJY7rpxBK+pPK5SzBMragVptS3yt7H8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Mar 06, 2025 at 09:37:49AM +0800, Yuquan Wang wrote: > On Wed, Mar 05, 2025 at 05:20:52PM -0500, Gregory Price wrote: First, thank you for bringing this up, this is exactly the type of ambiguiuty i was hoping others would contribute. It's difficult to figure out if the ACPI tables are "Correct", if there's unimplemented features, or we're doing something wrong - because some of this is undocumented theory of operation. > > ================== > > NUMA node creation > > =================== > > NUMA nodes are *NOT* hot-pluggable. All *POSSIBLE* NUMA nodes are > > identified at `__init` time, more specifically during `mm_init`. > > > > What this means is that the CEDT and SRAT must contain sufficient > > `proximity domain` information for linux to identify how many NUMA > > nodes are required (and what memory regions to associate with them). > > > Condition: > 1) A UMA/NUMA system that SRAT is absence, but it keeps CEDT.CFMWS > 2）Enable CONFIG_ACPI_NUMA > > Results: > 1) acpi_numa_init: the fake_pxm will be 0 and send to acpi_parse_cfmws() > 2）If dynamically create cxl ram region, the cxl memory would be assigned > to node0 rather than a fake new node. > This is very interesting. Can I ask a few questions: 1) is this real hardware or a VM? 2) By `dynamic creation` you mean leveraging cxl-cli (ndctl)? 2a) Is the BIOS programming decoders, or are you programming the decoder after boot? > Confusions: > 1) Does CXL memory usage require a numa system with SRAT? As you > mentioned in SRAT section: > > "This table is technically optional, but for performance information > to be enumerated by linux it must be present." > > Hence, as I understand it, it seems a bug in kernel. > It's hard to say if this is a bug yet. It's either a bug, or your system should have an SRAT to describe what the BIOS has done. > 2) If it is a bug, could we forbid this situation by adding fake_pxm > check and returning error in acpi_numa_init()? > > 3）If not, maybe we can add some kernel logic to allow create these fake > nodes on a system without SRAT? > I think we should at least provide a warning (if the SRAT is expected but missing) - but lets get some more information first. ~Gregory