From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B563C282EC for ; Tue, 11 Mar 2025 16:15:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 630F8280003; Tue, 11 Mar 2025 12:15:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B9D6280001; Tue, 11 Mar 2025 12:15:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 45B11280003; Tue, 11 Mar 2025 12:15:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 28103280001 for ; Tue, 11 Mar 2025 12:15:28 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 844E052138 for ; Tue, 11 Mar 2025 16:15:28 +0000 (UTC) X-FDA: 83209770336.30.87872C9 Received: from mail-qv1-f42.google.com (mail-qv1-f42.google.com [209.85.219.42]) by imf06.hostedemail.com (Postfix) with ESMTP id 2F18C180026 for ; Tue, 11 Mar 2025 16:15:25 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=lm2XCFM1; dmarc=none; spf=pass (imf06.hostedemail.com: domain of gourry@gourry.net designates 209.85.219.42 as permitted sender) smtp.mailfrom=gourry@gourry.net ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741709726; a=rsa-sha256; cv=none; b=gF1Q73OoEkR9nGAuByZq07/Dncz9Xl2tCUHZgBue3XiuuJOJCZjcARu296XoWOUuTcRxDQ hvEVlYQcWNpS7OC50Z2JUOiqJQGwclqY6SFHvpLaP2k/W0s6ABgD2ZG0fy4K0Yk6IVjG03 11Xk81Yara/RehLLIuLV3+1YM2G4QrA= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=lm2XCFM1; dmarc=none; spf=pass (imf06.hostedemail.com: domain of gourry@gourry.net designates 209.85.219.42 as permitted sender) smtp.mailfrom=gourry@gourry.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741709726; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+ctWxbKDFvJQjTOqLK3W0WOLogRld+0VnOA3SYeJG4o=; b=eM8UC0R1vFdwdkOLXHMkB4G1GaWC5ugRyi/E35BMgXok8V6yZi5VTglvQEJ3DVWJVOBJRz 6fgzY97+iLBIxojwStg5JRBgPbwTOabNEHzUs+vH+Ejq7CUicHommtKDVuHYHFd2J7hJyc XKFAsmd5OdprAODJQ33bgnA10UI2Em4= Received: by mail-qv1-f42.google.com with SMTP id 6a1803df08f44-6dd420f82e2so72165566d6.1 for ; Tue, 11 Mar 2025 09:15:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1741709725; x=1742314525; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=+ctWxbKDFvJQjTOqLK3W0WOLogRld+0VnOA3SYeJG4o=; b=lm2XCFM16Zz1GQMKFbsvsgFC3Tuf2bNaAuSoIGmziWDUFhIduS8WFhBZbYKWuCE4SY XcIDc7OFy0jSo+UKeF6oy8UYccQgp/ankqIEHzJF0gXyv/T8nmz1vwRxT1f3kwpERUYW Zzi0Ix0TwMOMW/IHRtdvYJ6YPSgZ17uXOi2mal3aiimF3fJOmlzu6hEdp6+zG5c4rLum AOjvnWPeZZNodNew6i9mcCmiOv3arjxRYgstrYg6azzT4uPtkkguPh+AJCfYiwfynBq0 nJ4xlfhBPFuoGiYlEpRf2tSLsjEggg5YVZ3mGkKOQDVsZTH3+9LiEKxqsCISRazlb464 G3NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741709725; x=1742314525; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+ctWxbKDFvJQjTOqLK3W0WOLogRld+0VnOA3SYeJG4o=; b=xPk7RbcZTwBS3TuMJGGTC9gmjymz4JA1oxhwRgMBB6rhSG61OXfmTRDna//lkJy0H/ Auut+NlBLvxCH0m18+ylBVQuN0s1mb26wTIZ0ica9Djr7fLbv2lqc2sXJaAEVOv7E3/N VpZIYZorKhgWiiLLQIWodc0glD0LD5RZEP+qMf6+tm9o+1NMcH1ZhYuAkLieJfx5/W25 5tvLj2CYzeKoZdLMDyCEA/FaFkkPCOuXvWWdT8CKJAnSt/sStrazVLUcfOcHV/PK+53w 1wk30IpdbEC33HAlwlvHw1pXibiWuZs33dPLMJq+YhJyuRnV6CwE+XRwy8VCMVPPoEvn 3xMQ== X-Forwarded-Encrypted: i=1; AJvYcCXqALZuoMg1/Y31HBJkC7c6c6b63cePawQagvBBrr+/OiY2vY94p4rcTK1AIj2LT+u11KyUjVK89g==@kvack.org X-Gm-Message-State: AOJu0YyQRu7o5T4O/Sj8twrN1D8km+MasHSqInfqqL7MuF0DhosGjXw3 4Xzh6nGAi+h1vlrn5OL1h0uX6DybYi/d879udzFyQ70J7UwL5/rwzpwqKkHMmOQ= X-Gm-Gg: ASbGncvoQACCfff5gURlzwY9PuHvmZ0J0IfptNzCySzVVDW5pqiQSr6NwhW2SGF8n6q xWaz3XhYjV+v9UbLrv4MUFsjwXmdneMOs2yfAQe52n00q/HC3cqxjC1eMmcpePWec4bGvZT0MZO 9wV6Aq5eKu45OlK+HhWSIlOyTylr4P1h5hGa4NUeAOAACtU2R5rDoVLesFP22UUjImucWR9p3pJ x2R8kPgCZ2BE2SkX7FHVoVnIOMHQxPh1ALU1ijrvIIO/13N1tmXU1Mx0eHGVvasBU2g/Wr2opYm +s0+tDMrZmA16wnLfYecFFnMogjHmPSBAejwV1fVD/aAuhsqqKpk7pL22VunmPUciuBbKWQWPiO V5RTKnC8/bAHG2MPAkEkm+9FW4Tk= X-Google-Smtp-Source: AGHT+IEngW+whjtSsGen92TRPPuxKluaJqI8wEtaAWqqtJh7Qsb0QErvZ+cCKG9luxZznTxhTIBdSA== X-Received: by 2002:a05:6214:c64:b0:6e8:f65a:67c5 with SMTP id 6a1803df08f44-6e90061f684mr206274106d6.11.1741709725208; Tue, 11 Mar 2025 09:15:25 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6e8f7094611sm74058526d6.50.2025.03.11.09.15.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Mar 2025 09:15:24 -0700 (PDT) Date: Tue, 11 Mar 2025 12:15:23 -0400 From: Gregory Price To: Zi Yan Cc: David Hildenbrand , Yang Shi , lsf-pc@lists.linux-foundation.org, linux-mm@kvack.org, linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: CXL Boot to Bash - Section 3: Memory (block) Hotplug Message-ID: References: <1b4c6442-a2b0-4290-8b89-c7b82a66d358@redhat.com> <4ae838ee-b079-408e-8799-e9530ca50417@redhat.com> <0C5749C4-B3EE-4393-A8EA-AA56B399E9E3@nvidia.com> <7A60A7B1-0BFF-45ED-8A11-06F0D5286723@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <7A60A7B1-0BFF-45ED-8A11-06F0D5286723@nvidia.com> X-Rspamd-Queue-Id: 2F18C180026 X-Rspamd-Server: rspam11 X-Stat-Signature: 4b7kryftjs8o3bpj95r5pn1agkcnj6ei X-Rspam-User: X-HE-Tag: 1741709725-52487 X-HE-Meta: U2FsdGVkX1/4sGYAnFpKD/NWETByu32rRaESizZYV8SySeDGhoDLEhIifJY0pOFnZjVnNAcOXIm2tX+XxyrtzpFrflkl/7kObjzVkulMp8TOofNecC0UKcCbtSMxxWcpPFv1zYu5fkkeY6r+hgp44u714/XmUvP22zjfziqCBXGvR9bAVBkm4Lf8/dlShPk9ARGw3hPmsdHZD9AItgpdIwJ+y1EQo1FlUxjQmizGVUA3wnZSqlUPbS/xZ/SFfTMTxRF4JkbblxoQZv/yRRR331juKz5VElz10y2YRItIpeDtXeTzqUilTRSYzjhcrmknCQdaFrtcM3H2wR33Vz/PuVdP1d6hUqiItrusBk2XhF90hc3rsdv3bwCTmd2PDaC1yt8+WLIQ3uyMwImEtQhaPQnBEHrRsxdyROikA+A4Y1kTIfxUXUOFoly8wdRW+87EsDWmleNUgNoU+3vG+dBxKG5q++4jURFI3lpTT+BxD0CLnw8wvkCGVAU6rHwWN12b3f/B6e8dRmgAj2nMBkpIrAaQSvR8QjWnD/+hL8Rk16wxSPyA3Nwz9q8TLeLav65/B6omFp1BJOsbfvGUfiswctGfwoPpoWW8diLEkuaECthSSKyDDq6RbnvBvU41WzJmL5llM9fgXF4+eG6vTTsoGxgt1AUrmVnvh2DfOjzweMIkNdpyUAmPSfebC5U0OTZBUSJ0AAzt39fuZfbqFjNoQSeqsLQUkpXSValc82I7w9k+ldg94My+45+tmAK+viG3CYZibqVDxi4nxPZXJ0WvalW0psIvHaVHmywCmmawSns4f4U1mO9d1x+UQ0yA5qlomUV596zjP9Es+cHcvXRvkIRLyY01VYuHJdB9DlGHFCFxZJLqRaCf/Xp8XFJx2aveHEynBBlUShy4Ssu7lKERAl4Jbjjq+YAHZTUm7PyNgBe/KSGH6rrcim8vm44N2T9EyGIGcE6PreYuBd54anO 5vqdbnQE qApVHffduSvO/1YAki9rti++KXg5v/7of6DxoKdEktfrlRmdwQzRLFs6qR5DGZ8gvpwVd9cyUjf1qUIKL36TSQIDbmanuSF8JL5cyGtIA9Y7Vj1Ol/m7Wdw8smdAYut8dVbYBhfcV3p1CtaPWWV2WtUo4safB/LEyW6xcAIzsYHL3A8iCPd0ZvjeLZbseHHudz30XXnuyvEe43xHCBwHabbprqVKUoPmzNYzBig0A22s7rSL62JuDM8rxwIB0bIEmHUm1vRdt7s0M0XkSnqQn5nHDettUw+7KMck6TUEMfjywswqkCwrHQwRvJXXxWXn8OnXQREuqUhd6ZttW18X9DmoybNfleMFSznsd X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Mar 11, 2025 at 12:08:03PM -0400, Zi Yan wrote: > On 11 Mar 2025, at 11:58, Gregory Price wrote: > > > On Tue, Mar 11, 2025 at 10:53:41AM -0400, Zi Yan wrote: > >> On 20 Feb 2025, at 14:26, David Hildenbrand wrote: > >> > >>> Yes, the only way to get some 1 GiB pages is by using larger memory blocks (e.g., 2 GiB on x86-64), which comes with a different set of issues (esp. hotplug granularity). > >> > >> An alternative I can think of is to mark a hot-plugged memory block dedicated > >> to memmap and use it for new memory block’s memmap provision. In this way, > >> a 256MB memory block can be used for 256MB*(256MB/4MB)=16GB hot plugged memory. > >> Yes, it will waste memory before 256MB+16GB is online, but that might be > >> easier to handle than variable sized memory block, I suppose? > >> > >>> > > > > The devil is in the details here. We'd need a way for the driver to > > tell hotplug "use this for memmap for some yet-to-be-mapped region" - > > rather than having that allocate naturally. Either this, or a special > > ZONE specifically for memmap allocations. > > Or a new option for memmap_on_memory like “use_whole_block”, then hotplug > code checks altmap is NULL or not when a memory block is plugged. > If altmap is NULL, the hot plugged memory block is used as memmap, > otherwise, the memory block is plugged as normal memory. The code also > needs to maintain which part of the altmap is used to tell whether > the memmap’d memory block can be offline or not. > Also just to be clear, this is only an issue if you intend to use CXL memory for something like 1GB Gigantic pages - which do not support ZONE_MOVABLE anyway. So for this to matter your system must: 1) Require smaller than 1GB alignment for memblocks, and 2) ZONE_NORMAL CXL memory. The whole thing is mitigated by telling your platform vendor to align memory base to 2GB and having DCD's allocate in 2GB aligned extents. Both of these are reasonable requirements for hardware, requiring a major kernel reconfiguration seems less reasonable. We should continue talking, but I've reached the conclusion that simply telling platform vendors to fix their alignment is a better overall solution. ~Gregory