From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5EF78C282D1 for ; Tue, 4 Mar 2025 01:32:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4797280003; Mon, 3 Mar 2025 20:32:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CF65B280001; Mon, 3 Mar 2025 20:32:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BBD75280003; Mon, 3 Mar 2025 20:32:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9F054280001 for ; Mon, 3 Mar 2025 20:32:55 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4CF51B71D4 for ; Tue, 4 Mar 2025 01:32:55 +0000 (UTC) X-FDA: 83182144710.16.3D157FC Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) by imf03.hostedemail.com (Postfix) with ESMTP id 76AB620002 for ; Tue, 4 Mar 2025 01:32:53 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b="M/6M9i7M"; spf=pass (imf03.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.178 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741051973; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r6xB6tZBkxBseJmMvpYQ2Hhx7HYfm01hEgI0/OHSmWI=; b=cQuE5KxucDhO4tYFbTh2Zf+ljABbDUV76Hujv0HBipP6SJHFaNg/ssmxmGCZEzVpXGXrRo M6ptsu9ZNDyTvJZ3XHUe0nzFHwuFn/SSlp6erNLd1JoBosR8IvyZhsuhmYtnksPGIx9sYW xlIJr737s3dP7kYyjMuP7RxIWgraI5A= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741051973; a=rsa-sha256; cv=none; b=zZ929nOhlcrWszGHgsSpehCMJqpe4JPgylt8/9FdHc6esFnPrxmPRhpRnuKWoQxoVa5arz Pw8HoOqoHCQlaotHd2rBs2QV9XtixI1WqbhdgMflZ9oSMrUr+BwaV21rzOpxkrq0QjCVnl MzrrN9pPXoh1K6ZJxZKj5piZ4o7ChzU= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b="M/6M9i7M"; spf=pass (imf03.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.178 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none Received: by mail-qk1-f178.google.com with SMTP id af79cd13be357-7c3ca86e8c3so61416285a.1 for ; Mon, 03 Mar 2025 17:32:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1741051972; x=1741656772; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=r6xB6tZBkxBseJmMvpYQ2Hhx7HYfm01hEgI0/OHSmWI=; b=M/6M9i7MauxOSiQteOteiY3j9urPFinmAbM1wCHkZQHjsIBC/TghKw9JfhsyjiEKMb yo+L9aWapaj8lnBtt7g/+LAxx2owkMmhWEsN2NVLSXUmvYR3jrs6xaGvTlVjRTesdOwg ng4O3OtHhB5T/sBORojxwhLoiu8LdpoaU2m5fMoNgbmgeZCl5a5NSdEqz0mnNtumHm5G xQqcKBdUewR/0cFltO2aN00DxQdoqvls8fi906I5y56ptct7NUsxxObmtQkIc5TelIab ifkwdrVAFy3SBoO6+4lhfl5mvEixtxsubVT5pt0i9e6JQpM6lU08Xfxju+PJBz0ISKgf Mpug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741051972; x=1741656772; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=r6xB6tZBkxBseJmMvpYQ2Hhx7HYfm01hEgI0/OHSmWI=; b=a/SaAecoz5EtBtCKmJPFX3mgKX2hKahOcBp0BpDXdAYfY7ifyp1hlFCKlRoTOw3/eV qHylvL0uk8DOGxWRIEYvyjQzv/26TalVQVMBKAuurHxqdzI3COHWSLPfjsSflmSr09Ij Co1NONMG6FUujj4/Vrw6aB50bapDIjp/9iAQgiZMfd251BQfy4S4MbINK2E/NNWxVcvz WUdBUAVZV7YHvc/fY/ZnAASZOICh+nIB8JqF1HnkWYnMo38Kv3JSK2ckaXvE8CvKLXbP /nBKUYFp4e9o1alg2XGDM//QoW5e6GrccNGl6RYy5PK3Bs4hV3eXvtHtAYQ4sw1QIYMJ mOiw== X-Gm-Message-State: AOJu0Yx5varJNy9Zi1KyVNBjMMJqUVeSif6mk+ILWVlzv2yXLj+0+al6 Ht8GfHpw+ZJXkIeolsuEIeQFMKIXUlKSVeT9bW8waxQUefudHtXCoYQ3eO0AIgFA8TNd6YkrT7t U X-Gm-Gg: ASbGncuoltaSNMdjodmAus3J9fE11wQhb2rjqJYDrDR7eN8Ix37ySQaq5KinfEswYQ3 tTjhx97iKPDhUajxRNj3JgPu4rvwnk/N3F+NGTuHFkZJcceR5ibB4A0sspuTCInFz7P2lSpHzK0 OyB9y2E1iDjpWoEb4Cww3q6HUVDstY5Ntmk6w7e/hE6sy7IysyERNlA1mfFyq6xf9Uqb0FW2cNc JnrMBVCNjhi1qktbAG6DIV/0uVi9TCbhEKkguZYebYJzoN8vXGbctDHu70Whl+XanJhaL6Fztj3 lLUcWsb582clxsawEormVG1+aRDgmQqnDLorixt0qDHc7WREVr4wi76mItyV0b9+9yUPhZZ0jrN iHYxMSYQu024eTdKjWjPBgXir2kQ= X-Google-Smtp-Source: AGHT+IG4dWln+CNbJieutk1fOc0kWi4nxKmsaP4G3WEAip9vevv4K/TkpbAdj3Eb+EfbBCSu65rNPQ== X-Received: by 2002:a05:620a:4492:b0:7c0:9f12:2b7e with SMTP id af79cd13be357-7c39c49fa6emr2001696785a.11.1741051972460; Mon, 03 Mar 2025 17:32:52 -0800 (PST) Received: from gourry-fedora-PF4VCD3F (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7c3b7366e26sm231363985a.87.2025.03.03.17.32.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Mar 2025 17:32:52 -0800 (PST) Date: Mon, 3 Mar 2025 20:32:50 -0500 From: Gregory Price To: lsf-pc@lists.linux-foundation.org Cc: linux-mm@kvack.org, linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: CXL Boot to Bash - Section 2: The Drivers Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: p9wmg1i98e7hqq5hc9u9rcohpgpo5qw1 X-Rspamd-Queue-Id: 76AB620002 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1741051973-458529 X-HE-Meta: U2FsdGVkX1/ZSckDo5Z8CatMkFBhJE7OZJ8zemUBsDiGh9emhoQTSXRPk/5eD3S9xZ4hlUxJvUsD4hICLfBWbD6FbhVR79ALduEs5tD42YbGq9DoGUIlHrv6TeVKz/NqEBvuUbVTspHU/vc9c1e8Zu4lhorWJDkaskoC460mTRLrpGywRQJxXH6tqc47eeNDDLELaFI9sHie/C9M9+aOUxQjGZLuX+Q3sY8eVlPwvs2RkltmQdc7QflbSeu+gzmb8en9qr0aM5dUg1g7zTeiMKl8OUYNhBQW6fAMOjkbX9Z6GGvqLqZmJk5CRnPsYTua21lsnAA9YuBEXfH5MBx7MMTB7OCNi2w/0hfZpBpnS92bf8GavIvv+cM3BTtr7mX0WosZPihmnds0jG92fhsCvfIreZSyCnfuFbkKz/m4dQHjsbcAu9b/35ChyS71W60lVgoI5b8hzFiTlr+yFdIOWTOkyr0dRNsb+CP7/4XCi2SwuuJCedPn49gGnk6skT4f9qQlR7zy9kq8SO0ejtq4hyJkwQJO4U7e0PVynSD4ajQXgJ5e6rZo7uY7pLKMQiFY9AN0NrXkCQFc4lyoB9fNeuuuhhPAz4ZIkjPDPaFa5wT/TP7AmY6v2z+AjkUteWSHzw2E02CYXciffR0NAxskfL9TcO9vLq7Y7Jp313vo5RXwaIRHStBTkIJo27OBz8H05cpgzqRh/sDkeoRdY3u2k+HNPHBcJFVBYDhTjWiB/Ojxgx/gtwqlvXprHDKbS7WIDbxj4BABSo5fDZu1Ez7aiX55jYF1hVOn5+vwuJ6JB8MRhNm9q1sLxp8wNReUY5vsntBRfqlqMEc2KK4Vh5A5Xgh6bzgO5FfZsGPyJTSHs6WfBL5xZ8iN7Gue5yffuCI0avhzFk7We0XbiItkNSyH1L8Kw+Rj+KUMPwyRpuSaOk1BkkFfG4Cuu9599YPs24GhNSFtfsOvN81pt77xFFQ +06H5Z8W JZHgZ0mEJIj3nFDMB/dgwW4EN8mhwTI6rf/NvK6vpCk9GwQhImgq4gLnVK7KHVTF0IabO4+sRNQDc1C44EcqWOC5pbjhSbv4hnxBWsq9d1VdEtLDjpzoxD0ofRv5daE7RYIy9yT7Z9UChR0j7VexsaChNBtqegrtGuiKHj8UpJq2IjDCFF8I62gNYqynacyCsQ7PF3j05tew0/qGog+NG8iCKdrDxxwPeVnDBqwNY72cm96J7cKpd92s3MU/lyXk1arVG0yeNhIs2qAskBF8Ww90QwEkFPxEQNhonuvqO9+N7XHMxHlnSbRXPHEX62oh2Ytn0HVrHpYYKeSg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.014253, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Feb 05, 2025 at 11:06:08AM -0500, Gregory Price wrote: > --------------------------------------------------------- > Second bit of nuanced complexity: Memory Block Alignment. > --------------------------------------------------------- > In section 1, we introduced CEDT / CFMW and how they map to iomem > resources. In this section we discussed out we surface memory blocks > to the kernel allocators. > > However, at no time did platform, arch code, and driver communicate > about the expected size of a memory block. In most cases, the size > of a memory block is defined by the architecture - unaware of CXL. > > On x86, for example, the heuristic for memory block size is: > 1) user boot-arg value > 2) Maximize size (up to 2GB) if operating on bare metal > 3) Use smallest value that aligns with the end of memory > > The problem is that [SOFT RESERVED] memory is not considered in the > alignment calculation - and not all [SOFT RESERVED] memory *should* > be considered for alignment. > > In the case of our working example (real system, btw): > > Subtable Type : 01 [CXL Fixed Memory Window Structure] > Window base address : 000000C050000000 > Window size : 0000003CA0000000 > > The base is 256MB aligned (the minimum for the CXL Spec), and the > window size is 512MB. This results in a loss of almost a full memory > block worth of memory (~1280MB on the front, and ~512MB on the back). > > This is a loss of ~0.7% of capacity (1.5GB) for that region (121.25GB). > Some additional nuance adding here: Dynamic Capacity Devices will also have problems with this issue unless the fabric manager is aware of the host memory block size configuration. Variable sized or sparse memory blocks may be possible in the future, but as of today they are not. ~Gregory