From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B47B7C369DC for ; Thu, 1 May 2025 18:22:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A7B36B0092; Thu, 1 May 2025 14:22:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 92F336B0093; Thu, 1 May 2025 14:22:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 783146B0096; Thu, 1 May 2025 14:22:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 571E16B0092 for ; Thu, 1 May 2025 14:22:05 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6F5F9141275 for ; Thu, 1 May 2025 18:22:06 +0000 (UTC) X-FDA: 83395158252.15.F0E1AE2 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf08.hostedemail.com (Postfix) with ESMTP id 82899160003 for ; Thu, 1 May 2025 18:22:04 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KqsEBZEB; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of kaleshsingh@google.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=kaleshsingh@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746123724; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=M4oEmD5pkFXHibsXA2WC273fo0mn5jrWwyr6sBq7SDo=; b=pxZlhlBZAU+EYa9skx/ISCExlZzNRbyomAfDf93yttiCFaxZ+idJafpZa80GGYiXNQ+ZS+ jlwF0xQMWNzNb8Pdo43B2+o3zjWB/P6cQIAbiNj4IfSOVj5uxR6s8aDljPw/lQPUrNfTt8 9VnAu+JH2m84L1wJ3gYMED3+ZNvpBbE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746123724; a=rsa-sha256; cv=none; b=fZueqIwIQwpWFmJicl7ZFdcBhJPz7v+T0AThsbYDkqQEUPB99b3LDMexN4DUB1FcT1HDN1 dwWhX6mxhh2BWyFgR3OOkeuKlTQ/GmXKxiX6PWwcxRxq+9xGDx0MibcsenZgXXfdJ6ybsI oCbmj5hjWJMFiB0+lwc0yrX5r8jbJYU= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KqsEBZEB; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of kaleshsingh@google.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=kaleshsingh@google.com Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2242ac37caeso16805ad.1 for ; Thu, 01 May 2025 11:22:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1746123723; x=1746728523; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=M4oEmD5pkFXHibsXA2WC273fo0mn5jrWwyr6sBq7SDo=; b=KqsEBZEBdWxx0RhZx1wQZNNAC/GBSxg2fgItdwWGohFm9INzm06fk/Tlvfhx2EUR6k GyBkJRsptxdw7FDZl0NW1cRYGNEVOUXraLoi1uHUjGFtKQmhAebCDxHkQaiT26B7C58+ 5sV/H4YMmcC6m7IEsqh+oGjfyBAaYkU0MBdos9csuKHYFqoSMSzZ7PRP/biUOuT8TpkS Un/uPnNnW9SnpEHIBlCKtDi6ZvkvucAaa7n3x9LLyF4ywvtmeDnywQmmr3Vtlnx4EAqg 9mJ6jBq17MEjg94y/RznJCImuXuMFucR8YPZiL/GUJr380l1yXESHfwX3Np7Rcit3VJ2 DZPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746123723; x=1746728523; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=M4oEmD5pkFXHibsXA2WC273fo0mn5jrWwyr6sBq7SDo=; b=NZdbhIj2i7W/U6kJDkF3QJPgmAuACxTDEbLtoHd+V0mlI8XlzF7dRlSE5SRBe5HB8E /D6UkLkcaZS54/orE2SZBbq040KWXs3QfbsZvIUH9ilxj3bPr1NXzazYgSvE80kVVVZB BMtytMOQ/MPKnYjHMRnJvlu4vO0hF4o2vaKDvBx4Jaf+opf/633sByjdHtr25rSHswYv lM9x168M71HnlFpHguhYX+0HrW6e/PR1Hqa+eu3JWVKjj2lZLlj9+hYSgUfz9gJ+9Rvx cifDrXYGncQMEgilktmOd89ETjRKIaR2I+kYG1g47FljspLWSeQoctnE22TnA6m4lrSE Erzg== X-Forwarded-Encrypted: i=1; AJvYcCUELN/JGiklwT59mLChfdY1LxVSa1BT7ia1XzqYWDOTdQ8fVOINZu93k4PNxLq9YkYnAmyGu6DPxA==@kvack.org X-Gm-Message-State: AOJu0YyxoY05r9cZn8ZYHpY9nbKOvvc6WThAN1HD6lzJSwqckA6TcbKR YQKHFqI/KSnXjRhkMEpnxRj12apUKAS7zg9gWl5kPyb91KxdPDCBLNICaRaRctdIanXfQ5Alk/7 msJsbJyzf+0N5mmmh6r0UeowCsaE7vPGg9eXh X-Gm-Gg: ASbGncucJQoSiRlXk76TNMwEH7PV6jsthTKX4ThGrrtXKzzQQdDEONgKJKGZmpSQ1yo CzPS9KS6BEd2FQEtkLTN3FpXIplsq1OwZKIxF34Zio21y+tjq9bn9i8HvgY/tmsix3Z25S/1vIG 3+jJuSnpMKOOd8AeCTIl9ufxRkTdR2NIYheR9+XhVnIWoW+suGVgA= X-Google-Smtp-Source: AGHT+IGw01QLKei1a1tqjRO1f1fstLHCdEGbwq/8Z+18IvDEQWgBjR9t4vbhB4thwSTGW5yezeLPGhT4pQ+nxyomE04= X-Received: by 2002:a17:902:e750:b0:21f:3e29:9cd1 with SMTP id d9443c01a7336-22e0784ed21mr2880445ad.1.1746123723117; Thu, 01 May 2025 11:22:03 -0700 (PDT) MIME-Version: 1.0 References: <20250501052532.1903125-1-jyescas@google.com> <3230A277-7D1D-4329-B871-5E43967E6A00@nvidia.com> In-Reply-To: From: Kalesh Singh Date: Thu, 1 May 2025 11:21:51 -0700 X-Gm-Features: ATxdqUELKjWvsXtfVkBZFM3lN1s1zGGvb5Be96Hj7WDDLkjDj34v7xW40EM0e3Y Message-ID: Subject: Re: [PATCH] mm: Add ARCH_FORCE_PAGE_BLOCK_ORDER to select page block order To: Juan Yescas Cc: Zi Yan , Catalin Marinas , Will Deacon , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, tjmercier@google.com, isaacmanjarres@google.com, surenb@google.com, Vlastimil Babka , "Liam R. Howlett" , Lorenzo Stoakes , David Hildenbrand , Mike Rapoport , Minchan Kim Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 82899160003 X-Rspam-User: X-Stat-Signature: bjjjiauowbpgot946uw4935nfyfqpid1 X-HE-Tag: 1746123724-281427 X-HE-Meta: U2FsdGVkX1/bZ0tpRfK7EzZAccTbNGKkQjKThkz7sA3BPnh0/nhaeSaDqXEHcG2+hKLhV2VpjqgIFdu0NbjHxfM0E+8T7SST9OsOePkfJ1O4I5vzzf+OOEyFNizmtletdu33TYB8CH5cpmgB/uAx03La6tx+if5i9qcur7E364LWm0581tEDLvWAfi9UCHjfasMaNSENbN+sOmjKOmLkm4M5U2G4YZHL0N4WxK64hv6DmkFhaD/TRu+fp2NWs+zlEe1aq2kavaMo6v93lgLkcRDIZAY4XsH8cNxIcp42JF0PVMiUcrUJFYzXgGOQa+Axhm+RPP9ZXn+o0X9AeBIFMeyFh5CgSaJwdHv1Jj61N669bEu6394eW0dst7bb7RTo6VNdgLnYO6owzllp5QIG7DTEx65RUXM7KKguAN0oLSYInhkVp3LFpmxpeX4413vqzg/a4LjZOMMAJEKVRnDVYVBDNI/NVW7nMF+TsAad4/f5sHXIvmk7Bdppsoj0upyaItIF2Pgd0oRcfgUKfnpgXYGOZs5gq7vT86gLBINEmGXjcVdoJETOArucS6ytt8vR3jKaCVf+lwbX7JvQSgFo2We+a7cNP3qPZGurNucptaDeVjnRqp2oanFgJqZJhb0TojOixdBVSFkW/YCRM4YyLDZjeLApc2O6txdZUgBwNP3QI1kmrlIWUaP3tPa2d2RznqJKsB8O0NzxrJG8fVsFjmoHg9A3pcraWfjPy1d7jzjH+QjRGuylhxH8NQPpXzWn/I0BDY5blZx6gN0l4gdhRy/S9ZhrscIifDBxXzMbZyxqg4+ryA2q3xCkwgo5vnupW0cI+HK6G55JOHroLYoUyob6r3ktSaYQaPidcwYd+ZQWb5hUMFwAvWoU5GSTeW4MLtq3Hxu0VTovDBjvAKO7g5m/zIaSH0Zo+LHVSLqAPsiQL2RWDmFYdwctrJ75cfCmHDyJ2PRFb29CYu1lTn2 eXvAAXZ5 t8DT952P/1XK2ItmoYjOJkTWn6JU05Y70t1gHcwjMID560ATA7Da3TWPP63XS8KPeGIS+QpD2LEdp60XKkC68+IKDP0njiFW6kHlNpUiyzMCM2C5L6i3Kb8M7IJEnt3PmWdBvt6V2+4fi+++xXsCwkWnpzPlPlAuiQeV9obTGKH7szLpSEaB6OqTBJhGACXA0VdQYk0gz5BsfnAFKnj/TnG9/PM49dwchWJVUzKSZe14oNzia0s0fjMYve0x46pthkYqhW5st1iTh0J1P8jFQhmfKRUUxEWP6L4LymBRAyvRERziQwtQPQmVOi96CHCn3DeRxnPxc3VTs78U6xY+o35tB9L9HnBluOMod X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, May 1, 2025 at 10:11=E2=80=AFAM Juan Yescas wr= ote: > > On Thu, May 1, 2025 at 7:24=E2=80=AFAM Zi Yan wrote: > > > > On 1 May 2025, at 1:25, Juan Yescas wrote: > > > > > Problem: On large page size configurations (16KiB, 64KiB), the CMA > > > alignment requirement (CMA_MIN_ALIGNMENT_BYTES) increases considerabl= y, > > > and this causes the CMA reservations to be larger than necessary. > > > This means that system will have less available MIGRATE_UNMOVABLE and > > > MIGRATE_RECLAIMABLE page blocks since MIGRATE_CMA can't fallback to t= hem. > > > > > > The CMA_MIN_ALIGNMENT_BYTES increases because it depends on > > > MAX_PAGE_ORDER which depends on ARCH_FORCE_MAX_ORDER. The value of > > > ARCH_FORCE_MAX_ORDER increases on 16k and 64k kernels. > > > > > > For example, the CMA alignment requirement when: > > > > > > - CONFIG_ARCH_FORCE_MAX_ORDER default value is used > > > - CONFIG_TRANSPARENT_HUGEPAGE is set: > > > > > > PAGE_SIZE | MAX_PAGE_ORDER | pageblock_order | CMA_MIN_ALIGNMENT_BYTE= S > > > ---------------------------------------------------------------------= -- > > > 4KiB | 10 | 10 | 4KiB * (2 ^ 10) =3D = 4MiB > > > 16Kib | 11 | 11 | 16KiB * (2 ^ 11) =3D = 32MiB > > > 64KiB | 13 | 13 | 64KiB * (2 ^ 13) =3D 5= 12MiB > > > > > > There are some extreme cases for the CMA alignment requirement when: > > > > > > - CONFIG_ARCH_FORCE_MAX_ORDER maximum value is set > > > - CONFIG_TRANSPARENT_HUGEPAGE is NOT set: > > > - CONFIG_HUGETLB_PAGE is NOT set > > > > > > PAGE_SIZE | MAX_PAGE_ORDER | pageblock_order | CMA_MIN_ALIGNMENT_BYT= ES > > > ---------------------------------------------------------------------= --- > > > 4KiB | 15 | 15 | 4KiB * (2 ^ 15) =3D 1= 28MiB > > > 16Kib | 13 | 13 | 16KiB * (2 ^ 13) =3D 1= 28MiB > > > 64KiB | 13 | 13 | 64KiB * (2 ^ 13) =3D 5= 12MiB > > > > > > This affects the CMA reservations for the drivers. If a driver in a > > > 4KiB kernel needs 4MiB of CMA memory, in a 16KiB kernel, the minimal > > > reservation has to be 32MiB due to the alignment requirements: > > > > > > reserved-memory { > > > ... > > > cma_test_reserve: cma_test_reserve { > > > compatible =3D "shared-dma-pool"; > > > size =3D <0x0 0x400000>; /* 4 MiB */ > > > ... > > > }; > > > }; > > > > > > reserved-memory { > > > ... > > > cma_test_reserve: cma_test_reserve { > > > compatible =3D "shared-dma-pool"; > > > size =3D <0x0 0x2000000>; /* 32 MiB */ > > > ... > > > }; > > > }; > > > > > > Solution: Add a new config ARCH_FORCE_PAGE_BLOCK_ORDER that > > > allows to set the page block order. The maximum page block > > > order will be given by ARCH_FORCE_MAX_ORDER. > > > > Why not use a boot time parameter to change page block order? > > That is a good option. The main tradeoff is: > > - The bootloader would have to be updated on the devices to pass the righ= t > pageblock_order value depending on the kernel page size. Currently, > We can boot 4k/16k kernels without any change in the bootloader. Once we change the page block order we likely need to update the CMA reservations in the device tree to match the new min alignment, which needs to be recompiled and flashed to the device. So there is likely not a significant process saving by making the page block order a boot parameter. -- Kalesh > > > Otherwise, you will need to maintain an additional kernel > > binary for your use case. > > > > Unfortunately, we still need 2 kernel binaries, one for 4k and another fo= r 16k. > There are several data structures that are aligned at compile time based = on the > PAGE_SIZE (__aligned(PAGE_SIZE)) that makes it difficult to have only one > binary. > > For example: > > static u8 idmap_ptes[IDMAP_LEVELS - 1][PAGE_SIZE] __aligned(PAGE_SIZE) > __ro_after_init, > kpti_ptes[IDMAP_LEVELS - 1][PAGE_SIZE] __aligned(PAGE_SIZE) __ro_after_i= nit; > > https://elixir.bootlin.com/linux/v6.14.4/source/arch/arm64/mm/mmu.c#L780 > > Thanks > Juan > > > -- > > Best Regards, > > Yan, Zi