From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECDFBC369DC for ; Thu, 1 May 2025 17:11:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 450D56B0088; Thu, 1 May 2025 13:11:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3FF6E6B0089; Thu, 1 May 2025 13:11:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2ED0E6B008A; Thu, 1 May 2025 13:11:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 135F56B0088 for ; Thu, 1 May 2025 13:11:14 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A3C7116125D for ; Thu, 1 May 2025 17:11:15 +0000 (UTC) X-FDA: 83394979710.15.9E0A498 Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com [209.85.160.178]) by imf11.hostedemail.com (Postfix) with ESMTP id BB7D440004 for ; Thu, 1 May 2025 17:11:13 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=JmMUQeq+; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of jyescas@google.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=jyescas@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746119473; a=rsa-sha256; cv=none; b=0dzJlbb+a1Ax3uOkMsXYvrzwkKhBCK4SO8dM8i/DSc2hO2xhALdodeHM7xhHUEfk/gzAlH kpNhB/yqUSq2SQfP1s8OpjSWw+58J0qGPcxUcTp32u8y2or3Dp1hu3uH+Ifj6mk4RDramj KMwyfTDj1xzc7WcV/GMfKhmAJiEld5M= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=JmMUQeq+; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of jyescas@google.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=jyescas@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746119473; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TyZUTfbjOtSI4Pg1zTRceh+Y3qETrTmwa1GNlHLrMsE=; b=kKtpMUBPPGUIxZ6U3bCwRtLyRIrkzFbq0ylckJLqtjSYg1/5TncKyhA2DdkV0KoTMSJhLX d+W7rx51Fiba5QmO2iUkLyt8jKrMiMajWDlMivocgEuFPzGtz6OJeGDHWhKbsowf2Mt0an BKr0lUZ+sdVHAIUPcFykLuv6yaN1KFE= Received: by mail-qt1-f178.google.com with SMTP id d75a77b69052e-48b7747f881so8911cf.1 for ; Thu, 01 May 2025 10:11:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1746119473; x=1746724273; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=TyZUTfbjOtSI4Pg1zTRceh+Y3qETrTmwa1GNlHLrMsE=; b=JmMUQeq+DN4/TfA7aX0TfneJdQUfTj39bESdoQD1jcIIJ1XOARxvqe7teEHFwfOz1w WJMLevKmdrtrn0iG/5TAEPZPguHBab8cjlHDf3lE6ZoJ/GKP/A2JxF0Xfmr5LiA3KNxV yFiT0aUUNQtKVUlOtBYiHjdv+Uue/EzDb1sxYuGh+i89ZAdHBeyWNRIe6hTDNu1kslRC YquTJGTbVYrJLeuvJgDM34GPpOqUGW863mDX0IePxjyYGvA10MQ9jGqv1d87jxEB/ckp ha2Icu46o/RS0OQhdHK8Mk5XNT7nnaWX93GAq7jpPuKzPQ5yHFKuXGKQiYvtdyutScXG HFcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746119473; x=1746724273; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TyZUTfbjOtSI4Pg1zTRceh+Y3qETrTmwa1GNlHLrMsE=; b=seP8Yny/DxOZ1v8s30MjmCoFiYRj0gcbJXS7Qkeojv2Zll3d0S2bhTcBgK0d7Tnk4C H6c9HqbodVlPHtYHXNf2vWA+fnvtlYYD+9h0EsF5UZlwAo50ld7AFg8uw/YGpL2uOMNb y4/1dMQ82yyerLY7JsRTzoCA6Zs2fN9uLoalKIfOeEhgnGdREsWg1DULjZaidCXlzm1b kb/W/7oP8OwdVXb4XXkKCwHoFlYwA06TD7GVcJQ/GWv2kwP59aTM30iNBQAYr9oOxDxi JNzXpzKqD6xjE3YOhqREsszih85QKhoLHSHtTOFxy+Sod37IBcCRXZrX+W8ZTEibxbIH bvYw== X-Forwarded-Encrypted: i=1; AJvYcCX2zp5e6L+TGihZYivqYfB/YyMD100qnYHIy/1Dup6sAnAkLr01ppEoXzt3XcY0E37SfOC9FX5Ugg==@kvack.org X-Gm-Message-State: AOJu0YwVFD+0Au+h8oJqjv+kidKw2bP84G6vhPVB8o7/mkwyUbFC8rRc onipYqoodxpIjKd80M3q/vwMrJ9vxKGJVDKqNx0GwibrZO/58nFz4yfcVN23GdENvl7DphhuOfS Y08rEC/GYWXYwr42kTeBw4aBir5KNHwCYhrRT X-Gm-Gg: ASbGncuycSOaqoR2vW6340PAGuiTZiULjIn/N+vH+DaeVbsMHI93Yg1ApEdQ2uXaQ65 sTXMJQDsC3iH6k29bNei5XtexCQudAeNkOjED5wOyBkyVdG/lVD3e6WDXLfpYMuZZVVBOLT+GqT YIG4ABQGKem6SMLIqzxFb9UwrMn45dYLDd6LXeAa2hThbM93WbjD+6751U X-Google-Smtp-Source: AGHT+IGv3eRGNECsUyk6JOE6O6LjupkXS1Vch7nD0P9cmrdw/DfOv9xrVQcy66j5Dit0TNIse4AjgxHkAuco5q3Aruo= X-Received: by 2002:a05:622a:104:b0:48a:42fa:78fa with SMTP id d75a77b69052e-48ae9f775aamr4522371cf.2.1746119472574; Thu, 01 May 2025 10:11:12 -0700 (PDT) MIME-Version: 1.0 References: <20250501052532.1903125-1-jyescas@google.com> <3230A277-7D1D-4329-B871-5E43967E6A00@nvidia.com> In-Reply-To: <3230A277-7D1D-4329-B871-5E43967E6A00@nvidia.com> From: Juan Yescas Date: Thu, 1 May 2025 10:11:01 -0700 X-Gm-Features: ATxdqUGuDLhfocEblH8_bII6KZwxowsdcLaFmo7SPnbUbxHXravB8vFsCWiGWlY Message-ID: Subject: Re: [PATCH] mm: Add ARCH_FORCE_PAGE_BLOCK_ORDER to select page block order To: Zi Yan Cc: Catalin Marinas , Will Deacon , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, tjmercier@google.com, isaacmanjarres@google.com, surenb@google.com, kaleshsingh@google.com, Vlastimil Babka , "Liam R. Howlett" , Lorenzo Stoakes , David Hildenbrand , Mike Rapoport , Minchan Kim Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: BB7D440004 X-Stat-Signature: prq93mx3cjxhshqsb3g9jjmt5siujur4 X-Rspam-User: X-HE-Tag: 1746119473-792676 X-HE-Meta: U2FsdGVkX1+NoecWNqaQF7RLUYyXzoVVlmMsJ86f2u4A6rNVb4FxoLcD8AZUZXM6ZNW/6NR16+ih2hqlNQhxPJdtNtZc2/f+VXgPHZE0JhgJuWnB+RiC1Y0wbAm2wf/HdZwFklkpM0Etn5tD7i42KgKIH66e8zwVraMYdxmMGd9i/Eqx8Y5UTCuhRFSH1VUw/q0zaWkJ7uwCGkhgc0kibapJF0uEY3mIC0sk+kIHRlI0Q1xHF81+4+fT5otRvu1UT6gtRHNXdqC4bjgo76wZriXg3iwxb8WROXpUmrwqvF1DToIshwiVYeq62LFS0f8BFcBsSifYkdt0p8go/3+epVdIXDCRFX/mlugAMEDVVk3XzyDlhDQj1MqasRIBYmrcFJTpaUw5mZtUxfLsc59RZQ2oRIUh3q8WXyBYKnc3lee6giDptmu29bxH9IvPG1ZEimFzi+pQL0B2XlpJ6XWH5KVESFjKj/CJKmZWs/45+e6LpO/XE2Q86+6JKNVC6im3evLYTEQJ9scCfoyxkvtTrfsLYurgKvOFL+VRVGKs1A+v6UwuEnjhdB6ercbXOSC2gvqg6Jb/13uwNR2QZBSaavjHp2pLq2P3RWTSNp9fS8vJBg7nJi9+dJKZXSx692uciPyvoFi0IuIF2Br4bQWc+EUrgRfdKJbNtv7yy9gNzgu/6LMo9g1OjtzD3feDzjjX5oFf7vT3BSeB3WBuHQg1lElDfTKvUFhw8W6r3kWL1cdZyjwpdMzCUpMu8e1OXroCClQ4/LLigI7EDJRohixiDvkfcsie9MJ5vvOcIFAzvgN17Ccvk8sjtW67MqzwwLWkJse4Ln3PYRlIZnX8/jwkdj9pNYG085wsEPXXjDmCI4/jcqp0kLOBIgkXbAhZREivNqs6aQux26ZsVnQfR4stiZwyIJnuy9AvDBAjGy3IxS/6w9yjCRF/7xK1wbNCEdaQaAWp3Vonf3O1AgguUN1 zAXDts36 5Qf1SnncDDcbB2CCb+v/27B1rkej+bA5iQo8Wc8cSNJXpCg0JSzQ1xapr/JTCVrv7ccdiXNtX8ifXWyHMuzpvr0u39hEXIOX4C8CqfCzN4gFvGFJSamKlQMaEbNsUwKJ/KAvVffW+xn0dQ3B8nGYbAU2wFNz0cz1TjIgS+8prc7aC2D8PWWp6AEeWhGwq0UmZu0KNvfNb1mMMVs0eIHuUP6SwgvXxdgo4jlaFypGTUDKMU6ZSN8PfvUh+g4qxlsLOPUNTkJFVlhk4rJXUjUYNtJrMFIWIN+8KgDkmcskY7GiFSx3FOg2keXuBzz0Z7g16E0iS X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, May 1, 2025 at 7:24=E2=80=AFAM Zi Yan wrote: > > On 1 May 2025, at 1:25, Juan Yescas wrote: > > > Problem: On large page size configurations (16KiB, 64KiB), the CMA > > alignment requirement (CMA_MIN_ALIGNMENT_BYTES) increases considerably, > > and this causes the CMA reservations to be larger than necessary. > > This means that system will have less available MIGRATE_UNMOVABLE and > > MIGRATE_RECLAIMABLE page blocks since MIGRATE_CMA can't fallback to the= m. > > > > The CMA_MIN_ALIGNMENT_BYTES increases because it depends on > > MAX_PAGE_ORDER which depends on ARCH_FORCE_MAX_ORDER. The value of > > ARCH_FORCE_MAX_ORDER increases on 16k and 64k kernels. > > > > For example, the CMA alignment requirement when: > > > > - CONFIG_ARCH_FORCE_MAX_ORDER default value is used > > - CONFIG_TRANSPARENT_HUGEPAGE is set: > > > > PAGE_SIZE | MAX_PAGE_ORDER | pageblock_order | CMA_MIN_ALIGNMENT_BYTES > > ----------------------------------------------------------------------- > > 4KiB | 10 | 10 | 4KiB * (2 ^ 10) =3D 4= MiB > > 16Kib | 11 | 11 | 16KiB * (2 ^ 11) =3D 32= MiB > > 64KiB | 13 | 13 | 64KiB * (2 ^ 13) =3D 512= MiB > > > > There are some extreme cases for the CMA alignment requirement when: > > > > - CONFIG_ARCH_FORCE_MAX_ORDER maximum value is set > > - CONFIG_TRANSPARENT_HUGEPAGE is NOT set: > > - CONFIG_HUGETLB_PAGE is NOT set > > > > PAGE_SIZE | MAX_PAGE_ORDER | pageblock_order | CMA_MIN_ALIGNMENT_BYTES > > -----------------------------------------------------------------------= - > > 4KiB | 15 | 15 | 4KiB * (2 ^ 15) =3D 128= MiB > > 16Kib | 13 | 13 | 16KiB * (2 ^ 13) =3D 128= MiB > > 64KiB | 13 | 13 | 64KiB * (2 ^ 13) =3D 512= MiB > > > > This affects the CMA reservations for the drivers. If a driver in a > > 4KiB kernel needs 4MiB of CMA memory, in a 16KiB kernel, the minimal > > reservation has to be 32MiB due to the alignment requirements: > > > > reserved-memory { > > ... > > cma_test_reserve: cma_test_reserve { > > compatible =3D "shared-dma-pool"; > > size =3D <0x0 0x400000>; /* 4 MiB */ > > ... > > }; > > }; > > > > reserved-memory { > > ... > > cma_test_reserve: cma_test_reserve { > > compatible =3D "shared-dma-pool"; > > size =3D <0x0 0x2000000>; /* 32 MiB */ > > ... > > }; > > }; > > > > Solution: Add a new config ARCH_FORCE_PAGE_BLOCK_ORDER that > > allows to set the page block order. The maximum page block > > order will be given by ARCH_FORCE_MAX_ORDER. > > Why not use a boot time parameter to change page block order? That is a good option. The main tradeoff is: - The bootloader would have to be updated on the devices to pass the right pageblock_order value depending on the kernel page size. Currently, We can boot 4k/16k kernels without any change in the bootloader. > Otherwise, you will need to maintain an additional kernel > binary for your use case. > Unfortunately, we still need 2 kernel binaries, one for 4k and another for = 16k. There are several data structures that are aligned at compile time based on= the PAGE_SIZE (__aligned(PAGE_SIZE)) that makes it difficult to have only one binary. For example: static u8 idmap_ptes[IDMAP_LEVELS - 1][PAGE_SIZE] __aligned(PAGE_SIZE) __ro_after_init, kpti_ptes[IDMAP_LEVELS - 1][PAGE_SIZE] __aligned(PAGE_SIZE) __ro_after_ini= t; https://elixir.bootlin.com/linux/v6.14.4/source/arch/arm64/mm/mmu.c#L780 Thanks Juan > -- > Best Regards, > Yan, Zi