From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DC9BCF9C5B for ; Mon, 23 Sep 2024 09:46:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B277A6B0085; Mon, 23 Sep 2024 05:46:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ADD116B0088; Mon, 23 Sep 2024 05:46:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99EAF6B0089; Mon, 23 Sep 2024 05:46:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 7F7E56B0085 for ; Mon, 23 Sep 2024 05:46:33 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2EA56411C9 for ; Mon, 23 Sep 2024 09:46:33 +0000 (UTC) X-FDA: 82595523066.12.891E073 Received: from imap4.hz.codethink.co.uk (imap4.hz.codethink.co.uk [188.40.203.114]) by imf19.hostedemail.com (Postfix) with ESMTP id 39E631A0007 for ; Mon, 23 Sep 2024 09:46:30 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=codethink.co.uk header.s=imap4-20230908 header.b=ZRWpPwx1; spf=pass (imf19.hostedemail.com: domain of ben.dooks@codethink.co.uk designates 188.40.203.114 as permitted sender) smtp.mailfrom=ben.dooks@codethink.co.uk; dmarc=pass (policy=reject) header.from=codethink.co.uk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727084635; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=M4vBfdM0E+Q1Eqn0D2bifpyInvcBhY39m5Q1LBNvNiI=; b=scuHvQjc5kRXpDHCRKe88IGWTjRXZoWovXGUHRLuMbJRaSQIJPN5dEPMQuM/+m6hZijjR1 SDJGstiUYTStb4vB1EirbJXRBFa/HByVh1NPdhJNRKaJoiSaUl2QnOZw/ajcrnIbZ8J4I4 7YLPWO53VBfsIIitxf1XkpyAAhGgvUM= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=codethink.co.uk header.s=imap4-20230908 header.b=ZRWpPwx1; spf=pass (imf19.hostedemail.com: domain of ben.dooks@codethink.co.uk designates 188.40.203.114 as permitted sender) smtp.mailfrom=ben.dooks@codethink.co.uk; dmarc=pass (policy=reject) header.from=codethink.co.uk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727084635; a=rsa-sha256; cv=none; b=XZcWlrIuRfq6TB4Cd/zLfoQjlqDmNNvBL2j/+0ZSlOnJGEmpZm9Jmdv0sSXN5EnwyDwjxk CiCAWpLbscZlHoXWaWS56pOyhUintrecDL1S4Wo3MBAyn60ZHVfpZX8jR2fumu1IATDPaH FoHDmU5YCVXvj4J9M/6t54Ihpiqm/Ik= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=codethink.co.uk; s=imap4-20230908; h=Sender:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:Cc:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=M4vBfdM0E+Q1Eqn0D2bifpyInvcBhY39m5Q1LBNvNiI=; b=ZRWpPwx15apUUlQepYIIJSHtdD X8nDeTPMqwoXUHxEIjHY1E908ND0DWfxNjbqmTRVb0rN4Z80y5jqu/WCZy1oN+2mRd7G42FxGqH4B bcONh82F6NIXyqTw2dD9yYM4hFV+pDMd6SpEGXg2xa7cb7N1267m9/nVtf+YaF1s7x0eXDhwYBcAa HnD9xHHg6rtecBYrpksR4mb2huHwju+FeTaZUykTLldykV/O6AH2yWJORlnFGfyLue7QL/oV1mGi3 lQ8n2UGWzAPF8xz+PptuCI4CjjjW6GC3BvgP+OETSEBERI+OknKictpggopydZw5NuTjdx1ywH0PV YrYsdaYQ==; Received: from [167.98.27.226] (helo=[10.35.6.157]) by imap4.hz.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1ssfe0-002GGl-P0; Mon, 23 Sep 2024 10:46:13 +0100 Message-ID: <575583c5-5a52-461b-944a-5e32bd4b627b@codethink.co.uk> Date: Mon, 23 Sep 2024 10:46:11 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] riscv: make ZONE_DMA32 optional To: Drew Fustini , Palmer Dabbelt Cc: dfustini@tenstorrent.com, Conor Dooley , vladimir.kondratiev@mobileye.com, Paul Walmsley , aou@eecs.berkeley.edu, akpm@linux-foundation.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: Content-Language: en-GB From: Ben Dooks Organization: Codethink Limited. In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 39E631A0007 X-Stat-Signature: dxg3u3wnsxhcxe6so9hyisgctzku7rhu X-Rspam-User: X-HE-Tag: 1727084790-766305 X-HE-Meta: U2FsdGVkX1863f7zlqPxvtCm+R3zx0gFJUtT9NuJm+sclvnZcQBEpqKBZL77CwaHwoCTkOEc+vXdv1tIzEfQLAFA4TstE5CTgQbPum4ryEZDbdF+8hW7sBsKpLBx8RS9UA34Ep+K860AU4HTwwnT3Th6nNnP9l8QtP8lwTBq2W+T55My710GjqKnbQ5Bipgu8NSMAXfVokUADIfFy20m2oprlFbXRLh5zF164S2nm9yFCP79zmtoM6+TiFH0JrmYlPIC2AD9vd9pZHKfSuA5lJseXSakULHfGTR2rXTwOQzXEkkuBBWKg89btcICTw0AeXgJfjWZPykGqEuPiD8ikHOjWXUayvIKcfg5Q6xycnOGbmLXqivD2ZwQ1qwRL1LYRO18PtB0qyPd/WAg0MT+5oym7lfX+zfHEYp9Vc1fp5HraXPwJKu+/uRo8hALaI5SgF/9NwE5PwIFQz65pEnU4XZxtDUHo6OadF6t1M6qS+Dv9FNZyTCwTLl3tMVOkeW3t1x22HznOnwfxh63G5ugP1CanUhBwwa2Iihd1epM4tiap4gfc8vMNRK2WyQvM1uq/xRxjB6yY5jLHWviVrCdqR/M0vy4bBa50WEndwUQk4uW39yvZ36RaccTMjzgGpHsp9uwiL3yDuW/7hDOGXHklHxLzidcddbEZUMEQBqXbtVB9pydlps9gBhrF2MEnySW9AVmQRZ7uUJc919Lyvtb5k/m/b8yDiDToGn3veF8ClchEgGh1Z/qQe836gMYLw+6QBmIKeXe4idN0fA7wIwmdEm5jQUZJ+ZII+7+NJWVnlXQ8r3O3gNRf5wjvdJXJ9vN6rAAuYsDEUYO5o2r9CW/1g+C+uYTBgptIeKiv3dZwMDJV7tfgziqMCzl4K/a6B/s8TqRZa36577LGMhj/hOFSRpdit0Cf4BmD/VBv9rmEqLbtamjDjsmr9Ly3H1W6jUpjR3qFG8e4yDN6xiSljs oehRu9KA m35zwvyp89r2SmeAdJMJfJY0aVWpy0eYpqjHFt6arGVGnWoAhvxqC5PPsLDWo3GIwmWdQ37MvjXR9otw58bP0JPT6MCre8dgk2f1SaZesXLujm49H0UNrj8GeNdY9dLhejLmIMvA2EPM0JtxodXnioW2TramjZ96vmcfIr1rct6Xsm9Hb5DcyEUoOn0Jq0LQcpDY+AIAyiw9lTOVjwc1GR8Pe5xJnmcF6gBcG+ymJlslQzpnXBNgX7hmh1MYhIMGHmKp4PUXZcNLBiznN0zgTMA1fZ4i97xnDohn+IiaXOq3TK2s3Hdfpe75Dut5paeOGoqSKjfb8EIXBcPz7WtorX5KP5G23C2cZROEhb1KUWOrTp4zVy7BmlH2iXIY4NC61mJBrzv99hbEX7bLVXuFv00JDmYB2iNvC2mslDK8zr0B5C1efC6XqEiTcPM2V7VBPJ1U5g/Us9ktsWkkMlB+3Yt0PTaYNvHyzZ0MPXPOlJab0fL6Galbsw6WB9UokTGe9vW0GlApdiwUDzqLAESA4IHsAhjGs86YRqlaEge4VFlii/W0RCCfEJ4fUE+QAN95SASBQE1F1TZbj75xaarV4eGtPmt8f2KDWzar6Dmcry96Jcs4qDLOhPKxRe79IWfF7wi2T X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 20/09/2024 14:18, Drew Fustini wrote: > On Fri, Sep 20, 2024 at 01:58:23AM -0700, Palmer Dabbelt wrote: >> On Tue, 27 Aug 2024 16:10:20 PDT (-0700), dfustini@tenstorrent.com wrote: >>> On Tue, Aug 27, 2024 at 02:36:11PM +0300, Vladimir Kondratiev wrote: >>>> It is not necessary any RISCV platform has ZONE_DMA32. >>>> >>>> Example - if platform has no DRAM in [0..4G] region, >>>> it will report failure like below each boot. >>>> >>>> [ 0.088709] swapper/0: page allocation failure: order:7, mode:0xcc4(GFP_KERNEL|GFP_DMA32), nodemask=(null),cpuset=/ >>>> [ 0.088832] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.11.0-rc5 #30 >>>> [ 0.088864] Call Trace: >>>> [ 0.088869] [] dump_backtrace+0x1c/0x24 >>>> [ 0.088910] [] show_stack+0x2c/0x38 >>>> [ 0.088957] [] dump_stack_lvl+0x52/0x74 >>>> [ 0.088987] [] dump_stack+0x14/0x1c >>>> [ 0.089010] [] warn_alloc+0xf4/0x176 >>>> [ 0.089041] [] __alloc_pages_noprof+0xc28/0xcb4 >>>> [ 0.089067] [] atomic_pool_expand+0x62/0x1f8 >>>> [ 0.089090] [] __dma_atomic_pool_init+0x46/0x9e >>>> [ 0.089115] [] dma_atomic_pool_init+0x96/0x11c >>>> [ 0.089139] [] do_one_initcall+0x5c/0x1b2 >>>> [ 0.089158] [] kernel_init_freeable+0x214/0x274 >>>> [ 0.089190] [] kernel_init+0x1e/0x10a >>>> [ 0.089209] [] ret_from_fork+0xe/0x1c >>>> >>>> Signed-off-by: Vladimir Kondratiev >>>> --- >>>> arch/riscv/Kconfig | 2 +- >>>> mm/Kconfig | 2 +- >>>> 2 files changed, 2 insertions(+), 2 deletions(-) >>>> >>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig >>>> index 0f3cd7c3a436..94a573112625 100644 >>>> --- a/arch/riscv/Kconfig >>>> +++ b/arch/riscv/Kconfig >>>> @@ -50,6 +50,7 @@ config RISCV >>>> select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST >>>> select ARCH_HAS_UBSAN >>>> select ARCH_HAS_VDSO_DATA >>>> + select ARCH_HAS_ZONE_DMA_SET if 64BIT >>>> select ARCH_KEEP_MEMBLOCK if ACPI >>>> select ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE if 64BIT && MMU >>>> select ARCH_OPTIONAL_KERNEL_RWX if ARCH_HAS_STRICT_KERNEL_RWX >>>> @@ -200,7 +201,6 @@ config RISCV >>>> select THREAD_INFO_IN_TASK >>>> select TRACE_IRQFLAGS_SUPPORT >>>> select UACCESS_MEMCPY if !MMU >>>> - select ZONE_DMA32 if 64BIT >>>> >>>> config CLANG_SUPPORTS_DYNAMIC_FTRACE >>>> def_bool CC_IS_CLANG >>>> diff --git a/mm/Kconfig b/mm/Kconfig >>>> index b72e7d040f78..97c85da98e89 100644 >>>> --- a/mm/Kconfig >>>> +++ b/mm/Kconfig >>>> @@ -1032,7 +1032,7 @@ config ZONE_DMA >>>> config ZONE_DMA32 >>>> bool "Support DMA32 zone" if ARCH_HAS_ZONE_DMA_SET >>>> depends on !X86_32 >>>> - default y if ARM64 >>>> + default y if ARM64 || (RISCV && 64BIT) >>>> >>>> config ZONE_DEVICE >>>> bool "Device memory (pmem, HMM, etc...) hotplug support" >>>> -- >>>> 2.37.3 >>>> >>> >>> Reviewed-by: Drew Fustini >>> >>> Thanks for sending this patch as I've also encountered that annoying >>> error on systems with DRAM above 4GB. >>> >>> I tested this patch by changing the qemu virt machine to have DRAM >>> starting at 2^32: >>> >>> diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c >>> index cef41c150aaf..3033a2560edb 100644 >>> --- a/hw/riscv/virt.c >>> +++ b/hw/riscv/virt.c >>> @@ -87,7 +87,7 @@ static const MemMapEntry virt_memmap[] = { >>> [VIRT_IMSIC_S] = { 0x28000000, VIRT_IMSIC_MAX_SIZE }, >>> [VIRT_PCIE_ECAM] = { 0x30000000, 0x10000000 }, >>> [VIRT_PCIE_MMIO] = { 0x40000000, 0x40000000 }, >>> - [VIRT_DRAM] = { 0x80000000, 0x0 }, >>> + [VIRT_DRAM] = { 0x100000000, 0x0 }, >>> }; >>> >>> /* PCIe high mmio is fixed for RV32 */ >> >> IIRC the ZONE_DMA32 stuff existed for some of the early SiFive systems, >> where the expansion daughterboard's PCIe controller (via a Xilinx FPGA) >> could only handle 32-bit DMA addreses. I think there's a similar quirk in >> the Microsemi PCIe controller on the PolarFire boards, but Conor would know >> for sure. The Fu740 SoC has a mirror of part of DRAM in 32bit space specifically I think for PCIe where something may be attached via a PCIe-PCI bridge. > I don't think this patch would affect those systems that need ZONE_DMA2. > I believe it just makes it possible to disable it in the kernel config. > The platform I'm working on has no memory below 4GB and all the PCIe > devices that I care about are not 32-bit constrained. Therefore I just > want to be able to turn it off in my .config. > > Thanks, > Drew I tried this a while ago and IIRC you run into issues with DMA allocations from certain drivers that expect this to exist. With new platforms that don't have any memory in DMA32 space this may become more common. I've already had to deal with this for an internal dev project, and it ended up having a couple of hacks into the allocation code to silence warnings/errors. This of course may have been fixed, given this was last year. -- Ben Dooks http://www.codethink.co.uk/ Senior Engineer Codethink - Providing Genius https://www.codethink.co.uk/privacy.html