From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE7B6E7B61D for ; Wed, 4 Oct 2023 13:50:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C6848D0072; Wed, 4 Oct 2023 09:50:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 176AD8D0027; Wed, 4 Oct 2023 09:50:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 03E148D0072; Wed, 4 Oct 2023 09:50:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E97A18D0027 for ; Wed, 4 Oct 2023 09:50:10 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 98E53A01F5 for ; Wed, 4 Oct 2023 13:50:10 +0000 (UTC) X-FDA: 81307912980.14.2A30A5C Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf05.hostedemail.com (Postfix) with ESMTP id AD657100013 for ; Wed, 4 Oct 2023 13:50:07 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696427408; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=efeGnsHXSYGspR6SFKewCWZMuVVRS4tREK0SH4ke6sU=; b=XOG865TtuPcsO31uny58ViH/gaLYbicaP+8EW+vwpZ4dQdLP5DzXKCjZQ5RGlcMv6/qBAD qqedHY+wU0u6vWS3dVuyGLkrVPSTGaP8nL6iQUQ6LghE6RvZMseaQ6+6ARwNsajqX7IFDd nNIWA7aGMCtI1R1Gaa8Q943jbIVSpLk= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696427408; a=rsa-sha256; cv=none; b=dJ8tutcIWT0BPshN+XQPZe1uePRv+hWUWGXN3vSP6VPDGqeGEigZIK5X9GAf+WxBg841NF 7PHAJo0DENG+jBIdhEMG4AddS1x6zOLq1rLV8UdVIiTG8zfEc66rmUlRPVSlH8qki1lmzc FNJ5+gXU9s5psHE3VUNSpsd42gheUGE= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2008CDA7; Wed, 4 Oct 2023 06:50:45 -0700 (PDT) Received: from [10.1.196.40] (e121345-lin.cambridge.arm.com [10.1.196.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 591733F59C; Wed, 4 Oct 2023 06:50:03 -0700 (PDT) Message-ID: Date: Wed, 4 Oct 2023 14:49:56 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux aarch64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 Subject: Re: [PATCH 0/6] RISC-V: Add eMMC support for TH1520 boards Content-Language: en-GB To: "Lad, Prabhakar" , Jisheng Zhang Cc: Drew Fustini , Christoph Hellwig , Lad Prabhakar , Robert Nelson , Ulf Hansson , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Adrian Hunter , Guo Ren , Fu Wei , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Jason Kridner , Xi Ruoyao , Han Gao , Icenowy Zheng , linux-mmc@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= , Alexandre Ghiti , Linux-MM References: <20230921-th1520-mmc-v1-0-49f76c274fb3@baylibre.com> From: Robin Murphy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: AD657100013 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 47ir6q6j3g1ixq75a878jo3gqhtghpyw X-HE-Tag: 1696427407-733603 X-HE-Meta: U2FsdGVkX1/YZq2IgISt/W+B1p89xT8Ts8v8xaQadYsh1F8HgrYcCG9q3YdL9EEOzCz8/s3677wGFjmslX/bd1vDdfdG/nQnGS/DFArph/VM0Kh5/HSuji9d665uaIG6Mv4sde/3XHNab6qRLGz8CifT4LjIANw+fuqhbWWyFOEHqwsE3JrIWJg/j3qik0y2El09CGhOAQ0wFWD9O3Tjeky2jjyTSNjM3CQdb2pgRnUTbHZFEcYRu7GjfR4pLY6QbrlSvE+4byTtnWiShZaVsEdZ5x338P2SEwVlE2/I/pII9JMuck3YYadOpL7HuNEXqmoIbKQpq9GCO8mmldKr1NgRVZ737+qtTbceWWAab8XZDtifxEbNuG4WzfadC5e2SKPMqNX/VyuLLXZgX6pA2NtoeTZgGUzd5XfNnyfHEBJbp7VMz8nmYyqdgkP89HQR8EGc12q0/JQbRkUieymKicur1R3IfGgzPd8byHQW1sVYJg4WwaW2e9gNbRifWISpw9RFg8c0sRSkYRER88QlShQp6ot5XZ5ioQmeNPyTKn71HrIgFUk6hEMuzixfEmWz7dkK5viAF1p+A4ZUnawD9jRJws3RVRffTrvPSIEwYGObooBYV7DFEee6NrnyUcMyoh9/wwRz/7FcI05vcU6awe6sILLeEXHhFmkbiiJRyCPnvG7DXdAQiCGtbMMgVDh/Ngg5kfUijunkit8rOm2/pMWIlKN0v3X5hSCiOjhOl6tvKl6KaC+TATBoqzgq8Hv7QplbbZ6QuZuL8QNmW0y+V1WI2Q0nztGrhPZ2pQgFtkG9QAhzOQPu8VKkwE2JVgd0HyhT1uSj1VjPU/Uclj3PhSm0RKFaI06arvsGSGDbQEPsbk3LsAY4/gPO9lQHzpGDnGqgU2c8b+6iMDAhukhTTsCGFWSxRb57HMJqiIfPkMBI5CizAgqucJz0rxXiPMGmAS4BqCPbToDoidmYn8J /zKTf/NM 0+ZZgKSkUNAOu06TXGplNW2Lytxkm8Nz04hjsLY8/cGVUihfkLLQZxbn3Z71d9z78Hy673GGDzyS0aoyGgC/RvGBmDRarzFBj2EsGITmjF6DOpGoqft9A8I99dS9zJpY8aohGrQ3jfhKtZlAcl2bUtSh7s7aeuSOMzgbbS1VocoNeZIXf1YaQOaGEc+o83DLjVULUHyyNM0ynvrRivx5X/tmlHSUyJ7MU/6wZuuuz7U/hsJ9NvqxXVUel+MHflfq4yOTGoMTWJRTadef/Tugo3sbHrc1vouuoOxWrVDunOHZV+isp9Ukihcag1QzfDLjQxmoLdyvlCUsyy1nzTwP1dkz1gaXDkWIvmbhB1ILSbMeYidI1XeNDdfCyDF8JBdUod/kVVe4UUPi9DzbkZcAdtibPDNKVeFnhNvpt X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 04/10/2023 2:02 pm, Lad, Prabhakar wrote: > + CC linux-mm and Robin Murphy > > On Wed, Oct 4, 2023 at 12:42 PM Jisheng Zhang wrote: >> >> On Mon, Oct 02, 2023 at 09:37:44PM -0700, Drew Fustini wrote: >>> On Fri, Sep 22, 2023 at 05:48:21PM -0500, Robert Nelson wrote: >>>> On Fri, Sep 22, 2023 at 2:08 PM Robert Nelson wrote: >>>>> >>>>> On Thu, Sep 21, 2023 at 8:51 PM Drew Fustini wrote: >>>>>> >>>>>> This series adds support for the eMMC on the BeagleV Ahead and the >>>>>> Sipeed LicheePi 4A. This allows the kernel to boot with the rootfs on >>>>>> eMMC. >>>>>> >>>>>> I tested on top of v6.6-rc2 with this config [1]. I was able to boot >>>>>> both the Ahead [2] and LPi4a [3] from eMMC. The following prerequisites >>>>>> are required: >>>>>> >>>>>> [PATCH v2] riscv: dts: thead: set dma-noncoherent to soc bus [4] >>>>>> >>>>>> I pushed a branch [5] with this patch series and the above patch for >>>>>> those that find a git branch easier to test. >>>>>> >>>>>> Please note that only the MMC controller connected to the eMMC device >>>>>> is enabled in the device trees for these two boards. I did not yet >>>>>> attempt to configure and use the microSD card slot. My preference is to >>>>>> address that in a future patch series. >>>>>> >>>>>> References: >>>>>> [1] https://gist.github.com/pdp7/5fbdcf2a65eb1abdd3a29d519c19cdd2 >>>>>> [2] https://gist.github.com/pdp7/91a801a5f8d1070c53509eda9800ad78 >>>>>> [3] https://gist.github.com/pdp7/1445c3c991e88fd69c60165cef65726a >>>>>> [4] https://lore.kernel.org/linux-riscv/20230912072232.2455-1-jszhang@kernel.org/ >>>>>> [5] https://github.com/pdp7/linux/tree/b4/th1520-mmc >>>>> >>>>> This patchset came out very nice! >>>>> >>>>> v6.6-rc2 with Last RFC v2: >>>>> >>>>> [ 4.066630] mmc0: SDHCI controller on ffe7080000.mmc >>>>> [ffe7080000.mmc] using PIO >>>>> >>>>> debian@BeagleV:~$ sudo hdparm -tT /dev/mmcblk0 >>>>> >>>>> /dev/mmcblk0: >>>>> Timing cached reads: 1516 MB in 2.00 seconds = 758.09 MB/sec >>>>> Timing buffered disk reads: 84 MB in 3.01 seconds = 27.94 MB/sec >>>>> >>>>> vs v6.6-rc2 with this patchset: >>>>> >>>>> [ 4.096837] mmc0: SDHCI controller on ffe7080000.mmc >>>>> [ffe7080000.mmc] using DMA >>>>> >>>>> debian@BeagleV:~$ sudo hdparm -tT /dev/mmcblk0 >>>>> >>>>> /dev/mmcblk0: >>>>> Timing cached reads: 1580 MB in 2.00 seconds = 790.97 MB/sec >>>>> Timing buffered disk reads: 418 MB in 3.00 seconds = 139.11 MB/sec >>>> >>>> Drew pointed out on Slack, this was not quite right.. After more >>>> digging by Drew, CONFIG_DMA_GLOBAL_POOL is causing a DMA limitation >>>> with the multiplatform defconfig. so with, >>>> >>>> ./scripts/config --disable CONFIG_ARCH_R9A07G043 >>>> >>>> (to remove CONFIG_DMA_GLOBAL_POOL)... another 2x in buffered reads.. >>>> >>>> [ 4.059242] mmc0: SDHCI controller on ffe7080000.mmc >>>> [ffe7080000.mmc] using ADMA 64-bit >>>> >>>> debian@BeagleV:~$ sudo hdparm -tT /dev/mmcblk0 >>>> >>>> /dev/mmcblk0: >>>> Timing cached reads: 1600 MB in 2.00 seconds = 800.93 MB/sec >>>> Timing buffered disk reads: 892 MB in 3.00 seconds = 297.06 MB/sec >>> >>> It seems CONFIG_DMA_GLOBAL_POOL=y causes ADMA buffer alloc to fail [1]: >>> >>> mmc0: Unable to allocate ADMA buffers - falling back to standard DMA >>> >>> Prabhakar's AX45MP non-coherent DMA support [2] series introduced the >>> selection of DMA_GLOBAL_POOL for ARCH_R9A07G043 and the riscv defconfig >>> selects ARCH_R9A07G043. >>> >>> Patch 5 in the series [3] states that: >>> >>> With DMA_GLOBAL_POOL enabled all DMA allocations happen from this >>> region and synchronization callbacks are implemented to synchronize >>> when doing DMA transactions. >>> >>> This example of a "shared-dma-pool" node was given: >>> >>> pma_resv0@58000000 { >>> compatible = "shared-dma-pool"; >>> reg = <0x0 0x58000000 0x0 0x08000000>; >>> no-map; >>> linux,dma-default; >>> }; >>> >>> I've copied that to th1520-beaglev-ahead.dts. The address of 0x58000000 >>> has no significance on th1520, but the existence of shared-dma-pool >>> seems to fix the problem. ADMA mode [4] is now working even though >>> CONFIG_DMA_GLOBAL_POOL=y. >> >> + Christoph, Lad >> >> IMHO, this is not TH1520 specific but a generic issue. >> >> I believe commit 484861e09f3e ("soc: renesas: Kconfig: Select the >> required configs for RZ/Five SoC") can cause regression on all >> non-dma-coherent riscv platforms with generic defconfig. This is >> a common issue. The logic here is: generic riscv defconfig selects >> ARCH_R9A07G043 which selects DMA_GLOBAL_POOL, which assumes all >> non-dma-coherent riscv platforms have a dma global pool, this assumption >> seems not correct. And I believe DMA_GLOBAL_POOL should not be >> selected by ARCH_SOCFAMILIY, instead, only ARCH under some specific >> conditions can select it globaly, for example NOMMU ARM and so on. >> >> Since this is a regression, what's proper fix? any suggestion is >> appreciated. I think the answer is to not select DMA_GLOBAL_POOL, since that is only designed for nommu cases where non-cacheable memory lives in a fixed place in the physical address map, and regular kernel pages can't be remapped. As far as I'm aware, RISCV_DMA_NONCOHERENT is the thing you want, such that DMA_DIRECT_REMAP can dynamically provide non-cacheable coherent buffers for non-hardware-coherent devices. Thanks, Robin. >> >> Thanks >> >>> >>> Thanks, >>> Drew >>> >>> [1] https://gist.github.com/pdp7/73041ed808bbc7dd445836fb90574979 >>> [2] https://lore.kernel.org/linux-riscv/20230818135723.80612-1-prabhakar.mahadev-lad.rj@bp.renesas.com/ >>> [3] https://lore.kernel.org/linux-riscv/20230818135723.80612-6-prabhakar.mahadev-lad.rj@bp.renesas.com/ >>> [4] https://gist.github.com/pdp7/91e72a663d3bb73eb28182337ad8bbcb