From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFEB6E784A9 for ; Wed, 4 Oct 2023 14:18:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 394996B01B1; Wed, 4 Oct 2023 10:18:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 346536B0266; Wed, 4 Oct 2023 10:18:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20D186B0269; Wed, 4 Oct 2023 10:18:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 122DA6B01B1 for ; Wed, 4 Oct 2023 10:18:24 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D5CF3801E4 for ; Wed, 4 Oct 2023 14:18:23 +0000 (UTC) X-FDA: 81307984086.03.AEAC18B Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf10.hostedemail.com (Postfix) with ESMTP id 02DF5C0003 for ; Wed, 4 Oct 2023 14:18:19 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=GByqZwZS; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf10.hostedemail.com: domain of jszhang@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=jszhang@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696429100; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lUL2qbahLpnlP5KRqib5RQIotVTuUvK370g3RyagrsU=; b=2fAuX1+RrkUrwFdqSOqvF/sQAY5VSKthargQ1iHJueWeI6Vu1ohHPUXAkt94WQ24Bms6L0 U2qU9nKV/xiwLTEoyGTTWcyiUqFLYgmP8tuA6jUyYmo92P+VU7xHOEFEyVOE6CYo+q2Ash CZmAZrEuoEOAnPVHMSvuE9HvUC4vgAc= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=GByqZwZS; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf10.hostedemail.com: domain of jszhang@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=jszhang@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696429100; a=rsa-sha256; cv=none; b=t2xUqeISNvAUEPaGKjhHMFn+FTkLxk8JHI4ket2bIXL5ME1oXFZ1AbQbusII4DP5Ce7Gug aGPN5lCFltA7na9PRqrgAgiRDiQBfppDrRBNUns+qRsHd2gjt4mTZCLXP03XpS3z438bGZ n/pNBtDOEtZjUAJze/1d72bAobbJfFU= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 15077CE1B1E; Wed, 4 Oct 2023 14:18:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 29B4AC433C7; Wed, 4 Oct 2023 14:18:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696429094; bh=lneHq+ASicfa1NYo5LSBGIqLFAZQwUrpIzoaUPbVdX8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=GByqZwZSWTBoMUWZd4ip6aQ/g+od7RJqstH+fCYHMETJdaH9oEQIgdqkTbhCsM5Gu a5EKlOxrxtutzNM1OjVtdluHAg02Swhb6yZjLeWNiU4+3sfzJp6qVTUFZ3qONksapD WZmJgNtCPuynPvN2QJRoQl+2gsvBndMPCgqi7jHzF+GpVjYUhg93lvW12HO5Oap3j5 2o0RjZDynGQX/LPjZ4Ecizli/lir4Tws4hKkPA99LJSYaFcMLWyRx5C64yZ8gdxRdM 2GJB7bgplpmbtQJmYZfEgLv1yjPEoeE8CsvMtkzcQ6gA+T7xYFg5ZoZzf6BIKunYle gkMNFpW0Y8CFQ== Date: Wed, 4 Oct 2023 22:06:03 +0800 From: Jisheng Zhang To: Robin Murphy Cc: "Lad, Prabhakar" , Drew Fustini , Christoph Hellwig , Lad Prabhakar , Robert Nelson , Ulf Hansson , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Adrian Hunter , Guo Ren , Fu Wei , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Jason Kridner , Xi Ruoyao , Han Gao , Icenowy Zheng , linux-mmc@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, =?utf-8?B?QmrDtnJuIFTDtnBlbA==?= , Alexandre Ghiti , Linux-MM Subject: Re: [PATCH 0/6] RISC-V: Add eMMC support for TH1520 boards Message-ID: References: <20230921-th1520-mmc-v1-0-49f76c274fb3@baylibre.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 02DF5C0003 X-Stat-Signature: 5bzza59jq66rzhpddnugam4a68fz3itj X-Rspam-User: X-HE-Tag: 1696429099-319138 X-HE-Meta: U2FsdGVkX1/J/XouzBnopgg/ED++++5/yzg4qqS7/9UiWXbM3lor3rRGcGEPHoE5lT54x4bNGyWsBvykyQvFE/Zz/JuU+WkeJnBuO2+wCKaLQhkR0aCptjTksqyt23llnG5X5bYvSy+VDDX/v5F5sSwJ7FG+k449LjQ+46EmD8rlYe46BwxpSfAjE3uttYTN4lJvENBKCuIMkOqcrJPzhOUCaNIj6JKNsXnHyn4kNKHdyK47X5Jh5ErV1vr4dKbpqy4rfoAikLajdt2fcY1I+9p8WDlvJFszCiwqczfCDbHsXH8+QX8qsYRBgfDIHJzxOqzX1Vm1OL8BCGOd8DtiUTi1mBYC/XcYzra1TFX6sKmdM4v8LXuqrq+BmhugwnGjWLJ2/aaKy/CoIaAVpDDviw6jcURss81u3hAywqjK5LIkEyqZK4YBT3AYyl7b9rrn/mm2OiGO3TeFS4JK70IjLAbLYj3uAfBEDzPD+H6xjYa3s5f+EkDUytfDzmxnDQ82Pr+BpRVua2gr/E5z8IhKM+MbW4X9gCHjz37vupoKI/DJGotUkk9QAOlRzk1/58t/Z6lRXuQ7zEbGbLLuOPR/PBazzn8KR9I9Qj4SpHZ3VRkXBcN0yN9UjxCclNOZchfqdHA4dxr7x9Gof9a/wOFlvCZrMLK7XCfxkQvIkDfqLxbqoBIeUBPbTVFzGGJYnO8TsxkHOaeqrHBy2LJxpbe4aFGmKnADFeWwffEA/bpGvC5L1B6ZxhTR6crx+7hHQDnDZd5jA4trRDNOwmGY+Amn2ROnD+RYFc5x0auZo+mb4LVatrDAw9dI3quX3DAbVS0St7kKbKGGoMm6/Q9OsZqhjoDg3X9uXCYuEaAlM/dyHbF2BEq47LpNSF33S4XJko7A5NqN7MxcShJz1QAkJt/PocF2Xt1dsVssE6Rw3Unua20/2uWq7HH+JSyUInr+zz3f6xqV20XLSFgS1DcNWPp lsO55Iqp LQm1xLSVsrxyJv32FKRVZbl/xN3a5k86zqZeZ8D+Mk7qFUgb+QQXOjBhUflICCbqfBp3T8sCW4p2OBbkR0frFtFEOYvPMOmFj5mDtbairSTfMB9DxwZItTGWsuvqcU1E92i/sjDBG74aU2DFAU39lYfmZSduFXsrc8Rl50MZdBfQbCcVGEpaPPsNOWH0qQvJmvscgw8EwZdCXSuiQKMujvGOaYpX7IEJgo9qCXvbeeu9spTqmOzcZI+oOVP67+Wzs3yq+ynXGmqS6UO869TX75aZ24k4jYrjAxvMboucZhcGMnecLxqrIsa3/08CiGD3N1ybN3uXMYAliZGo1jHroZ9SXxf3u2PwH7ehMHiLaVQzd/e9z8PIF5zyKni3D/ryXJ5W+TDCx8/4el+Ssh4emwz/tmAYNSi+djcWEUl/loRyQVhP0oI4pjxfLmS0ncXDyh3bouJd611zSftNsB3w9AEl+JiwLQVhYR5g8sCK6yfZtZ8RIO55HViG1OMn4Y/ZUsQJb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Oct 04, 2023 at 02:49:56PM +0100, Robin Murphy wrote: > On 04/10/2023 2:02 pm, Lad, Prabhakar wrote: > > + CC linux-mm and Robin Murphy > > > > On Wed, Oct 4, 2023 at 12:42 PM Jisheng Zhang wrote: > > > > > > On Mon, Oct 02, 2023 at 09:37:44PM -0700, Drew Fustini wrote: > > > > On Fri, Sep 22, 2023 at 05:48:21PM -0500, Robert Nelson wrote: > > > > > On Fri, Sep 22, 2023 at 2:08 PM Robert Nelson wrote: > > > > > > > > > > > > On Thu, Sep 21, 2023 at 8:51 PM Drew Fustini wrote: > > > > > > > > > > > > > > This series adds support for the eMMC on the BeagleV Ahead and the > > > > > > > Sipeed LicheePi 4A. This allows the kernel to boot with the rootfs on > > > > > > > eMMC. > > > > > > > > > > > > > > I tested on top of v6.6-rc2 with this config [1]. I was able to boot > > > > > > > both the Ahead [2] and LPi4a [3] from eMMC. The following prerequisites > > > > > > > are required: > > > > > > > > > > > > > > [PATCH v2] riscv: dts: thead: set dma-noncoherent to soc bus [4] > > > > > > > > > > > > > > I pushed a branch [5] with this patch series and the above patch for > > > > > > > those that find a git branch easier to test. > > > > > > > > > > > > > > Please note that only the MMC controller connected to the eMMC device > > > > > > > is enabled in the device trees for these two boards. I did not yet > > > > > > > attempt to configure and use the microSD card slot. My preference is to > > > > > > > address that in a future patch series. > > > > > > > > > > > > > > References: > > > > > > > [1] https://gist.github.com/pdp7/5fbdcf2a65eb1abdd3a29d519c19cdd2 > > > > > > > [2] https://gist.github.com/pdp7/91a801a5f8d1070c53509eda9800ad78 > > > > > > > [3] https://gist.github.com/pdp7/1445c3c991e88fd69c60165cef65726a > > > > > > > [4] https://lore.kernel.org/linux-riscv/20230912072232.2455-1-jszhang@kernel.org/ > > > > > > > [5] https://github.com/pdp7/linux/tree/b4/th1520-mmc > > > > > > > > > > > > This patchset came out very nice! > > > > > > > > > > > > v6.6-rc2 with Last RFC v2: > > > > > > > > > > > > [ 4.066630] mmc0: SDHCI controller on ffe7080000.mmc > > > > > > [ffe7080000.mmc] using PIO > > > > > > > > > > > > debian@BeagleV:~$ sudo hdparm -tT /dev/mmcblk0 > > > > > > > > > > > > /dev/mmcblk0: > > > > > > Timing cached reads: 1516 MB in 2.00 seconds = 758.09 MB/sec > > > > > > Timing buffered disk reads: 84 MB in 3.01 seconds = 27.94 MB/sec > > > > > > > > > > > > vs v6.6-rc2 with this patchset: > > > > > > > > > > > > [ 4.096837] mmc0: SDHCI controller on ffe7080000.mmc > > > > > > [ffe7080000.mmc] using DMA > > > > > > > > > > > > debian@BeagleV:~$ sudo hdparm -tT /dev/mmcblk0 > > > > > > > > > > > > /dev/mmcblk0: > > > > > > Timing cached reads: 1580 MB in 2.00 seconds = 790.97 MB/sec > > > > > > Timing buffered disk reads: 418 MB in 3.00 seconds = 139.11 MB/sec > > > > > > > > > > Drew pointed out on Slack, this was not quite right.. After more > > > > > digging by Drew, CONFIG_DMA_GLOBAL_POOL is causing a DMA limitation > > > > > with the multiplatform defconfig. so with, > > > > > > > > > > ./scripts/config --disable CONFIG_ARCH_R9A07G043 > > > > > > > > > > (to remove CONFIG_DMA_GLOBAL_POOL)... another 2x in buffered reads.. > > > > > > > > > > [ 4.059242] mmc0: SDHCI controller on ffe7080000.mmc > > > > > [ffe7080000.mmc] using ADMA 64-bit > > > > > > > > > > debian@BeagleV:~$ sudo hdparm -tT /dev/mmcblk0 > > > > > > > > > > /dev/mmcblk0: > > > > > Timing cached reads: 1600 MB in 2.00 seconds = 800.93 MB/sec > > > > > Timing buffered disk reads: 892 MB in 3.00 seconds = 297.06 MB/sec > > > > > > > > It seems CONFIG_DMA_GLOBAL_POOL=y causes ADMA buffer alloc to fail [1]: > > > > > > > > mmc0: Unable to allocate ADMA buffers - falling back to standard DMA > > > > > > > > Prabhakar's AX45MP non-coherent DMA support [2] series introduced the > > > > selection of DMA_GLOBAL_POOL for ARCH_R9A07G043 and the riscv defconfig > > > > selects ARCH_R9A07G043. > > > > > > > > Patch 5 in the series [3] states that: > > > > > > > > With DMA_GLOBAL_POOL enabled all DMA allocations happen from this > > > > region and synchronization callbacks are implemented to synchronize > > > > when doing DMA transactions. > > > > > > > > This example of a "shared-dma-pool" node was given: > > > > > > > > pma_resv0@58000000 { > > > > compatible = "shared-dma-pool"; > > > > reg = <0x0 0x58000000 0x0 0x08000000>; > > > > no-map; > > > > linux,dma-default; > > > > }; > > > > > > > > I've copied that to th1520-beaglev-ahead.dts. The address of 0x58000000 > > > > has no significance on th1520, but the existence of shared-dma-pool > > > > seems to fix the problem. ADMA mode [4] is now working even though > > > > CONFIG_DMA_GLOBAL_POOL=y. > > > > > > + Christoph, Lad > > > > > > IMHO, this is not TH1520 specific but a generic issue. > > > > > > I believe commit 484861e09f3e ("soc: renesas: Kconfig: Select the > > > required configs for RZ/Five SoC") can cause regression on all > > > non-dma-coherent riscv platforms with generic defconfig. This is > > > a common issue. The logic here is: generic riscv defconfig selects > > > ARCH_R9A07G043 which selects DMA_GLOBAL_POOL, which assumes all > > > non-dma-coherent riscv platforms have a dma global pool, this assumption > > > seems not correct. And I believe DMA_GLOBAL_POOL should not be > > > selected by ARCH_SOCFAMILIY, instead, only ARCH under some specific > > > conditions can select it globaly, for example NOMMU ARM and so on. > > > > > > Since this is a regression, what's proper fix? any suggestion is > > > appreciated. > > I think the answer is to not select DMA_GLOBAL_POOL, since that is only > designed for nommu cases where non-cacheable memory lives in a fixed place > in the physical address map, and regular kernel pages can't be remapped. As > far as I'm aware, RISCV_DMA_NONCOHERENT is the thing you want, such that > DMA_DIRECT_REMAP can dynamically provide non-cacheable coherent buffers for > non-hardware-coherent devices. Thank Robin! AFAIK, ARCH_R9A07G043 needs the dma global pool to handle its CMO. So it looks like ARCH_R9A07G043 can't be enabled in riscv generic defconfig. And we also need a special solution to prevent random config from selecting ARCH_R9A07G043 by chance for other platforms Thanks > > Thanks, > Robin. > > > > > > > Thanks > > > > > > > > > > > Thanks, > > > > Drew > > > > > > > > [1] https://gist.github.com/pdp7/73041ed808bbc7dd445836fb90574979 > > > > [2] https://lore.kernel.org/linux-riscv/20230818135723.80612-1-prabhakar.mahadev-lad.rj@bp.renesas.com/ > > > > [3] https://lore.kernel.org/linux-riscv/20230818135723.80612-6-prabhakar.mahadev-lad.rj@bp.renesas.com/ > > > > [4] https://gist.github.com/pdp7/91e72a663d3bb73eb28182337ad8bbcb