From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 340BCC432C3 for ; Thu, 21 Nov 2019 16:35:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BA9A520692 for ; Thu, 21 Nov 2019 16:35:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=xenosoft.de header.i=@xenosoft.de header.b="XO6yh2Db" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BA9A520692 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xenosoft.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4D3796B0353; Thu, 21 Nov 2019 11:35:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 484206B0358; Thu, 21 Nov 2019 11:35:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 34CCC6B035B; Thu, 21 Nov 2019 11:35:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0133.hostedemail.com [216.40.44.133]) by kanga.kvack.org (Postfix) with ESMTP id 1AE176B0353 for ; Thu, 21 Nov 2019 11:35:08 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id C43068249980 for ; Thu, 21 Nov 2019 16:35:07 +0000 (UTC) X-FDA: 76180834254.30.wine07_f7674074b616 X-HE-Tag: wine07_f7674074b616 X-Filterd-Recvd-Size: 13549 Received: from mo4-p01-ob.smtp.rzone.de (mo4-p01-ob.smtp.rzone.de [85.215.255.51]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Thu, 21 Nov 2019 16:35:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1574354104; s=strato-dkim-0002; d=xenosoft.de; h=In-Reply-To:Date:Message-ID:From:References:Cc:To:Subject: X-RZG-CLASS-ID:X-RZG-AUTH:From:Subject:Sender; bh=MR6HbPNBWmyhl8iShSyS/i2w9Bm/K5EVySERuVVBGj4=; b=XO6yh2DbzZY+vgCzgyXv4crcqtfHJn0VGh6VjcZhiXFXV6GGlU5FIbzZXqcWB3gCef iWDrzI+GB5LkJ6FQeUcdl2YWtjtLuaHV3+tGUODGOe34r88H/zcDtbWvHaQGQloeO1fz ngIwqrDNGKguf2wLgCvdC8TbrA4JKaw/2MW9BSFRLLsOrZoDBi+4Hlpzsy2ifyYlHtmg H03A+WBJw1a5HLWnM5GtwomkBtCo236HkvrRYM9IdtTP4Opf/NK7ih2Q3Zu470wUGjUx qYqkbaSwyeRQKPRiu75W/EuOudHlJ7OOIj+82TUlonTMA+5uItboi91gtuyQN45JObG+ PWEg== X-RZG-AUTH: ":L2QefEenb+UdBJSdRCXu93KJ1bmSGnhMdmOod1DhGM4l4Hio94KKxRySfLxnHfJ+Dkjp5DdBJSrwuuqxvPgBcsBrTF1qGB6TwVFx4Pq4s7A=" X-RZG-CLASS-ID: mo00 Received: from [IPv6:2a02:8109:89c0:ebfc:bd57:573a:d50f:b5] by smtp.strato.de (RZmta 44.29.0 AUTH) with ESMTPSA id q007c8vALGYnkb1 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (curve secp521r1 with 521 ECDH bits, eq. 15360 bits RSA)) (Client did not present a certificate); Thu, 21 Nov 2019 17:34:49 +0100 (CET) Subject: Re: Bug 205201 - Booting halts if Dawicontrol DC-2976 UW SCSI board installed, unless RAM size limited to 3500M To: Robin Murphy , Christoph Hellwig Cc: linux-arch@vger.kernel.org, darren@stevens-zone.net, mad skateman , Benjamin Herrenschmidt , linux-kernel@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org, Rob Herring , paulus@samba.org, rtd2@xtra.co.nz, "contact@a-eon.com" , linuxppc-dev , nsaenzjulienne@suse.de References: <20191121072943.GA24024@lst.de> <6eec5c42-019c-a988-fc2a-cb804194683d@xenosoft.de> From: Christian Zigotzky Message-ID: Date: Thu, 21 Nov 2019 17:34:48 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------CA5FBA987424D6E471C8129F" Content-Language: de-DE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is a multi-part message in MIME format. --------------CA5FBA987424D6E471C8129F Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Am 21.11.19 um 14:33 schrieb Robin Murphy: > On 21/11/2019 12:21 pm, Christian Zigotzky wrote: >> On 21 November 2019 at 01:16 pm, Christian Zigotzky wrote: >>> On 21 November 2019 at 08:29 am, Christoph Hellwig wrote: >>>> On Sat, Nov 16, 2019 at 08:06:05AM +0100, Christian Zigotzky wrote: >>>>> /* >>>>> =C2=A0 *=C2=A0 DMA addressing mode. >>>>> =C2=A0 * >>>>> =C2=A0 *=C2=A0 0 : 32 bit addressing for all chips. >>>>> =C2=A0 *=C2=A0 1 : 40 bit addressing when supported by chip. >>>>> =C2=A0 *=C2=A0 2 : 64 bit addressing when supported by chip, >>>>> =C2=A0 *=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 limited to 16 segments of 4 = GB -> 64 GB max. >>>>> =C2=A0 */ >>>>> #define=C2=A0=C2=A0 SYM_CONF_DMA_ADDRESSING_MODE=20 >>>>> CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE >>>>> >>>>> Cyrus config: >>>>> >>>>> CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=3D1 >>>>> >>>>> I will configure =E2=80=9C0 : 32 bit addressing for all chips=E2=80= =9D for the=20 >>>>> RC8. Maybe this is the solution. >>>> 0 means you are going to do bounce buffering a lot, which seems >>>> generally like a bad idea. >>>> >>>> But why are we talking about the sym53c8xx driver now?=C2=A0 The las= t issue >>>> you reported was about video4linux allocations. >>>> >>> Both drivers have the same problem. They don't work if we have more=20 >>> than 3.5GB RAM. I try to find a solution until you have a good=20 >>> solution. I have already a solution for V4L but I still need one for=20 >>> the sym53c8xx driver. >> OK, you mean that "0" is a bad idea but maybe it works until you have=20 >> a solution. ;-) > > Is this on the same machine with the funny non-power-of-two=20 > bus_dma_mask as your other report? If so, does Nicolas' latest=20 > patch[1] help at all? > > Robin. > > [1]=20 > https://lore.kernel.org/linux-iommu/20191121092646.8449-1-nsaenzjulienn= e@suse.de/T/#u > Robin, I modified the patch and compiled a new RC8 of kernel 5.4 today. (patch=20 attached) We have to wait to Rolands test results with his SCSI PCI card. I tested=20 it today but my TV card doesn't work with this patch. Thanks --------------CA5FBA987424D6E471C8129F Content-Type: text/x-patch; name="dma-v1.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="dma-v1.patch" diff -rupN a/arch/powerpc/sysdev/fsl_pci.c b/arch/powerpc/sysdev/fsl_pci.c --- a/arch/powerpc/sysdev/fsl_pci.c 2019-11-17 23:47:30.000000000 +0100 +++ b/arch/powerpc/sysdev/fsl_pci.c 2019-11-21 15:32:50.216488955 +0100 @@ -115,8 +115,8 @@ static void pci_dma_dev_setup_swiotlb(st { struct pci_controller *hose = pci_bus_to_host(pdev->bus); - pdev->dev.bus_dma_mask = - hose->dma_window_base_cur + hose->dma_window_size; + pdev->dev.bus_dma_limit = + hose->dma_window_base_cur + hose->dma_window_size - 1; } static void setup_swiotlb_ops(struct pci_controller *hose) @@ -135,7 +135,7 @@ static void fsl_pci_dma_set_mask(struct * mapping that allows addressing any RAM address from across PCI. */ if (dev_is_pci(dev) && dma_mask >= pci64_dma_offset * 2 - 1) { - dev->bus_dma_mask = 0; + dev->bus_dma_limit = 0; dev->archdata.dma_offset = pci64_dma_offset; } } diff -rupN a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c --- a/drivers/iommu/dma-iommu.c 2019-11-17 23:47:30.000000000 +0100 +++ b/drivers/iommu/dma-iommu.c 2019-11-21 15:32:50.216488955 +0100 @@ -405,8 +405,7 @@ static dma_addr_t iommu_dma_alloc_iova(s if (iova_len < (1 << (IOVA_RANGE_CACHE_MAX_SIZE - 1))) iova_len = roundup_pow_of_two(iova_len); - if (dev->bus_dma_mask) - dma_limit &= dev->bus_dma_mask; + dma_limit = min_not_zero(dma_limit, dev->bus_dma_limit); if (domain->geometry.force_aperture) dma_limit = min(dma_limit, domain->geometry.aperture_end); diff -rupN a/drivers/of/device.c b/drivers/of/device.c --- a/drivers/of/device.c 2019-11-17 23:47:30.000000000 +0100 +++ b/drivers/of/device.c 2019-11-21 15:32:50.216488955 +0100 @@ -93,7 +93,7 @@ int of_dma_configure(struct device *dev, bool coherent; unsigned long offset; const struct iommu_ops *iommu; - u64 mask; + u64 mask, end; ret = of_dma_get_range(np, &dma_addr, &paddr, &size); if (ret < 0) { @@ -148,12 +148,13 @@ int of_dma_configure(struct device *dev, * Limit coherent and dma mask based on size and default mask * set by the driver. */ - mask = DMA_BIT_MASK(ilog2(dma_addr + size - 1) + 1); + end = dma_addr + size - 1; + mask = DMA_BIT_MASK(ilog2(end) + 1); dev->coherent_dma_mask &= mask; *dev->dma_mask &= mask; - /* ...but only set bus mask if we found valid dma-ranges earlier */ + /* ...but only set bus limit if we found valid dma-ranges earlier */ if (!ret) - dev->bus_dma_mask = mask; + dev->bus_dma_limit = end; coherent = of_dma_is_coherent(np); dev_dbg(dev, "device is%sdma coherent\n", diff -rupN a/include/linux/device.h b/include/linux/device.h --- a/include/linux/device.h 2019-11-17 23:47:30.000000000 +0100 +++ b/include/linux/device.h 2019-11-21 15:32:50.216488955 +0100 @@ -1186,8 +1186,8 @@ struct dev_links_info { * @coherent_dma_mask: Like dma_mask, but for alloc_coherent mapping as not all * hardware supports 64-bit addresses for consistent allocations * such descriptors. - * @bus_dma_mask: Mask of an upstream bridge or bus which imposes a smaller DMA - * limit than the device itself supports. + * @bus_dma_limit: Limit of an upstream bridge or bus which imposes a smaller + * DMA limit than the device itself supports. * @dma_pfn_offset: offset of DMA memory range relatively of RAM * @dma_parms: A low level driver may set these to teach IOMMU code about * segment limitations. @@ -1270,7 +1270,7 @@ struct device { not all hardware supports 64 bit addresses for consistent allocations such descriptors. */ - u64 bus_dma_mask; /* upstream dma_mask constraint */ + u64 bus_dma_limit; /* upstream dma constraint */ unsigned long dma_pfn_offset; struct device_dma_parameters *dma_parms; diff -rupN a/include/linux/dma-direct.h b/include/linux/dma-direct.h --- a/include/linux/dma-direct.h 2019-11-17 23:47:30.000000000 +0100 +++ b/include/linux/dma-direct.h 2019-11-21 15:37:40.091564417 +0100 @@ -28,7 +28,7 @@ static inline bool dma_capable(struct de return false; return addr + size - 1 <= - min_not_zero(*dev->dma_mask, dev->bus_dma_mask); + min_not_zero(*dev->dma_mask, dev->bus_dma_limit); } #endif /* !CONFIG_ARCH_HAS_PHYS_TO_DMA */ diff -rupN a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h --- a/include/linux/dma-mapping.h 2019-11-17 23:47:30.000000000 +0100 +++ b/include/linux/dma-mapping.h 2019-11-21 15:32:50.220488949 +0100 @@ -693,7 +693,7 @@ static inline int dma_coerce_mask_and_co */ static inline bool dma_addressing_limited(struct device *dev) { - return min_not_zero(dma_get_mask(dev), dev->bus_dma_mask) < + return min_not_zero(dma_get_mask(dev), dev->bus_dma_limit) < dma_get_required_mask(dev); } diff -rupN a/kernel/dma/direct.c b/kernel/dma/direct.c --- a/kernel/dma/direct.c 2019-11-17 23:47:30.000000000 +0100 +++ b/kernel/dma/direct.c 2019-11-21 15:50:09.570609847 +0100 @@ -27,10 +27,10 @@ static void report_addr(struct device *d { if (!dev->dma_mask) { dev_err_once(dev, "DMA map on device without dma_mask\n"); - } else if (*dev->dma_mask >= DMA_BIT_MASK(32) || dev->bus_dma_mask) { + } else if (*dev->dma_mask >= DMA_BIT_MASK(32) || dev->bus_dma_limit) { dev_err_once(dev, - "overflow %pad+%zu of DMA mask %llx bus mask %llx\n", - &dma_addr, size, *dev->dma_mask, dev->bus_dma_mask); + "overflow %pad+%zu of DMA mask %llx bus limit %llx\n", + &dma_addr, size, *dev->dma_mask, dev->bus_dma_limit); } WARN_ON_ONCE(1); } @@ -51,15 +51,14 @@ u64 dma_direct_get_required_mask(struct } static gfp_t __dma_direct_optimal_gfp_mask(struct device *dev, u64 dma_mask, - u64 *phys_mask) + u64 *phys_limit) { - if (dev->bus_dma_mask && dev->bus_dma_mask < dma_mask) - dma_mask = dev->bus_dma_mask; + u64 dma_limit = min_not_zero(dma_mask, dev->bus_dma_limit); if (force_dma_unencrypted(dev)) - *phys_mask = __dma_to_phys(dev, dma_mask); + *phys_limit = __dma_to_phys(dev, dma_limit); else - *phys_mask = dma_to_phys(dev, dma_mask); + *phys_limit = dma_to_phys(dev, dma_limit); /* * Optimistically try the zone that the physical address mask falls @@ -69,9 +68,9 @@ static gfp_t __dma_direct_optimal_gfp_ma * Note that GFP_DMA32 and GFP_DMA are no ops without the corresponding * zones. */ - if (*phys_mask <= DMA_BIT_MASK(ARCH_ZONE_DMA_BITS)) + if (*phys_limit <= DMA_BIT_MASK(ARCH_ZONE_DMA_BITS)) return GFP_DMA; - if (*phys_mask <= DMA_BIT_MASK(32)) + if (*phys_limit <= DMA_BIT_MASK(32)) return GFP_DMA32; return 0; } @@ -79,7 +78,7 @@ static gfp_t __dma_direct_optimal_gfp_ma static bool dma_coherent_ok(struct device *dev, phys_addr_t phys, size_t size) { return phys_to_dma_direct(dev, phys) + size - 1 <= - min_not_zero(dev->coherent_dma_mask, dev->bus_dma_mask); + min_not_zero(dev->coherent_dma_mask, dev->bus_dma_limit); } struct page *__dma_direct_alloc_pages(struct device *dev, size_t size, @@ -88,7 +87,7 @@ struct page *__dma_direct_alloc_pages(st size_t alloc_size = PAGE_ALIGN(size); int node = dev_to_node(dev); struct page *page = NULL; - u64 phys_mask; + u64 phys_limit; if (attrs & DMA_ATTR_NO_WARN) gfp |= __GFP_NOWARN; @@ -96,7 +95,7 @@ struct page *__dma_direct_alloc_pages(st /* we always manually zero the memory once we are done: */ gfp &= ~__GFP_ZERO; gfp |= __dma_direct_optimal_gfp_mask(dev, dev->coherent_dma_mask, - &phys_mask); + &phys_limit); page = dma_alloc_contiguous(dev, alloc_size, gfp); if (page && !dma_coherent_ok(dev, page_to_phys(page), size)) { dma_free_contiguous(dev, page, alloc_size); @@ -110,7 +109,7 @@ again: page = NULL; if (IS_ENABLED(CONFIG_ZONE_DMA32) && - phys_mask < DMA_BIT_MASK(64) && + phys_limit < DMA_BIT_MASK(64) && !(gfp & (GFP_DMA32 | GFP_DMA))) { gfp |= GFP_DMA32; goto again; diff -rupN a/arch/powerpc/include/asm/dma-direct.h b/arch/powerpc/include/asm/dma-direct.h --- a/arch/powerpc/include/asm/dma-direct.h 2019-11-17 23:47:30.000000000 +0100 +++ b/arch/powerpc/include/asm/dma-direct.h 2019-11-21 16:18:13.316815445 +0100 @@ -8,7 +8,7 @@ static inline bool dma_capable(struct de return false; return addr + size - 1 <= - min_not_zero(*dev->dma_mask, dev->bus_dma_mask); + min_not_zero(*dev->dma_mask, dev->bus_dma_limit); } static inline dma_addr_t __phys_to_dma(struct device *dev, phys_addr_t paddr) --------------CA5FBA987424D6E471C8129F--