From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15897C38A02 for ; Sun, 30 Oct 2022 16:43:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F13306B0071; Sun, 30 Oct 2022 12:43:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E9BD46B0073; Sun, 30 Oct 2022 12:43:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D63B26B0074; Sun, 30 Oct 2022 12:43:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C36986B0071 for ; Sun, 30 Oct 2022 12:43:30 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7D8F71205C8 for ; Sun, 30 Oct 2022 16:43:30 +0000 (UTC) X-FDA: 80078186580.03.9AD9294 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf17.hostedemail.com (Postfix) with ESMTP id F2FC44000C for ; Sun, 30 Oct 2022 16:43:28 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E712B60F0B; Sun, 30 Oct 2022 16:43:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0D946C433C1; Sun, 30 Oct 2022 16:43:24 +0000 (UTC) Date: Sun, 30 Oct 2022 16:43:21 +0000 From: Catalin Marinas To: Christoph Hellwig Cc: Greg Kroah-Hartman , Linus Torvalds , Arnd Bergmann , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH v2 2/2] treewide: Add the __GFP_PACKED flag to several non-DMA kmalloc() allocations Message-ID: References: <20221025205247.3264568-3-catalin.marinas@arm.com> <20221030084718.GC5278@lst.de> <20221030091349.GA5600@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221030091349.GA5600@lst.de> ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667148209; a=rsa-sha256; cv=none; b=1oUKXXhU6glLXbP7FDXLFN5uNeqgkplTa9a3uISrkgLH0UoHQpusgq3XEQAbIEXKYQp7TZ p5HejLD97dPstlEwqNsjfM6ZPeCLZXkwmOK3XdAasfj2o291sZc5ELauI4cwTML9Wc++91 peQvapuzCccIY2nIVCnxNi21b0FbDWA= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf17.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667148209; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pyKVMhTQMN4tOo0wEBH/c27Do0C7jhsqE6+bPFAOsCQ=; b=XSq1HYHJwNqIMo/nLSyJ2EKA4H+SHzzk+3RcIGBwwwVQ3zcl3hmVfCX9DqsLk4UFjHZ6cz uZbv4STT6hCL3e7X9OO8JHE9+Tb7DqolQnflUXexfyieaJbKHPu1mfBBw5xJSn6HKPyFsn RAieAUmyWIC1c/5jqvmQxFOI9s5Pb9U= X-Stat-Signature: h6ijts8ytb4bf31rnb6omssd6m4r1a9t X-Rspamd-Queue-Id: F2FC44000C X-Rspam-User: Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf17.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org X-Rspamd-Server: rspam09 X-HE-Tag: 1667148208-636056 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, Oct 30, 2022 at 10:13:49AM +0100, Christoph Hellwig wrote: > On Sun, Oct 30, 2022 at 10:02:53AM +0100, Greg Kroah-Hartman wrote: > > Ah, my fault, sorry, you are right. Is there a sparse tag that just > > means "enforce this void * type?" I guess we could do that with something > > like: > > typedef void *dmaptr; > > > > but that feels icky. > > That's because it is :) I find the concept of the DMA pointes pretty > strange to be honest. I've been trying to come up with what the semantics of __dma should be but couldn't get to any clear conclusion. We can avoid the noderef part as it doesn't make sense but it still implies a different address space. Once I made the ptr in dma_map_single() a 'void __dma *', sparse complained not just about the callers of this function but the callees like is_vmalloc_addr(). So I have a suspicion we'd end up with __dma annotations in lots of places unrelated to DMA or drivers. Then we have structures like 'catc' with a 'ctrl_dr' that ends up in the DMA API (so far as TO_DEVICE, so not that problematic). The alternative is for dma_map_*() to (dynamically) verify at the point of DMA setup rather than at the point of allocation (and bounce). > It only affects a subset of dma (when devices are not attached in a > DMA coherent way). So count me in as someone who would be much more > happy about figuring out a way to simplify bounce buffer for > non-coherent DMA if the start and length are not aligned to the cache > line size over any kind of special annotation all over the kernel. That's definitely less daunting if we find the right solution. The problem with checking the alignment of both start and length is that there are valid cases where the start is aligned but the length isn't. They don't need bouncing since they come from a sufficiently aligned slab. We have a few options here: a) Require that the dma_map_*() functions (including sg) get a size multiple of cache_line_size() or the static ARCH_DMA_MINALIGN. b) Always bounce small sizes as they may have come from a small kmalloc cache. The alignment of both ptr and length is ignored. This assumes that in-struct alignments (crypto) use ARCH_DMA_MINALIGN and we can reduce ARCH_KMALLOC_MINALIGN independently. c) Similar to (b) but in addition check whether the object comes from a small kmalloc cache, e.g. using ksize(). (a) has the advantage that it works by default even if drivers don't use the correct alignment. We can optimise them afterwards. However, it feels wrong to get the driver to align the length to a cache_line_size() when it doesn't control the coherency of the bus (and always worked fine on x86). (b) is what I attempted on one of my branches (until I got stuck on bouncing for iommu). A slight overhead on dma_map_*() to check the length and we may keep a check on the start alignment (not length though). With (c) we can avoid some bouncing if the driver uses the right kmalloc cache (maybe via a newly introduces dma_kmalloc()) but such check would add a bit of overhead (and I'm not sure it works with slob). The advantage is that we can avoid a bounce buffer if the drivers in question are adapted to use dma_kmalloc(). -- Catalin