From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D3CDC433FE for ; Fri, 14 Oct 2022 16:25:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D2136B0074; Fri, 14 Oct 2022 12:25:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 781676B0075; Fri, 14 Oct 2022 12:25:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 649496B0078; Fri, 14 Oct 2022 12:25:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 520CD6B0074 for ; Fri, 14 Oct 2022 12:25:41 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 27E0C407CE for ; Fri, 14 Oct 2022 16:25:41 +0000 (UTC) X-FDA: 80020080882.10.9252711 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf15.hostedemail.com (Postfix) with ESMTP id A7B44A002D for ; Fri, 14 Oct 2022 16:25:39 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A53FF61B97; Fri, 14 Oct 2022 16:25:38 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 78F26C433C1; Fri, 14 Oct 2022 16:25:34 +0000 (UTC) Date: Fri, 14 Oct 2022 17:25:30 +0100 From: Catalin Marinas To: Saravana Kannan Cc: Isaac Manjarres , Herbert Xu , Ard Biesheuvel , Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds , Linux Memory Management List , Linux ARM , Linux Kernel Mailing List , "David S. Miller" , kernel-team@android.com Subject: Re: [PATCH 07/10] crypto: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf15.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665764739; a=rsa-sha256; cv=none; b=nBzlnCUzl3jr7w88vgpuCjtvnXLF48vUqC2jsDLiSMdrzNyOQRn6atmm0mVgQIsbaGeL9d dcHcjnUQQq7tw0KHYOYHlMOaWVyclA1Mn8ouHyusIt2kzdae+xmPGSAbA1ThaqUvN6CzM3 20Edz4bhWowNQSPk08tY4iJNz3A5P7E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665764739; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SOuySbzNUl0QzTzHJTFyXaK6zIExcJxbBxEtta1Bs0I=; b=fNEMKEBfsc04T13/EOvXvP51To3NBkoJaEbK3lDNgqGNtk26drBvIWWK4/1hcdvY3mQEG5 i6Vv4LF2khGIQHKzCsbKrDGbd3l39GQ6sSikUJWRzbivNGtpAuNZYH6r3ag7HdBVv6V1Xi OjcXFXfiC91yUDdEQr76tuLDU1SF7jI= X-Rspamd-Server: rspam05 X-Rspam-User: Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf15.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org X-Stat-Signature: fwnwc4uoeyamgdimj9yx9diwjwfiemib X-Rspamd-Queue-Id: A7B44A002D X-HE-Tag: 1665764739-850894 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Oct 13, 2022 at 11:58:22AM -0700, Saravana Kannan wrote: > On Thu, Oct 13, 2022 at 9:57 AM Catalin Marinas wrote: > > > If so, would there be concerns that the memory savings we get back from > > > reducing the memory footprint of kmalloc might be defeated by how much > > > memory is needed for bounce buffering? > > > > It's not necessarily about the saved memory but also locality of the > > small buffer allocations, less cache and TLB pressure. > > Part of the pushback we get when we try to move some of the Android > ecosystem from 32-bit to 64-bit is the memory usage increase. So, > while the main goal might not be memory savings, it'll be good to keep > that in mind too. I'd definitely not want this patch series to make > things worse. Ideally, it'd make things better. 10MB is considered a > lot on some of the super low speced devices. Well, we can still add the option to skip allocating from the small kmalloc caches if there's no swiotlb available. > > I wonder whether swiotlb is actually the best option for bouncing > > unaligned buffers. We could use something like mempool_alloc() instead > > if we stick to small buffers rather than any (even large) buffer that's > > not aligned to a cache line. Or just go for kmem_cache_alloc() directly. > > A downside is that we may need GFP_ATOMIC for such allocations, so > > higher risk of failure. > > Yeah, a temporary kmem_cache_alloc() to bounce buffers off of feels > like a better idea than swiotlb. Especially for small allocations (say > 8 byte allocations) that might have gone into the kmem-cache-64 if we > hadn't dropped KMALLOC_MIN_ALIGN to 8. I now remembered why this isn't trivial. On the dma_unmap_*() side, we can't easily tell whether the buffer was bounced and it needs freeing. The swiotlb solves this by checking whether the address is within the pre-allocated swiotlb range. With kmem_caches, we could dig into the slab internals and check whether the pointer is part of a slab page, then check with kmem_cache that was. It looks too complicated and I'm rather tempted to just go for swiotlb if available or prevent the creation of small kmalloc caches if no swiotlb (TBH, even the initial series was an improvement dropping KMALLOC_MIN_ALIGN from 128 to 64). -- Catalin