From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 792E0C433FE for ; Fri, 14 Oct 2022 20:23:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9DFE66B0074; Fri, 14 Oct 2022 16:23:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 968166B0075; Fri, 14 Oct 2022 16:23:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E1CB6B0078; Fri, 14 Oct 2022 16:23:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 636B96B0074 for ; Fri, 14 Oct 2022 16:23:41 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E168AC0ECE for ; Fri, 14 Oct 2022 20:23:40 +0000 (UTC) X-FDA: 80020680600.29.B69E469 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) by imf15.hostedemail.com (Postfix) with ESMTP id 77EF8A0030 for ; Fri, 14 Oct 2022 20:23:40 +0000 (UTC) Received: by mail-pj1-f51.google.com with SMTP id a6-20020a17090abe0600b0020d7c0c6650so8975744pjs.0 for ; Fri, 14 Oct 2022 13:23:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=EpzOL2I39WBx4qybm1ABKOUs+/6ZbeAJj9uKHcYl/y0=; b=rqSEwkAGNU0A9qJXTxV51byK3octUtAdTJqjspf3Zo48mhS2+uaup2U//kMV6Qoi62 syQwaRn9nsRagawb/MKhDXVvCpux2lI+M4XEw1iDZC0o2HhDC9k5j5PExZukHB5G/Jww j6ZItS6qhvBaudYRlPHGRKhhJBskxFwmJ78+J+X/HkBc6g758120CU0/QzDCX9fV81e4 Eq4ii2IUaIpi9bIJt80Lspql5uGcRZWW9z8VL+EdbkVPXwT4p5ov9Q0wzUY09iwHM/MJ H5vBxIMqou+Z9ewA89od4rWZTGb9L0mvjcwT/loTUek4I3GIZ9PlVAhIZ1LDxwyojwlB VDvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=EpzOL2I39WBx4qybm1ABKOUs+/6ZbeAJj9uKHcYl/y0=; b=ak32yNgn0ul20rxJVSxEUZ8XjFGKlPFn/zRIPQKs26kSjvAjKc1bOcfU2f7e6rXhxf E+wOQzjH/gbTWT7an18k+9NXlK2Rhzq2LC/agihvO2k9nf9vUlox9iEIzZYbWn3rkFz8 YKBztoVQzJPwpm6RpK1CEtl8pYVp/LQsM9PRGamY6w+i+OVFRrsMKHv5hpa3qqhRXUER OuKrIpgHtfj0De8TvdXhZG0x2dpdY/gmBu7UtNoa8QfrGNsQUDRaaglxjUUxd7aKjKO+ mB+wNR1EF1u9KVXSAeN7VFapvsobmh7aN90mF6z/9cVtDG5flRTcDiOzIneQxVVtDSC2 aUkQ== X-Gm-Message-State: ACrzQf2cMgH3D8noZ/yFEfUGumFVVB0gPDNCog7AC0LqShVCU0mpBZgv LP+G+ObGMufBlXVEpTZOS2YhPFpIQk7fTE2q2h3j3w== X-Google-Smtp-Source: AMsMyM5uLYl1iWzreHrWhR8NbPrXI1BfqD4vW2V0ddYzchLa9asDA1jsxo2N0h00jt4kdett3c5jH+ZmRt7EltGyW0o= X-Received: by 2002:a17:902:f786:b0:180:6f9e:23b with SMTP id q6-20020a170902f78600b001806f9e023bmr7264149pln.37.1665779019201; Fri, 14 Oct 2022 13:23:39 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Saravana Kannan Date: Fri, 14 Oct 2022 13:23:02 -0700 Message-ID: Subject: Re: [PATCH 07/10] crypto: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN To: Catalin Marinas Cc: Isaac Manjarres , Herbert Xu , Ard Biesheuvel , Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds , Linux Memory Management List , Linux ARM , Linux Kernel Mailing List , "David S. Miller" , kernel-team@android.com Content-Type: text/plain; charset="UTF-8" ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=rqSEwkAG; spf=pass (imf15.hostedemail.com: domain of saravanak@google.com designates 209.85.216.51 as permitted sender) smtp.mailfrom=saravanak@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665779020; a=rsa-sha256; cv=none; b=aWT0tAwiptuvb1v91glFeUkzBDqsjpP2I3l5k/oCyj2T6QffgejPyK0PE1yEhFdga4sXd6 yUeb4Nh9p+cYPMk1s601gsMR2HrOvnKdsgGR7reMR74N1slpFS2joL2jyArqjdBZmieT+u zib/de2b9UsHpXAoSiHIiOZLJsJFPA0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665779020; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EpzOL2I39WBx4qybm1ABKOUs+/6ZbeAJj9uKHcYl/y0=; b=Ow+VWonuAeVPqN+sK/VXNWimpJ1lBgigcSDj4MPc1CIMyd0DvhZk0neNcojGgH8doqBQSv ZkZzNKHXndhFYjkf8QIdqfuRgKjRDbYX0zG8Sd76OCY9KQ50wJyB2NEJHW08xlwBZG/Er9 jtXn+GvahIAP4Cltty75/QOLPKjHn1U= X-Rspamd-Server: rspam05 X-Rspam-User: Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=rqSEwkAG; spf=pass (imf15.hostedemail.com: domain of saravanak@google.com designates 209.85.216.51 as permitted sender) smtp.mailfrom=saravanak@google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: imedse5saq66yrzff9xe3jwy1b6ykqsp X-Rspamd-Queue-Id: 77EF8A0030 X-HE-Tag: 1665779020-651323 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Oct 14, 2022 at 9:25 AM Catalin Marinas wrote: > > On Thu, Oct 13, 2022 at 11:58:22AM -0700, Saravana Kannan wrote: > > On Thu, Oct 13, 2022 at 9:57 AM Catalin Marinas wrote: > > > > If so, would there be concerns that the memory savings we get back from > > > > reducing the memory footprint of kmalloc might be defeated by how much > > > > memory is needed for bounce buffering? > > > > > > It's not necessarily about the saved memory but also locality of the > > > small buffer allocations, less cache and TLB pressure. > > > > Part of the pushback we get when we try to move some of the Android > > ecosystem from 32-bit to 64-bit is the memory usage increase. So, > > while the main goal might not be memory savings, it'll be good to keep > > that in mind too. I'd definitely not want this patch series to make > > things worse. Ideally, it'd make things better. 10MB is considered a > > lot on some of the super low speced devices. > > Well, we can still add the option to skip allocating from the small > kmalloc caches if there's no swiotlb available. This seems like a good compromise. > > > I wonder whether swiotlb is actually the best option for bouncing > > > unaligned buffers. We could use something like mempool_alloc() instead > > > if we stick to small buffers rather than any (even large) buffer that's > > > not aligned to a cache line. Or just go for kmem_cache_alloc() directly. > > > A downside is that we may need GFP_ATOMIC for such allocations, so > > > higher risk of failure. > > > > Yeah, a temporary kmem_cache_alloc() to bounce buffers off of feels > > like a better idea than swiotlb. Especially for small allocations (say > > 8 byte allocations) that might have gone into the kmem-cache-64 if we > > hadn't dropped KMALLOC_MIN_ALIGN to 8. > > I now remembered why this isn't trivial. On the dma_unmap_*() side, we > can't easily tell whether the buffer was bounced and it needs freeing. > The swiotlb solves this by checking whether the address is within the > pre-allocated swiotlb range. With kmem_caches, we could dig into the > slab internals and check whether the pointer is part of a slab page, > then check with kmem_cache that was. It looks too complicated and I'm > rather tempted to just go for swiotlb if available or prevent the > creation of small kmalloc caches if no swiotlb (TBH, even the initial > series was an improvement dropping KMALLOC_MIN_ALIGN from 128 to 64). Agreed. Even allowing a 64-byte kmalloc cache on a system with a 64-byte cacheline size saves quite a bit of memory. -Saravana