From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD464C433F5 for ; Thu, 14 Apr 2022 22:26:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5807C6B0071; Thu, 14 Apr 2022 18:26:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E1C16B0073; Thu, 14 Apr 2022 18:26:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3354A6B0074; Thu, 14 Apr 2022 18:26:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.27]) by kanga.kvack.org (Postfix) with ESMTP id 210D16B0071 for ; Thu, 14 Apr 2022 18:26:21 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E92CD27983 for ; Thu, 14 Apr 2022 22:26:20 +0000 (UTC) X-FDA: 79356919320.10.72487C7 Received: from mail-lj1-f182.google.com (mail-lj1-f182.google.com [209.85.208.182]) by imf13.hostedemail.com (Postfix) with ESMTP id 7A3EF20009 for ; Thu, 14 Apr 2022 22:26:20 +0000 (UTC) Received: by mail-lj1-f182.google.com with SMTP id bn33so7731709ljb.6 for ; Thu, 14 Apr 2022 15:26:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=grIrFiDIYej8A2nEyShR/f/AA0BjSrqUafhM5e+bJC8=; b=PDOMyhu4Ot0RITPmSamRezvWlV9rWJSil6/c4p4BGDd0iBOTga7RB6uavJaMpTmOXV 5jwUTMm/+Dz9yaqMjzZq8YxJ8+ff2e0+Be5W/cj0EWXwFsBB0vRDLZihMAtRe/vKVQQW GkKv2Ydf1Q2D3rndpsIMIBShWkf0ZsFxSqc0k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=grIrFiDIYej8A2nEyShR/f/AA0BjSrqUafhM5e+bJC8=; b=MeLE+KaRFJsICcrTnysBeYamJEET+wUjCq4RF2BAo7kUXYO9PGMOYyo/oTfuUyZ4/c uixhtoDnEc2Mb35CfcESDINCcpf/542wXmqYIO1LY7uM8Op80WaNCkJe9H+G2bHkx0TN rTRJljxENKzX7SZXfYAKIVlwPO9Qk5R3sId/s5lx2IG/nsY5usr4esEraOnaNprlxAXB 6+bFXpy10WMbl7p6EqUSUJOHxTWL722/1XdJsHre3KzJWHOAeU+ss/lcm3AySH69y8Q0 x4/vXEa9OvksEHtAvbDVtLTjwZuffjDoZlzYo1jVqq1Mb3/hrLOSaOK521PyyqgfFpCe OrWA== X-Gm-Message-State: AOAM532fEMtjP2uznMHaOYxZkyYBNalexAcSBGd8VQwXCecGVmWGfdlM uTZLPopv29CIt1+Ccl478rzksaJNPn1c6lty X-Google-Smtp-Source: ABdhPJzhWxN1bgu+cZcy2Agy80PHiU+paQW6VJktwlUbJovhfyddCyFxMNz1NhQYoZ26jEhv279/9w== X-Received: by 2002:a2e:3c19:0:b0:24d:a5ad:e7d3 with SMTP id j25-20020a2e3c19000000b0024da5ade7d3mr25154lja.426.1649975178173; Thu, 14 Apr 2022 15:26:18 -0700 (PDT) Received: from mail-lf1-f47.google.com (mail-lf1-f47.google.com. [209.85.167.47]) by smtp.gmail.com with ESMTPSA id f20-20020a056512229400b0044a6ac1af69sm126824lfu.181.2022.04.14.15.26.16 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 14 Apr 2022 15:26:16 -0700 (PDT) Received: by mail-lf1-f47.google.com with SMTP id y32so11426003lfa.6 for ; Thu, 14 Apr 2022 15:26:16 -0700 (PDT) X-Received: by 2002:a05:6512:3055:b0:44a:3914:6603 with SMTP id b21-20020a056512305500b0044a39146603mr3273417lfb.435.1649975175853; Thu, 14 Apr 2022 15:26:15 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Linus Torvalds Date: Thu, 14 Apr 2022 15:25:59 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 07/10] crypto: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN To: Catalin Marinas Cc: Ard Biesheuvel , Herbert Xu , Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linux Memory Management List , Linux ARM , Linux Kernel Mailing List , "David S. Miller" Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: 7d1ft1edgwimn3jepjy648dp3pywpm3i X-Rspam-User: Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=google header.b=PDOMyhu4; dmarc=none; spf=pass (imf13.hostedemail.com: domain of torvalds@linuxfoundation.org designates 209.85.208.182 as permitted sender) smtp.mailfrom=torvalds@linuxfoundation.org X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 7A3EF20009 X-HE-Tag: 1649975180-73629 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Apr 14, 2022 at 12:49 PM Catalin Marinas wrote: > > It's a lot worse, ARCH_KMALLOC_MINALIGN is currently 128 bytes on arm64. > I want to at least get it down to 64 with this series while preserving > the current kmalloc() semantics. So here's a thought - maybe we could do the reverse of GFP_DMA, and add a flag to the places that want small allocations and know they don't need DMA? Because even 64 bytes is _huge_ for those small allocations. And 128 bytes is just insane. Maybe we don't have a ton of them, but I just checked my desktop, and if my laptop had 17k 8-byte allocation, on my desktop it's currently sitting at 43k of them. And I've got a lot of 16- and 32-byte ones too: kmalloc-32 51529 51712 32 128 1 : kmalloc-16 45056 45056 16 256 1 : kmalloc-8 43008 43008 8 512 1 : Don't ask me what they are. It's probably fairly easy to find out, and it's probably something silly like sysfs private pointer data or whatever. If I did my math right, with a 128-byte minimum allocation, that is 16MB of wasted memory. Yeah, yeah, I've got 64GB or RAM in this thing, and there are things that take a lot more memory than the above (mem_map etc), and 64MB is peanuts at just 0.1% of RAM. Maybe nobody cares. But I really feel bad about that kind of egregious waste. The mem_map[] array may be big, it may use 1.5% of the memory I have, but at least it *does* something. And it could be that if I have 150k of those smallish allocations, a server with lots of active users might have millions. Not having looked at where they come from, maybe that isn't the case, but it *might* be. Maybe adding something like a static int warn_every_1k = 0; WARN_ON(size < 32 && (1023 & ++warn_every_1k)); to kmalloc() would give us a statistical view of "lots of these small allocations" thing, and we could add GFP_NODMA to them. There probably aren't that many places that have those small allocations, and it's certainly safer to annotate "this is not for DMA" than have the requirement that all DMA allocations must be marked. Then just teach kmalloc() something like align = (gfp_flags & __GFP_NODMA) ? 8 : 128; on arm. Hmm? Linus