From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69F3BC3600C for ; Tue, 8 Apr 2025 05:27:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3F54D6B0028; Tue, 8 Apr 2025 01:27:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A2476B0029; Tue, 8 Apr 2025 01:27:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2427B6B002A; Tue, 8 Apr 2025 01:27:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 065416B0028 for ; Tue, 8 Apr 2025 01:27:38 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 046FFB7E4F for ; Tue, 8 Apr 2025 05:27:39 +0000 (UTC) X-FDA: 83309744280.07.837908B Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) by imf23.hostedemail.com (Postfix) with ESMTP id CD539140002 for ; Tue, 8 Apr 2025 05:27:37 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=ZtYCSfro; spf=pass (imf23.hostedemail.com: domain of ptesarik@suse.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=ptesarik@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744090058; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mlPuhzudsvQ1/IcNbANi4lVHY7itb57IMm9L+ku/K1I=; b=C4D94qVuIQ/ZQgw1dOJtkftHZEBBqKLBrYdRiGCPBqzaTXlkuc66HY6bTR6lGS3xlvzuG/ HMzco52C5KtiuRPCxBgXIhuGkDX1oTSsC0titCmZeDjWSxkxiTQQLYkUztRRiNNGS6inXI cy6QpJlLYsMpvg6LQWRPyTN5fgY1KFA= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=ZtYCSfro; spf=pass (imf23.hostedemail.com: domain of ptesarik@suse.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=ptesarik@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744090058; a=rsa-sha256; cv=none; b=TcCyAKpQypK32VOTTT+qfms/RpE6PJ0yql83Yb2hwpF9QQl7WgvtMPx5lEqju5FXaDQi0z nIDaOLMQg4NELH7Koq+nLAfVLfPq8F+rwoL9ZxJoH0t/lx5bIxaAzLEXiTYEVmkf9zqnaD /UTQqhFCLYE2cT3jwMAK1E0ys/avozQ= Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-ac2c7a367d9so97253366b.3 for ; Mon, 07 Apr 2025 22:27:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1744090056; x=1744694856; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=mlPuhzudsvQ1/IcNbANi4lVHY7itb57IMm9L+ku/K1I=; b=ZtYCSfroHA+FHyfvT0CjGBMPezj++KldrZ0rMp5/BIcSjHjF8DrE40xlKR8ESgHQWM nQre4UcQbXbHyUW468vcEukKMpoQtU/hMB7h6OcfLYK+7mfTSQjinXksEZP+lbDuieLe kCt57Y46M8/pDVmPwJNv7XZvrpCrw/SJphQMyIekFkYgua442R4x4gVtwA3gTGEy0h5N OzHuyz9Ki+cp4qNHsbHsQp/wkRtQUPPd4KqQdY0Q1ZRov/nCnBR6cMGZQ2aiLtpJQcB6 3l4sotzkGO0Suf/PHNNZ3DoElVnhUIpIQgXXXXicgfBfqJkZikTJ0JAY8gZMdywYqTGg LTtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744090056; x=1744694856; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mlPuhzudsvQ1/IcNbANi4lVHY7itb57IMm9L+ku/K1I=; b=xKSdQcldleN0Ac1pOgd7gvasD+RU8/2LgxC6TLJ68LZx/chh0wYR0Aq+A68OQXhG7D tboxSsmZ5iQfACRL0aZwW9lWfuuLGDIKGsJcIl3o6EtB1GLFepniB6W+a1Gc6EAPRGmt mM7/0lqnECEslWfRVf3AY+xo/kD+NJ890Ixkkj5oiiF4Y/9IlRw1dHHpNNLLP5X3FLB8 PFym55eE8L48Hq4mQUppYGCmywIBcTkoLXoORtLyA/P2BEClSdjc9jX6ps5HLowr9I0E 8fOhSrYcMTQGgFYLpC/25DvDRTdaOvZlKSNeFnh4W3p+S4Kv8jkbeI1/voso+Pu3G382 RCng== X-Forwarded-Encrypted: i=1; AJvYcCUPOtrGIWrHmaGSILKjgSnzvXJpZLGdvR1794uLN80AGNmEHuBtIW0eVUaPZphN+Y5q8ti4oybtBw==@kvack.org X-Gm-Message-State: AOJu0YwI7XlrNzBDzgjdwwUvCW/1IqfZhAl3dOVHz7SpExfKgmAk2c1k cOlVC/16cIJOL554zGCzQpvVwYzc0srs7Dol3d6Vzs8DRjIoYNi+gP2GrDatW/c= X-Gm-Gg: ASbGnctMguk3kfK3tVRArwng28fUbSeS2Mz/t7Ms6g+5EBMcnCtqF8BUruubdwwrv1H bFPEXwFTWRiVRAulNxNIpj6hZulnjabcHXomdz7Q9mbBdWI67rUwbZerc+8z3fHzT6OhtDvwQP0 HffxZWqaIK2KNY0KpFj4KSuSFo7+W7DDw64Id+ycpWqBkfYccKLIxFqW/1vcMi4GGrupvS4FlUZ w1pPAC9iffehUBW+HY6OB6UDM84NiZXRBWgeJufLOrWfjDYuztL7zXnHbu2r+rXV+0duJ5yPFwj KDLw5ISQKrNR X-Google-Smtp-Source: AGHT+IFz0dkIWbIe5MebU5W+KUE5d0MlEEsxwXRj0gbxwU80usCDooD80xrcYdt20kSweON6pAHo+w== X-Received: by 2002:a17:907:2cc4:b0:abb:eec3:3937 with SMTP id a640c23a62f3a-ac7d19d8fd9mr351150366b.10.1744090056158; Mon, 07 Apr 2025 22:27:36 -0700 (PDT) Received: from mordecai ([213.235.133.107]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac7c01c2c2dsm860491866b.173.2025.04.07.22.27.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Apr 2025 22:27:35 -0700 (PDT) Date: Tue, 8 Apr 2025 07:27:32 +0200 From: Petr Tesarik To: Catalin Marinas Cc: Vlastimil Babka , Feng Tang , Harry Yoo , Peng Fan , Hyeonggon Yoo <42.hyeyoo@gmail.com>, David Rientjes , Christoph Lameter , "linux-mm@kvack.org" Subject: Re: slub - extended kmalloc redzone and dma alignment Message-ID: <20250408072732.32db7809@mordecai> In-Reply-To: References: <20250404131239.2a987e58@mordecai> <20250404155303.2e0cdd27@mordecai> <39657cf9-e24d-4b85-9773-45fe26dd16ae@suse.cz> X-Mailer: Claws Mail 4.3.1 (GTK 3.24.48; x86_64-suse-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: CD539140002 X-Stat-Signature: ce5buf9iephotr5c9zq5bqhd3w49e41k X-HE-Tag: 1744090057-838630 X-HE-Meta: U2FsdGVkX18JrjB56iInvdoMXPoLbx+Un1MgxkLtb52VHbBFMjF8Zg1cPSAdqkJ9jh90ms2IDYf2GclewNc5Cd2vT+0Z+UuP3KEoO2LuHAc5k3xoMJMpeUno3oEq/G+IqE597pY5fbJsLWWvKy7Na3QxmYy4wpeHkx1rzfb14DfYnshIyW6spy6Y3pqCcxniHmf/cS4q1FGfoy36VpCMm76LwSajcxb6taWHaycEjuYTi8AbbqBDP9O/GcRGiJXKY6ZkdpT4JpnSROMSTA3YdVLx4bmNo0VDVQNCNLBaNDwbYrleXLRxtkIP1BfPdfqK9yLtFpvX3LeI0gbvotJIE35uWWbkImZFvVEXezYarKQBDYpv9odAvQCzcFG/DadU6Gterdp4QUnD0QeM9nLwDM+LMEHwT8sZ8KbbYB9/Cw6dHXviUEZxmmeofSRWzMVSmQ5wzmMWleX2IEAop55sRd4HKu4AebPRYiAxnSRKxMr7ot8JRx24Jf03SaEnq5R/R9nvfQwqU5qXSu0HwVPNd8Nrhaoz0qd9XYajCAnrICYBBi6nY3M0woqVqJkXErqeBoOye0pQYLgxmEwhyeBCBbnxOTqnmwpycJoCGE3LsaK4RZWMEtgO8JQuAc7ufvhTlZTEk1vqA3EobsYcQkaCu+jpvDrjCOzyW0ictNmqyWBOgvsVSrl2fhdYozqHlMdY8iGTWtzJsVz9pmHqfl7kwzMvr70+a3CJ5pD2t33J+LSdflbSxmuMUYCtlJQxCdfOiCk5Zl9TG9Z2v2EtYOYqRjgHx5KMP58ANLkuYWwvbZg/X+DQ+klCUmUjjpQANxMQEsh2st4nFDuwwe9i8L62E8BoA8GmtfxRCf8ni/wSJsJg6R5rw+AXhdZTizqTO7/yWbPZLTpItMLalr2VtPc8WgKi4npCdxIxNNK3JryQ9gSyF/mMlh2d5U7gWChd37tHsq2FszCFe2CbA8PFldv rjlsITn5 GWsCK/blg8iJk3yokHsdh51cW6UbOD36AR6gsq7CzK0alMuyfNvDD5JsSEufjNbJ+kmX1lFGK5lSKu1i29YBCSeQSpZYk58HAXj8bK7+aV4WMOufGo4WoZ2sUbDmX88uc8YRoWHxxXXM5tU/KZ8xyHLsynNdtv9uSmMN0fJZo+Fh0l2rdTBE3xWYdA4hmrjNBjswA0tcnxUWGG9dElZ4XR49TuX8SDoS9n0ck2QViCGdDYnO6mdiGw+msU4CyXnv6YldM2BsXKkvU7DWYs3giHPQMdzLU8N1+rx9+ak+IIYlVJbssji2KzxobZxgDRLgbBTBqbLlsPcBAi1RODnow76HUEcVV53274c6rylLuD2UWB3K9i42e64dRJbOrEOuBtN58ADaqGKdWc7c3NRNgZcycw4FLDyyfTkWnpA5vSF5ufOSp7BQFe9HMSZDw+G4UJVI3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, 7 Apr 2025 18:12:09 +0100 Catalin Marinas wrote: > Hi Vlastimil, Feng, > > Thanks for looping me in. I'm just catching up with this thread. > > On Mon, Apr 07, 2025 at 09:54:41AM +0200, Vlastimil Babka wrote: > > On 4/7/25 09:21, Feng Tang wrote: > > > On Sun, Apr 06, 2025 at 10:02:40PM +0800, Feng Tang wrote: > > > [...] > > >> > I can remember this series, as well as my confusion why > > >> > 192-byte kmalloc caches were missing on arm64. > > >> > > > >> > Nevertheless, I believe ARCH_DMA_MINALIGN is required to avoid > > >> > putting a DMA buffer on the same cache line as some other data > > >> > that might be _written_ by the CPU while the corresponding > > >> > main memory is modified by another bus-mastering device. > > >> > > > >> > Consider this layout: > > >> > > > >> > ... | DMA buffer | other data | ... > > >> > ^ ^ > > >> > +-------------------------+-- cache line boundaries > > >> > > > >> > When you prepare for DMA, you make sure that the DMA buffer is > > >> > not cached by the CPU, so you flush the cache line (from all > > >> > levels). Then you tell the device to write into the DMA > > >> > buffer. However, before the device finishes the DMA > > >> > transaction, the CPU accesses "other data", loading this cache > > >> > line from main memory with partial results. Worse, if the CPU > > >> > writes to "other data", it may write the cache line back into > > >> > main memory, racing with the device writing to DMA buffer, and > > >> > you end up with corrupted data in DMA buffer. > > Yes, cache evictions from 'other data; can override the DMA. Another > problem, when the DMA completed, the kernel does a cache invalidation > to remove any speculatively loaded cache lines from the DMA buffer > but that would also invalidate 'other data', potentially corrupting > it if it was dirty. > > So it's not safe to have DMA into buffers less than ARCH_DMA_MINALIGN > (and unaligned). It's not safe to DMA into buffers that share a CPU cache line with other data, which could be before or after the DMA buffer, of course. > What I did with reducing the minimum kmalloc() > alignment was to force bouncing via swiotlb if the size passed to the > DMA API is small. It may end up bouncing buffers that did not > originate from kmalloc() or have proper alignment (with padding) but > that's some heuristics we were willing to accept to be able to use > small kmalloc() caches on arm64 - see dma_kmalloc_needs_bounce(). > > Does redzoning apply to kmalloc() or kmem_cache_create() (or both)? I > haven't checked yet but if the red zone is within ARCH_DMA_MINALIGN > (or rather dma_get_cache_alignment()), we could have issues with > either corrupting the DMA buffer or the red zone. [...] I'm sorry if I'm being thick, but IIUC the red zone does not have to be protected. Yes, we might miss red zone corruption if it happens to race with a DMA transaction, but I have assumed that this is permissible. I regard the red zone as a useful debugging tool, not a safety measure that is guaranteed to detect any write beyond the buffer end. >[...] > > > I'm not familiar with DMA stuff, but Vlastimil's idea does make it > > > easier for driver developer to write a driver to be used on > > > different ARCHs, which have different DMA alignment requirement. > > > Say if the minimal safe size is 8 bytes, the driver can just > > > request 8 bytes and ARCH_DMA_MINALIGN will automatically chose > > > the right size for it, which can save memory for ARCHs with > > > smaller alignment requirement. Meanwhile it does sacrifice part > > > of the redzone check ability, so I don't have preference here :) > > > > Let's clarify first who's expected to ensure the word alignment for > > DMA, if it's not kmalloc() then I'd rather resist moving it there > > :) > > In theory, the DMA API should handle the alignment as I tried to > remove it from the kmalloc() code. Are we talking about the alignment of the starting address, or buffer size, or both? > With kmem_cache_create() (or kmalloc() as well), if the object size is > not cacheline-aligned, is there risk of redzoning around the object > without any alignment restrictions? The logic in > dma_kmalloc_size_aligned() would fail for sufficiently large buffers > but with unaligned red zone around the object. This red zone is the extra memory that is normally wasted by kmalloc() rounding up the requested size to the bucket size. But dma_kmalloc_size_aligned() already uses kmalloc_size_roundup(size), so it seems to be covered. Petr T