From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98401C30658 for ; Tue, 2 Jul 2024 16:41:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3009E6B009F; Tue, 2 Jul 2024 12:41:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2B08C6B00A0; Tue, 2 Jul 2024 12:41:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 177C76B00A2; Tue, 2 Jul 2024 12:41:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id EC8596B009F for ; Tue, 2 Jul 2024 12:41:15 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 9DE1312030E for ; Tue, 2 Jul 2024 16:41:15 +0000 (UTC) X-FDA: 82295377710.01.17046BA Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by imf27.hostedemail.com (Postfix) with ESMTP id D5A7640007 for ; Tue, 2 Jul 2024 16:41:13 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=cGb7k9q3; spf=pass (imf27.hostedemail.com: domain of aliceryhl@google.com designates 209.85.221.45 as permitted sender) smtp.mailfrom=aliceryhl@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719938451; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cdEhN7egg4GTetScTgi2yWO5StQG3EhkGlVLI6a6mXM=; b=YKWMnX+1l1d+bVxm0IWBMrnRjamJw6zuc9lb69qcop2RaRXfqLykjdJ8DDTjWhj58QBzF6 0WPzUPAunzCVnRaNn7mCENXJsLFbVMGlOA7kqi4P/zEcsmGNAr/+qJ6Uvso1Mkna0CB2EG W4yClF+KYyff8BLsUJJccLAGOuHW6lk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719938451; a=rsa-sha256; cv=none; b=8F+SvK1B7fPWWGc2alF449S3uioCQD7ss316LkgtKwcRFCNFSbSIrtEZna78mYaAK2hoq8 62LHD++pQjfddDAT3Q9WLPi0qsoQ8cFP1x0hR45y39x3jWpDAhXL3U2RnWeWPmabpOBin+ 5yt1MyYFmn1mIVu09fclgP7I2PP5390= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=cGb7k9q3; spf=pass (imf27.hostedemail.com: domain of aliceryhl@google.com designates 209.85.221.45 as permitted sender) smtp.mailfrom=aliceryhl@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-3629c517da9so3482952f8f.2 for ; Tue, 02 Jul 2024 09:41:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1719938472; x=1720543272; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=cdEhN7egg4GTetScTgi2yWO5StQG3EhkGlVLI6a6mXM=; b=cGb7k9q3p2JMalufh7dJJiYB1EOUtXQ3wsQXpuUdlwaXXLeqDI1b32e8n+WCIrRxT2 EVl6gtWvt/C2QJd0tK4Le29A5IR2B2DNE2uRv2lpYUARS5BQYAdPY3cL7x54lMKvpIUR RyXCveRGqP1rKX8A3qqXhXRaLeIrlGR0/CriQNTEDyx5tCFMEc1M2HeyJ0J9gNC8VxYo n+8/JSD/xq8RJkSVQMl6rlQRATsaB5JNiRxIcE3AVTxMleAY1P/YQ0AFvaloUVaJ+Zhz aQ+YMxi9bbddjCXfEcn19RoSHyB+gnE14p8IkPmZpNbhMFBtwHB19if0h47NfyEwK2yM LXvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719938472; x=1720543272; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cdEhN7egg4GTetScTgi2yWO5StQG3EhkGlVLI6a6mXM=; b=WKca/aadX900hn9NF11BDfbb0143lM+EoCPA7ofXrx11TWLZmevc/zLffs83hyhL+0 u98uo9BlwvJ03CKeZVLExAg4R1OlP/ciWzCdmmSn7U/ffVqZVgSLbWVj6KgRr0V1CGWJ e21rO8prBDKpsm2mGIyyFhdD5SvI6TK+QA5qnw+dhNygyZ+QuovOEwxiikVSYgyMPmss iv51rL3qI+HpOiGKdJQhFCtGG74LJvAmb8+PbH0EN7+b8WHfIVItCWRHc/vdjt9GbBQP XJ8e/1R1PbCb2ce40047i6vWRNL+Cr/1+bkkFxKIPX8KbVcFCHkNS5ZDEtxoIcLCtP6W 1lWg== X-Gm-Message-State: AOJu0YxxeyTSeiFw8gvklblnmuOqe0K6YCOXPjGVyJx/MPDL01CYKvnz DF/wEC4y8WsXMuKSBaTEHui3n9YpuE8bgGRjeTJOItigx27QWrfgSlR8DLk7eW/x7Zf40VyB2W/ JuAmoD4kDVdMwaxGdGPQqflqp9LvuWsnCfsMn X-Google-Smtp-Source: AGHT+IHmYsZn/Vy3aZXo1dvsiv+ZReJUZGbhaq1QATW22XsOqABArbPhv8Sbvt3RE2ym1iLvf5P0H4DKXuYSyJ0kisU= X-Received: by 2002:a5d:5889:0:b0:366:eba0:8d8c with SMTP id ffacd0b85a97d-36775724976mr8558555f8f.54.1719938472236; Tue, 02 Jul 2024 09:41:12 -0700 (PDT) MIME-Version: 1.0 References: <20240702155800.166503-2-vbabka@suse.cz> In-Reply-To: <20240702155800.166503-2-vbabka@suse.cz> From: Alice Ryhl Date: Tue, 2 Jul 2024 18:40:59 +0200 Message-ID: Subject: Re: [PATCH] mm, slab: extend kmalloc() alignment for non power-of-two sizes To: Vlastimil Babka Cc: linux-mm@kvack.org, David Rientjes , Christoph Lameter , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Roman Gushchin , Kees Cook , Boqun Feng , rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, patches@lists.linux.dev Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D5A7640007 X-Stat-Signature: m6ph3sjy41fupmdg6awmfspxrr6w3jze X-HE-Tag: 1719938473-66566 X-HE-Meta: U2FsdGVkX1+NKDm5NR8Bq2C0CA3qV1d4EC+l2zd5UP0ylKpkaqpj7khtZpjbytVPikX0Fdd2C7Xw57pb6sKsTlBzJTulVBRRnRhe6AxCWQ1ln6zAEqTprqTpG448cRlNfH986zUD1ZzvXvcMktrAuikBdA+juIexiuMeY+dfMm7UGG3EEIroi0ZtKaQNtvh256Hsueq4Rk5qFJAxx/FXefRaBvcRJWA2vd38w2o6jchPBMvKseRaoEHMU1RqPhf/Tn2JRfBYjqiZJ6EY3C6rQooc0ckIs0KLte1vVRpSTgMECmOE37PjgjY9mYmx1dOXB1+FXTp9D4z4HmbZ604IIStxlBQXotpF3u54+gbKmU22VeR2sjNOjCXx5UURJkAgeg79mhyZya7qS4sRdc09VT+21/W6DoHA2eYDeGUl16h7VYYceF/2F/ghX/ZaQXuLjjeC/t2WcF8SgvxUC0m93tLaxGSs45cHV8+9U1Boeyx0NdnX9a4uew3Aqkq3r2yM7KO0bNy+JPP/2uhhcec8Vm/TyEVB4U7sxTQ/OWdUpX+bcM6r13TkHYR4oLpSbSVbdMqf+LHo7l1w/J/Thrxs0ATPYAvmZzBPmAI+oiYsAJIAlCL3L993u7lNKY18otuUv8zNEIfCA2iZo037kJ4rQ7sJqTRfO4qzup7YgWlGcg6HuAIry7LmdBgZM6tEfKp6wgjqKv+kkSU7qf330yxn0B9kd0+sd0xxsFTs2ZKoxQrD4R7ggWuiCYjhkRSwpYuN+U0ReznskAyuhD4H3/OvJzugpPpOTSfft5WSyYmbJPRWoYsn0OHi+U+CXV4UZMIdCVRr+HwjeAhPOycPenqm/n+ZQAKO7nlYecIHESHCDy/euW3E9b6+zbX43fsippWwtukGDY/PWOb9XiQzoxox3Lk5eqdLz1gIx0VsZtIWSKgERhM8D4jUYA1nKNZfdUQIL8Yls3yGG9HKc+QV/aH cOHYL9qv TMlO8bHcmi8C4W9Tj0S98zi3Ef7ZS2/YKJYK+e6LGtWYcCTj0/FEX0xD1MBa9ABS0d8I7A0Wp+B2cxOh0RAukphS22kblZKLNAGj989iqrAyD4q8R+HjaHDVpxs8RX4V46gHW1kUsBUkhLE9o73urxGof9DwKvPrEiqWtXvxWtmhIHxtdwkhN7qJlQQB4/xq4wK8kHr+UGfcpN9dHPjCWmmtxWc9ROyZ4mpml/O5K3+MX2/Gvw8pRKAYeMtXRnslG/1sGAQDMFsQeFFokle9RpjOOi9sybHytV6Qyw3/mCt2rICRTYEJgHi6kMqnalViHfj1O X-Bogosity: Ham, tests=bogofilter, spamicity=0.009132, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jul 2, 2024 at 5:58=E2=80=AFPM Vlastimil Babka wro= te: > > Slab allocators have been guaranteeing natural alignment for > power-of-two sizes since commit 59bb47985c1d ("mm, sl[aou]b: guarantee > natural alignment for kmalloc(power-of-two)"), while any other sizes are > aligned only to ARCH_KMALLOC_MINALIGN bytes. > > Rust's allocator API specifies size and alignment per allocation, which > have to satisfy the following rules, per Alice Ryhl [1]: > > 1. The alignment is a power of two. > 2. The size is non-zero. > 3. When you round up the size to the next multiple of the alignment, > then it must not overflow the signed type isize / ssize_t. > > In order to map this to kmalloc()'s guarantees, some requested > allocation sizes have to be enlarged to the next power-of-two size [2]. > For example, an allocation of size 96 and alignment of 32 will be > enlarged to an allocation of size 128, because the existing kmalloc-96 > bucket doesn't guarantee alignent above ARCH_KMALLOC_MINALIGN. Without > slab debugging active, the layout of the kmalloc-96 slabs however > naturally aligns the objects to 32 bytes, so extending the size to 128 > bytes is wasteful. > > To improve the situation we can extend the kmalloc() alignment > guarantees in a way that > > 1) doesn't change the current slab layout (and thus does not increase > internal fragmentation) when slab debugging is not active > 2) reduces waste in the Rust allocator use case > 3) is a superset of the current guarantee for power-of-two sizes. > > The extended guarantee is that alignment is at least the largest > power-of-two divisor of the requested size. For power-of-two sizes the > largest divisor is the size itself, but let's keep this case documented > separately for clarity. > > For current kmalloc size buckets, it means kmalloc-96 will guarantee > alignment of 32 bytes and kmalloc-196 will guarantee 64 bytes. > > This covers the rules 1 and 2 above of Rust's API as long as the size is > a multiple of the alignment. The Rust layer should now only need to > round up the size to the next multiple if it isn't, while enforcing the > rule 3. > > Implementation-wise, this changes the alignment calculation in > create_boot_cache(). While at it also do the calulation only for caches > with the SLAB_KMALLOC flag, because the function is also used to create > the initial kmem_cache and kmem_cache_node caches, where no alignment > guarantee is necessary. > > Link: https://lore.kernel.org/all/CAH5fLggjrbdUuT-H-5vbQfMazjRDpp2%2Bk3%3= DYhPyS17ezEqxwcw@mail.gmail.com/ [1] > Link: https://lore.kernel.org/all/CAH5fLghsZRemYUwVvhk77o6y1foqnCeDzW4WZv= 6ScEWna2+_jw@mail.gmail.com/ [2] > Signed-off-by: Vlastimil Babka Thanks! Since this change is motivated by Rust, why not also include the Rust change in this patch? You would need to remove the if inside krealloc_aligned in rust/kernel/alloc/allocator.rs and update the comments. Alice