From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09338C46CA1 for ; Mon, 18 Sep 2023 12:08:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8CBDD6B0322; Mon, 18 Sep 2023 08:08:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 87C806B0327; Mon, 18 Sep 2023 08:08:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 74F0C6B0322; Mon, 18 Sep 2023 08:08:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 64F5B6B0322 for ; Mon, 18 Sep 2023 08:08:57 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 2F1D11A0C28 for ; Mon, 18 Sep 2023 12:08:57 +0000 (UTC) X-FDA: 81249597114.03.B4D0AB1 Received: from mail-qv1-f53.google.com (mail-qv1-f53.google.com [209.85.219.53]) by imf21.hostedemail.com (Postfix) with ESMTP id 722D51C000D for ; Mon, 18 Sep 2023 12:08:55 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=2IxmmmGZ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf21.hostedemail.com: domain of matteorizzo@google.com designates 209.85.219.53 as permitted sender) smtp.mailfrom=matteorizzo@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695038935; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ziZvMGLK6JgmUUboWqZiWtfE7gVK2DDQAWWdwk2gDB8=; b=iiYLYf9GkR5/og0LppssBl31SojZURCGPSoOAE9zj7Gd2U9brQnzCqrJ3cYyZQLKWgFyd+ 0fLRVIbys3LQ68Atbrq9gYk26TxyUnKzODWVm5uAYLcH+Rfxa8YyoANSKQedN0aO5vPbH5 uVoeBzXk5HweroU8bDYUnol5O6B/WR0= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=2IxmmmGZ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf21.hostedemail.com: domain of matteorizzo@google.com designates 209.85.219.53 as permitted sender) smtp.mailfrom=matteorizzo@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695038935; a=rsa-sha256; cv=none; b=jGhyn1+2uHgk2qIVszIMGLvv4PA6E5hG8qmc35aXlWU6QSWvkZfbDwiMQ1sC3doG4i7MUn zVwgeY+QbulqLdIJWCkKGxiMD7v3BwHlnWvRUN/wsQJiaXvzRyT5cqPcW/rzNveaPeakYO 9EmPwZIYLJtDlYYTJv+pUZq3uKiIItU= Received: by mail-qv1-f53.google.com with SMTP id 6a1803df08f44-6561c09ead6so23965246d6.1 for ; Mon, 18 Sep 2023 05:08:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695038934; x=1695643734; darn=kvack.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ziZvMGLK6JgmUUboWqZiWtfE7gVK2DDQAWWdwk2gDB8=; b=2IxmmmGZ1WmA1kzidpZZuLwzKjwwtiV/PuL+acKjzmEFNaYNwkiM9Ag+zIX3WNAB6k isP5bPRzJtXznD91y0QSTIQ1/S0zSeOSsTnytUecXe2d2ulWJb22ZtXOXIENnef72fmO rXVnqGaVfYTFT9FBrK/UVhvsEthrfFtpB1riJvA0Dh1aiZYAY4cqdhM9mruEF3IabFrF /7hr5w5C8uHC6PKfgzlHAEJh1tc22bT3FD3YMzc3WYAqSprMZhQPUrG2KCrph8u1J/U9 CAitX0W08uwACVbCwxU/xuXZ9YNM2veTQFMGH8EsDw/x/xUOg2msHvd4cWyApOE8CMo2 QX4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695038934; x=1695643734; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ziZvMGLK6JgmUUboWqZiWtfE7gVK2DDQAWWdwk2gDB8=; b=LmpFMLtiiLnjf9q8qzOmkaO2QyJ6RpIIN/4aOvH9BHxKOlCnzvBbe4WQpoHmiwmYoC PhS6hfHkBahtbU7jqpMSMJSUFCfMPhXOuhskj/u1y1UZPR26AM6fc0yN+OB7mFPCPFN0 wbky/Vb0oB8VF5f+Vrvvz10hIk1JSp6yQzefdzdtt2mCZrxRag8yUf40zY85Zv9FezFI KVdvqYFDUSslQaSwt1rGhtocYLBbVvrJquOUFPNbTjn6omirpH7xoMbrMZjEjL15rs+j wKzAqklWqkmAL+QBMmK8A4NGyVWYfhOUtdtJSw2lvQYDZwqGEMhaNBj1dErKe1L47nJX q6Gg== X-Gm-Message-State: AOJu0YyJw0246VxbW7l4hr36zZe4Ji97e1EBERCFlXQVPiBB9C6zTViu 79Wzge6gb8xIfWTYWw8tjTPAMNajj+fGlprllQ7zSg== X-Google-Smtp-Source: AGHT+IGe6eycPkCnJorto9OJHO+hQCundE/h1zKNUGuoGj4gpWrDhAWQ0sCOlXxSU9kbzdmJ9bi8fQ6B0MF4HXGsu54= X-Received: by 2002:a0c:cc82:0:b0:656:5337:e7bd with SMTP id f2-20020a0ccc82000000b006565337e7bdmr4263366qvl.3.1695038934426; Mon, 18 Sep 2023 05:08:54 -0700 (PDT) MIME-Version: 1.0 References: <20230915105933.495735-1-matteorizzo@google.com> <7a4f5128-28fd-3c5f-34c2-1c34f4448174@intel.com> <1d7573c0-ebbc-6ed2-f152-1045eb0542f9@os.amperecomputing.com> In-Reply-To: <1d7573c0-ebbc-6ed2-f152-1045eb0542f9@os.amperecomputing.com> From: Matteo Rizzo Date: Mon, 18 Sep 2023 14:08:43 +0200 Message-ID: Subject: Re: [RFC PATCH 00/14] Prevent cross-cache attacks in the SLUB allocator To: "Lameter, Christopher" Cc: Dave Hansen , penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, keescook@chromium.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-hardening@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, corbet@lwn.net, luto@kernel.org, peterz@infradead.org, jannh@google.com, evn@google.com, poprdi@google.com, jordyzomer@google.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 722D51C000D X-Stat-Signature: ke71bg66tr8mnd5yammybzpxe6wj1ei4 X-Rspam-User: X-HE-Tag: 1695038935-976989 X-HE-Meta: U2FsdGVkX1+DGfGIm4tIVb+F80tURb6C99BkUsPpqBEAWoAOM3pobr//2G5Ei7qJ7oTX1/z4FVIlWuZa5FuZUFkon62F81VYTNpAYn5Q/XnUOrSTWRJxSjV7UAgyI/MKxjIWx9bkqFpye9siHF1dnACEwgQr0gdENWzctbZU62SGj5Nbb8HKCd7ZTYbvajECut7jkhxbMdkPcX0cUr1fGHg6dEmE845OPyn6e7QsXhDlQa1S0O9uyXxdW22l28YT6pMh8qSXqjOoAICjkXH3GyEgAhi2oxpZhJcJ+MZqtn5YnSlrllDdPndRaAVpXczw9yW9OYTU7caJSkkK7U/FdtliChwtrhmyD2J/9hJDYsntABPJwlqHheuRlWoaOs6s2HRYEysVXn1AolsvkjmRckABxM1WHcW+SqINFqCzryNeGVosWwVQLoVwFCUjOcTTrUobN83jQMCCIfazUUdwp/5rRQ8rtRh2P6KSUJ18ARIvBdtYO6g62E7mElGdgmei2EPrWKNAD5BOlxGi1BSOr7rjMeVKWKc8w517Ob/aAP+q8kmdu5AEHq673gRMp8Uz0I0Na+TZJubnX+uW4V7XP9ZgtYF5qf0B0xNCwKMawVcGK6aEmjBkrfjAy+fTpDCxCxeDkpwHW+tNOGYkMRmsRQkknzgJW47qTvqM/Utb/56ViW1zFxNWTYpnZLdoHqpkwQ1kbVPBet59DMMFWOPmNGw/uvfFI51IO+h8Sv7rj4nsd5zz3cLsCbv8me5fZBdUTUWhoEnodfd2/A3xrXx/NQaQVKtql99a1K9UaOALCMHyH9tBSdSx8kZR7LES0duF0YqFtO94cUpNEuPdShsPWbP4Pf4zziRnnCni7vYxpmu5qRVdV/xwBOn8lYuIXAegCOSvcbMyhpWScW+Hp+gRWTDeWnON/2PKxNcDUak9jfPV+PQ1NvzS03U4+5TVusdwuf3nwuEz9R3xP2KLaUf UiWMghC+ 6tyoVopqF2k+Zx/i5Y3xlQ/FLNXrlrLr2gkBZBWlWonZAcF5Ls22fQ81+W9XvWf8ezSF6YnCaNu/92odmUSDPapSTbC3Mc5W4NFchDARRj0eAVGYakBJU2Z/BLyAqEnBKlwAss5l7v3HLsZn3GigS/bUsXNxRV2mJ6B8RUqtwwfAx9K7R1Jdx772hNNIqaCua18/nrp8NQYQCqL4IMeeKId0A9uI+pYKJ6O/Ivfy83GPy4d0OZp8S/uNzEajTgXU4Aor2TWitk5rmIKbz8RPLAde0bOyllIaSF4jJP+gTmpSrQGQQ7HTx20xk6LpgpUXTrbt6reU3ZH0Q1yWEAemorSzpSoX4VCz+qnVSH/OitItodMI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, 15 Sept 2023 at 18:30, Lameter, Christopher wrote: > > On Fri, 15 Sep 2023, Dave Hansen wrote: > > > What's the cost? > > The only thing that I see is 1-2% on kernel compilations (and "more on > machines with lots of cores")? I used kernel compilation time (wall clock time) as a benchmark while preparing the series. Lower is better. Intel Skylake, 112 cores: LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV ---------------+-------+---------+---------+---------+---------+-------- SLAB_VIRTUAL=n | 150 | 49.700s | 51.320s | 50.449s | 50.430s | 0.29959 SLAB_VIRTUAL=y | 150 | 50.020s | 51.660s | 50.880s | 50.880s | 0.30495 | | +0.64% | +0.66% | +0.85% | +0.89% | +1.79% AMD Milan, 256 cores: LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV ---------------+-------+---------+---------+---------+---------+-------- SLAB_VIRTUAL=n | 150 | 25.480s | 26.550s | 26.065s | 26.055s | 0.23495 SLAB_VIRTUAL=y | 150 | 25.820s | 27.080s | 26.531s | 26.540s | 0.25974 | | +1.33% | +2.00% | +1.79% | +1.86% | +10.55% Are there any specific benchmarks that you would be interested in seeing or that are usually used for SLUB? > Problems: > > - Overhead due to more TLB lookups > > - Larger amounts of TLBs are used for the OS. Currently we are trying to > use the maximum mappable TLBs to reduce their numbers. This presumably > means using 4K TLBs for all slab access. Yes, we are using 4K pages for the slab mappings which is going to increase TLB pressure. I also tried writing a version of the patch that uses 2M pages which had slightly better performance, but that had its own problems. For example most slabs are much smaller than 2M, so we would need to create and map multiple slabs at once and we wouldn't be able to release the physical memory until all slabs in the 2M page are unused which increases fragmentation. > - Memory may not be physically contiguous which may be required by some > drivers doing DMA. In the current implementation each slab is backed by physically contiguous memory, but different slabs that are adjacent in virtual memory might not be physically contiguous. Treating objects allocated from two different slabs as one contiguous chunk of memory is probably wrong anyway, right? -- Matteo