From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67524CD37AD for ; Fri, 15 Sep 2023 21:22:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF0D16B03A2; Fri, 15 Sep 2023 17:22:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CA18D6B03A3; Fri, 15 Sep 2023 17:22:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B68BD6B03A4; Fri, 15 Sep 2023 17:22:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A4A526B03A2 for ; Fri, 15 Sep 2023 17:22:24 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 788BA12081C for ; Fri, 15 Sep 2023 21:22:24 +0000 (UTC) X-FDA: 81240105408.17.000C083 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf15.hostedemail.com (Postfix) with ESMTP id 98DCAA0036 for ; Fri, 15 Sep 2023 21:22:22 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="Np3+/Fnw"; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf15.hostedemail.com: domain of keescook@chromium.org designates 209.85.210.171 as permitted sender) smtp.mailfrom=keescook@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694812942; a=rsa-sha256; cv=none; b=JpnRBPYydE4RV9l5NBs8Mu3o9WJnKMl9kMmI1SdZwvRhARmWCXri0F790Eac+nkfkky/td eq/hvI4TfK7z8DJ9KSXlr8w4d6e/txp92a2s0byjf4TMxnqSuM61j8yo/gjKHkfjgVkKb4 qgN8X7T96pgdEDjuXFVu2/2QnlyVA94= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="Np3+/Fnw"; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf15.hostedemail.com: domain of keescook@chromium.org designates 209.85.210.171 as permitted sender) smtp.mailfrom=keescook@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694812942; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=D0kMZ8d93qRdQCVI6CVDzaa2b8htk/6nsTOwdqxIFLQ=; b=GGwnWcnw85wbjast4s0paAuxplI3WxlH7fipCJ8cp3gX01CS5LeJEc/0Xx/HUd5zvzwqWy CQUN8TOUrsq4r2KQ32s9Lq6b7C1OpBEDLHkQWOgpjTGwVjWNIWFeClei6Y6wQfO3N5mbs5 JdHZL8MUVuoNkAsWeVcXp/qZxIxW4p8= Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-690471b5889so964172b3a.0 for ; Fri, 15 Sep 2023 14:22:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1694812941; x=1695417741; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=D0kMZ8d93qRdQCVI6CVDzaa2b8htk/6nsTOwdqxIFLQ=; b=Np3+/FnweJcK8JJt0fQprwcUjOKJ9xqxP/73rHc1n3y88G1kwS83zBJtrHAj7clg6H +FrH6uC1Dpjg0P5rzMmUToFGLNC24VwlSHELaSP1zNyuaoNaNuTUIeN31v8xeHrykN1N jMKrDkMblE56XVek08ZgVTChEYoQRqYR7+GVQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694812941; x=1695417741; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=D0kMZ8d93qRdQCVI6CVDzaa2b8htk/6nsTOwdqxIFLQ=; b=DH/n1mMQBpTCg6qPeHyj/sd0M2wZGmAYLpaOpADCZeHQd7v4TQNUzoB4/q+f2usvBh oiFUH9UdmX6DPcUrLLRz6zEauSa9J7oOMoH39YQubo2LYUJwgpf56/J5XxZnWUQqrx5w cnnaIXIX2eFcJ/CSU/w1VIojkIMgfUEsDJoFKnG0RFJumHD8YDFcEmLiyfXWnzdBiVfi E5vYxmgiSdoBE86Zu7A4IYilCEVxePAxkFH9EdTcjRlbXWKW92bKcFDh4DATvpRt+yQH 4INGrwCScPl/txbWi7cCK41rFUGvP89+UvZJLSw7R92S0efXlg6YinF1PnzujQqmlDoZ 2FwQ== X-Gm-Message-State: AOJu0YyevTMJbzfL4rUuW76zCmCMtShuDvZo8OO6uzFTq4K6iv64FoKN rby8BXMhaUSNL7PARBD0Vmqocw== X-Google-Smtp-Source: AGHT+IFXhYnUgA00/Q6g5wBduIWHI2eXNBUpE7oWPl1Z19J8jTpjt+T/QZNudB9t3IWA9FQ6aC/fsg== X-Received: by 2002:a05:6a00:2344:b0:68e:43ed:d30b with SMTP id j4-20020a056a00234400b0068e43edd30bmr2745959pfj.21.1694812941429; Fri, 15 Sep 2023 14:22:21 -0700 (PDT) Received: from www.outflux.net (198-0-35-241-static.hfc.comcastbusiness.net. [198.0.35.241]) by smtp.gmail.com with ESMTPSA id 15-20020aa7910f000000b006877b0b31c2sm3486022pfh.147.2023.09.15.14.22.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 14:22:20 -0700 (PDT) Date: Fri, 15 Sep 2023 14:22:20 -0700 From: Kees Cook To: Matteo Rizzo Cc: cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-hardening@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, corbet@lwn.net, luto@kernel.org, peterz@infradead.org, jannh@google.com, evn@google.com, poprdi@google.com, jordyzomer@google.com Subject: Re: [RFC PATCH 11/14] mm/slub: allocate slabs from virtual memory Message-ID: <202309151413.A1166F314A@keescook> References: <20230915105933.495735-1-matteorizzo@google.com> <20230915105933.495735-12-matteorizzo@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230915105933.495735-12-matteorizzo@google.com> X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 98DCAA0036 X-Stat-Signature: pia65tr8mrxc9z75wc784n5xf6yzaobo X-HE-Tag: 1694812942-411850 X-HE-Meta: U2FsdGVkX1//s4Owno/h1fuY6zB4anWcnJvgDJWS7GKLVRsf26MLpIIYLNZH810HXkrLKzGqqOhB9PW+vI2JEHjxiWGC3SWKe+/5/2BNH7W4cqXh7YyRo78Sz8k8DqXtFrdBTcPZym3Djm7u3kdzEMO/9soW+e15Yg29KAMIQ2dnGlp5yhdvJQgL4MwkOx/i4FKWmTb6Mn0sJ9ztit/j+rh2QG5Mr9BciWo644zrKdxPGFBN/lSleB7a+loPpHFamJ9RwXxjW7fTsF/g9zRz6Qhs780cxaV0ZYx8S1aSSh4ZnitJTO3S5arxZEk91eNgNfmEv8ZkpBEfUJ5tpNya33Smb8VJKKas9joditn6pIwBwBxi9gPoG0cxFLJovA7X8b+ZPW6ed9s+mvv8gMClkPwQiCTwgPLb/FZomtEyEq7+1nv4Sj0WcF6LynrW4b33CLDtkfc1o86MVUtbVDxqB5I6kGlbx7FQu1IZ54IChaK87ZLeAI89w63O81Es3XNRjFlNmU9O24sX73P/xB3ySYcP8uu5IMHuuSFp5PwJK4SJW+A2+msZarFqjkEHrFVW6scZqSQgxGWmeE7ESSXp3vQCUo+MYPvzX+j1peaon7Mu6waavtauJZ3VeNhB2FfBl/PfpJ9C2wrGBkAcVo4yDZ+EXYXAlAwnWeAM40oDF54N4ys1JYMsM9qENR7M5NiQOBhwY6hLMjW8DQNi2CTMY0cFZEs7idAELvHDiCFxgSkhsjdFHJiRiZWiIKpc+x1Srz1FSiUayOaekzDoZ63VRkM4eQdNagdQJNe3KQ8pp5s34Kdc/OSRtdka1BCKDpicBMDt4yiydu9uxTDv0uAc8Hf7w8a5XMHRiItMmDB2dpVpJOtXfRTf1QbNEBUesFcx99xkISPZ3FktXJWu8NtZ5WPz6SV96ieLPddV5mDh/i4MtTUWuQBgfaQ3+iSFbmMVhp4C0unr4E6IGcwIAwO TOn5FdcJ BuC/GDrJOmX3ru7i+9kQQcYF/Z6Pim0ofHNhQ3dE030eGYRuVzQ6fCEK95561fyouR474DGsQTuWVDyCnWMfp8zvE+1H1USiWm6ECmi3vcndyjkk16GsTKE8nBXeYFhdjFqm6fpwMTR2SC8NfE6LXdwGu7HZrzpAN+u0y3ECAU10twWdeONzexFy5FNS65c4GPaaeuA9x40zk51qZwSJfWpPuHOX3ZkIT+1tjzHaQI8j1Co1CLg4sHZSSPn56xsm7QoG5FwnI2sti+8qvOmrt/GOKaR2BAMIlz4LprrPdivX4yrsmrUXj2gcISAeN8Y4aOuuh1OFPoh0rEGXiPdu/klS76RZJ0bUoG3ySTcuXpjPPLFWZW1IaLqYdHQfpEI2CfBFienYSEp4qdhd8K67kztPWk07v8Am8RaRyHj64nYN93sxGzIKfriJdZ4a0AfZRnlvFaA02VxXPpUBVkTZs96JGGeKEpNBH2zBHjMvjHL8gqX4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Sep 15, 2023 at 10:59:30AM +0000, Matteo Rizzo wrote: > From: Jann Horn > > This is the main implementation of SLAB_VIRTUAL. With SLAB_VIRTUAL > enabled, slab memory is not allocated from the linear map but from a > dedicated region of virtual memory. The code ensures that once a range > of virtual addresses is assigned to a slab cache, that virtual memory is > never reused again except for other slabs in that same cache. This lets > us mitigate some exploits for use-after-free vulnerabilities where the > attacker makes SLUB release a slab page to the page allocator and then > makes it reuse that same page for a different slab cache ("cross-cache > attacks"). > > With SLAB_VIRTUAL enabled struct slab no longer overlaps struct page but > instead it is allocated from a dedicated region of virtual memory. This > makes it possible to have references to slabs whose physical memory has > been freed. > > SLAB_VIRTUAL has a small performance overhead, about 1-2% on kernel > compilation time. We are using 4 KiB pages to map slab pages and slab > metadata area, instead of the 2 MiB pages that the kernel uses to map > the physmap. We experimented with a version of the patch that uses 2 MiB > pages and we did see some performance improvement but the code also > became much more complicated and ugly because we would need to allocate > and free multiple slabs at once. I think these hints about performance should be also noted in the Kconfig help. > In addition to the TLB contention, SLAB_VIRTUAL also adds new locks to > the slow path of the allocator. Lock contention also contributes to the > performance penalty to some extent, and this is more visible on machines > with many CPUs. > > Signed-off-by: Jann Horn > Co-developed-by: Matteo Rizzo > Signed-off-by: Matteo Rizzo > --- > arch/x86/include/asm/page_64.h | 10 + > arch/x86/include/asm/pgtable_64_types.h | 5 + > arch/x86/mm/physaddr.c | 10 + > include/linux/slab.h | 7 + > init/main.c | 1 + > mm/slab.h | 106 ++++++ > mm/slab_common.c | 4 + > mm/slub.c | 439 +++++++++++++++++++++++- > mm/usercopy.c | 12 +- > 9 files changed, 587 insertions(+), 7 deletions(-) Much of this needs review by MM people -- I can't speak well to the specifics of the implementation. On coding style, I wonder if we can get away with reducing the amount of #ifdef code by either using "if (IS_ENABLED(...)) { ... }" style code, or, in the case of the allocation function, splitting it out into two separate files, one for standard page allocator, and one for the new virt allocator. But, again, MM preferences reign. :) -- Kees Cook