From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2C9DC021A4 for ; Mon, 24 Feb 2025 20:53:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 57C466B0095; Mon, 24 Feb 2025 15:53:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 52CC36B0096; Mon, 24 Feb 2025 15:53:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41C426B0098; Mon, 24 Feb 2025 15:53:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 23FF56B0095 for ; Mon, 24 Feb 2025 15:53:02 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id AC39C120176 for ; Mon, 24 Feb 2025 20:53:01 +0000 (UTC) X-FDA: 83156037762.23.EFC4C35 Received: from out-170.mta0.migadu.com (out-170.mta0.migadu.com [91.218.175.170]) by imf23.hostedemail.com (Postfix) with ESMTP id A5A2D140010 for ; Mon, 24 Feb 2025 20:52:59 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b="GJ/xwPJe"; spf=pass (imf23.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.170 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740430380; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZxMpDa0xMnH1ZIMUjjLgz553O/WGtdR5AKi8JCJQOAI=; b=5XqJxR0BzoDQTjGs+4Y8Nimvv181iyAEBHQdBzsgk1Vg7imkR6zVHfgEIhW8gfdQ+7/pT3 NBrwXu4jXoKVIzS1Wtw2LyNNMx1YplLn5KfaBYsTDXJ0mMU7+hSKyvb/jsMEyfCoP9kSIs Sj8Ff8amcxLaQY7dCmVmoUYcvHX9sbE= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b="GJ/xwPJe"; spf=pass (imf23.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.170 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740430380; a=rsa-sha256; cv=none; b=fTgtugSf7Pg4jrITwaIaf1MF3uLItMhRjatWx8kv847yVn/YMFV5eUVO/0dSsuMntKfmnK K48cIs4NiEZxFKgOBWfIKWykV+LUXCQxeleL7Mrrp72crY+h0Ee1ChwsQLYDlLuSra7BuT MCMDX3HIzL5bNhGVS15ARnZOqCTyKJ0= Date: Mon, 24 Feb 2025 12:52:53 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1740430377; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ZxMpDa0xMnH1ZIMUjjLgz553O/WGtdR5AKi8JCJQOAI=; b=GJ/xwPJelPfSC8eMwZ4zP4uLV5EZKY9maAjtWDIju3E4ZgxCkkfhgwnM/X8n/F0F73DOBY HUSo4gttew5+DYUQ2sGM1SMLCtGfv6UFLQ2uVU3Sys/Yf8KFB6WRW9tz1IOkzAaagTaFnn Ud1R1Q9olhOGRFjFsONi4tBtjoYH3Ho= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Shakeel Butt To: Vlastimil Babka Cc: lsf-pc@lists.linux-foundation.org, linux-mm@kvack.org, bpf , Christoph Lameter , David Rientjes , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Uladzislau Rezki (Sony)" , Alexei Starovoitov Subject: Re: [LSF/MM/BPF TOPIC] SLUB allocator, mainly the sheaves caching layer Message-ID: References: <14422cf1-4a63-4115-87cb-92685e7dd91b@suse.cz> <704ba4a7-37ec-4c6b-9de4-0c662e5c5ce1@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <704ba4a7-37ec-4c6b-9de4-0c662e5c5ce1@suse.cz> X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A5A2D140010 X-Stat-Signature: wbyot1zf9bi8fat8sam9w5fq58368c1w X-Rspam-User: X-HE-Tag: 1740430379-4142 X-HE-Meta: U2FsdGVkX1/EYl7YBO1GiC21QCZfVEsHRBgTbIuaSpDfLqTOICy7MBZlEkqtBWHqbtK1E8istELLYcP2CY6QfxguSNNde0A7ENhY5nQO1b+UYtcfp6nf+eDwfCmRHm06wBynfsHXEMKVLe/rIfPsos775mbwJjRt60LnW/roQYdJ2/wyTc4RDPu50bymdHPOe4Ov2C6X8S+R4ifdVySzezzdNpxo7Jk78cluBFjK5ObxqI7X7ztd3ifyo8fKAop8KJ3va1MWhuFayT6QH/tyvCMGd08VrgdxGV2NZdGrNJIqSVIr6j+9cXeROG+XxpGBs3XLRajRHWoNZAeCkjmcGKutCPSAyMb+yrcnrDdx+t+wch3atI9C8bXvlUoBm4LZxVIRW96bqk4Gr+7k0BAxBBhQCIieCebwF+U7PoGh3bznMfdZKFPe7ayIiXPFl462MFpSP7IuKNkEVxLZ906QKhS7yA3jQPe7/djeMblZen1IqfWcvbT8bNJ3od0+zT27ivdzRa2oEaQT9B8bCWAKMrWewtmrHWhGYQCmCd46WYNrEhT8mXrXTFjJSko/FSuNumKN8N7BkqY4KYIXwnAo/uyjQ0upKjLhrt5v5WzOuLlwPDimcXmKzqt2VRmhtBFBc84fRC0StfuQc5RvKFswDJAKG5kkuVjzefhAEvrTrc7oFlXIBRHeF392N5+YSzc2WhNrIJQ/JNemXbTZyb2sB6Jbm8aLECz5PDi55lI97EAibw8g/xpr6z6F5IAdjoe4x4RQatn/s7QtjSzAD5FuUgHDxJ2uYzkYycU6hLKR1E093Qi8goP4h5TcihB4NETlwH5tRidmkU9fjs1vK/faY/pLSWacxpgODaBNndp14zrgaimsepimNNpDSuq7PrQkdpUKJr4WhM2Oa/7IpdnvLG87Gx57/SQ7WDvSowsVRY6/EKLuwmtS22l8qv1sDpc7Os50OWLcRKZVEbeo510 4hl+pqF1 8K85jHZDt1Oki8cXTh1jPqd2FMCT2kjGcoO9q6flAsdzE+OxEdh7kTguy7P1Gcjq6ehRzyLCR9c/NEqq1S0O3E9kSP/kU67x+B9/kKL/jiyEd/kLy7WqbU9A31JsLF0Fbs0igQ2q+jbPjRdBRPuehbJVlA9LhAAlbLCewTrT13rt1b4CvVU6ESBRiDWDj4jwPo/yNextvigYAdZg92LtT9k32JmPNLePwBwnzJo231ISMDtqn59lVbGOl/SllgWIUN+xs X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Feb 24, 2025 at 07:15:16PM +0100, Vlastimil Babka wrote: > On 2/24/25 19:02, Shakeel Butt wrote: > > On Mon, Feb 24, 2025 at 05:13:25PM +0100, Vlastimil Babka wrote: > >> Hi, > >> > >> I'd like to propose a session about the SLUB allocator. > >> > >> Mainly I would like to discuss the addition of the sheaves caching layer, > >> the latest RFC posted at [1]. > >> > >> The goals of that work is to: > >> > >> - Reduce fastpath overhead. The current freeing fastpath only can be used if > >> the same target slab is still the cpu slab, which can be only expected for a > >> very short term allocations. Further improvements should come from the new > >> local_trylock_t primitive. > >> > >> - Improve efficiency of users such as like maple tree, thanks to more > >> efficient preallocations, and kfree_rcu batching/reusal > >> > >> - Hopefully also facilitate further changes needed for bpf allocations, also > >> via local_trylock_t, that could possibly extend to the other parts of the > >> implementation as needed. > >> > >> The controversial discussion points I expect about this approach are: > >> > >> - Either sheaves will not support NUMA restrictions (as in current RFC), or > >> bring back the alien cache flushing issues of SLAB (or there's a better idea?) > >> > >> - Will it be possible to eventually have sheaves enabled for every cache and > >> replace the current slub's fastpaths with it? Arguably these are also not > >> very efficient when NUMA-restricted allocations are requested for varying > >> NUMA nodes (cpu slab is flushed if it's from a wrong node, to load a slab > >> from the requested node). > >> > >> Besides sheaves, I'd like to summarize recent kfree_rcu() changes and we > >> could discuss further improvements to that. > >> > >> Also we can discuss what's needed to support bpf allocations. I've talked > >> about it last year, but then focused on other things, so Alexei has been > >> driving that recently (so far in the page allocator). > > > > What about pre-memcg-charged sheaves? We had to disable memcg charging > > of some kernel allocations > > You mean due to bad performance? Which ones for example? Was the overhead > due to accounting of how much is charged, or due to the associating memcgs > with objects? > I know of the following two cases but we do hear frequently that kmemcg accounting is not cheap. 3754707bcc3e ("Revert "memcg: enable accounting for file lock caches"") 0bcfe68b8767 ("Revert "memcg: enable accounting for pollfd and select bits arrays"") > > and I think sheaves can help in reenabling > > it. > > You mean by mean having separate sheaves per memcg? Wouldn't that mean > risking that too many objects could be cached in them, we'd have to flush > eventually e.g. the least recently used ones, etc? Or do you mean some other > scheme? > As you pointed out a simple scheme of separate sheaves per memcg might not work. Maybe targeting specific kmem caches or allocation sites would be a first step. I will need to think more on this.