From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HK_RANDOM_FROM,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6013EC433EF for ; Mon, 20 Sep 2021 11:55:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D983560F70 for ; Mon, 20 Sep 2021 11:55:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D983560F70 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 395CE900003; Mon, 20 Sep 2021 07:55:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 34615900002; Mon, 20 Sep 2021 07:55:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20E1F900003; Mon, 20 Sep 2021 07:55:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id 1301A900002 for ; Mon, 20 Sep 2021 07:55:43 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C65FE2488E for ; Mon, 20 Sep 2021 11:55:42 +0000 (UTC) X-FDA: 78607797324.08.9016303 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf24.hostedemail.com (Postfix) with ESMTP id 8122FB0000BF for ; Mon, 20 Sep 2021 11:55:42 +0000 (UTC) Received: by mail-pf1-f176.google.com with SMTP id y4so14461148pfe.5 for ; Mon, 20 Sep 2021 04:55:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=ZKZIHqQbaushXF3880STu4eW1r+MpmKGfOUqwk8tg/I=; b=ecfAL32e1Oan/lTQAHB5xbX4D4fuf130CL99g2ZYFbU1ehlQgoL+ix/nf2B/MKjw0R KdJuxD8zxyd8Jmi5h/FEhkkAKHHW/bNs6t9CGiglJC8Flr4Vn7lMVpXlpopDWyrKwwFG cCGUuU67mG7e/3Bs1Lr2+aJrb+BiUBb8miHLRPJYCqbJJVIF6nCNw+dNmTjP2lrHmAlv dAhd+7UEC2esEhlMT+rUel9wjgHvlyUj12dkiA0TSm2YSZCZ4WnNRAn9w3KT3Tp00Sz5 abNa5G1u+FrZI2MfrDsku/Qk90szWsY25q76qenb4Ckz6FaLIBDF3G59UyBQZyuAo9dS b6JA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=ZKZIHqQbaushXF3880STu4eW1r+MpmKGfOUqwk8tg/I=; b=lBlkWsEWX4YPdza8lk1H7U47NPfZTgIQtbhOShDKzZdqtxzYK3KiiNlzIHcLNbE6BW UaBmYe+3tDfnAiq0f/oOIGrWwnERUmsvvFPYPYyrHmmJ6TCgv6kq7hmGEFY0lesxM8YV 9W4WzaZ8n2bTs4uN54b3jDOlwMhDpEgIPsaHQxQkvwdHnvk2X36yDYLsLPMW8N2CDs29 3sW6Wg+Yal9LXK+/9u/N96hZhd41/8kZNtk3LSSthr+4vqNiZ+DRW9vkDp0LN3DLblr4 2iU8CMTNcszzo3tUCzZPefh3ZA1QTQklg/W5Nqi7s/pF09pf+DFSwzk82IQ/fTcpLm8h jXuQ== X-Gm-Message-State: AOAM531fFdX2vg0CwK4IO/aJAoNiQLGRdoLFoWihCLdAfr9IQ8pFUr8n UteuMmmd2xVgUBzkbv1avOE= X-Google-Smtp-Source: ABdhPJx2ax+msLhKrXItxio1F/lGp05ksCDXkCMJVfHpQGBqDn9jqBHQ6Gvkvv5vbpASWpoN9ffKGQ== X-Received: by 2002:a63:e64a:: with SMTP id p10mr23103490pgj.263.1632138941547; Mon, 20 Sep 2021 04:55:41 -0700 (PDT) Received: from kvm.asia-northeast3-a.c.our-ratio-313919.internal (252.229.64.34.bc.googleusercontent.com. [34.64.229.252]) by smtp.gmail.com with ESMTPSA id t14sm14887742pga.62.2021.09.20.04.55.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 04:55:41 -0700 (PDT) Date: Mon, 20 Sep 2021 11:55:36 +0000 From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Vlastimil Babka Cc: Matthew Wilcox , Christoph Lameter , David Rientjes , Joonsoo Kim , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Jens Axboe Subject: Re: [RFC PATCH] Introducing lockless cache built on top of slab allocator Message-ID: <20210920115536.GA3117@kvm.asia-northeast3-a.c.our-ratio-313919.internal> References: <20210919164239.49905-1-42.hyeyoo@gmail.com> <20210920010938.GA3108@kvm.asia-northeast3-a.c.our-ratio-313919.internal> <432da236-4d8c-1013-cd57-42c352281862@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <432da236-4d8c-1013-cd57-42c352281862@suse.cz> X-Stat-Signature: onsjjz4dbwzog8kabqehtbmuozkin69b Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ecfAL32e; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 8122FB0000BF X-HE-Tag: 1632138942-668563 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Sep 20, 2021 at 11:07:36AM +0200, Vlastimil Babka wrote: > On 9/20/21 03:53, Matthew Wilcox wrote: > > On Mon, Sep 20, 2021 at 01:09:38AM +0000, Hyeonggon Yoo wrote: > >> Hello Matthew, Thanks to give me a comment! I appreciate it. > >> Yeah, we can implement lockless cache using kmem_cache_alloc_{bulk, free} > >> but kmem_cache_alloc_{free,bulk} is not enough. > >> > >> > I'd rather see this be part of the slab allocator than a separate API. > >> > >> And I disagree on this. for because most of situation, we cannot > >> allocate without lock, it is special case for IO polling. > >> > >> To make it as part of slab allocator, we need to modify existing data > >> structure. But making it part of slab allocator will be waste of memory > >> because most of them are not using this. > > > > Oh, it would have to be an option. Maybe as a new slab_flags_t flag. > > Or maybe a kmem_cache_alloc_percpu_lockless(). > > I've recently found out that similar attempts (introduce queueing to SLUB) > have been done around 2010. See e.g. [1] but there will be other threads to > search at lore too. Haven't checked yet while it wasn't ultimately merged, > I guess Christoph and David could remember (this was before my time). There was attempt on SLUB with queueing as you said. I searched a bit and found [2] and [3]. - SLUB with queueing (V2) beats SLAB netperf TCP_RR, 2010-07 [2] https://lore.kernel.org/lkml/alpine.DEB.2.00.1007121010420.14328@router.home/T/#m5a31c7caa28b93a00de3af6d547b79273449f5ba - The Unified slab allocator (V4), 2010-10 [3] https://linux-mm.kvack.narkive.com/e595iCuz/unifiedv4-00-16-the-unified-slab-allocator-v4#post47 Looking at [3], There was still some regression comparing "SLUB with queueing" with SLAB. And I couldn't find patch series after [3] yet. I'll add link if I find. > I guess making it opt-in only for caches where performance improvement was > measured would make it easier to add, as for some caches it would mean no > improvement, but increased memory usage. But of course it makes the API more > harder to use. Do you mean "lockless cache" it should be separate from slab because some caches doesn't benefit at all? > I'd be careful about the name "lockless", as that's ambiguous. Is it "mostly > lockless" therefore fast, but if the cache is empty, it will still take > locks as part of refill? It is actually "mostly lockless" so it is ambiguous. Can you suggest a name? like try_lockless or anything? > Or is it lockless always, therefore useful in > contexts that can take no locks, but then the caller has to have fallbacks > in case the cache is empty and nothing is allocated? > > [1] https://lore.kernel.org/linux-mm/20100804024531.914852850@linux.com/T/#u