From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC33BC433F5 for ; Mon, 20 Sep 2021 09:07:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4FCCA608FB for ; Mon, 20 Sep 2021 09:07:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4FCCA608FB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id DF0FE900003; Mon, 20 Sep 2021 05:07:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DA0E1900002; Mon, 20 Sep 2021 05:07:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C68AF900003; Mon, 20 Sep 2021 05:07:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0041.hostedemail.com [216.40.44.41]) by kanga.kvack.org (Postfix) with ESMTP id B6A2A900002 for ; Mon, 20 Sep 2021 05:07:39 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 74AFC181AEF07 for ; Mon, 20 Sep 2021 09:07:39 +0000 (UTC) X-FDA: 78607373838.30.3B8F4E1 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf20.hostedemail.com (Postfix) with ESMTP id F24BCD000650 for ; Mon, 20 Sep 2021 09:07:38 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 90BD7200A9; Mon, 20 Sep 2021 09:07:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1632128857; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GxefN77fejNNXNAqyPi3T1kMi4HccQjMetP8mDr03sM=; b=KbGDApwCysahfDp4ht3j8oFcOzW9V+lNsWU8bUlG10RA+C3sPJovhVPL+8Mi/hk8xbxwvX 4b8j0t6CWw1ch4z70k10gExno1UAXsusvuUdMo1lIChQUKfBLu6oHXOyVVFTVIsqCARlOu HTNRUJ6dR7vQWGqysLJq75k6WHSPTgQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1632128857; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GxefN77fejNNXNAqyPi3T1kMi4HccQjMetP8mDr03sM=; b=ZHeRt1RtsMQcQySCnnpl3U8w0ZtifDbI2ra2eV80ZkhLCxmYsQRn9dd0CSBdkuIrswNGid EQ06xDBttMhP/vDg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 4D1E313A81; Mon, 20 Sep 2021 09:07:37 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 33AUEllPSGHnNAAAMHmgww (envelope-from ); Mon, 20 Sep 2021 09:07:37 +0000 Message-ID: <432da236-4d8c-1013-cd57-42c352281862@suse.cz> Date: Mon, 20 Sep 2021 11:07:36 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.0 Content-Language: en-US To: Matthew Wilcox , Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: Christoph Lameter , David Rientjes , Joonsoo Kim , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Jens Axboe References: <20210919164239.49905-1-42.hyeyoo@gmail.com> <20210920010938.GA3108@kvm.asia-northeast3-a.c.our-ratio-313919.internal> From: Vlastimil Babka Subject: Re: [RFC PATCH] Introducing lockless cache built on top of slab allocator In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: F24BCD000650 X-Stat-Signature: dtjbkdk9cgh8h1u4z8n8ckfydxtw7s4k Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=KbGDApwC; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=ZHeRt1Rt; spf=pass (imf20.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none X-HE-Tag: 1632128858-796708 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 9/20/21 03:53, Matthew Wilcox wrote: > On Mon, Sep 20, 2021 at 01:09:38AM +0000, Hyeonggon Yoo wrote: >> Hello Matthew, Thanks to give me a comment! I appreciate it. >> Yeah, we can implement lockless cache using kmem_cache_alloc_{bulk, free} >> but kmem_cache_alloc_{free,bulk} is not enough. >> >> > I'd rather see this be part of the slab allocator than a separate API. >> >> And I disagree on this. for because most of situation, we cannot >> allocate without lock, it is special case for IO polling. >> >> To make it as part of slab allocator, we need to modify existing data >> structure. But making it part of slab allocator will be waste of memory >> because most of them are not using this. > > Oh, it would have to be an option. Maybe as a new slab_flags_t flag. > Or maybe a kmem_cache_alloc_percpu_lockless(). I've recently found out that similar attempts (introduce queueing to SLUB) have been done around 2010. See e.g. [1] but there will be other threads to search at lore too. Haven't checked yet while it wasn't ultimately merged, I guess Christoph and David could remember (this was before my time). I guess making it opt-in only for caches where performance improvement was measured would make it easier to add, as for some caches it would mean no improvement, but increased memory usage. But of course it makes the API more harder to use. I'd be careful about the name "lockless", as that's ambiguous. Is it "mostly lockless" therefore fast, but if the cache is empty, it will still take locks as part of refill? Or is it lockless always, therefore useful in contexts that can take no locks, but then the caller has to have fallbacks in case the cache is empty and nothing is allocated? [1] https://lore.kernel.org/linux-mm/20100804024531.914852850@linux.com/T/#u