From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C0F3C47258 for ; Tue, 23 Jan 2024 08:24:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 876E38D0001; Tue, 23 Jan 2024 03:24:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 828F26B0093; Tue, 23 Jan 2024 03:24:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6C8348D0001; Tue, 23 Jan 2024 03:24:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5D8BB6B0092 for ; Tue, 23 Jan 2024 03:24:36 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id D2E07120225 for ; Tue, 23 Jan 2024 08:24:35 +0000 (UTC) X-FDA: 81709889310.17.1D0B3F3 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf19.hostedemail.com (Postfix) with ESMTP id 5EF9A1A0004 for ; Tue, 23 Jan 2024 08:24:33 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Q5oqOpBR; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=NaDyXDrk; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Q5oqOpBR; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=NaDyXDrk; spf=pass (imf19.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705998273; a=rsa-sha256; cv=none; b=enwygpCrOge7ki75YPTvecvjuVk68DRIKGltcKl2UfoHL2OL85W8WgJSVjYxo2CBHUykAQ 6beb8cKABSM1Gqaoc2q9uf0/Xu6ZDyJGs9ybLHT5ymOj/4DjfqsjF0CBfzMW+SIt8vtsLE NDLEiYSiJCrf0+D8yxTvd6e5veczKmw= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Q5oqOpBR; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=NaDyXDrk; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Q5oqOpBR; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=NaDyXDrk; spf=pass (imf19.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705998273; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qF9pTM2iZocrtZVj2sV+edGXZT+tYrsByVnKG403QCk=; b=ljxIt2siamfjwvw242GK0SQm+KxQsPrKvgMvzzWp1NRi48qiot6qNTPHgjDsX9BzKLkZ88 ml9vuanjf4Xe9b2OXDoNN1DEYXFVfycZizUCtQ+l/MzKvRYo/ssqzkcnHZbHtrHZZ4OL8h QgUqw+m6UCkDwIW7r23YOYsaP0eB5yg= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 8DE811F76B; Tue, 23 Jan 2024 08:24:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1705998271; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qF9pTM2iZocrtZVj2sV+edGXZT+tYrsByVnKG403QCk=; b=Q5oqOpBRfYsOK9W2ZmmZRKdqfxnIJYTZb7b/LgwHU0/R19Qvdfl3c15OBI8Bd1xw6Tl3Gz WXp6vVo8X59UKcqUHvcfueP7eIH1zcuK79BgzYjs0Hlo7XfD3zjwifxNXS3GdgjqwiK9Gj /6MoaqHkLTJnbylUcipN84bOwqrqfe0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1705998271; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qF9pTM2iZocrtZVj2sV+edGXZT+tYrsByVnKG403QCk=; b=NaDyXDrkSWboFPGhW2kgv0r+9725XNgEZdRzOuRrihNQwhuiPmUAyOV71f7yuovZQLQR4j ODL5E+7FNcT+r1Aw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1705998271; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qF9pTM2iZocrtZVj2sV+edGXZT+tYrsByVnKG403QCk=; b=Q5oqOpBRfYsOK9W2ZmmZRKdqfxnIJYTZb7b/LgwHU0/R19Qvdfl3c15OBI8Bd1xw6Tl3Gz WXp6vVo8X59UKcqUHvcfueP7eIH1zcuK79BgzYjs0Hlo7XfD3zjwifxNXS3GdgjqwiK9Gj /6MoaqHkLTJnbylUcipN84bOwqrqfe0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1705998271; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qF9pTM2iZocrtZVj2sV+edGXZT+tYrsByVnKG403QCk=; b=NaDyXDrkSWboFPGhW2kgv0r+9725XNgEZdRzOuRrihNQwhuiPmUAyOV71f7yuovZQLQR4j ODL5E+7FNcT+r1Aw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 754D713786; Tue, 23 Jan 2024 08:24:31 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 7G5CHL93r2VxcAAAD6G6ig (envelope-from ); Tue, 23 Jan 2024 08:24:31 +0000 Message-ID: <0974c3b7-a964-44b6-a588-e08c6f79eec9@suse.cz> Date: Tue, 23 Jan 2024 09:24:31 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 1/3] mm/slub: directly load freelist from cpu partial slab in the likely case To: Chengming Zhou , "Christoph Lameter (Ampere)" Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, Joonsoo Kim , Pekka Enberg , Andrew Morton , Roman Gushchin , David Rientjes , linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20240117-slab-misc-v1-0-fd1c49ccbe70@bytedance.com> <20240117-slab-misc-v1-1-fd1c49ccbe70@bytedance.com> <76641777-1918-2b29-b6aa-bda9b5467aa3@gentwo.org> <412b8618-0941-4d9d-85df-ee480695e7f7@bytedance.com> <36964450-f45a-4f35-a187-dc493246ef59@bytedance.com> Content-Language: en-US From: Vlastimil Babka In-Reply-To: <36964450-f45a-4f35-a187-dc493246ef59@bytedance.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 5EF9A1A0004 X-Stat-Signature: o9oerij3dhkoh3z1zrxzujq8io1cpqbo X-Rspam-User: X-HE-Tag: 1705998273-244414 X-HE-Meta: U2FsdGVkX18hZPwwWj0+eoRV+noPO47/UNv5RUVpHUWIzrWrRZNFEeHMxEc+8faJBymiWJRlioALsLABcSHsviBGy6tCJQQkeT0hLbvF2YwV7zt8OAdVGclN6zIgCLpjVoyFRU8o2LiAhpHOemsSDONedSRcYdY0JgLp8U/Zc0vznEYp6bpnQAYlCTxEkg4rH3nARlX76cGgF2e9yIFh9bv56mMiFvIiy+Ky4V3iC0t5krTLwauf5QiM+80hw+w5v+eB8G4FwbGT0pGoGRcH42MC6McEaQzJ7oWhothuZs/S8OZDzE+kFhE8NpOkb9flNNqBW5NyAipZL1OW6Vli6+LmUDiMkjZehmiZ+tiz/11JNnFbyVRavF16s3CVSma+KvRUQEuNM5vLJNKX6DmWd6OC5ISB4uCx+evnikjaGvlCs9sjQSVZkAPbdMa/tethV9n/JLzEsxsTYxw6OgYCMVv2VXrkQGl8P9ppL2ysCF6uYXugrDAdgYofooTMnoC+4G1sWRfmksJJ6mFirLX9kqLf+sM9uJRwUxRDgTB2sQzU2v2lsVIDv1TNy/7ySF9VbtdxvVjZ9gg0IEXj26QWhS/D0rh6g4qWbkcOItwHUsVjBi06GuYT8rVCZTfwM/bZCYFdkmEjdUwXMHzKZLA0q78r3BgnnfSiJtTjMC3mudRifpFYMhH9GB2OM44aalnMq2GrFZWacAIySmNPvqWtYcwCmXf8obYFgMtI5JWsDYdP+yd6xEUCxqXMBpsiM0DdC5qcq75M4v3oN3lqWZIaVDyzjcpyP5RzEUzrBsQnzVYYwLKoIVOS55mYUXx9yhBfssd5WfHHjuZZOsy7iRWUct4MxptcArMryqUNB5mRxdIGCHuQ3oVPYbR/3FRl2GbErnH8aEa3+8EpzcYgfmOMnoZmuzaBOXo2GjkaQOri1P4CAtJBzyie3FKv8T1najX0H4cQvjYJ0tga8tMlfyx 8pp0/30x kVGHp+d9ZuLTOfqsBDqz9WQoKUjl1NiMJm5cnNqnenduGtPGc2OtMAxVRf/y/OxqMHNZrIgnH/GoWlaAZKLCi90nBAUG2AqOevhYf4qlZ0fJgOcOeIANs5GYlWh+zh6ceHDABpRjJpJB4HxZq5ktgNLTEFPYTINW8+Y8KFJcP0RmtngD7SFmJg7ZSgMkbjqhgZXm6O8mPHKXpsctSy+cl27m5RpGo3sKWv7Gf1aeL1Q+k+k5Ng1uT2h3gLL631Svg7aMbCL+a8qC8IhfXlPsMB24T5h/opoxuyxb2O1gEkT1+rlDDk1Z7RrsLqL3LWzqtZiWHh2LqFQR8QQE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 1/23/24 03:51, Chengming Zhou wrote: > On 2024/1/23 01:13, Vlastimil Babka wrote: >> On 1/19/24 04:53, Chengming Zhou wrote: >>> On 2024/1/19 06:14, Christoph Lameter (Ampere) wrote: >>>> On Thu, 18 Jan 2024, Chengming Zhou wrote: >>>> >>>>> So get_freelist() has two cases to handle: cpu slab and cpu partial list slab. >>>>> The latter is NOT frozen, so need to remove "VM_BUG_ON(!new.frozen)" from it. >>>> >>>> Right so keep the check if it is the former? >>>> >>> >>> Ok, I get it. Maybe like this: >> >> I think that's just too ugly for a VM_BUG_ON(). I'd just remove the check >> and be done with that. > > Ok with me. > >> >> I have a somewhat different point. You reused get_freelist() but in fact >> it's more like freeze_slab(), but that one uses slab_update_freelist() and >> we are under the local_lock so we want the cheaper __slab_update_freelist(), >> which get_freelist() has and I guess that's why you reused that one. > > Right, we already have the lock_lock, so reuse get_freelist(). > >> >> However get_freelist() also assumes it can return NULL if the freelist is >> empty. If that's possible to happen on the percpu partial list, we should >> not "goto load_freelist;" but rather create a new label above that, above >> the "if (!freelist) {" block that handles the case. >> >> If that's not possible to happen (needs careful audit) and we have guarantee > > Yes, it's not possible for now. > >> that slabs on percpu partial list must have non-empty freelist, then we >> probably instead want a new __freeze_slab() variant that is like >> freeze_slab(), but uses __slab_update_freelist() and probably also has >> VM_BUG_ON(!freelist) before returning it? >> > > Instead of introducing another new function, how about still reusing get_freelist() > and VM_BUG_ON(!freelist) after calling it? I feel this is simpler. Could you measure if introducing new function that sets new.frozen = 1; has any performance benefit? If not, we can reuse get_freelist() as you say. Thanks! > Thanks!