From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9A5CC46CD2 for ; Tue, 23 Jan 2024 02:51:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E51B8D0002; Mon, 22 Jan 2024 21:51:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 26E4B6B008C; Mon, 22 Jan 2024 21:51:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E97B8D0002; Mon, 22 Jan 2024 21:51:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id EFD5C6B008A for ; Mon, 22 Jan 2024 21:51:28 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6958FA1CDF for ; Tue, 23 Jan 2024 02:51:28 +0000 (UTC) X-FDA: 81709049856.29.BB1B96E Received: from mail-oo1-f54.google.com (mail-oo1-f54.google.com [209.85.161.54]) by imf19.hostedemail.com (Postfix) with ESMTP id 8B84D1A0003 for ; Tue, 23 Jan 2024 02:51:25 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=e7IcE+mR; spf=pass (imf19.hostedemail.com: domain of zhouchengming@bytedance.com designates 209.85.161.54 as permitted sender) smtp.mailfrom=zhouchengming@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705978286; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=F0ibQ4RvPFjMpe1eoTckSq1lEolGNvZR2rp4j16CDEY=; b=HdJ5/DgonejvUKh2+Os3IXJLIW0rK4nI3Vojx1rswfgFVsTh4bUWVRlYlnaC9o5oRZQFxb dWlFTF/ZNXWbizYXuXrpeHPMcC45Qc+i6Oqr8ZvTT4l1d+2F62zFaH/rYC+xyVA4lYYfw/ w0CrAeHBKvONg59EOZxy2HkG2VjcYnw= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=e7IcE+mR; spf=pass (imf19.hostedemail.com: domain of zhouchengming@bytedance.com designates 209.85.161.54 as permitted sender) smtp.mailfrom=zhouchengming@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705978286; a=rsa-sha256; cv=none; b=ObxSnNhGxeQYkCHczl+6/NQnZIbA7j9ctf4Xtapb68Bg1nnl/B0npEIYorOEJglGl+wOsN CaiLKpNufA+NXYT3Mdl9YWEoU3kBWuBxOqzDl4GrWfopotRQM3HNtN2hzi6oz8Vqxx+O53 BckzdLC+tO1b40Li6HVxSwxGkFtHJu8= Received: by mail-oo1-f54.google.com with SMTP id 006d021491bc7-598f3e9f2e3so2505559eaf.3 for ; Mon, 22 Jan 2024 18:51:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1705978284; x=1706583084; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=F0ibQ4RvPFjMpe1eoTckSq1lEolGNvZR2rp4j16CDEY=; b=e7IcE+mRjPtp4sf3q9uVz5EQ86wS90e/d8WXkskBaAtWyndvpwoT/tNgIk1iNnDiwU Itt6mWEgQMtE2yl/BMvLQiSToKiWg2D7+FKBQWRaZsqXvSvcQjPyRi+PVq81PFqWU38V UFUblrA4W5JCy5anafG0yF1oGs2/jnwH0x+Fg3UG6fafAWCtp6C7shk8q4+EP72jugmL e+3d3Sdj9QrWlOwN69zP4pjOb4kfGcN6gKI6DoWIudf/6pSj9903vdEuwgXHkhNVlW1h +q/khmhWysVNWSD3yNUR7zWbTS6qdWdXu0fLOVDQHFUSkm0tUlTJmFVK+QWG2CsHoaNF AjEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705978284; x=1706583084; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=F0ibQ4RvPFjMpe1eoTckSq1lEolGNvZR2rp4j16CDEY=; b=TxcLwT176Nf4l21XVo33B4ys3tjQ0RKQWpcen9KdFAlCPjD594iMDnijR8BvMGZV6r +aqkaNpiL+tGnczj0sKKQjA3evLrkU71DY6kDpWhBhOFyt7yAxVnj7zSNsv8navDG1N7 Q/QgAiyo1W3PLD7VZGyhLmfPC7cY3Lfc88v0K2FeezB5Yw4VinIGiX7G3Sc9R1NVKVx6 huD1H3i/7EykNkqA4K1EoOiiWWCP2Hfyc6FIuX2kS05VWIk3z869CJ7hmI4asxdtpZ2A mBJZTrMmCXgpTv1QQuxRrJIumYydFRayTVzU5QnBS/J3pTY4mak75tpGjWJnkSNdntLB 3mTQ== X-Gm-Message-State: AOJu0YwneRyvV3P1EjRs6EuPF56K47xPDy1MkN6btxSVccmzxNXFz4ia SwAWW5FZp9ejCn+95wyN+Uj12D98Zo0MLjT/GK3HtCxOsa3vhcip6nXLLz6u7HM= X-Google-Smtp-Source: AGHT+IGE52Gz24GUs56VfplhYi/ooi487/aHOzJj6ZZ2QKzP2AYiHSVxocx0WD9sJCXfLB8D3QeAbg== X-Received: by 2002:a05:6358:d3a7:b0:176:53d4:d1a4 with SMTP id mp39-20020a056358d3a700b0017653d4d1a4mr2646584rwb.3.1705978284347; Mon, 22 Jan 2024 18:51:24 -0800 (PST) Received: from [10.254.227.61] ([139.177.225.243]) by smtp.gmail.com with ESMTPSA id ph16-20020a17090b3bd000b0028e87ce1de0sm10308939pjb.51.2024.01.22.18.51.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 22 Jan 2024 18:51:23 -0800 (PST) Message-ID: <36964450-f45a-4f35-a187-dc493246ef59@bytedance.com> Date: Tue, 23 Jan 2024 10:51:17 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 1/3] mm/slub: directly load freelist from cpu partial slab in the likely case Content-Language: en-US To: Vlastimil Babka , "Christoph Lameter (Ampere)" Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, Joonsoo Kim , Pekka Enberg , Andrew Morton , Roman Gushchin , David Rientjes , linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20240117-slab-misc-v1-0-fd1c49ccbe70@bytedance.com> <20240117-slab-misc-v1-1-fd1c49ccbe70@bytedance.com> <76641777-1918-2b29-b6aa-bda9b5467aa3@gentwo.org> <412b8618-0941-4d9d-85df-ee480695e7f7@bytedance.com> From: Chengming Zhou In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 8B84D1A0003 X-Rspam-User: X-Stat-Signature: 91hukwed55s3yhimycpekmed7tesb4m7 X-Rspamd-Server: rspam01 X-HE-Tag: 1705978285-306981 X-HE-Meta: U2FsdGVkX1+6B7G1ZP8qwTgiVvIHSka1DgDEWkFhdU8xdc6D1gCNrNtkazwoRh/KRFWY6Hb+Ypg2W/yFeivUIgA/p5bWM3K4HQmo5mMLjALsersjbAYplWiIv6/wzNi4NxLbtq1ZEjLfh3/c6IDNn0MajPyjVtxnoDMUvJMUDWHxcHvyK7/cJlGuBFo4yRJa89d6RTYmbTV9qXt9D89lI3f5mPbEObnJ+//rJt5Ar2B0XAqOzvej9B1ng6F9X4anyssgj2q6RMapLdMFJj9dPxYIfImFEpYazxGmJ7vSnfbsY0A17o5d5eZk2vLjWVJZjgOgqxCaxn89/GKyBhM543Mf6iaAT8uscogG0vvVlrs4HocaZXV/CF+X/g/1lATEBbhRdZbsnNKvCJntBb4fJP1czDa+e7ymZbhUtYaHpmnVyezfBbeuwCWXk6AQbX/49YV0/MTOP9SO05TBkEyS+13KFBSnT/mYbaXJvvYQoxKYUSL/DQW+lj9WocvNnsuuyBafhn5XMKjo9NzjXVW+fwBoWeBwlswloopnspLiXS6PT8YpzRCVOZSrpCMIfvnTSf0ag1Rt4fHevoM8IJOjMBsS0l7byLORNw0Ou/TVLFPxxF8zfI0RvnXxAfsiTUSzjhvgofl+dof+IoDDs28qlV9GZKbxl1gjAchqN6JH0pRqFkVUWJsrgRTnqcctXjgJ9/lptsqVHsHYSfPNTnSDtX2G+Ikqq1skEZwyo1UYBPpOZLlp1Y1+2JD/N090XNNXvWATLOoFOWF1UysYQMWaW5+IUgSfTKpfJJcb5vdb3GqZRoebTaLCRY2xDouG0z7oIrZk/XF4VzJQgmD8LDKvxs2j7lV6sMB33G9LyQPuP5iVKApg34s7KT4XeTbXnV/YyJfLCnI9erYwYPfH2qcAY72bgftkygQ1bMJ7soQFRXDeifMm4Ps4lEL01QMB2n6U2k9otLKHg36EQNNDL1O Ibj7qSQJ PhNAQX6bhMJcdPaymAyjz8ZE3uGeZI3VUlRjLmCiDZtJg+8jZEr6ApkkrhumnMHlKxOmxMGYAI6tXqZxBKbWuLbWQ7bcp/Fi0qIdzh39eJnrCXdh4/m+ocVI0XARxuXXaL7xW+Q3gEp4HbsctmhOhG2b2+HdDSjtKi+uZ9Ov/tC1xmzPEMiP8Y+IL3XpPVdtDDcd5Vc536mDsgBoriho793q8qPFOhG18oldd2tVCeAqtzo7F9fkO3c7sAEfa6oj3dtdEUiSb7aXgu9/jzIIbDFlftEDlGfD8mkPWjEdLj3F3ZrIySTpSxizJuiTi41SwkMiNeqjQzBGGsWn9bV9AiZa3rKL5nJWw8aTcTXmMRWhdL3LuGd+FxhlDy+8rCLx1HYd44XPswYlRaBqKBwNeIwSmvEQeHsWc0d+Pu7VvO9SdSATxVeC9BvIECFLvVMcbeJfdbPv2SxirOcIv5wD/r0+RFCNweLk3Lmkc X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/1/23 01:13, Vlastimil Babka wrote: > On 1/19/24 04:53, Chengming Zhou wrote: >> On 2024/1/19 06:14, Christoph Lameter (Ampere) wrote: >>> On Thu, 18 Jan 2024, Chengming Zhou wrote: >>> >>>> So get_freelist() has two cases to handle: cpu slab and cpu partial list slab. >>>> The latter is NOT frozen, so need to remove "VM_BUG_ON(!new.frozen)" from it. >>> >>> Right so keep the check if it is the former? >>> >> >> Ok, I get it. Maybe like this: > > I think that's just too ugly for a VM_BUG_ON(). I'd just remove the check > and be done with that. Ok with me. > > I have a somewhat different point. You reused get_freelist() but in fact > it's more like freeze_slab(), but that one uses slab_update_freelist() and > we are under the local_lock so we want the cheaper __slab_update_freelist(), > which get_freelist() has and I guess that's why you reused that one. Right, we already have the lock_lock, so reuse get_freelist(). > > However get_freelist() also assumes it can return NULL if the freelist is > empty. If that's possible to happen on the percpu partial list, we should > not "goto load_freelist;" but rather create a new label above that, above > the "if (!freelist) {" block that handles the case. > > If that's not possible to happen (needs careful audit) and we have guarantee Yes, it's not possible for now. > that slabs on percpu partial list must have non-empty freelist, then we > probably instead want a new __freeze_slab() variant that is like > freeze_slab(), but uses __slab_update_freelist() and probably also has > VM_BUG_ON(!freelist) before returning it? > Instead of introducing another new function, how about still reusing get_freelist() and VM_BUG_ON(!freelist) after calling it? I feel this is simpler. Thanks!