From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5FC5FA3741 for ; Tue, 1 Nov 2022 00:18:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EDB1580007; Mon, 31 Oct 2022 20:18:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E8AD36B0075; Mon, 31 Oct 2022 20:18:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2BF780007; Mon, 31 Oct 2022 20:18:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C084E6B0074 for ; Mon, 31 Oct 2022 20:18:47 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7CB3A1C6737 for ; Tue, 1 Nov 2022 00:18:47 +0000 (UTC) X-FDA: 80082962694.11.27CD134 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by imf12.hostedemail.com (Postfix) with ESMTP id 1CC7E40004 for ; Tue, 1 Nov 2022 00:18:46 +0000 (UTC) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 7E4A85C00DA; Mon, 31 Oct 2022 20:18:46 -0400 (EDT) Received: from imap46 ([10.202.2.96]) by compute5.internal (MEProxy); Mon, 31 Oct 2022 20:18:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com.au; h=cc:cc:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1667261926; x=1667348326; bh=fi mXTSH5ZLpHfGKWM0j4zT+1so1IP4NpiYE5OssGC/c=; b=J1pUU7BERazf1IeLBd jAWNGQAB2kKYNThwgOANWSBBBAbnUxGN9q8hSYqVwMI4gYFnfyN3UQzun4eLndEd IwCpynC7CK/cLg+QktJAiVQ8FimTS7jnB2iT8OKV5IylFxdHuZexPndnPJ9KFnZa ekksJw1NoKXHNH7GmqVdgNRtHQkqB04ySo8BOi3agwwS0BbxFiSo/cwubK/RuQm4 kWW/w/WGH/yRkWTBo2WLo8vnDwxPl5N2OKH7GHTtExEI+MU8MWS4POPweskpvect 7IhaXIBh+G0u+ERvlnQ7inrwxxKnKWBJz8G5PhTlIZjzq9UPrCUmp7WC72r3eO6+ CdEQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1667261926; x=1667348326; bh=fimXTSH5ZLpHfGKWM0j4zT+1so1I P4NpiYE5OssGC/c=; b=Z3P5fypnqKglxy1Mw/0alIqLLvyCi1HVyzxSDnZUIyDA XDYW6nDULoHv3jXTEv8CHHdB5YHQX9qxPzDC/3F5WkGI9QvgTaHc3XhMfr3VNMWM oleabt+4AMlPllGgscypqUTZigHmVPaURtZuQBf5PCqPC5sXiZhMV40oin9+eaNx jAQh+fg33NfDaFtEKElq15W2vVlE/CA/1fLa32BmN2xeZeGXSpVm9L9tfLWx2Xzs XUTyPV09IF7ioG5c4tqrK4XOAUNPMmDT9cPRCRQUKCghaGBT6Y1ngTGyfHF5/4rI R9CzV04lyJR8CfR9FkHOiRQQx7NoUP3XujvTboTFGA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrudeggddvudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefofgggkfgjfhffhffvvefutgesthdtredtreertdenucfhrhhomhepfdflohhh nhcuvfhhohhmshhonhdfuceolhhishhtshesjhhohhhnthhhohhmshhonhdrfhgrshhtmh grihhlrdgtohhmrdgruheqnecuggftrfgrthhtvghrnhepkeefhffguddtgeegjedtvedt vddvkeevvdehfeehieejhffhhfejkeejgfelveegnecuvehluhhsthgvrhfuihiivgeptd enucfrrghrrghmpehmrghilhhfrhhomheplhhishhtshesjhhohhhnthhhohhmshhonhdr fhgrshhtmhgrihhlrdgtohhmrdgruh X-ME-Proxy: Feedback-ID: ia7894244:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id A370A2A20080; Mon, 31 Oct 2022 20:18:44 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.7.0-alpha0-1087-g968661d8e1-fm-20221021.001-g968661d8 Mime-Version: 1.0 Message-Id: <9b71ae3e-7f53-4c9e-90c4-79d3d649f94c@app.fastmail.com> In-Reply-To: References: <20220913065423.520159-1-feng.tang@intel.com> <20220913065423.520159-2-feng.tang@intel.com> Date: Tue, 01 Nov 2022 00:18:19 +0000 From: "John Thomson" To: "Feng Tang" Cc: "Vlastimil Babka" , "Andrew Morton" , "Christoph Lameter" , "Pekka Enberg" , "David Rientjes" , "Joonsoo Kim" , "Roman Gushchin" , "Hyeonggon Yoo" <42.hyeyoo@gmail.com>, "Dmitry Vyukov" , "Jonathan Corbet" , "Andrey Konovalov" , "Hansen, Dave" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , "kasan-dev@googlegroups.com" , "Robin Murphy" , "John Garry" , "Kefeng Wang" Subject: Re: [PATCH v6 1/4] mm/slub: enable debugging memory wasting of kmalloc Content-Type: text/plain ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667261927; a=rsa-sha256; cv=none; b=u5mkQWoNBxEO6TWm+BSR5oeNEPbD9H7wHpG5r662ovA3jr+A4VMd8vNxUVVRm9yFvxGTON 8xHJ4YcNhM3PIsU9KQ67IFG2OuTAGARnDHMUZHzZPl/mlboLkCmbIdued4Fu0BMAF0fuoR MAQTJwEuWtXO2jri/yFEPIXeAgfxhGU= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=fastmail.com.au header.s=fm3 header.b=J1pUU7BE; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=Z3P5fypn; dmarc=pass (policy=none) header.from=fastmail.com.au; spf=pass (imf12.hostedemail.com: domain of lists@johnthomson.fastmail.com.au designates 66.111.4.25 as permitted sender) smtp.mailfrom=lists@johnthomson.fastmail.com.au ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667261927; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fimXTSH5ZLpHfGKWM0j4zT+1so1IP4NpiYE5OssGC/c=; b=d40uj2dJDNPb6WlbWcH+G57I5GvSkytIQ4pp7RjZ+QsCseEvOUV64w2iR7ljD7CGGluJfW mtZK1iYzzbO02NQCVFF0JZ2r6t16NMH7hZToBFY5gcD+b5DHrquBHQLOn4HCkuO53jpxnK rg38mN1sM+xeybr7gj5eNMAGBXpXifs= X-Rspamd-Server: rspam02 X-Rspam-User: Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=fastmail.com.au header.s=fm3 header.b=J1pUU7BE; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=Z3P5fypn; dmarc=pass (policy=none) header.from=fastmail.com.au; spf=pass (imf12.hostedemail.com: domain of lists@johnthomson.fastmail.com.au designates 66.111.4.25 as permitted sender) smtp.mailfrom=lists@johnthomson.fastmail.com.au X-Stat-Signature: gdxebw5xb5866iyimktp3reks7gh3gi4 X-Rspamd-Queue-Id: 1CC7E40004 X-HE-Tag: 1667261926-277193 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, 31 Oct 2022, at 11:42, Feng Tang wrote: > On Mon, Oct 31, 2022 at 10:05:58AM +0000, John Thomson wrote: >> On Mon, 31 Oct 2022, at 02:36, Feng Tang wrote: >> >> > >> >> > possibly relevant config options: >> >> > grep -E '(SLUB|SLAB)' .config >> >> > # SLAB allocator options >> >> > # CONFIG_SLAB is not set >> >> > CONFIG_SLUB=y >> >> > CONFIG_SLAB_MERGE_DEFAULT=y >> >> > # CONFIG_SLAB_FREELIST_RANDOM is not set >> >> > # CONFIG_SLAB_FREELIST_HARDENED is not set >> >> > # CONFIG_SLUB_STATS is not set >> >> > CONFIG_SLUB_CPU_PARTIAL=y >> >> > # end of SLAB allocator options >> >> > # CONFIG_SLUB_DEBUG is not set >> >> >> >> Also not having CONFIG_SLUB_DEBUG enabled means most of the code the >> >> patch/commit touches is not even active. >> >> Could this be some miscompile or code layout change exposing some >> >> different bug, hmm. >> >> Yes, it could be. >> >> >> Is it any different if you do enable CONFIG_SLUB_DEBUG ? >> >> No change >> >> >> Or change to CONFIG_SLAB? (that would be really weird if not) >> >> This boots fine >> >> > I haven't found any clue from the code either, and I compiled >> > kernel with the config above and tested booting on an Alder-lake >> > desktop and a QEMU, which boot fine. >> > >> > Could you provide the full kernel config and demsg (in compressed >> > format if you think it's too big), so we can check more? >> >> Attached >> >> > Thanks, >> > Feng >> >> vmlinux is bigger, and entry point is larger (0x8074081c vs 0x807407dc revert vs 0x8073fcbc), >> so that may be it? Or not? >> revert + SLUB_DEBUG + SLUB_DEBUG_ON is bigger still, but does successfully boot. >> vmlinux entry point is 0x8074705c > > Thanks for prompt info! > > As I can't reproduce it locally yet, could you help try 3 tests separately: > * change the O2/O3 compile option to O1 > * try the attached 0001 patch (which cut part of commit) > * try attached 0001+0002 patch None of these changed my outcome. I also tried compiling the same linux tree & config with the Bootlin toolchain (mipsel-buildroot-linux-gnu-gcc.br_real (Buildroot 2021.11-4428-g6b6741b) 12.2.0) with the same results. I will look into finding or building a mips clang toolchain. No JTAG capability to debug, sorry. I get the same outcome with either the ZBOOT vmlinuz, or vmlinux Same happening with 6.1-rc3 After some blind poking around changing (how much of the commit affected) mm/slub.c, I may have got lucky. it appears as though this is all I need to boot: (against 6.1-rc3), and with the Bootlin toolchain. Will test my other build system as well. --- a/mm/slub.c +++ b/mm/slub.c @@ -3276,7 +3276,7 @@ static void *__slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, c = slub_get_cpu_ptr(s->cpu_slab); #endif - p = ___slab_alloc(s, gfpflags, node, addr, c, orig_size); + p = ___slab_alloc(s, gfpflags, node, addr, c, 0); #ifdef CONFIG_PREEMPT_COUNT slub_put_cpu_ptr(s->cpu_slab); #endif Would like to hear your thoughts, but I will keep digging. > > Thanks! > > > - Feng > > > > > Attachments: > * 0001-reduced-slub-patch.patch > * 0002-reorder-the-partial_context-initialization.patch -- John Thomson