From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B1F1C27C54 for ; Thu, 6 Jun 2024 16:53:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9485A6B00AA; Thu, 6 Jun 2024 12:53:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F7FC6B00AB; Thu, 6 Jun 2024 12:53:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7BFF46B00AC; Thu, 6 Jun 2024 12:53:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5CD556B00AA for ; Thu, 6 Jun 2024 12:53:53 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 857A514179A for ; Thu, 6 Jun 2024 16:53:52 +0000 (UTC) X-FDA: 82201060704.19.604B648 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf12.hostedemail.com (Postfix) with ESMTP id 811414000B for ; Thu, 6 Jun 2024 16:53:48 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=DSQS1HFl; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf12.hostedemail.com: domain of vbabka@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=vbabka@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717692829; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CXkQIcH8/l+6fWsuP2KqStMF4NvMD2uUROMEiEGLHQw=; b=CVVvev0NjFgDPp0sL9Cg5bSeVdirPRlUaeitwAIziIuol8xqEHTKB4cIiDgiYXirsJm5H9 lZ356Kh6Tq6yOjuQ+q9RxWNyO57Ij4rq8HF7VVoWAPTcdYdbrW3XfAJflZaqmApxb3Abkl qiYxWWB4wulVLSBHcTeKAFobRXvyomA= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=DSQS1HFl; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf12.hostedemail.com: domain of vbabka@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=vbabka@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717692829; a=rsa-sha256; cv=none; b=VlSeIBwB9WW71muLnnp6u8gw/lMAwCnz72Wj4sdfklmd7DhJ6z9VperPrBU3PxGnJiiHVd A1C3I43EGGZF1YPRcspOqpupM4KFdCdfaJ9pkQdBNsTuum2z7V6xsAGmDUb8iWBviBxjwI wMRuk8+lqpRK2M4JcApd0IOzg7xMq7g= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 46A2FCE1BFC; Thu, 6 Jun 2024 16:53:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0919AC2BD10; Thu, 6 Jun 2024 16:53:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717692822; bh=UfPRa8N1NpQXy0hUsyfCoV9/IzKPawQ3WxpkEv4gwoQ=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=DSQS1HFlpBRZ83O1xKUSI5diJtuOlVYBr2LgczQEvleXWNd6HpjbexSOBzjkR5WbY BSqiVya+7Z5Bxpd4p20MoH6QOAgx/Yz2qeT8KVm2+c702w87lV+lvfZKQtx/IHYQBm 9SXDanaV5WWdi9qzuEQcE+lIZWxbw3xOX44VoXmhi7nWvpqr+DTOSjMf7gkEHSIHGR KTWPQ+OvIoPA48E78fLk923VjkKy2Oi09ajoH1Hs9dRIYbCMCpdyOhVxh7Bbkx9rWV dcJHbFamIVno+gXC6hCG/CPK3BIgvvhxAsmR4o9jYW5ToB/sh5wGsfDcMSSReJG8FP +54Rq6Ik/+LXg== Message-ID: Date: Thu, 6 Jun 2024 18:53:36 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: kswapd0: page allocation failure: order:0, mode:0x820(GFP_ATOMIC), nodemask=(null),cpuset=/,mems_allowed=0 (Kernel v6.5.9, 32bit ppc) Content-Language: en-US To: Erhard Furtner Cc: Yosry Ahmed , Yu Zhao , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Johannes Weiner , Nhat Pham , Chengming Zhou , Sergey Senozhatsky , Minchan Kim , Michal Hocko References: <20240508202111.768b7a4d@yea> <20240515224524.1c8befbe@yea> <20240602200332.3e531ff1@yea> <20240604001304.5420284f@yea> <20240604134458.3ae4396a@yea> <20240604231019.18e2f373@yea> <20240606010431.2b33318c@yea> <20240606153210.18ef5299@yea> From: "Vlastimil Babka (SUSE)" In-Reply-To: <20240606153210.18ef5299@yea> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 811414000B X-Stat-Signature: bhsuzq9t46wgipwc387sxm3p9495o87u X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1717692828-207304 X-HE-Meta: U2FsdGVkX18Kh07kN78DJNifXmhQFZ5BBQBzexolT2cguyJ+GqznOlx99NgDnzNzYP5CxIz/YIhU3IQ39A0IwaledXjC8Ae38Zar9+K6mT8/x9Nm4I7JTuz5lyT78F3lsJxVfTkAfwwdSGNw++sZNAtcssII872dVEalQhpEhKD8hT8prIrgaCrkzAWC8ApwtYhv7Sb5FMALUeMFxMdAm+OTWWr1D5EacWrEtRd78epRiowd80lvllG9jrzsDoXAgBpSvIRFMLNXP3l6rZAjhwWND9fD1v31Ybx9LI5YY9bviYXJudgT9T5n4q5YHFBQ7aO4x9m5nC/SMGAT1lcIcZbQB/dklIKZrbcusNnHe07VfVIDkjf/SWlqQrUKjIwcHv8AvoDgfY1ZweSO39JG99IvLLVrfNVcwoNaYCXy+9HqQ5pv/NJAbGr2lf2kO1ByfSkj1k+HxZ0RwCYeo0wYSjpdYXVmlxsDPAECBdLnTtxIuwvhvWaOuYOpHO1gtVNk/aJTcHrAx/k/DKaFXvVF7/sJF4BgN+YNX3Wuw+i1NbrHNrx4Kd2iCggSggHRnlN8yULnwyZ48TdigcjjUY9xg3fAuDt5FTTVtLzAHX5ChH1ii4sbtj84UndpYbPs6iYxezt20FG80YNpNATcXG6UKmUmz8ZLD8b3SneAO47XAi5zKptGMwywFObFH5frgCWbKZzNWDpvgPHhFXQHos1abjmbqWLT4tZFSrDbLNNQ+T2LNX6gpM0s3eiGAXm7oQMzJ6zBBagiqZypeSOR65vTNRt085zIS6c78/Me77E9oc6bE8GVokv+FdAxkpR82s5IUmTYJYCOtBXzofVbrfk2Om//FNv+fhqTLeKQvY6HqhZCjh8tGwLj5U/ZvGNyE9UwrRWJSt6IZT7+WZXMEcXSwfz7ah9lBif6TDP4KxfHrmCRQeDkJPZVaqc0k7D6vqhEiKy5XvGkDl47vBmLb6r fIWrfX12 +5xVrbdFCHm8xL6DPr3JVfIS4nWGSQXswVlt4WE8NQvjE4R7wbakl3Egf7BOa5CpYJeILk3ZogbSmoHa+a3aZ4Kq30FaFUFlrsxjkk53YY7+DfnCF6jo2LURiq4gIpYKOQ9oC7LNsmOhE6du8Ptmz99rg3u/DEuVJ403XCq/RKuLUQBZLKztXHnE9Uz64XASm48DSDVd7qjBZwhByuskWHT8SC7MS7TtCweYuW5gi0tceeopp4uBsY0okHEl8DJxwTPMWPfJ2nqI324XWXgZhqEF9gdrLimvTMdm00YuChTiTwLtgd0zJELVcXtb1raIlPtgqisFhNCRIx4w= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 6/6/24 3:32 PM, Erhard Furtner wrote: > On Thu, 6 Jun 2024 09:24:56 +0200 > "Vlastimil Babka (SUSE)" wrote: > >> Besides the zpool commit which might have just pushed the machine over the >> edge, but it was probably close to it already. I've noticed a more general >> problem that there are GFP_KERNEL allocations failing from kswapd. Those >> could probably use be __GFP_NOMEMALLOC (or scoped variant, is there one?) >> since it's the case of "allocating memory to free memory". Or use mempools >> if the progress (success will lead to freeing memory) is really guaranteed. >> >> Another interesting data point could be to see if traditional reclaim >> behaves any better on this machine than MGLRU. I saw in the config: >> >> CONFIG_LRU_GEN=y >> CONFIG_LRU_GEN_ENABLED=y >> >> So disabling at least the second one would revert to the traditional reclaim >> and we could see if it handles such a constrained system better or not. > > I set RANDOM_KMALLOC_CACHES=n and LRU_GEN_ENABLED=n but still hit the issue. > > dmesg looks a bit different (unpatched v6.10-rc2). What caught my eye, but it's also in some of the previous dmesg with MGRLU, is that in one case there's: DMA free:0kB That means many allocations went through that are allowed to just ignore all reserves, and depleted everything. That would mean __GFP_MEMALLOC or PF_MEMALLOC, which I suggested earlier for the GFP_KERNEL failure, is being used somewhere, but not leading to the expected memory freeing. > Regards, > Erhard