From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42D2BC61DF4 for ; Fri, 24 Nov 2023 09:34:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C3A088D006D; Fri, 24 Nov 2023 04:33:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BE7A68D0063; Fri, 24 Nov 2023 04:33:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A8A2D8D006D; Fri, 24 Nov 2023 04:33:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9906B8D0063 for ; Fri, 24 Nov 2023 04:33:59 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 74E49140C44 for ; Fri, 24 Nov 2023 09:33:59 +0000 (UTC) X-FDA: 81492336198.11.D92DAA9 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf22.hostedemail.com (Postfix) with ESMTP id 33EB1C000F for ; Fri, 24 Nov 2023 09:33:56 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=li51CbE1; spf=pass (imf22.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.131 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700818437; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dN2IlGcDTrVpBzasxMgeU6tNQKLc4MOOnU/SQr2kNZ8=; b=BlVM4UzUlofU9gcZYVgVSzUmVtAApP+jfijRLo2uOAjVQ9BhwdhYoYmVRnvd/Dh6d5aq9T nY2DOb37BKNv5TPdZRphrOaFn/sNZIKso3q5za8LaISzhAWc3x4c3tdHwaF91ZRA4XmxQP 8Jlh0zk96JngFXo4wwQPKs6b+Tzkgb8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700818437; a=rsa-sha256; cv=none; b=hR9MUkq+dcL8QES8NFvBVDEiBBaDQGLSRcuIc4TCfZK5+xjYd+4d1GOB9O63szlqztBtH7 PXo6taBU5n3L4jcBbp0B7dNKi7ON2e7oZjDpwjX3lZkGf2U5RRHYAG2/LmqH9v7ugbHYDy nFxw67uB9/3lePl7mVHOp3hQF3uwgIU= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=li51CbE1; spf=pass (imf22.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.131 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 9633E1FB78; Fri, 24 Nov 2023 09:33:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1700818435; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dN2IlGcDTrVpBzasxMgeU6tNQKLc4MOOnU/SQr2kNZ8=; b=li51CbE1Ey2FwgQ5N4urURDRSCIZsScNQ/vTVsPQXR4C/DA09W9DONYIEgUHELe++94ldO Urxq9IJUtliCzBfkOTFX3bRW/QkKpGYj6DeImNB25eMaHyf/6m0/zBLU8SadA6+gqsaMd1 XI+9fz6GlQXYcR6EybQlDdhPhJz344A= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 7705013A98; Fri, 24 Nov 2023 09:33:55 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id NPeuGQNuYGXTZwAAD6G6ig (envelope-from ); Fri, 24 Nov 2023 09:33:55 +0000 Date: Fri, 24 Nov 2023 10:33:54 +0100 From: Michal Hocko To: gaoxu Cc: Andrew Morton , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Suren Baghdasaryan , yipengxiang Subject: Re: =?utf-8?B?5Zue5aSNOiBbUEFUQ0hdIG1tLG9v?= =?utf-8?B?bV9yZWFwZXI6?= avoid run queue_oom_reaper if task is not oom Message-ID: References: <400d13bddb524ef6af37cb2220808c75@hihonor.com> <20231122134715.487677f38de74a6f8e0111cd@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: 33EB1C000F X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: wytmm3swn3sk9rgmrerhok9tpiq6y7g8 X-HE-Tag: 1700818436-917451 X-HE-Meta: U2FsdGVkX1+bSE6JfdodXxz6aA91senPIdJBDHAmGQrYDzUMqtaJ4vOMDuxk5j3pJcnrzycSnhTZ4G/dc7QZOXQ8Mu5iwfHnrbHTmUtoekemqp8gVVoXy3qvf96+ib3Cs8d+wpGXzWn3apKk9YJ8MBHKYl0Oqid6seReC7S+JJCvviEK4+z8GerSiYrtit/rXTgh/G++M7jQjUboy6gYRaeR+ytgYFA26iSly8E3Cu6Ox5SIXBiaYkdvyvkA5+adSl/BFbG2iGihHYMBi8jjIEqR7yjVT/EMxM/ZVweo3GRFY9s5i65RgSL2VC0IwQvpvavQjugeqeurnsW4abvL74wpBhWZ3VQ+PDZFPIEMdLBJJpL8m9fCysYpqoDq6mYGmhlXPKBR4SXMF9OYS+YZwo4APgYpg1BCb3XvJcdVTcPW6T0d+B6dw5W8zHSra4nTRh/KHVNGj9RPLvjl4mjL/ZvEoFMBvl0KewF0xy1cBajY6X8F77Twsbgdaj8pUoEzR6K77jtQgbO2pi+kIgStE+JLouoQEyV2HieB3Cl4O8PKSD8uuwgpMhnB9pi3h3VrkZzGbik6bBjkZKq3kIGGN38JyXyIf30Nz3ldP7mweTM82vxv1+J9x6EURyxei/CGYeGh/FQV4Xlmz+epY5zXD68qHYz+xO0vV/y93JSF8aIH9CVnkxkPGADSXDkp/Pb1uzsGcOeJ2jMxhDnyw77OVv0esZz4TfgKp7BYvP5no49x1c3lk1H77qtf0w7IwXKaf24JnIUGeZeXhmJputFeIq7ZYJvqms0VsmNRM8AYwdxVTKgrsx4jAwj9XVzEp5i70QdpV/TutjzCvQPSMPnXNz9Ni1MsqKgbAUQfw+4P6cdJJPvdHp0vk0z5Z9tE7UApscIDCxndSnk/VOaErSZ2EeXJ7r8Kx6mZCANZbdliNXQRnJLEllC4Xfv2GrPG5ksbJA7trJsBobjairRz1oA rxQpiAfb nr9BEh8mRpart5icvQShFtOOk/wu7NXpXRkS0b1fWwgnlqXrQs+ALipm6DJ14mNkR/mhHIGz3rw2cfDuww8yfadlLG8vJ80SHWj0hJJwc73PpEKJJIGTAbnsU1MQ77vCuf5z6ItGuGgd7pYwH5ZhdbnljjolR1sqeRCfudgySDX3b/tpcRfukIHcoycueUy/wf5uPl8FnXpogFCWp46J5n7d5drGo8UKPAmPmhok62iaV+tHwEVV/ChLoda3f/bRVCB9Twk0xkutQCsBcfevQn+A5LEdNWH8+6AGMDdZUN76jb2texOBKrqv1rS5c1rMOhJIAeJ27yBF0qBcIZ+xKRAsq/XS+fNoApviHt1fnz4k4FEm+D5FogAkDWBM9g+OKkseq X-Bogosity: Ham, tests=bogofilter, spamicity=0.000050, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri 24-11-23 02:52:34, gaoxu wrote: > On Web, 22 Nov 2023 21:47:44 +0000 Andrew Morton wrote: > > On Wed, 22 Nov 2023 12:46:44 +0000 gaoxu wrote: > > >> The function queue_oom_reaper tests and sets tsk->signal->oom_mm->flags. > >> However, it is necessary to check if 'tsk' is an OOM victim before > >> executing 'queue_oom_reaper' because the variable may be NULL. > >> > >> We encountered such an issue, and the log is as follows: > >> [3701:11_see]Out of memory: Killed process 3154 (system_server) > >> total-vm:23662044kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB, > >> UID:1000 pgtables:4056kB oom_score_adj:-900 > >> [3701:11_see][RB/E]rb_sreason_str_set: sreason_str set null_pointer > >> [3701:11_see][RB/E]rb_sreason_str_set: sreason_str set unknown_addr > >> [3701:11_see]Unable to handle kernel NULL pointer dereference at > >> virtual address 0000000000000328 > > > Well that isn't good. How frequently does this happen and can you suggest why some quite old code is suddenly causing problems? > > What is your workload doing that others' do not do? > This is a low probability issue. We conducted monkey testing for a month, > and this problem occurred only once. > The cause of the OOM error is the process surfaceflinger has encountered dma-buf memory leak. > > I have not found the root cause of this problem. > The physical memory of the process killed by OOM has been released, indicating that the issue may have occurred due to a concurrency problem > between process termination and OOM kill. > oom kill logļ¼š > Out of memory: Killed process 3154 (system_server) total-vm:23662044kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB, > UID:1000 pgtables:4056kB oom_score_adj:-900 > > >> [3701:11_see]user pgtable: 4k pages, 39-bit VAs, pgdp=00000000821de000 > >> [3701:11_see][0000000000000328] pgd=0000000000000000, > >> p4d=0000000000000000,pud=0000000000000000 > >> [3701:11_see]tracing off > >> [3701:11_see]Internal error: Oops: 96000005 [#1] PREEMPT SMP > >> [3701:11_see]Call trace: > >> [3701:11_see] queue_oom_reaper+0x30/0x170 [3701:11_see] > >> __oom_kill_process+0x590/0x860 [3701:11_see] > >> oom_kill_process+0x140/0x274 [3701:11_see] out_of_memory+0x2f4/0x54c > >> [3701:11_see] __alloc_pages_slowpath+0x5d8/0xaac > >> [3701:11_see] __alloc_pages+0x774/0x800 [3701:11_see] > >> wp_page_copy+0xc4/0x116c [3701:11_see] do_wp_page+0x4bc/0x6fc > >> [3701:11_see] handle_pte_fault+0x98/0x2a8 [3701:11_see] > >> __handle_mm_fault+0x368/0x700 [3701:11_see] > >> do_handle_mm_fault+0x160/0x2cc [3701:11_see] do_page_fault+0x3e0/0x818 > >> [3701:11_see] do_mem_abort+0x68/0x17c [3701:11_see] el0_da+0x3c/0xa0 > >> [3701:11_see] el0t_64_sync_handler+0xc4/0xec [3701:11_see] > >> el0t_64_sync+0x1b4/0x1b8 [3701:11_see]tracing off > >> > >> Signed-off-by: Gao Xu > > > I'll queue this for -stable backporting, assuming review is agreeable. > > Can we please identify a suitable Fixes: target to tell -stable maintainers which kernels need the fix? It looks like this goes back a long way. > The problem occurred on Linux version 5.15.78, There is no difference between the latest kernel version code and Linux version 5.15.78 in the > Function __oom_kill_process, so this problem is likely common to both versions. __oom_kill_process is not the only involved part. The exit path plays a really huge role there as well. I do understand that this was one off and likely hard to reproduce but without knowing that the current Linus tree can trigger this, we cannot really do much, I am afraid. -- Michal Hocko SUSE Labs