From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42855C43217 for ; Wed, 2 Nov 2022 16:11:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6C96D8E0002; Wed, 2 Nov 2022 12:11:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 652978E0001; Wed, 2 Nov 2022 12:11:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A6CB8E0002; Wed, 2 Nov 2022 12:11:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 343DC8E0001 for ; Wed, 2 Nov 2022 12:11:56 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 09D301211A6 for ; Wed, 2 Nov 2022 16:11:56 +0000 (UTC) X-FDA: 80088993432.22.E8B7E15 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf21.hostedemail.com (Postfix) with ESMTP id 566FB1C0011 for ; Wed, 2 Nov 2022 16:11:55 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C52C222D61; Wed, 2 Nov 2022 16:11:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1667405513; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=o/a40auuQ0L/AJHav0351LK7cNnLgaQeg9TE6eFJvOk=; b=d8bCk+71C3xkLcEx92hV4+fPX8PwQwV4ZNBEE/KopsBFkoqxVj0Cl68+2Xh8Mqnjg1NUdi KUupgZ8txk+a/5GyzatujSbtyyJ1dDabwNQKiXWpfs+JeqYmFr4iWz540QK3qQqMJeeWbE tRG6Y7bHU14OBS6iCjEUWhLCGTCEMVA= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id A70F6139D3; Wed, 2 Nov 2022 16:11:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 3mGrJ8mWYmNTGgAAMHmgww (envelope-from ); Wed, 02 Nov 2022 16:11:53 +0000 Date: Wed, 2 Nov 2022 17:11:52 +0100 From: Michal Hocko To: Yang Shi Cc: Zach O'Keefe , akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] mm: don't warn if the node is offlined Message-ID: References: <20221031183122.470962-1-shy828301@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667405515; a=rsa-sha256; cv=none; b=T+O2XK4EMoKuycfPgLqOqyXoRtB3IaQTEQTfPrQUGPoW7Tv7t+7MLZ2i3KLQm4MdF8zra7 jHXc39n0M1NzuGCR+GNfTs9ZqcwAOD68OZCm4NxeF4ecjB4dDEYlHZimy/O8wwr5NL6AEG j+cMlymVD8L4mFHUKn8WFxk8OVR4on8= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=d8bCk+71; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf21.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667405515; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=o/a40auuQ0L/AJHav0351LK7cNnLgaQeg9TE6eFJvOk=; b=ABtM5heQ6wtxe3xRElD0BZkTlbdIyR6DlIybAOLNENd3FtaxsL+tEHWrTmeMvcZF8HUOcd JC5LGW9+sGMhR6458vyR6Xh9aoH7h09FC97nf68i3GLwKpQ57YdvcPWZ1Hp0MXOEsoXiFi B6xMiMRHCmF6ZObLb+YLj5PknzqI5bE= X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 566FB1C0011 X-Rspam-User: Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=d8bCk+71; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf21.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com X-Stat-Signature: h9kqf7erpsua8s3iw7dx6nb31dxpxjy4 X-HE-Tag: 1667405515-754529 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 02-11-22 08:58:09, Yang Shi wrote: > On Wed, Nov 2, 2022 at 12:14 AM Michal Hocko wrote: > > > > On Tue 01-11-22 10:12:49, Yang Shi wrote: > > > On Tue, Nov 1, 2022 at 12:54 AM Michal Hocko wrote: > > > > > > > > On Mon 31-10-22 17:05:06, Zach O'Keefe wrote: > > > > > On Mon, Oct 31, 2022 at 3:08 PM Michal Hocko wrote: > > > > > > > > > > > > On Mon 31-10-22 11:31:22, Yang Shi wrote: > > > > > > > Syzbot reported the below splat: > > > > > > > > > > > > > > WARNING: CPU: 1 PID: 3646 at include/linux/gfp.h:221 __alloc_pages_node include/linux/gfp.h:221 [inline] > > > > > > > WARNING: CPU: 1 PID: 3646 at include/linux/gfp.h:221 hpage_collapse_alloc_page mm/khugepaged.c:807 [inline] > > > > > > > WARNING: CPU: 1 PID: 3646 at include/linux/gfp.h:221 alloc_charge_hpage+0x802/0xaa0 mm/khugepaged.c:963 > > > > > > > Modules linked in: > > > > > > > CPU: 1 PID: 3646 Comm: syz-executor210 Not tainted 6.1.0-rc1-syzkaller-00454-ga70385240892 #0 > > > > > > > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/11/2022 > > > > > > > RIP: 0010:__alloc_pages_node include/linux/gfp.h:221 [inline] > > > > > > > RIP: 0010:hpage_collapse_alloc_page mm/khugepaged.c:807 [inline] > > > > > > > RIP: 0010:alloc_charge_hpage+0x802/0xaa0 mm/khugepaged.c:963 > > > > > > > Code: e5 01 4c 89 ee e8 6e f9 ae ff 4d 85 ed 0f 84 28 fc ff ff e8 70 fc ae ff 48 8d 6b ff 4c 8d 63 07 e9 16 fc ff ff e8 5e fc ae ff <0f> 0b e9 96 fa ff ff 41 bc 1a 00 00 00 e9 86 fd ff ff e8 47 fc ae > > > > > > > RSP: 0018:ffffc90003fdf7d8 EFLAGS: 00010293 > > > > > > > RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 > > > > > > > RDX: ffff888077f457c0 RSI: ffffffff81cd8f42 RDI: 0000000000000001 > > > > > > > RBP: ffff888079388c0c R08: 0000000000000001 R09: 0000000000000000 > > > > > > > R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 > > > > > > > R13: dffffc0000000000 R14: 0000000000000000 R15: 0000000000000000 > > > > > > > FS: 00007f6b48ccf700(0000) GS:ffff8880b9b00000(0000) knlGS:0000000000000000 > > > > > > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > > > > > > CR2: 00007f6b48a819f0 CR3: 00000000171e7000 CR4: 00000000003506e0 > > > > > > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > > > > > > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 > > > > > > > Call Trace: > > > > > > > > > > > > > > collapse_file+0x1ca/0x5780 mm/khugepaged.c:1715 > > > > > > > > > > > > This is quite weird, isn't it? alloc_charge_hpage is selecting the most > > > > > > busy node (as per collapse_control). How come this can be an offline > > > > > > node? Is a parallel memory hotplug happening? > > > > > > > > > > TBH -- I did not look closely at the syzbot reproducer (let alone > > > > > attempt to run it) and assumed this was the case. Taking a quick look, > > > > > at least memory hot remove is enabled: > > > > > > > > > > CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y > > > > > CONFIG_ARCH_ENABLE_MEMORY_HOTREMOVE=y > > > > > CONFIG_MEMORY_HOTPLUG=y > > > > > CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=y > > > > > CONFIG_MEMORY_HOTREMOVE=y > > > > > > > > > > But looking at the C reproducer, I don't immediately see anywhere > > > > > where we offline nodes. I'll try to run this tomorrow to make sure I'm > > > > > not missing something real here. > > > > > > > > Looking slightly closer at hpage_collapse_scan_file I think that it is > > > > possible that xas_for_each simply doesn't find any entries in the page > > > > cache and with khugepaged_max_ptes_none == HPAGE_PMD_NR we can fall back > > > > to collapse_file even without any real entries. > > > > > > The khugepaged_max_ptes_none can't be HPAGE_PMD_NR, it must be <= > > > (HPAGE_PMD_NR - 1), but MADV_COLLAPSE does ignore it. > > > > OK, I see. > > > > > But a closer look at the code about how to pick up the preferred node, > > > there seems to be a corner case for MADV_COLLAPSE. > > > > > > The code tried to do some balance if several nodes have the same hit > > > record. Basically it does conceptually: > > > * If the target_node <= last_target_node, then iterate from > > > last_target_node + 1 to MAX_NUMNODES (1024 on default config) > > > * If the max_value == node_load[nid], then target_node = nid > > > > Correct > > > > > So assuming the system has 2 nodes, the target_node is 0 and the > > > last_target_node is 1, if MADV_COLLAPSE path is hit, then it may > > > return 2 for target_node, but it is actually not existing (offline), > > > so the warn is triggered. > > > > How can node_load[2] > 0 (IIUC max_value > 0 here) if the node is > > offline (other than a race with hotplug)? > > The max_value may be 0 if there is no entry in page cache for that range IIUC. Ohh, I am blind. I believe you have already mentioned that but I must have overlooked it. I have only now noticed cc->is_khugepaged part of the check. Supposedly, the primary idea here being that madvise calls should de-facto create a brand new THP in that case. A creative way to call collapsing but I can see some point in that. See my other reply on how to deal with that in a more reasonable (IMHO) way. -- Michal Hocko SUSE Labs