From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1022EB64DA for ; Thu, 20 Jul 2023 12:13:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5C2662800F9; Thu, 20 Jul 2023 08:13:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 54B6D28004C; Thu, 20 Jul 2023 08:13:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C5A42800F9; Thu, 20 Jul 2023 08:13:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2A8D028004C for ; Thu, 20 Jul 2023 08:13:31 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D03B8C035C for ; Thu, 20 Jul 2023 12:13:30 +0000 (UTC) X-FDA: 81031880580.26.37FDA17 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf19.hostedemail.com (Postfix) with ESMTP id C00B61A0004 for ; Thu, 20 Jul 2023 12:13:27 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=RZi4dFEk; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf19.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689855208; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=z8W41ZNTIwKt+2Nk7HZ0tWIn8uIgWXqjgIN9VIsXFfg=; b=svbuBPYM3aL33lzLHbSpsqh34GyXBEeR1Y2IgpX3fWMFtmUhNp+FrexT3ZRoBxbyCK16Xg v1EY2bay09ELTb004nHWnfMzKGUh8Ce9d3bROpCfchS+rRy8R4pMrAo9sIhQ+2fSI7zAqv IjPpmdMzIg7X5eD/UUnBThnwdm39RDM= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=RZi4dFEk; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf19.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689855208; a=rsa-sha256; cv=none; b=Ko9BaND/uO7v2d32LOxvXJdhrjLRdpd7dEhCxQFUkd9p0hmt9Z/k8i1uIStrknBrUG8E5O Vss06T9O2iYV5fBrgcUyWrdD1GfNuLTLas2llAcOxG3h9fBw+LAMAlwkj6uzoDTe0pWDru Hdnl8+hFMYB/YlGfoQzVjHbeyjWLH7E= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0BF3D20505; Thu, 20 Jul 2023 12:13:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1689855206; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=z8W41ZNTIwKt+2Nk7HZ0tWIn8uIgWXqjgIN9VIsXFfg=; b=RZi4dFEkrVEPhleKd9t89JcanOoz6QOVGM9Zk9Nu70dWLuPSu8pIZm8QbloSRQj4hOSJi0 jzLuqZXdxLNS9Xe27DpPVygI+TwySrUAn8Kxcee7aMc6e/h8oqNBpLYmcIngd/oB+NiEt6 zQg1I2w6HoV4qftlFEbWYu4M2SXceLc= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id EF9C1138EC; Thu, 20 Jul 2023 12:13:25 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id tuj6N+UkuWQ5JAAAMHmgww (envelope-from ); Thu, 20 Jul 2023 12:13:25 +0000 Date: Thu, 20 Jul 2023 14:13:25 +0200 From: Michal Hocko To: Ross Zwisler Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Mike Rapoport , Andrew Morton , Matthew Wilcox , Mel Gorman , Vlastimil Babka , David Hildenbrand Subject: Re: collision between ZONE_MOVABLE and memblock allocations Message-ID: References: <20230718220106.GA3117638@google.com> <20230719224821.GC3528218@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230719224821.GC3528218@google.com> X-Rspam-User: X-Stat-Signature: jsmujw55fynsh6nu6g81g43m5ubfwp4i X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C00B61A0004 X-HE-Tag: 1689855207-608551 X-HE-Meta: U2FsdGVkX1/34v9sz9zzuYEU2q93XdtCvWqiowv5WWYLkSPORuYs7+PIocohL8Hgu87A012gB8ySKgmHFn9pvy6UHs2m+EGQYKz1O4Fm/VDMJkJNR211wZ1PN8rZX+/WJ+UWuQ4/aYwmxZMQQ2vmOMkEqb7pHB5k1Su56ZITdWxK50+wWFT9GawfkKlQWoECE+xPcZFoADg0vE30UP6W3Q8g/vqjZy8Q1BCkzdMtjEXa76In6RZhAyMcREL4v1yL7+fx6yCiZLqm3eJI6o7dr533K07u9y+OtMXcsVFmuzZyegkhEyOXM0zOJzRMd+qQXQrZKptoioqAUJ7qLbU1ezpjKpZF9Mn62MrzGCjn0RcCqdaywIQanK65w/UV6APAEQWl0+alKbNtswCJv6O6nRgXd6kDcRIwXSJazZiG35JY/aG6CFRJ67Ox8mUeI+Mzb2TXmxZlGPKqQpIu9g4CjHkmXaewFigNwNWkaDG+go7DEkMiksu60QYdyS+uAuHFLhby65F8KOZk6PWenA5pLPw6FAze4IaZZ5X9P5wFLBoDQHubaYXGhaCcnRJCR6+LahYkNBh2lAZ9KPSnZ3raWIuz4hmBtLuM/Mb9+gQBFlIvUo07qfkZ8nL5VSVo3O9gAN8jXs8R3iQjSCoSgUIftJP52tQjUBELA2zVkXkKvY7ajfvbgfsrBTpUApQ7x9Z7OGXzAXOVvTu4qTqUTOlKP7IHtc6Jho/3pyCEHJOGRrkPr0nsVp9J/cTPOHXHvtByF6T5Q4jlyBsXvKJn1Iv1cJVTM8aCRY0X+7ItlWvo++pjeIhqMix2or1GGfZYhBN2wX1kRu+ViDXNZLqnt0dBjNhLkMMF3c6IUo2OK7IuGV8WVqWmHGMqVFRRKGa74Jd/bJu1O5RIf+9Wdj9VOuX5GikM2a8Ock6sPZueVMq7YjS6N14Plz3OxuIzYFWT3Y1sreRJ92quf6dzjS0XbEm sjm7Zd6E Rn88jgurL8JB5zybte67ibpCME71qYf/7wNcmWHlsxtDLIxuV4aUeHxD0bSeYDiCwn1uM3eGBQK1xRI61iyOnGSoZeWFYfkN9Q7zZKYEyzX+3SSnhLuNtMgH624vNtUNYkFS4KsGpUmVIYDCgp7GrJei+z4S+0i7B/lqrb8DKBZ3ZDWvFoH0mAHWScdfSVHNzZu0o X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 19-07-23 16:48:21, Ross Zwisler wrote: > On Wed, Jul 19, 2023 at 08:14:48AM +0200, Michal Hocko wrote: > > On Tue 18-07-23 16:01:06, Ross Zwisler wrote: > > [...] > > > I do think that we need to fix this collision between ZONE_MOVABLE and memmap > > > allocations, because this issue essentially makes the movablecore= kernel > > > command line parameter useless in many cases, as the ZONE_MOVABLE region it > > > creates will often actually be unmovable. > > > > movablecore is kinda hack and I would be more inclined to get rid of it > > rather than build more into it. Could you be more specific about your > > use case? > > The problem that I'm trying to solve is that I'd like to be able to get kernel > core dumps off machines (chromebooks) so that we can debug crashes. Because > the memory used by the crash kernel ("crashkernel=" kernel command line > option) is consumed the entire time the machine is booted, there is a strong > motivation to keep the crash kernel as small and as simple as possible. To > this end I'm trying to get away without SSD drivers, not having to worry about > encryption on the SSDs, etc. > > So, the rough plan right now is: > > 1) During boot set aside some memory that won't contain kernel allocations. > I'm trying to do this now with ZONE_MOVABLE, but I'm open to better ways. > > We set aside memory for a crash kernel & arm it so that the ZONE_MOVABLE > region (or whatever non-kernel region) will be set aside as PMEM in the crash > kernel. This is done with the memmap=nn[KMG]!ss[KMG] kernel command line > parameter passed to the crash kernel. > > So, in my sample 4G VM system, I see: > > # lsmem --split ZONES --output-all > RANGE SIZE STATE REMOVABLE BLOCK NODE ZONES > 0x0000000000000000-0x0000000007ffffff 128M online yes 0 0 None > 0x0000000008000000-0x00000000bfffffff 2.9G online yes 1-23 0 DMA32 > 0x0000000100000000-0x000000012fffffff 768M online yes 32-37 0 Normal > 0x0000000130000000-0x000000013fffffff 256M online yes 38-39 0 Movable > > Memory block size: 128M > Total online memory: 4G > Total offline memory: 0B > > so I'll pass "memmap=256M!0x130000000" to the crash kernel. > > 2) When we hit a kernel crash, we know (hope?) that the PMEM region we've set > aside only contains user data, which we don't want to store anyway. We make a > filesystem in there, and create a kernel crash dump using 'makedumpfile': > > mkfs.ext4 /dev/pmem0 > mount /dev/pmem0 /mnt > makedumpfile -c -d 31 /proc/vmcore /mnt/kdump > > We then set up the next full kernel boot to also have this same PMEM region, > using the same memmap kernel parameter. We reboot back into a full kernel. Btw. How do you ensure that the address range doesn't get reinitialized by POST? Do you rely on kexec boot here? -- Michal Hocko SUSE Labs