From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 825B1E732FD for ; Thu, 28 Sep 2023 17:13:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EF1028D00BD; Thu, 28 Sep 2023 13:13:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA10E8D0023; Thu, 28 Sep 2023 13:13:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D8F938D00BD; Thu, 28 Sep 2023 13:13:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id CAF998D0023 for ; Thu, 28 Sep 2023 13:13:04 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 8DCC11A0364 for ; Thu, 28 Sep 2023 17:13:04 +0000 (UTC) X-FDA: 81286651488.20.18D510B Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by imf07.hostedemail.com (Postfix) with ESMTP id DCC2E40015 for ; Thu, 28 Sep 2023 17:13:02 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linux.microsoft.com header.s=default header.b=BN+O+Hzs; dmarc=pass (policy=none) header.from=linux.microsoft.com; spf=pass (imf07.hostedemail.com: domain of skinsburskii@linux.microsoft.com designates 13.77.154.182 as permitted sender) smtp.mailfrom=skinsburskii@linux.microsoft.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695921183; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WHD7w8p4ig6hZvJ2HDlS2yknFytIbx+O5dBVV0TdS9o=; b=nKpH7tRJi8ma8mxYo6HwsQ/NcLtkQZscg/DWoHlS9Ww1pgP8NRXK/HZcvzvyTmAKgI9nGE fYdZaucUjgbjevYF0IC/QSNSr/KVu/O93fmUnmHFToeN7PtHusZOxjYzDPDzseYAEULYXY SCf3rvSJgsK9V1/SMbzZNgLpwx4UOQM= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=linux.microsoft.com header.s=default header.b=BN+O+Hzs; dmarc=pass (policy=none) header.from=linux.microsoft.com; spf=pass (imf07.hostedemail.com: domain of skinsburskii@linux.microsoft.com designates 13.77.154.182 as permitted sender) smtp.mailfrom=skinsburskii@linux.microsoft.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695921183; a=rsa-sha256; cv=none; b=xG3WgGVzWrJ0Nkrp2oJ/qhVqMiQo7clLipbneiTnq0ffjbpQXvmH6oQIk9ChoMp7TjiEEL OyMGifc9KSeckvKhtschBuIa2fB7HctcyAj9r5Pc5kDP8SsC/tXSWM78qS3tGhb5pRTs0w O73o70RiXwIwOZiia+e8P98N7hRoJGk= Received: from skinsburskii. (c-67-170-100-148.hsd1.wa.comcast.net [67.170.100.148]) by linux.microsoft.com (Postfix) with ESMTPSA id 3105020B74C0; Thu, 28 Sep 2023 10:13:01 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 3105020B74C0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1695921181; bh=WHD7w8p4ig6hZvJ2HDlS2yknFytIbx+O5dBVV0TdS9o=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=BN+O+Hzs4pIM424F5eKC0n/EsDjESogGkGqVZh9mS53ithFDLcEae9I207mN3piB3 iYLiX52ica6itYY25Bf8TDkXOeQEX15tNQ+E7se//LrdKtJXjlQSNI+CXpZI1sl4IS Bj+9f33ilTHKzi/UIx20ACz8Wa6bWPz2ig9ieBRU= Date: Wed, 27 Sep 2023 16:25:48 -0700 From: Stanislav Kinsburskii To: Dave Hansen Cc: Baoquan He , tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, ebiederm@xmission.com, akpm@linux-foundation.org, stanislav.kinsburskii@gmail.com, corbet@lwn.net, linux-kernel@vger.kernel.org, kexec@lists.infradead.org, linux-mm@kvack.org, kys@microsoft.com, jgowans@amazon.com, wei.liu@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, graf@amazon.de, pbonzini@redhat.com Subject: Re: [RFC PATCH v2 0/7] Introduce persistent memory pool Message-ID: <20230927232548.GA20221@skinsburskii.> References: <01828.123092517290700465@us-mta-156.us.mimecast.lan> <20230927161319.GA19976@skinsburskii.> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: DCC2E40015 X-Stat-Signature: 5acfsjygwg5q541dw3ushdu7qancpn7o X-Rspam-User: X-HE-Tag: 1695921182-396756 X-HE-Meta: U2FsdGVkX199E/1M6vXafrqx0gK6lvQCU7zb8LZe1HideODmomOS+XAUCwy8jxjQaU4EKGlE9AGKgcePuQDDIZckgpf8+i30oom/gLC/eHVvYkJpY+wX2YWtmyLy27NtYDIcINNYesDGOHDvsg9uEUCMgNVgRj2h5Fwvy8SUIYYrYbZIoCUBAKayGaZ/RvdXHknCYilqkCN80Nig14rXyclPSmTDwicenL9oOW6uwtas4GsbE2CKjg/deRMY3v2GAdhaDqD+GsTyOxX88b1KdCjP8tNpW7+9O2meWwzChsrD2ApHVZyXF34WVWTabeg8g7fdxSggNWkX0zYNoMRwx3aTebq+vgBWsDPl7pIkfMwV9YeN7JlVEskod6rqEBXA2lJX+ipn+g7TKh+j578D+fmDocQ4fcFzwp3k5ZW2yv9wuZjLfQBW8OH9EQTL9Fs37yETwBl0cyu3wbd9Vb6NSyuiFD8ibVHn4urLgGPfZLFOjZF4CFqnUx5K5LRX25bJAOK+vaBJDxS0Pn+gtMIZ2/LK1Zg8/kjzA7rTYJhwzggkXpP3rUmYaMDm4+UhkgptiiMWYk0x/nzD5DVmsAJPPGi5qBGzVLfmW0Zf5ISM9sMgz/joKzPWBDSwQJxzo026GbdDEWzHXwwl9v8NMLFWsGFElbE6drYabNgV4YtVFy1CfOGmhl/fmegG5acYH0yZlmIBt83SwGH7fZ4LhORwHKisC0B2z10+0WkYuopkimQ6DnoHSfHlDoXUKhKQO1E5zxe9Srw0zIZKLKjfFadGmk5CIoa7vmTku2/3HYyzPVfN8ByPWjZvqEsEawJ5SafORhORYo2z64ekRt8r6o/f5gs+I+nuK0xztWkLrEPP6CD5v8HURGev9woRKDePJYecR08FtYcZSycgWXUM0oCTSFgTw1FKpTSqcDBh74pPe4jj7KUrFNEUF99sPcrrUKWjUOpfG7i5EZOsEoGXp+o VCUy24dk IZO2Fst7VOW4uUio5pbXe3hlSD6WqXxTmXiytAEYqX1CzKPScKTZ+4bCVjZbkCER06s1+W4+Wr1lk5FsuaTmmUkxrX9KTTsf7uLg3Wv0X64dPCrgEt/+115T/JOVOtY/jur6w0n3DCnZ6vzSXh6hJuofBInZMB7mi5gIToeYmPWHbb/CXmyPpCAPk/evMvyj1pDSPh6slrNWzcU+NJHN52V2piKq+wxnPiS7IXM6CKSj77RqJqRL4e3cKdxmGAEg+jzfc X-Bogosity: Ham, tests=bogofilter, spamicity=0.000159, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Sep 28, 2023 at 06:22:54AM -0700, Dave Hansen wrote: > On 9/27/23 09:13, Stanislav Kinsburskii wrote: > > Once deposited, these pages can't be accessed by Linux anymore and thus > > must be preserved in "used" state across kexec, as hypervisor state is > > unware of kexec. > > If Linux can't access them, they're not RAM any more. I'd much rather > remove them from the memory map and move on with life rather than > implement a bunch of new ABI that's got to be handed across kernels. Could you elaborate more on the new ABIs? FDT is handled by x86 already, and passing it over kexec looks like a natural extension. Also, adding more state to it also doens't look like a new ABI. Or does it? Let me also comment on removing this regions from the memory map. The major peculiarity here is that hypervisor distinguish between the pages, deposited for guests to rnu and the pages deposited for the Linux root partition to keep the guest-related portion of hypervisor state in the root partition. And the latter is the matter in question. We can indeed isolate and deposit a excessive amount of memory upfront in hope that hypervisor will never get into the situation, when it needs more memory. However, it's not reliable, as the amount of memory will always be an estimation, depending on the number of expected guests, guest-attached devices, etc. And this becomes even a bigger problem when most of the memory is already removed from the memory map to host guest partitions. It's also not efficient as the amount of memory required by hypervisor can grow or shrink depending on the use case or host configuration, and deposting excessive amount of memory will be a waste. But, actually, the idea of removing the pages from memory map was reflected to some extent in the first version of this proposal, so let me elaborate on it a bit. Effectively, instead of reserving and depositing a lot of memory to hypervisor upfront, the memory can be allocated from kernel memory when needed and then returned back when unused. This would still require pages removal from the memory map upon kexec, but that's another problem. Thanks, Stanislav