From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91799C00140 for ; Fri, 12 Aug 2022 14:31:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8FEA86B0073; Fri, 12 Aug 2022 10:31:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8871A6B0075; Fri, 12 Aug 2022 10:31:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7010A8E0001; Fri, 12 Aug 2022 10:31:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 5E4146B0073 for ; Fri, 12 Aug 2022 10:31:00 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2C3C31608C0 for ; Fri, 12 Aug 2022 14:31:00 +0000 (UTC) X-FDA: 79791177480.27.6A6E962 Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by imf26.hostedemail.com (Postfix) with ESMTP id 5E4B91401A9 for ; Fri, 12 Aug 2022 14:30:58 +0000 (UTC) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 555105C0046; Fri, 12 Aug 2022 10:30:56 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Fri, 12 Aug 2022 10:30:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov.name; h=cc:cc:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm2; t=1660314656; x=1660401056; bh=M3 MbplBIwl853hk2/PIOPlXrFz0V9VywUt08s+uBFWc=; b=qDLFPe+gLm+/z45jUp gD1UvOGWYdhSo5E9SwWBLGldN1kUuLTENtVPbHIdv3hf8qvJeBvPfiRf5YclZRX/ RLn75eWjWu3W8emeTTnw8aTXUulREPgharGzwtUvg1fNzXZyI0N2nRyynX0q+HeY 7U/ZDoyPNff3L86fomnW7vkKEtgQa3Gos4SxdyDEBe48rak8I0xku+rSsC+bOw8M saHAHh04yQquyC+JkWW5yjyREoVpwc3IGAQJ/ScGqcHEAMEVYahRdgEf8hH2vQZ8 v94zBjExxClOApEhuxJH6GjhfdMMnL95VEx+wzc3N4SI5jgdAFNORBbHwoGZ2Qi9 kU+Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1660314656; x=1660401056; bh=M3MbplBIwl853hk2/PIOPlXrFz0V 9VywUt08s+uBFWc=; b=cG9MWGAal32lE95MfUxQ/10J6t2iXDTt1UN6kMXynFqU G3Q3nVWNM12MIAF9aM44YEFZB/aCC9R6qapvdo6o3bl9lv35OU527D+1Ry780WQ3 cpemd/Bmh29v2uqNr2a5GZjIZGda68AzECthe1fwRxsCfgcQg5KzF1J6zt2guETh cu8WldLbTReoBKK5K5j9SkdO2TEY6oO/59kXpZGV83wc5umqvAOTRd0Vjo2nGPz4 pSewfnRC5zv4T3d/oyfurbm8gALCMRUf2KUq58UDChCl9PMF9n4IaUVCxc5h7Ss+ 2FjVMhXN9yul6fQOiPiXJ9kocjc2UNDD7GqwpU3TWg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdegiedgjeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvvefukfhfgggtuggjsehttd dttddttddvnecuhfhrohhmpedfmfhirhhilhhlucetrdcuufhhuhhtvghmohhvfdcuoehk ihhrihhllhesshhhuhhtvghmohhvrdhnrghmvgeqnecuggftrfgrthhtvghrnhephfeige fhtdefhedtfedthefghedutddvueehtedttdehjeeukeejgeeuiedvkedtnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepkhhirhhilhhlsehshh huthgvmhhovhdrnhgrmhgv X-ME-Proxy: Feedback-ID: ie3994620:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 12 Aug 2022 10:30:55 -0400 (EDT) Received: by box.shutemov.name (Postfix, from userid 1000) id E33AD104A2F; Fri, 12 Aug 2022 17:33:56 +0300 (+03) Date: Fri, 12 Aug 2022 17:33:56 +0300 From: "Kirill A. Shutemov" To: Matthew Wilcox Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: Re: State of the Page (August 2022) Message-ID: <20220812143356.4kv5cycwbcy2t7ul@box.shutemov.name> References: <20220812101639.ijonnx7zeus7h2hn@box.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660314658; a=rsa-sha256; cv=none; b=l1j5dNO6UJAlwkxxrskQtV+Fh5lI1t7wWuamzq1wHDflaq26VdaOQ2EWO1+gB+sDAlW9Na VDwxcy3Kf4ylnT8KphkqL8BRjVrrNJK3/tcm72BLtsRstmzjWhEVP8Aa2VOyxaQjn/iDGR N8LiW0F247QSwx/Ftm6XFjCf49ZnQ/A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660314658; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=M3MbplBIwl853hk2/PIOPlXrFz0V9VywUt08s+uBFWc=; b=fEkhAv82SOlhttuXrYMdeT4oAOEGRlH8FOZwFWRK7+suunMTjol/UDGjxrOcbH+oWMIlUe ci+4uXoGZPitsgolx7u6+vA4Vu+G2kuFup01A47wVC20Bpk4e/VCpyph8KrcB2WRy/ESWh tjzIVe5MkUx82OJ7srs8aZqVY+7B2Bc= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=shutemov.name header.s=fm2 header.b=qDLFPe+g; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=cG9MWGAa; spf=pass (imf26.hostedemail.com: domain of kirill@shutemov.name designates 66.111.4.28 as permitted sender) smtp.mailfrom=kirill@shutemov.name; dmarc=none X-Stat-Signature: q48ukuc918mnhifopj1xa9kcyicx1e75 X-Rspamd-Queue-Id: 5E4B91401A9 Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=shutemov.name header.s=fm2 header.b=qDLFPe+g; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=cG9MWGAa; spf=pass (imf26.hostedemail.com: domain of kirill@shutemov.name designates 66.111.4.28 as permitted sender) smtp.mailfrom=kirill@shutemov.name; dmarc=none X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1660314658-769235 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Aug 12, 2022 at 02:34:53PM +0100, Matthew Wilcox wrote: > On Fri, Aug 12, 2022 at 01:16:39PM +0300, Kirill A. Shutemov wrote: > > On Thu, Aug 11, 2022 at 10:31:21PM +0100, Matthew Wilcox wrote: > > > ============================== > > > State Of The Page, August 2022 > > > ============================== > > > > > > I thought I'd write down where we are with struct page and where > > > we're going, just to make sure we're all (still?) pulling in a similar > > > direction. > > > > > > Destination > > > =========== > > > > > > For some users, the size of struct page is simply too large. At 64 > > > bytes per 4KiB page, memmap occupies 1.6% of memory. If we can get > > > struct page down to an 8 byte tagged pointer, it will be 0.2% of memory, > > > which is an acceptable overhead. > > > > Right. This is attractive. But it brings cost of indirection. > > It does, but it also crams 8 pages into a single cacheline instead of > occupying one cacheline per page. If you really need info about these pages and reference their memdesc it is likely be 9 cache lines that scattered across memory instead of 8 cache lines next to each other in the same page. And it's going to be two cachelines instead of one if we need info about one page. I think it is the most common case. Initially, I thought we can offset the cost by caching memdescs instead of struct page/folio. Like page cache store memdesc, but it would require memdesc_to_pfn() which is not possible, unless we want to store pfn explicitly in memdesc. I don't want to be buzzkill, I like the idea a lot, but abstractions are often costly. Getting it upstream without noticeable performance regressions going to be a challenge. > > It can be especially painful for physical memory scanning. I guess we can > > derive some info from memdesc type itself, like if it can be movable. But > > still looks like an expensive change. > > I just don't think of physical memory scanning as something we do > often, or in a performance-sensitive path. I'm OK with slowing down > kcompactd if it makes walking the LRU list faster. > > > Do you have any estimation on how much CPU time we will pay to reduce > > memory (and cache) overhead? RAM size tend to grow faster than IPC. > > We need to make sure it is the right direction. > > I don't. I've heard colourful metaphors from the hyperscale crowd about > how many more VMs they could sell, usually in terms of putting pallets > of money in the parking lot and setting them on fire. But IPC isn't the > right metric either, CPU performance is all about cache misses these days. As I said above, I don't expect the new scheme to be cache-friendly either. -- Kiryl Shutsemau / Kirill A. Shutemov