From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0117C54798 for ; Fri, 1 Mar 2024 03:28:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 36E046B0080; Thu, 29 Feb 2024 22:28:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 31EA06B00A5; Thu, 29 Feb 2024 22:28:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E6A16B00A6; Thu, 29 Feb 2024 22:28:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0DB446B00A5 for ; Thu, 29 Feb 2024 22:28:48 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id D56E0A1749 for ; Fri, 1 Mar 2024 03:28:47 +0000 (UTC) X-FDA: 81847038294.29.D9A2960 Received: from out-187.mta1.migadu.com (out-187.mta1.migadu.com [95.215.58.187]) by imf20.hostedemail.com (Postfix) with ESMTP id EAE471C000A for ; Fri, 1 Mar 2024 03:28:44 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=wqDaVUra; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf20.hostedemail.com: domain of kent.overstreet@linux.dev designates 95.215.58.187 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709263725; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RzrkXcFl3rQIY+ikNeM0Wr3ys6+rWyOMJgwkAPjSi+8=; b=D7NVhbAWDReE4gP3lerYpy7+/7uoW6XxAstLaUNwTyWsbCaG+qyOCeXeIdZ3ndlAKebu9g a2wUCQ5e1mzobQtQE83MufctpPIlubWDGVbhR7U+RjHHBIopLp56scSmMBJ3Ihy9W3sk3L u6zCIa50hHk9Lf6c0VZXAJEkQ26ILQg= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=wqDaVUra; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf20.hostedemail.com: domain of kent.overstreet@linux.dev designates 95.215.58.187 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709263725; a=rsa-sha256; cv=none; b=s7jdsPJ4d7klY9LwJZYuJPPYGGDpNXykjEXfhczkM9R8S752zJhZ1v09thNemlBz3LpeVR vJY/E2MPawvWsVUi9j3hXNdckqKxIk1V6fousBtd6ofUkZLCMGU8QC2cTrRbETgYF/Bzef U+wIUjSRf3SvT5z9sponCTEkVr4GfDs= Date: Thu, 29 Feb 2024 22:28:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1709263723; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RzrkXcFl3rQIY+ikNeM0Wr3ys6+rWyOMJgwkAPjSi+8=; b=wqDaVUraQk8wLdVKd/NBwm6DJ31A9k5JoDnQ9iXq+aUw7EydKsONhiYrviC7CB4qsesuwv wlY5pEQ4kVh9/+245RcRhqie5fBt25nGxhG40M8df5Tfov/+CA9xzynGF8bVlklfIXmXBu v0tFltbA0tKQdlwpUnu5RgoWjKr0eQY= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Kent Overstreet To: "Paul E. McKenney" Cc: Amir Goldstein , lsf-pc@lists.linux-foundation.org, linux-mm@kvack.org, linux-fsdevel , Matthew Wilcox , Jan Kara Subject: Re: [Lsf-pc] [LSF/MM/BPF TOPIC] Reclamation interactions with RCU Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Stat-Signature: dcocjpstwc4rg1dtiwr3axwfk7om6yhj X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: EAE471C000A X-HE-Tag: 1709263724-755769 X-HE-Meta: U2FsdGVkX1/9xb7iBl87MhlTOm8bJCq7yQdtugBB0Icy4yFYpJWllOJM+OB/OqVDZOXGPSRHQstOramcNySaURCtenzE0gzre0gerxHgHRjxCGiXUMAm86PJPalDCvdhbKrHJus/NU1qiGUpKHokrDcvw2sSJwwme/HACSJWDF02sbzmoqAWQbeAFc7K8IIApmIUr7fiUHyiJcggfVgQ2J3xsJkk7/7PRDQonD168WZkCdjNLBkd6KXnAt98nt41Ke0ywOHJvpg+8KXSXEPoJ1c2y9X/uzEb6jBSQXiB8b/X+LYOIl8QQbtwXC7bo7fT+70foWfvcBPB2XBGV8EiPk5yXr4bZFQhPvF99esJnQHcKprDisvhjBnST7cr62JUJiWvU4b+duaU5L/n9JVxMK0aXhvRoeejIEgEjjVXuqtW920x4SLR5AzpC/3J8Lkfb9VmN9UW4WZyXWgLkO2TcfNsLRtr0LaWk4dKzvRtFBAcI2ftEu4DkC+zrC/CwpWGk28bDRMeKOSyyhzWm+CZffO4JQONpelt8HNyB7qAKqVDMJ+ZzeSoSK5EGoCggmznkOoSvpdI7ZIJbVC3mf8Lmv/y6O7VO1a+lSdCtYzzsRrkfIInSPQUTaXbmlj+lRgF+JDeIyh1obGIDuAMyfv9u1S93BrvtiUnSnY8AbRT1eBLgVklxu00nNFZVW2Azm+EQZp6nx2ieMNFtcY3N5HrethK2Fd4MAy7aat5K+tdWsY2EJ7dUMWH3mInUGA4d5KwTYZHesomn0OWy/hReT8ziP+tLN/5Z8isixy0k5QvzMbYuKtN5L4reS6fPF8OQasBeBeidktkzMLp8qLxyhUScXJIE/wp005rYpo2Uif3WVYhvEfnKPWeq2nrXdCCF5pstvX7pnbPjC4nYO3CP1+CxG10h0pqqtsQpi6e4k8x4EDJ2yJV8h9sE794jRC3+/loexDl5/+A0jATpVKtr9o 0fmsfGfR XmOnoNMVT7oeWy8qhg0piCOojnt2rsxnb2UcZCRZiB98Mz5O1lrDo5tKurDg0ahm2MIsMAcEn+Ky9z7+YyyNxq4rRmOkH6PFzVM71IdjJ/VVabWuC2iklf04TgV28E3/Jw8AVqzqFM+J6vjTA6kOKFYWK2G3/C4Ick+CS1ThXy9lNATLL2/c3BXyh2BoyQ3U5NqAU3LWGsNr6wQ63GMIhar02TZKiTMM4KeXVSdhOWZmAwT8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Feb 27, 2024 at 02:59:33PM -0800, Paul E. McKenney wrote: > On Tue, Feb 27, 2024 at 09:19:47PM +0200, Amir Goldstein wrote: > > On Tue, Feb 27, 2024 at 8:56 PM Paul E. McKenney wrote: > > > > > > Hello! > > > > > > Recent discussions [1] suggest that greater mutual understanding between > > > memory reclaim on the one hand and RCU on the other might be in order. > > > > > > One possibility would be an open discussion. If it would help, I would > > > be happy to describe how RCU reacts and responds to heavy load, along with > > > some ways that RCU's reactions and responses could be enhanced if needed. > > > > Adding fsdevel as this should probably be a cross track session. > > The more, the merrier! ;-) Starting to look at the practicalities of freeing pages via rcu. Where to stick the rcu_head might be a problem. I think by the time we're in free_unref_page() enough of struct page isn't in use that we can just... throw it in the struct page union somewhere (ok, actually there's one already for ZONE_DEVICE pages; we just have to make sure it doesn't collide with anything - probably just compound page metadata). But it'd be more efficient if we weren't using linked lists for this. It's premature to ask for this before I've tested or profiled anything, but I just wanted to throw it out there that we might want to look using a dequeue for pending frees (and perhaps other things?). Willy and I were just chatting about this and I think the conclusion we're coming to is that we'll probably want to try just RCU freeing _everything_ at first, the reason being that anonymous pages are probably going to get included and that may end up being most of the allocations and frees - so why not just do everything. Meaning - _lots _of memory cycling through RCU, not bounded by device bandwidth like I initially thought. Another relevant topic: https://kernelnewbies.org/MatthewWilcox/Memdescs The plan has been to eventually get an actual enum/type tag for our different page types, and that would be really helpful for system visibility/introspection in general, and in particular for this project where we're going to want to be able to put numbers on different things for performance analysis and then later probably excluding things from RCU freeing in a sane standard way (slab, at least). I wonder if anyone wants to solve this. The problem with doing it now is that we've got no more page flags on 32 bit, but it'd be _very_ helpful in future cleanups if we could figure out a way to make it hapen. That'd help with another wrinkle, which is non compound higher order pages. Those are tricky to handle, because how do we know the order from the RCU callback? If we just don't RCU free them, that's sketchy without the page type enum because we have no way to have a master definition/list of "these types of pages are RCU freed, these aren't". Possibly we can just fake-compound them - store the order like it is in compound pages but don't initialize every page like happens for compound pages.