From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D559C5478C for ; Tue, 27 Feb 2024 00:43:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F071A4401D2; Mon, 26 Feb 2024 19:43:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EB76544017F; Mon, 26 Feb 2024 19:43:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D57EC4401D2; Mon, 26 Feb 2024 19:43:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C5F3A44017F for ; Mon, 26 Feb 2024 19:43:35 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 844D11208D0 for ; Tue, 27 Feb 2024 00:43:35 +0000 (UTC) X-FDA: 81835735590.03.AD4CE8B Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by imf06.hostedemail.com (Postfix) with ESMTP id B4D97180012 for ; Tue, 27 Feb 2024 00:43:33 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=fromorbit-com.20230601.gappssmtp.com header.s=20230601 header.b=QAtwyaeg; spf=pass (imf06.hostedemail.com: domain of david@fromorbit.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=david@fromorbit.com; dmarc=pass (policy=quarantine) header.from=fromorbit.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708994613; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZrngvPKRu7DYWkRTK5Ps5V2RuXh9L2ADzFSIUkVyZxA=; b=tZ8JAdmZej1MbBRhaHrQ2GbzhqJk3BB7VUzYO2HJD8SmoVmvUSINl37OgMoYsZb5PfhkuX FnPZ2L7CyTGj60y2VqiGFgu6TNGBXR6L88nYxXnxECvFu6lzHhlyoIc8BPJKZaX9Yjx/g1 YWW+YUG8bXOEj+NwIfGj2ZmPbA/KFGg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708994613; a=rsa-sha256; cv=none; b=plyBtjtea69pvOYs4npoPtxx/PAbRwbOkBKCoihXbNwlwSk5vCLcNiaPxdviMNnlC+DEFd 9kpAmG0WjrbQ9AkjsPBdX2P57yGtAAFK9rpZ0tGbO2Le/AgYLviTw3TNOVQJoXaRVY5poh AculAJIz2eP+qvjxCc32DV5UVzCRsqM= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=fromorbit-com.20230601.gappssmtp.com header.s=20230601 header.b=QAtwyaeg; spf=pass (imf06.hostedemail.com: domain of david@fromorbit.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=david@fromorbit.com; dmarc=pass (policy=quarantine) header.from=fromorbit.com Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-6e08dd0fa0bso3199948b3a.1 for ; Mon, 26 Feb 2024 16:43:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20230601.gappssmtp.com; s=20230601; t=1708994612; x=1709599412; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ZrngvPKRu7DYWkRTK5Ps5V2RuXh9L2ADzFSIUkVyZxA=; b=QAtwyaeg80B5UgN8VEzR/wiskdQsGaFwqZRMhW3xOVF5CS1mzr1oBhUQZJMfxI0fhx Lh7FRWnw2ieqGysmi/QK530V+BZfA6XNqPvVVxUdmUa4wzBtqorHy+3EynXtr7oDjl8/ 7/EjWrj8meO146VnPELA05mYixd/kkp3i6ymSaJe6o2MSe13j6AOY5s16ubEf+GH0jpo F1zls5fiqBepGki7zoTixmdwzBuutrbEMQMzDl1xWEl7fPHaFOxrp8kVt0KMNw+AaNir Qzvbr2/5Vi1+se1xxiZoa1UaIAL68rRXexISumHvC+dyGO4YBqp9RgjaBBDQcmKIaOoy U17A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708994612; x=1709599412; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=ZrngvPKRu7DYWkRTK5Ps5V2RuXh9L2ADzFSIUkVyZxA=; b=Hp8LduvRvbXO4K0hpNMQo/qS92ha5enVO8Ve9rik+j8X09uSKl3V075lZMDbpPPEoq KIz5tE3mBKCXZOLLlbLRnZiZ6OPhp4AV8MC1DVBFTYg5o9oMDYd/G0oqtYnB+Xf1SbYx YGSjegwiMVTGdZndTt7o3ces54NTyEd+c4SueGLdO/1lpd3Resv2L20yYiRKeYkwnnTs BPqOS5oKf/CjeOjD7EE5PUwozk2QFcAq+5tPjKpZVkMqiGNvPudIZ6obo5ocxunfR1vz FMNM2PZHs/sGJ2yjTdgsRAAg0rdIPyczpffNbhiEod7svGgpALwKGA7/wYzlflngeXWi yFPw== X-Forwarded-Encrypted: i=1; AJvYcCX4lTJv8JdaEjPq2adDZTXYpAodPeYdchLRYHN0sVGuD6CMp1Gb+ozfuW1ErPC+KXRyQrip42Vdggmwltf5flrhhWY= X-Gm-Message-State: AOJu0Yw9GArHKziq3Xb+BWS2Tuek6Y0jnzyp2mBoxQ5CN2pyQY0b8yeO HIcodTCwSyBV2ypHa/AiEcpmiL3BAf45L6vibtJ+MsEsPHqEpgDXEZGk8q3SLgM= X-Google-Smtp-Source: AGHT+IF8FLEW5uhlJVJSeS+3Q6ieOhlDW+d89tKJv6iRDVhfnxjTJO9Q8GfP9tJ9ZP+BT/0s1SJomw== X-Received: by 2002:a05:6a00:808:b0:6e5:2633:5d78 with SMTP id m8-20020a056a00080800b006e526335d78mr6008961pfk.4.1708994612450; Mon, 26 Feb 2024 16:43:32 -0800 (PST) Received: from dread.disaster.area (pa49-181-247-196.pa.nsw.optusnet.com.au. [49.181.247.196]) by smtp.gmail.com with ESMTPSA id jw8-20020a056a00928800b006e4e4c80e3fsm4124770pfb.29.2024.02.26.16.43.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 16:43:32 -0800 (PST) Received: from dave by dread.disaster.area with local (Exim 4.96) (envelope-from ) id 1relZB-00Bz3U-2I; Tue, 27 Feb 2024 11:43:29 +1100 Date: Tue, 27 Feb 2024 11:43:29 +1100 From: Dave Chinner To: Kent Overstreet Cc: "Paul E. McKenney" , Matthew Wilcox , Linus Torvalds , Al Viro , Luis Chamberlain , lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-mm , Daniel Gomez , Pankaj Raghav , Jens Axboe , Christoph Hellwig , Chris Mason , Johannes Weiner Subject: Re: [LSF/MM/BPF TOPIC] Measuring limits and enhancing buffered IO Message-ID: References: <5c6ueuv5vlyir76yssuwmfmfuof3ukxz6h5hkyzfvsm2wkncrl@7wvkfpmvy2gp> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5c6ueuv5vlyir76yssuwmfmfuof3ukxz6h5hkyzfvsm2wkncrl@7wvkfpmvy2gp> X-Rspamd-Queue-Id: B4D97180012 X-Rspam-User: X-Stat-Signature: o38yznpbih6j3o3wwweyy4otbqzx8fbm X-Rspamd-Server: rspam03 X-HE-Tag: 1708994613-501702 X-HE-Meta: U2FsdGVkX1+oAhA/KlrQ1QYI+tYufmCj7rcNVVVS8UbffT5i358zX1hj28OnKKjFig+ku554YAXZgOl77gpr7IYxTxWvCn/iV6rehI1rYi6MvNTOHFkPOWuZunO2WHuict7jhDxqNL6s0h1QezXv8UkheacAV1J2Hs/KEjA2uYVvAeIU0DLD0HoYg2O6hPaHqgaZj5X+kgrG1Sj0LNamvB0LFPYpVsKFN8hqQPPo1yLyRPHYuRjWL1BcptifHn+6/0gO2phQ2l6c01PpEvVrbOmrhAaWduxBNjcRv2NqZGkdfrVbKRm0qMGfhFMaGY4X8wc9f4we6AQOEKHibXOrg1nzjin/UbHcQATnhywq4aE1NyFllRQDuAeXLxtss2f5nhW7So05H74Joa+chNJ+963gnhG7quNiddnlRLsQDN8vKKyRi4mgM63jy9PZGaUQGAy5/y9QGUN+Kmfwt9Ec59sgzUhfKmQq7DLJSQdCYUJoHNpP8QZsciWCoVv1lK+7s6HXaI4edhWJ7FlI5jnoXsLnDKBRzWSqkAivzZ63EwCjDr1nrn97imZV0hVB8Ce1jcDPjGgyd5CXLyVQ9ksOG8FT/Ot6P2peBvnlxBLyB73TBn5whrsdjlgSC6AoLlj3/4NV9jHHLiKb7g+nFY9KmVq8R8ahpPXNddYa/GqAVXveHlpdepwZoT9QbX9a1aBvGI5+cbxm1YeVQcxfBw3ImT3wBzyIFcqoZPzUupj2V12LG0cm2zTZkc+T8iW1qJBpTTx77uq0x07ym7BQVQMrwdbieJki45XsywfpWPGtBXlfb7JWzrnl9rY2veShJJHVuU9/3QutF1d8wIXlgCdWd9oPbqd/nzrqRrR4N22jKxsQ+dLI3Y93tPkW0S185P0huuULtDkGWiJRzBNze2PceyOB/ic5WlGKBP836eNkkXB4dl7QXfE4JtgjLZ59hkApG2YRymC2ov0RP6K4Jiz bSJhyacE hJjxpvKbMnFWjochVsHy6SyYtIiizgfks2KAq3g8A8Olc/nUQZb01kOipN0XeniajmCguGcycFStbnQ/yk5GB1NcaNngtNXIiXFaLohY9xOwjCuJkP/xHacEH3w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Feb 26, 2024 at 06:29:43PM -0500, Kent Overstreet wrote: > On Mon, Feb 26, 2024 at 01:55:10PM -0800, Paul E. McKenney wrote: > > On Mon, Feb 26, 2024 at 04:19:14PM -0500, Kent Overstreet wrote: > > > > RCU allocating and freeing of memory can already be fairly significant > > > > depending on workload, and I'd expect that to grow - we really just need > > > > a way for reclaim to kick RCU when needed (and probably add a percpu > > > > counter for "amount of memory stranded until the next RCU grace > > > > period"). > > > > There are some APIs for that, though the are sharp-edged and mainly > > intended for rcutorture, and there are some hooks for a CI Kconfig > > option called RCU_STRICT_GRACE_PERIOD that could be organized into > > something useful. > > > > Of course, if there is a long-running RCU reader, there is nothing > > RCU can do. By definition, it must wait on all pre-existing readers, > > no exceptions. > > > > But my guess is that you instead are thinking of memory-exhaustion > > emergencies where you would like RCU to burn more CPU than usual to > > reduce grace-period latency, there are definitely things that can be done. > > > > I am sure that there are more questions that I should ask, but the one > > that comes immediately to mind is "Is this API call an occasional thing, > > or does RCU need to tolerate many CPUs hammering it frequently?" > > Either answer is fine, I just need to know. ;-) > > Well, we won't want it getting hammered on continuously - we should be > able to tune reclaim so that doesn't happen. If we are under sustained memory pressure, there will be a relatively steady state of "stranded memory" - every rcu grace period will be stranding and freeing roughly the same amount of memory because that reclaim progress across all caches won't change significantly from grace period to grace period. I really haven't seen "stranded memory" from reclaimable slab caches (like inodes and dentries) ever causing issues with allocation or causing OOM kills. Hence I'm not sure that there is any real need for expediting the freeing of RCU memory in the general case - it's probably only when we get near OOM (i.e. reclaim priority is approaching 0) that expediting rcu_free()d memory may make any difference to allocation success... > I think getting numbers on the amount of memory stranded waiting for RCU > is probably first order of business - minor tweak to kfree_rcu() et all > for that; there's APIs they can query to maintain that counter. Yes, please. Get some numbers that show there is an actual problem here that needs solving. -Dave. -- Dave Chinner david@fromorbit.com