From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB053C48BF6 for ; Sat, 24 Feb 2024 19:11:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 221BF6B00B3; Sat, 24 Feb 2024 14:11:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1AB866B00B4; Sat, 24 Feb 2024 14:11:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 024566B00B5; Sat, 24 Feb 2024 14:11:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E004D6B00B3 for ; Sat, 24 Feb 2024 14:11:51 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 61AEBA03FD for ; Sat, 24 Feb 2024 19:11:51 +0000 (UTC) X-FDA: 81827642022.21.FFA3727 Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by imf08.hostedemail.com (Postfix) with ESMTP id 4BD34160008 for ; Sat, 24 Feb 2024 19:11:49 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=google header.b="gJJZ/R22"; dmarc=none; spf=pass (imf08.hostedemail.com: domain of torvalds@linuxfoundation.org designates 209.85.128.54 as permitted sender) smtp.mailfrom=torvalds@linuxfoundation.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708801909; a=rsa-sha256; cv=none; b=hjse+SAd0ONst1fwx0XqQZofvRlasH2nrSVaIypQPAFzTpXYAjKHp8Tm9/6M2YLbj6b/YU OmndVVhSJT2ng6mKCzdtNkt19rkKZEs/A40vpGdYSN7Xm06pS+s5O5lk5DleW+kOaYBBvT RvxtfE8mYhxtUJsfUgnAHbevCZObMNw= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=google header.b="gJJZ/R22"; dmarc=none; spf=pass (imf08.hostedemail.com: domain of torvalds@linuxfoundation.org designates 209.85.128.54 as permitted sender) smtp.mailfrom=torvalds@linuxfoundation.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708801909; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=v3rIRZE6qd8PRkBRyLQqB1Dk89aejWHzm1lt6LZxklY=; b=i52BMGaXtTqMo9gKX6SmhU2jGSULFtXp309HLZwTLRZ3sGPbsQSusDv2MduuBCxkf9hKY/ iw4qo7kstD+/MlVFImunvgwiP1+Dw7dNn/ThYwiFUXobBeUgUeSab2xwr9V/w+inWc2mW1 Yn4O3PZ43mD0QZBjfm7Ai/goOCFJIS4= Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-412960dbb0eso10590465e9.1 for ; Sat, 24 Feb 2024 11:11:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; t=1708801907; x=1709406707; darn=kvack.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=v3rIRZE6qd8PRkBRyLQqB1Dk89aejWHzm1lt6LZxklY=; b=gJJZ/R22wCEe7APyaN9LdGjkpTf5bQE7u1cLazUYp50jfxP81rCVO7yvMDReT7BBz3 wgUJcJ1UW5frNjQUoXRH3/+0lExgBU0ToU8JVGm/MsgjBCin3YA5ADCcqoX4tfUuf8HF JfJefyRGfCY0yKh5cEPy3VLWMzsBgVlJq+q0M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708801907; x=1709406707; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=v3rIRZE6qd8PRkBRyLQqB1Dk89aejWHzm1lt6LZxklY=; b=rGS/f6bSX+tGQuvobIG0vXzSgp/8BWQB/ffVj2sqWaLyeYu+98KXitcHl+PqkFWZuE 0NasFqMJbpzH2e4tKJi/q3d0KIQJroUbeVUrf/dzgnE913oB03DONmMtIUmPwgml9DYs /CYm8XMk+zzJSkx5BlQOtqhMIltSUKJ1oscME4PjFFfppDZExj9T2g9LRItRuB4RRRul 3+d9/6kaqLctlhBavXuYW4ad6ZoeJvrB65+GjntwgzDFQJ55wEEtXWEXDQ20IP/USuWy YohByFMSxyDx0ANJqdRVrhgU8BKnXLqOHO0qK8kYrEeyqe59JP7clzmJ3o5JQ8Jh+gew l48Q== X-Forwarded-Encrypted: i=1; AJvYcCXyEcb+knfIsc2wYQhri0AlNpT2zj1PRdjk1tSH8TSxvuMDUK0Nk8bEZP181LTn8i5/3sf8ISXCmprUQfPrHduSHGE= X-Gm-Message-State: AOJu0Yzc9s6sVMyVcEFz2rHDRH6SOqlX4joA+Hhg/Al8sjkAUo2naOSA sFZHt29N/PFUVl5mnngNzfn1Mcm/hwukBQ+3geNLBQCNquFpXLYqRW527JGjGPjcyLsptvyVL0G I/Es= X-Google-Smtp-Source: AGHT+IGWQDYdVsPVKvqE5eur4TVkqGn0zKw15Cuf9p0DiltdMClZCR3E2sh5RWThSq74u1xGsybEEw== X-Received: by 2002:adf:e2cc:0:b0:33d:3098:c1c with SMTP id d12-20020adfe2cc000000b0033d30980c1cmr2267550wrj.5.1708801907520; Sat, 24 Feb 2024 11:11:47 -0800 (PST) Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com. [209.85.218.48]) by smtp.gmail.com with ESMTPSA id k10-20020a17090646ca00b00a3d004237ebsm814325ejs.212.2024.02.24.11.11.45 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 24 Feb 2024 11:11:45 -0800 (PST) Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-a3f4464c48dso219276366b.3 for ; Sat, 24 Feb 2024 11:11:45 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCXQm/P8YPkIEc/ihsHWIfYrLwEH8+opP5FPVp/gSKT1BV9wWYYd+6wj5ufFi1MPEVrWDxBU28uSqrwQ3HpYFmJl8dQ= X-Received: by 2002:a17:906:260d:b0:a3e:d2ea:ff5e with SMTP id h13-20020a170906260d00b00a3ed2eaff5emr2238168ejc.58.1708801905524; Sat, 24 Feb 2024 11:11:45 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Linus Torvalds Date: Sat, 24 Feb 2024 11:11:28 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [LSF/MM/BPF TOPIC] Measuring limits and enhancing buffered IO To: Matthew Wilcox Cc: Luis Chamberlain , lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-mm , Daniel Gomez , Pankaj Raghav , Jens Axboe , Dave Chinner , Christoph Hellwig , Chris Mason , Johannes Weiner Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 4BD34160008 X-Stat-Signature: 3ojbqbiuq4jjrqi8metm5cd7kryk61ns X-HE-Tag: 1708801909-530254 X-HE-Meta: U2FsdGVkX18tqpO+sIQOPzIYPyX8gwQUYVx5WcKE0kD/9N8XExM8aj4U2qzYiB/0jusEhnKMKQh5jsNuxK84sKniM8BhrfUZA1jj+W5QPaeT+VGamlsIRCg+9fL2y/cVZJqqUFyrCpoRF0QAsAZHZi+Vpyk2/CZiZzWXaxqXapoUug+TPFY5Crawl3DS20CsndX6cEzXKSHNSh5UnbI26LTf04gvDJPoEgX8FZbL/dM7g2hau5czHbYFWFki/vpPH2IkkeVLt1C3bxxVQIz5FyrShf7O/HEkeAtCpFoWCL5ZUMKhnDC8waonGMCsOdhT83zfOcnYOmXfah/2hYdPTQiUOvWBELqKKfrZsIwRs4f7uQdBy8rUhPPtXdESNAuLJu8G9ZdXYt60PQtdPYVflJzKEjTnRzzgTpfcM/Fp9tqo450UWrAaXXQ/JsLX+F5jQ+sYiBh//9csXA/Br49C6yI5sWTIaVRIbZoVPKC0vQBfieo0lBB7uKn5Z44Kg8tPk4TQuTxtVwoostcVsVfpNnpSryn6NA3UhYuC/dotmwj+p3UnrEPOEp2Jw3jRxU1jv1XJvEpDkVcaVpyL+2Tye2wBMi+tWzBglQSPlfyATmpXbl48Sgz1xeOx8s+7UcnsgLmb9v8idsj6ssaf70nQvddF+wppDpAVhR1OQvfe4b2BZKAHzzDZrNxclRlF5HBN1TP1MG93a//nllb6hMBx95/rp2Ci16pcCg/i5y4JwTlBQmJViTu2KgwffDRZ4e+V/EvfWLS9mMssmc3ieCPnHg8sSRmgRMBEgYVu/xlfr2lXI0e03W5H9akcYaku8p4cyVNkIi9EaJNg9bdXSy/VHpr6ehJ/55IKqc/Fv8834pvDacb05gu8ZCGaCnHsqMWp4KfaObxAT89OafYWfNl3+lK8z78ARJPtbM8hHo+rnIsLLm4cbFrX6BQoRk9u5lp/A5gAqw1jBpF8Q17zjGX RkdbPcF6 kjVCoe/UBYFVnD7l3/+a3TqWr9Wd9nmnpV25IyKxjG+D85DoFIAez06agdwDo8EhEUJp+mQK6dc3Y/up2rTUsnSWextwopyi+7XDkm3n6YwAsQyikYqrxuRbLHxJviyqD9ECZoVTujwghQfu4UVkuJjXos9uQ31wzTRkjWeWYPEcA6VFdEZ9BvVrBOF8hs3EOuiCZxGQx1/7D9WBrdX5VztQ4DDYaKNGBBizA3MQrwT+KFyot6STPNnqgxWbhQPN3Mr6kDL8Fus7FV3AoOCwNEaqcKCDklNVg0Ly+dmBj8T5wUvymSRe6f4pIF1lOQQAUvhQMz+bWjLMzQE3iKtrUVYuIcnZujUDg+yMgp+nmf0h61MFI76EHyCiW9A9PRAtKYRc5x6ekL+R13D3cIboJ5jNjKWJO5lkN1GRHnHhn3qhLn3EQxla3oy+viJPADSNEh4aJZlpNTQ5U+RHBR9+5ICeFyT98DtrI1dKFEjXbCrcuMs2wyH9VV/twhh8xKuOdJvRodQtN4sjVrPA19Tw0oXQs4oLwcEJnNvKziK1gAME5HwN3Y0cPdaMMyVrcS1TEVQimBGoXJfTWblY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, 24 Feb 2024 at 10:20, Linus Torvalds wrote: > > If somebody really cares about this kind of load, and cannot use > O_DIRECT for some reason ("I actually do want caches 99% of the > time"), I suspect the solution is to have some slightly gentler way to > say "instead of the throttling logic, I want you to start my writeouts > much more synchronously". > > IOW, we could have a writer flag that still uses the page cache, but > that instead of that > > balance_dirty_pages_ratelimited(mapping); I was *sure* we had had some work in this area, and yup, there's a series from 2019 by Konstantin Khlebnikov to implement write-behind. Some digging in the lore archives found this https://lore.kernel.org/lkml/156896493723.4334.13340481207144634918.stgit@buzz/ but I don't remember what then happened to it. It clearly never went anywhere, although I think something _like_ that is quite possibly the right thing to do (and I was fairly positive about the patch at the time). I have this feeling that there's been other attempts of write-behind in this area, but that thread was the only one I found from my quick search. I'm not saying Konstanti's patch is the thing to do, and I suspect we might want to actually have some way for people to say at open-time that "I want write-behind", but it looks like at least a starting point. But it is possible that this work never went anywhere exactly because this is such a rare case. That kind of "write so much that you want to do something special" is often such a special thing that using O_DIRECT is generally the trivial solution. Linus