From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 819B6C02180 for ; Tue, 14 Jan 2025 00:56:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E6256B0083; Mon, 13 Jan 2025 19:56:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 197596B0089; Mon, 13 Jan 2025 19:56:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05E156B008A; Mon, 13 Jan 2025 19:56:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DA2286B0083 for ; Mon, 13 Jan 2025 19:56:45 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5076AC0505 for ; Tue, 14 Jan 2025 00:56:45 +0000 (UTC) X-FDA: 83004242370.20.FB61BF3 Received: from mail-il1-f174.google.com (mail-il1-f174.google.com [209.85.166.174]) by imf27.hostedemail.com (Postfix) with ESMTP id 34BD340006 for ; Tue, 14 Jan 2025 00:56:43 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=PwyDwpLX; dmarc=none; spf=pass (imf27.hostedemail.com: domain of axboe@kernel.dk designates 209.85.166.174 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736816203; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aFRwrXz42i3Ex6VxVXT/+/UsrVhslXKijtqwvVA4fO8=; b=o7sf2jtAcnsgrgLnQ6rz5BQdBjsSFSOCY0T8eRkfT5Tl4aqdAtnrIaWpD5RE883H1CAxLy sDPxAwk4Z4ieiXuD2jCXtsyLebGB3kAogEK3N1su51yf0mY4jHhNlpFo/WR6jBmGk5g07Y eBocpyq3GY16tcv+XEtclRuYJmaVfUY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736816203; a=rsa-sha256; cv=none; b=mJIOj/AfJf7McUJXSzW04/Qetz3y3izBS6Y7Oi4o17xt44O/E0MwLJkPgrm5u/+MqErX+N SPnZb5iBmDXzMFWSxFpbsu7Z7Kgvji4LzVKjY/sw9AR1eeA1dwFQSaUgVWsT0Uz8MWLnO7 GEZHTpu/YklEdjT2bEG7HB+m7iyu5ds= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=PwyDwpLX; dmarc=none; spf=pass (imf27.hostedemail.com: domain of axboe@kernel.dk designates 209.85.166.174 as permitted sender) smtp.mailfrom=axboe@kernel.dk Received: by mail-il1-f174.google.com with SMTP id e9e14a558f8ab-3a9c9f2a569so33408865ab.0 for ; Mon, 13 Jan 2025 16:56:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1736816202; x=1737421002; darn=kvack.org; h=content-transfer-encoding:in-reply-to:content-language:from :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=aFRwrXz42i3Ex6VxVXT/+/UsrVhslXKijtqwvVA4fO8=; b=PwyDwpLXkSSlp6CMVXuO7q9HrLtmcsd9G+2lJEhFkdrGxMOBBUpksCYfIkwvJspkaS ifVs6maiKEJa3orhy9/VCWbf/Evo89a76mlYaZYYr//GmlKTrMFqSXmRFR0V3T3+LBLa LjmWNkCwKcUiN00Y+sSyYBE1Kk35OuL6YGhbfddbm0Y15EhZ/WVcUbV0TKPSTa/eIWIV x7bT6HNMrElMLx9LOeFW/K7LP7soTw9QtNdfH2mDxB0wus+5Yu0nKbiFzfRK9r4xWsRy KY178J40wx/49sjC4f6jENSdgMspAptTVHTkMZQROG4t8d7sfqrN15Zveb53C6vPCSmn qE6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736816202; x=1737421002; h=content-transfer-encoding:in-reply-to:content-language:from :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=aFRwrXz42i3Ex6VxVXT/+/UsrVhslXKijtqwvVA4fO8=; b=HOX0oysDurS4XMqqpqczr3Ytx/gSSCObnUkI/YKApb7iEoLTgbl3s4YkN/dhG7GbaM 2SUbZKi8bU6WgeaPTWAw7sHqBf2fmT4nLN7SS2Bf/Mkpx3w9nOHY9vTM/Yw3Pq14a7+f 6Zkmsy1siGI9e2wDjXOblxHMa8kNvu5Y+ZwuOJaI8i2hv4ELZS/nx6hBkIgD987MTeIg 2eKgKQCaA/+Wm/MWEBfs6qD1733bSPVWrjWmtA53sR9vId254XaYH2F94/Oxxg7sfjA1 lrqA6E/HkBe77CT81rebu1RiSc2haR3nxd1KYlKKA1fc1Y3T5dLmQWhaLk2SpiJkNdRv YAqw== X-Gm-Message-State: AOJu0YxjjDvLraXrL/r/b+pTuT/nUneBf8nFVaYAQZvzp6V+NGbkuSQM /NiR0va93hA79kVAbXHM4wfAfqgsDMfJbghLN5/AT9LZFTKsyr2MN7uCeCO7gGM= X-Gm-Gg: ASbGnctsd2hg34HFn9ssX/Z6GXrkPlH3653Seqmk13pDupuTbvf8JFngEczcKL+gWuZ 83Md4mS0iRaoniA/6QDTSGlS1dzzHkcSZSLmPIg6zYWB0wevNbPZADh1wA132g2vUKvpV43Oy2o L8Nrz5H4+gfuH8G8N4iTNgsxHopgYFtAr6ja94OwStKpq4nPd7E83wnBraC1Si4DIbTO6iXbIzt fOutcbBBVS7vj+6nLWFBdFdAL8CeqGSA67ZSc7uqMaLbG03eVyjNw== X-Google-Smtp-Source: AGHT+IHfVig9z9X28a8RJmX8cXjlHvlOE8g17fwAl0H867Dl3tHMXc6GXh5dVP+DOBfwAE8WmycnJw== X-Received: by 2002:a05:6e02:194c:b0:3a7:7ec0:a3dc with SMTP id e9e14a558f8ab-3ce3a88a2femr191007565ab.14.1736816202027; Mon, 13 Jan 2025 16:56:42 -0800 (PST) Received: from [192.168.1.150] ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4ea1b718bafsm3134733173.96.2025.01.13.16.56.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 13 Jan 2025 16:56:41 -0800 (PST) Message-ID: <010513c8-7c17-454f-b6e0-d03fe7795eb2@kernel.dk> Date: Mon, 13 Jan 2025 17:56:40 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCHSET v8 0/12] Uncached buffered IO To: Andrew Morton Cc: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com References: <20241220154831.1086649-1-axboe@kernel.dk> <20250107193532.f8518eb71a469b023b6a9220@linux-foundation.org> <3cba2c9e-4136-4199-84a6-ddd6ad302875@kernel.dk> <20250113164650.5dfbc4f77c4b294bb004804c@linux-foundation.org> From: Jens Axboe Content-Language: en-US In-Reply-To: <20250113164650.5dfbc4f77c4b294bb004804c@linux-foundation.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Stat-Signature: 9mhgep7m6m5zjhbkty6fpoc8o6mdqwhw X-Rspamd-Queue-Id: 34BD340006 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1736816203-687285 X-HE-Meta: U2FsdGVkX1/HrOiMoK2jOqDjts0phguTwo1H9uUeVJMLzC6DHN35fHGoPBU1NxAIGwyA6ZtgZi5Sqvcv9oZSyU2OJ1WZCCdRzOu8MkBcgf9ZavSMvkxOL68GuFmxvXUGFuZYUdAVTRyYvF01HzqMVeARckdPLsURUXHf0eSbRPSU3NNyDeMJ87fzsDsq8ERpAkg41L9R/PH09IGPgJiAU0Aw09I/zxU5ZgL7dL5o35YrXTKWN2N1m+fFOJJemQaq23toJfElCm7bQjOL37KkoYylMAGk9S6I1sbNB/v7Z53vgvRB2CabMFORF1ZhVMOZt0YQrwy9ZiLnORLfBFXutG/Rt9tjoem1RXcQjfcL39O/Irmh49UaQyWVaJYsmuc27U3z/eHG367JUbocStqgakDPisbOiY3mtVL14quvSVHYuke7RqmAW3GysOuAZq2Fr0Iv9fVbX7La53R+LK9md+SnxH+xqE9P+b2KRu3dwrUcejsm1AEnwcSVE4mH8huWMBhrAsMBtDp423lNIkJlhR4/u0JFTpWZQbi2sxfMrYwYbDeBKuq0lPOErRvkorJrtbkWoHBncrvTNv6JFckLkY5RjM4JdFQVuLwGi9tGDEj+5lSgJXj5RQj5PyDrVvaxss663iD/f21EwQqehPcpNJjJQiW7iPgADViOeiB9+fiIcR4dInAShw7D5+EGa+u0CSuxHJ1EsY3yMepBoEmQnhEH1U3RLr03kQYA02cmqQcBMr+HMj6F2hFpcOOY/NtlXV5xUMuf9dw1HvWeUT8bBnMughRtih7fFJ75l3ShfDu1ZhlIHNK5bYNo77hxTZduwcYINwQhgQnc99pMOZoGs9Ylv4UN429vUIdNbSsDsicrzU2+Muy8Kb7j2edrnShb7Ma/PPUe6ISBk9undxc5zuu04QbV4UUqhUoxAvEHiRUFvInVQYT/JlZP6n9SHyERC780R+Wo76AwFoPsHHI D1crIXEE OVNN2rCdij8KgMuYrKeqFi8hc2ud+BS4NVlSr4mVm9ZuJyHMssvDgN/qdfM5iEN0j8VusWkcNAtTtzcSUbJHm3isjRZE6+7pySmhfUagJoE53SllOhPjqQHHPIT8ehNZQ2eBMwNrK8p+o9I0XmytaCq3iY38MiV2P37NMwp5m6w4jd8ryZbQ0njxeL5IvFIhI/4awaVKKv1zNkjX+OCLQSCJ8z/1o4rmBL4fATfncd9x/QnXicR83DaSEFX4tx1VRu8HzKQzww6Zo/TayKS6kjrjyVlOr0WQ/clUc24zQLNSmNvuTRavhlltptYTIoTp41gOZYCanls52z4TMS+Q/QDTCdfeQdd2jTjWw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 1/13/25 5:46 PM, Andrew Morton wrote: > On Mon, 13 Jan 2025 08:34:18 -0700 Jens Axboe wrote: > >>> >> >> ... >> >>> Of course, we're doing something here which userspace could itself do: >>> drop the pagecache after reading it (with appropriate chunk sizing) and >>> for writes, sync the written area then invalidate it. Possible >>> added benefits from using separate threads for this. >>> >>> I suggest that diligence requires that we at least justify an in-kernel >>> approach at this time, please. >> >> Conceptually yes. But you'd end up doing extra work to do it. Some of >> that not so expensive, like system calls, and others more so, like LRU >> manipulation. Outside of that, I do think it makes sense to expose as a >> generic thing, rather than require applications needing to kick >> writeback manually, reclaim manually, etc. >> >>> And there's a possible middle-ground implementation where the kernel >>> itself kicks off threads to do the drop-behind just before the read or >>> write syscall returns, which will probably be simpler. Can we please >>> describe why this also isn't acceptable? >> >> That's more of an implementation detail. I didn't test anything like >> that, though we surely could. If it's better, there's no reason why it >> can't just be changed to do that. My gut tells me you want the task/CPU >> that just did the page cache additions to do the pruning to, that should >> be more efficient than having a kworker or similar do it. > > Well, gut might be wrong ;) A gut this big is rarely wrong ;-) > There may be benefit in using different CPUs to perform the dropbehind, > rather than making the read() caller do this synchronously. > > If I understand correctly, the write() dropbehind is performed at > interrupt (write completion) time so that's already async. It does, but we could actually get rid of that, at least when called via io_uring. From the testing I've done, doing it inline it definitely superior. Though it will depend on if you care about overall efficiency or just sheer speed/overhead of the read/write itself. -- Jens Axboe