From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9371BC433E0 for ; Sat, 20 Jun 2020 19:15:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 394E820786 for ; Sat, 20 Jun 2020 19:15:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Za6zjjv9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 394E820786 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 77E8C6B0055; Sat, 20 Jun 2020 15:15:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 72E326B005A; Sat, 20 Jun 2020 15:15:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 645176B005C; Sat, 20 Jun 2020 15:15:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0252.hostedemail.com [216.40.44.252]) by kanga.kvack.org (Postfix) with ESMTP id 49F406B0055 for ; Sat, 20 Jun 2020 15:15:29 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id BCDF12DFD for ; Sat, 20 Jun 2020 19:15:28 +0000 (UTC) X-FDA: 76950543936.24.field68_370c98a26e24 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 961601A4A5 for ; Sat, 20 Jun 2020 19:15:28 +0000 (UTC) X-HE-Tag: field68_370c98a26e24 X-Filterd-Recvd-Size: 5179 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Sat, 20 Jun 2020 19:15:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=In-Reply-To:Content-Type:MIME-Version :References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=QHhdZsEwflik6dhlpG9lDRGf2hUbiRTywvrsqPNIe50=; b=Za6zjjv9SW/fcKUx1GQuuZ9KSj 3wWuDUlczJhnxt7HcGZtrKs/RQAJ2AxNVABPlYlAUBMiiPlezmVC/KuFD6oTzv+hc1dMlkYzLmgp1 BPrTo0wrA4qsogg8qYSC3XksacQyySS6ckSNjRripeZ512qO4mTc7KGXKNg8VQ5SVqp77jaNsBQCD +bna6zliSxrpte3Z5JYZigx6V/DCULCJaNKwASL42oxHzh09Glukgz997JBujOxmA6MEj2P223Zis hG0Xdtepr3u9WW7EyIJ91zJac8F2UV9HEp7UHIfZXswsSbuBZIPMuRls7H1qrI9KG2uToOIRL6u0B oFWJCYJg==; Received: from willy by bombadil.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1jmixW-0004zd-13; Sat, 20 Jun 2020 19:15:22 +0000 Date: Sat, 20 Jun 2020 12:15:21 -0700 From: Matthew Wilcox To: Amir Goldstein Cc: linux-fsdevel , Linux MM , Andreas Gruenbacher , linux-kernel Subject: Re: [RFC] Bypass filesystems for reading cached pages Message-ID: <20200620191521.GG8681@bombadil.infradead.org> References: <20200619155036.GZ8681@bombadil.infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 961601A4A5 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, Jun 20, 2020 at 09:19:37AM +0300, Amir Goldstein wrote: > On Fri, Jun 19, 2020 at 6:52 PM Matthew Wilcox wrote: > > This patch lifts the IOCB_CACHED idea expressed by Andreas to the VFS. > > The advantage of this patch is that we can avoid taking any filesystem > > lock, as long as the pages being accessed are in the cache (and we don't > > need to readahead any pages into the cache). We also avoid an indirect > > function call in these cases. > > XFS is taking i_rwsem lock in read_iter() for a surprising reason: > https://lore.kernel.org/linux-xfs/CAOQ4uxjpqDQP2AKA8Hrt4jDC65cTo4QdYDOKFE-C3cLxBBa6pQ@mail.gmail.com/ > In that post I claim that ocfs2 and cifs also do some work in read_iter(). > I didn't go back to check what, but it sounds like cache coherence among > nodes. That's out of date. Here's POSIX-2017: https://pubs.opengroup.org/onlinepubs/9699919799/functions/read.html "I/O is intended to be atomic to ordinary files and pipes and FIFOs. Atomic means that all the bytes from a single operation that started out together end up together, without interleaving from other I/O operations. It is a known attribute of terminals that this is not honored, and terminals are explicitly (and implicitly permanently) excepted, making the behavior unspecified. The behavior for other device types is also left unspecified, but the wording is intended to imply that future standards might choose to specify atomicity (or not)." That _doesn't_ say "a read cannot observe a write in progress". It says "Two writes cannot interleave". Indeed, further down in that section, it says: "Earlier versions of this standard allowed two very different behaviors with regard to the handling of interrupts. In order to minimize the resulting confusion, it was decided that POSIX.1-2017 should support only one of these behaviors. Historical practice on AT&T-derived systems was to have read() and write() return -1 and set errno to [EINTR] when interrupted after some, but not all, of the data requested had been transferred. However, the US Department of Commerce FIPS 151-1 and FIPS 151-2 require the historical BSD behavior, in which read() and write() return the number of bytes actually transferred before the interrupt. If -1 is returned when any data is transferred, it is difficult to recover from the error on a seekable device and impossible on a non-seekable device. Most new implementations support this behavior. The behavior required by POSIX.1-2017 is to return the number of bytes transferred." That explicitly allows for a write to be interrupted by a signal and later resumed, allowing a read to observe a half-complete write. > Because if I am not mistaken, even though this change has a potential > to improve many workloads, it may also degrade some workloads in cases > where case readahead is not properly tuned. Imagine reading a large file > and getting only a few pages worth of data read on every syscall. > Or did I misunderstand your patch's behavior in that case? I think you did. If the IOCB_CACHED read hits a readahead page, it returns early. Then call_read_iter() notices the read is not yet complete, and calls ->read_iter() to finish the read. So it's two calls to generic_file_buffered_read() rather than one, but it's still one syscall.