From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F5E3C46CD4 for ; Tue, 26 Dec 2023 16:54:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9F3026B006E; Tue, 26 Dec 2023 11:54:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 97C336B0071; Tue, 26 Dec 2023 11:54:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81BF16B0072; Tue, 26 Dec 2023 11:54:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 6B8656B006E for ; Tue, 26 Dec 2023 11:54:53 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 00C97A1237 for ; Tue, 26 Dec 2023 16:54:52 +0000 (UTC) X-FDA: 81609568824.26.76746D2 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf30.hostedemail.com (Postfix) with ESMTP id 9B3C48000A for ; Tue, 26 Dec 2023 16:54:50 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=hyoEySQd; spf=pass (imf30.hostedemail.com: domain of nathan@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=nathan@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703609691; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SzWLAITtwK73H2cwqmpqBYB6tMuu5L8mbxLj0upJJ8M=; b=Z9B9fgvD0ByRAmuDn0CioTkREJFQsJiaO4ohmwUU8aF922C7eaTIQvyCjSAPPr+VOdfV7g 7JEae4f7p4dS/wSfH0nyHN+bcDvmeD+W6z2TM9tdv1GxA+FPLXhGJO0i4zZ2mluC3Spjkk RmXZAHOGw0xzF0qnp5cBSwrKIxf3XC0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703609691; a=rsa-sha256; cv=none; b=DLPsDTNmzpGgMrXzF+BilV6IYnOx3Gh059FYl/i6y1/fGeAD2L9cVtSkokdYnFYBF17WuZ 8JQGr4TU+xvnTv5P/zxedZM6p6WoKmHsa69Nby5WYEVQ7FigerKGsbGC+qyJPNnnh3gzUQ F5NmEMmXI3U10uYFAlZruNRmQ99OaAM= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=hyoEySQd; spf=pass (imf30.hostedemail.com: domain of nathan@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=nathan@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 18ECFCE0F55; Tue, 26 Dec 2023 16:54:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 08B5FC433C8; Tue, 26 Dec 2023 16:54:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1703609685; bh=3qM2GHh77DRCqEQvJaFuJ3OPumR+rWFKPmfNaFK09E8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=hyoEySQdPRqvopBhVvKfA83jPoJIz0P6THQ1jeUtceYWcDZeRZW1jcFBAvdLWkqMe frKLy7VUSRSjJQfc+reCBX7hp9ANBukZNVXEwM4p/dU8O2geuf8PC57yDjHEPNsvlI E4vKIvBhMpQdA3341bpZkfJ7JHjoRmNhXyyO7fdFjPxHkb8rcFGSlGXjrsYbVxi0dZ SBFfT1xUJKTeDW+oKt+mmjGGSV3PNdjQDn2oRYEUWiwe+EsfhMQYlo4r+cxZ/bmj66 TZFcKGOwzCsSvPV0AwndM3MugXdYA2GSDrjrm+Zki38tWEUoO8HbiBJjk8RiqrDPOg 1uHSFsFaNMGEA== Date: Tue, 26 Dec 2023 09:54:42 -0700 From: Nathan Chancellor To: David Howells Cc: Jeff Layton , Steve French , Matthew Wilcox , Marc Dionne , Paulo Alcantara , Shyam Prasad N , Tom Talpey , Dominique Martinet , Eric Van Hensbergen , Ilya Dryomov , Christian Brauner , linux-cachefs@redhat.com, linux-afs@lists.infradead.org, linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org, ceph-devel@vger.kernel.org, v9fs@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev Subject: Re: [PATCH v5 15/40] netfs: Add support for DIO buffering Message-ID: <20231226165442.GA1202197@dev-arch.thelio-3990X> References: <20231221132400.1601991-1-dhowells@redhat.com> <20231221132400.1601991-16-dhowells@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231221132400.1601991-16-dhowells@redhat.com> X-Rspamd-Queue-Id: 9B3C48000A X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: ak17cahc3eo4wczej65z6wfnbyyj3ums X-HE-Tag: 1703609690-463630 X-HE-Meta: U2FsdGVkX184k4WDvZYhiPhx2KWFqRMkCe65ZjMEtMYmFlcLIZcQpHEN8DuuIaAwbqaoFnKcYr+0g6xdxPaDOw69tR+SM+4/RNCVP4Ocz/xK8saOP8g1QaUjzE5FsZCVuezercHMq/o9IlmZ6MUFXsU8y6ydcLE6Z7O9+LckOWGjZFsCiTiO8KCSRGMSKkZ8IEi9xdLAt/yMlEf028DVDdCq5D/vsxLDUzguYnt3NVk+qmZnBRBd5uZ94fjnwJp4BMHjYHyUjxwiY46vPRQbeFBsdrmYLPniYdZh2b6ijJ1dG/hvWbKFDsbjVEgx8EnY7zOpkPbqYXkWNQHxcu5CsPGgteORMEaSNdn8v2Rrntw/6hmm3nat3DgbJpb5RE7s7cOVua/tlqgbgHssnfxRgbJ6izh3scMCuufUtUAZbBPPaE8Rs8cgfJRobAnznu3SdixwxSgydxKNEO4KNShgH2TfqaZ0mY9y1TrlPlcvZ5S4ex0TjgEMxcBadWgklNKZ1ca0sKL/932JXr65auKzEVsyjhSVwYyBG+dtAfBMq+d3PEXPy2cedQfwazHSl9aX/13EDmev1ZSXy4qfhoxZhpkEVebz4x85Ldaf4t0Y4PNCOrubKhn9RjecaaNflO32P3HKRIPZDKqDVsE1CWOZiJ50PaTDPiiN2B9JUZgkiB2ViVXJa066qTQ3G1+fWjU0JOxk6JWOkXm942pnSPhYG6otIe42uD560481MHmYB3WzMNUFEojrJwiimGeTmViWdOZ3stkd9VakHy4MY8bIVXThdS6qq7kKtagUD63m/NhfEtK3n0HhHY+OURnvLFfgLycaRiZ2U3FtSPnfPNXKZp4uhW1qOuCBF//GV1/fNNVl5LInme7z7bPggXypGNwIrvOeirS+vBmeCZ9JrzUR18PF5r20ulP5CPzmzdk6tGnXHiB58JlsEHMirStV3ziJ73x+AhbPLnj/Z7OEXxI NpMva2gn vdo/x8dxsxvBzU8iYgur5t588oFh9MkoBGxjfeGaCj4UT8YQkuZQqSk1sIWllIVKhFf2XZxW2NKwKEcg5LgiEZ2/UG7JpGgAVKHCyd3TuHsXz8CqxmY/PDWzOhQKv7/pArp2ENouRt78EuYsN4zI8ujw4h86iMFan11HcUvIxV0NXQ7eNG5so7lXWm+77JB/dZnnjPRL982NArnyE5JAVSFQoe9iF9gNe6T3XhVgUOsRPGE0mLB5XsQkcTvRD/A0+gwsifyvYpSu7mAd2oJQRwnlabWjnLeF6uP5sEfAWsIAANO28//D+25Rii+WCcpD9WGzmrwqyU1UHbDudngAesvwllG4vI27HY9ux X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Dec 21, 2023 at 01:23:10PM +0000, David Howells wrote: > Add a bvec array pointer and an iterator to netfs_io_request for either > holding a copy of a DIO iterator or a list of all the bits of buffer > pointed to by a DIO iterator. > > There are two problems: Firstly, if an iovec-class iov_iter is passed to > ->read_iter() or ->write_iter(), this cannot be passed directly to > kernel_sendmsg() or kernel_recvmsg() as that may cause locking recursion if > a fault is generated, so we need to keep track of the pages involved > separately. > > Secondly, if the I/O is asynchronous, we must copy the iov_iter describing > the buffer before returning to the caller as it may be immediately > deallocated. > > Signed-off-by: David Howells > Reviewed-by: Jeff Layton > cc: linux-cachefs@redhat.com > cc: linux-fsdevel@vger.kernel.org > cc: linux-mm@kvack.org > --- > fs/netfs/objects.c | 10 ++++++++++ > include/linux/netfs.h | 4 ++++ > 2 files changed, 14 insertions(+) > > diff --git a/fs/netfs/objects.c b/fs/netfs/objects.c > index 1bd20bdad983..4df5e5eeada6 100644 > --- a/fs/netfs/objects.c > +++ b/fs/netfs/objects.c > @@ -76,6 +76,7 @@ static void netfs_free_request(struct work_struct *work) > { > struct netfs_io_request *rreq = > container_of(work, struct netfs_io_request, work); > + unsigned int i; > > trace_netfs_rreq(rreq, netfs_rreq_trace_free); > netfs_proc_del_rreq(rreq); > @@ -84,6 +85,15 @@ static void netfs_free_request(struct work_struct *work) > rreq->netfs_ops->free_request(rreq); > if (rreq->cache_resources.ops) > rreq->cache_resources.ops->end_operation(&rreq->cache_resources); > + if (rreq->direct_bv) { > + for (i = 0; i < rreq->direct_bv_count; i++) { > + if (rreq->direct_bv[i].bv_page) { > + if (rreq->direct_bv_unpin) > + unpin_user_page(rreq->direct_bv[i].bv_page); > + } > + } > + kvfree(rreq->direct_bv); > + } > kfree_rcu(rreq, rcu); > netfs_stat_d(&netfs_n_rh_rreq); > } > diff --git a/include/linux/netfs.h b/include/linux/netfs.h > index 3da962e977f5..bbb33ccbf719 100644 > --- a/include/linux/netfs.h > +++ b/include/linux/netfs.h > @@ -190,6 +190,9 @@ struct netfs_io_request { > struct iov_iter iter; /* Unencrypted-side iterator */ > struct iov_iter io_iter; /* I/O (Encrypted-side) iterator */ > void *netfs_priv; /* Private data for the netfs */ > + struct bio_vec *direct_bv /* DIO buffer list (when handling iovec-iter) */ > + __counted_by(direct_bv_count); This will break the build with versions of clang that have support for counted_by (as it has been reverted in main but reapplication to main is being actively worked on) because while annotating pointers with this attribute is a goal of the counted_by attribute, it is not ready yet. Please consider removing this and adding a TODO to annotate it when support is available. In file included from /builds/linux/fs/ceph/crypto.c:14: In file included from /builds/linux/fs/ceph/super.h:20: /builds/linux/include/linux/netfs.h:259:19: error: 'counted_by' only applies to C99 flexible array members 259 | struct bio_vec *direct_bv /* DIO buffer list (when handling iovec-iter) */ | ^~~~~~~~~ 1 error generated. Some additional context from another recent error like this: https://lore.kernel.org/CAKwvOdkvGTGiWzqEFq=kzqvxSYP5vUj3g9Z-=MZSQROzzSa_dg@mail.gmail.com/ Cheers, Nathan > + unsigned int direct_bv_count; /* Number of elements in direct_bv[] */ > unsigned int debug_id; > atomic_t nr_outstanding; /* Number of ops in progress */ > atomic_t nr_copy_ops; /* Number of copy-to-cache ops in progress */ > @@ -197,6 +200,7 @@ struct netfs_io_request { > size_t len; /* Length of the request */ > short error; /* 0 or error that occurred */ > enum netfs_io_origin origin; /* Origin of the request */ > + bool direct_bv_unpin; /* T if direct_bv[] must be unpinned */ > loff_t i_size; /* Size of the file */ > loff_t start; /* Start position */ > pgoff_t no_unlock_folio; /* Don't unlock this folio after read */ >