From: David Howells <dhowells@redhat.com>
To: Jeff Layton <jlayton@kernel.org>, Steve French <smfrench@gmail.com>
Cc: David Howells <dhowells@redhat.com>,
Matthew Wilcox <willy@infradead.org>,
Marc Dionne <marc.dionne@auristor.com>,
Paulo Alcantara <pc@manguebit.com>,
Shyam Prasad N <sprasad@microsoft.com>,
Tom Talpey <tom@talpey.com>,
Dominique Martinet <asmadeus@codewreck.org>,
Eric Van Hensbergen <ericvh@kernel.org>,
Ilya Dryomov <idryomov@gmail.com>,
Christian Brauner <christian@brauner.io>,
linux-cachefs@redhat.com, linux-afs@lists.infradead.org,
linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org,
ceph-devel@vger.kernel.org, v9fs@lists.linux.dev,
linux-fsdevel@vger.kernel.org, linux-mm@kvack.org,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH v3 14/59] netfs: Provide tools to create a buffer in an xarray
Date: Thu, 7 Dec 2023 21:21:21 +0000 [thread overview]
Message-ID: <20231207212206.1379128-15-dhowells@redhat.com> (raw)
In-Reply-To: <20231207212206.1379128-1-dhowells@redhat.com>
Provide tools to create a buffer in an xarray, with a function to add new
folios with a mark. This will be used to create bounce buffer and can be
used more easily to create a list of folios the span of which would require
more than a page's worth of bio_vec structs.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
---
fs/netfs/internal.h | 13 +++++++
fs/netfs/misc.c | 81 +++++++++++++++++++++++++++++++++++++++++++
include/linux/netfs.h | 4 +++
3 files changed, 98 insertions(+)
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 2bd99989dd7f..1f25e8622b8a 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -56,6 +56,19 @@ static inline void netfs_proc_add_rreq(struct netfs_io_request *rreq) {}
static inline void netfs_proc_del_rreq(struct netfs_io_request *rreq) {}
#endif
+/*
+ * misc.c
+ */
+#define NETFS_FLAG_PUT_MARK BIT(0)
+#define NETFS_FLAG_PAGECACHE_MARK BIT(1)
+int netfs_xa_store_and_mark(struct xarray *xa, unsigned long index,
+ struct folio *folio, unsigned int flags,
+ gfp_t gfp_mask);
+int netfs_add_folios_to_buffer(struct xarray *buffer,
+ struct address_space *mapping,
+ pgoff_t index, pgoff_t to, gfp_t gfp_mask);
+void netfs_clear_buffer(struct xarray *buffer);
+
/*
* objects.c
*/
diff --git a/fs/netfs/misc.c b/fs/netfs/misc.c
index d946d85764de..96014c3d1683 100644
--- a/fs/netfs/misc.c
+++ b/fs/netfs/misc.c
@@ -8,6 +8,87 @@
#include <linux/swap.h>
#include "internal.h"
+/*
+ * Attach a folio to the buffer and maybe set marks on it to say that we need
+ * to put the folio later and twiddle the pagecache flags.
+ */
+int netfs_xa_store_and_mark(struct xarray *xa, unsigned long index,
+ struct folio *folio, unsigned int flags,
+ gfp_t gfp_mask)
+{
+ XA_STATE_ORDER(xas, xa, index, folio_order(folio));
+
+retry:
+ xas_lock(&xas);
+ for (;;) {
+ xas_store(&xas, folio);
+ if (!xas_error(&xas))
+ break;
+ xas_unlock(&xas);
+ if (!xas_nomem(&xas, gfp_mask))
+ return xas_error(&xas);
+ goto retry;
+ }
+
+ if (flags & NETFS_FLAG_PUT_MARK)
+ xas_set_mark(&xas, NETFS_BUF_PUT_MARK);
+ if (flags & NETFS_FLAG_PAGECACHE_MARK)
+ xas_set_mark(&xas, NETFS_BUF_PAGECACHE_MARK);
+ xas_unlock(&xas);
+ return xas_error(&xas);
+}
+
+/*
+ * Create the specified range of folios in the buffer attached to the read
+ * request. The folios are marked with NETFS_BUF_PUT_MARK so that we know that
+ * these need freeing later.
+ */
+int netfs_add_folios_to_buffer(struct xarray *buffer,
+ struct address_space *mapping,
+ pgoff_t index, pgoff_t to, gfp_t gfp_mask)
+{
+ struct folio *folio;
+ int ret;
+
+ if (to + 1 == index) /* Page range is inclusive */
+ return 0;
+
+ do {
+ /* TODO: Figure out what order folio can be allocated here */
+ folio = filemap_alloc_folio(readahead_gfp_mask(mapping), 0);
+ if (!folio)
+ return -ENOMEM;
+ folio->index = index;
+ ret = netfs_xa_store_and_mark(buffer, index, folio,
+ NETFS_FLAG_PUT_MARK, gfp_mask);
+ if (ret < 0) {
+ folio_put(folio);
+ return ret;
+ }
+
+ index += folio_nr_pages(folio);
+ } while (index <= to && index != 0);
+
+ return 0;
+}
+
+/*
+ * Clear an xarray buffer, putting a ref on the folios that have
+ * NETFS_BUF_PUT_MARK set.
+ */
+void netfs_clear_buffer(struct xarray *buffer)
+{
+ struct folio *folio;
+ XA_STATE(xas, buffer, 0);
+
+ rcu_read_lock();
+ xas_for_each_marked(&xas, folio, ULONG_MAX, NETFS_BUF_PUT_MARK) {
+ folio_put(folio);
+ }
+ rcu_read_unlock();
+ xa_destroy(buffer);
+}
+
/**
* netfs_dirty_folio - Mark folio dirty and pin a cache object for writeback
* @mapping: The mapping the folio belongs to.
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index bbb33ccbf719..c05150f51beb 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -109,6 +109,10 @@ static inline int wait_on_page_fscache_killable(struct page *page)
return folio_wait_private_2_killable(page_folio(page));
}
+/* Marks used on xarray-based buffers */
+#define NETFS_BUF_PUT_MARK XA_MARK_0 /* - Page needs putting */
+#define NETFS_BUF_PAGECACHE_MARK XA_MARK_1 /* - Page needs wb/dirty flag wrangling */
+
enum netfs_io_source {
NETFS_FILL_WITH_ZEROES,
NETFS_DOWNLOAD_FROM_SERVER,
next prev parent reply other threads:[~2023-12-07 21:23 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-07 21:21 [PATCH v3 00/59] netfs, afs, 9p, cifs: Delegate high-level I/O to netfslib David Howells
2023-12-07 21:21 ` [PATCH v3 01/59] netfs, fscache: Move fs/fscache/* into fs/netfs/ David Howells
2023-12-07 21:21 ` [PATCH v3 02/59] netfs, fscache: Combine fscache with netfs David Howells
2023-12-07 21:21 ` [PATCH v3 03/59] netfs, fscache: Remove ->begin_cache_operation David Howells
2023-12-07 21:21 ` [PATCH v3 04/59] netfs, fscache: Move /proc/fs/fscache to /proc/fs/netfs and put in a symlink David Howells
2023-12-13 15:19 ` Jeff Layton
2023-12-13 15:32 ` David Howells
2023-12-07 21:21 ` [PATCH v3 05/59] netfs: Move pinning-for-writeback from fscache to netfs David Howells
2023-12-07 21:21 ` [PATCH v3 06/59] netfs: Add a procfile to list in-progress requests David Howells
2023-12-07 21:21 ` [PATCH v3 07/59] netfs: Allow the netfs to make the io (sub)request alloc larger David Howells
2023-12-07 21:21 ` [PATCH v3 08/59] netfs: Add a ->free_subrequest() op David Howells
2023-12-07 21:21 ` [PATCH v3 09/59] afs: Don't use folio->private to record partial modification David Howells
2023-12-07 21:21 ` [PATCH v3 10/59] netfs: Provide invalidate_folio and release_folio calls David Howells
2023-12-07 21:21 ` [PATCH v3 11/59] netfs: Implement unbuffered/DIO vs buffered I/O locking David Howells
2023-12-07 21:21 ` [PATCH v3 12/59] netfs: Add iov_iters to (sub)requests to describe various buffers David Howells
2023-12-07 21:21 ` [PATCH v3 13/59] netfs: Add support for DIO buffering David Howells
2023-12-07 21:21 ` David Howells [this message]
2023-12-07 21:21 ` [PATCH v3 15/59] netfs: Add bounce buffering support David Howells
2023-12-07 21:21 ` [PATCH v3 16/59] netfs: Add func to calculate pagecount/size-limited span of an iterator David Howells
2023-12-07 21:21 ` [PATCH v3 17/59] netfs: Limit subrequest by size or number of segments David Howells
2023-12-07 21:21 ` [PATCH v3 18/59] netfs: Export netfs_put_subrequest() and some tracepoints David Howells
2023-12-07 21:21 ` [PATCH v3 19/59] netfs: Extend the netfs_io_*request structs to handle writes David Howells
2023-12-07 21:21 ` [PATCH v3 20/59] netfs: Add a hook to allow tell the netfs to update its i_size David Howells
2023-12-07 21:21 ` [PATCH v3 21/59] netfs: Make netfs_put_request() handle a NULL pointer David Howells
2023-12-07 21:21 ` [PATCH v3 22/59] netfs: Make the refcounting of netfs_begin_read() easier to use David Howells
2023-12-07 21:21 ` [PATCH v3 23/59] netfs: Prep to use folio->private for write grouping and streaming write David Howells
2023-12-07 21:34 ` Matthew Wilcox
2023-12-07 21:21 ` [PATCH v3 24/59] netfs: Dispatch write requests to process a writeback slice David Howells
2023-12-07 21:21 ` [PATCH v3 25/59] netfs: Provide func to copy data to pagecache for buffered write David Howells
2023-12-07 21:21 ` [PATCH v3 26/59] netfs: Make netfs_read_folio() handle streaming-write pages David Howells
2023-12-07 21:21 ` [PATCH v3 27/59] netfs: Allocate multipage folios in the writepath David Howells
2023-12-07 21:21 ` [PATCH v3 28/59] netfs: Implement support for unbuffered/DIO read David Howells
2023-12-07 21:21 ` [PATCH v3 29/59] netfs: Implement unbuffered/DIO write support David Howells
2023-12-07 21:21 ` [PATCH v3 30/59] netfs: Implement buffered write API David Howells
2023-12-07 21:21 ` [PATCH v3 31/59] netfs: Allow buffered shared-writeable mmap through netfs_page_mkwrite() David Howells
2023-12-07 21:21 ` [PATCH v3 32/59] netfs: Provide netfs_file_read_iter() David Howells
2023-12-07 21:21 ` [PATCH v3 33/59] netfs, cachefiles: Pass upper bound length to allow expansion David Howells
2023-12-07 21:21 ` [PATCH v3 34/59] netfs: Provide a writepages implementation David Howells
2023-12-07 21:21 ` [PATCH v3 35/59] netfs: Provide minimum blocksize parameter David Howells
2023-12-07 21:21 ` [PATCH v3 36/59] netfs: Make netfs_skip_folio_read() take account of blocksize David Howells
2023-12-07 21:21 ` [PATCH v3 37/59] netfs: Perform content encryption David Howells
2023-12-07 21:21 ` [PATCH v3 38/59] netfs: Decrypt encrypted content David Howells
2023-12-07 21:21 ` [PATCH v3 39/59] netfs: Support decryption on ubuffered/DIO read David Howells
2023-12-07 21:21 ` [PATCH v3 40/59] netfs: Support encryption on Unbuffered/DIO write David Howells
2023-12-07 21:21 ` [PATCH v3 41/59] netfs: Provide a launder_folio implementation David Howells
2023-12-07 21:21 ` [PATCH v3 42/59] netfs: Implement a write-through caching option David Howells
2023-12-07 21:21 ` [PATCH v3 43/59] netfs: Rearrange netfs_io_subrequest to put request pointer first David Howells
2023-12-07 21:21 ` [PATCH v3 44/59] netfs: Optimise away reads above the point at which there can be no data David Howells
2023-12-07 21:21 ` [PATCH v3 45/59] afs: Use the netfs write helpers David Howells
2023-12-07 21:21 ` [PATCH v3 46/59] 9p: Use netfslib read/write_iter David Howells
2023-12-07 21:21 ` [PATCH v3 47/59] cifs: Replace cifs_readdata with a wrapper around netfs_io_subrequest David Howells
2023-12-07 21:21 ` [PATCH v3 48/59] cifs: Share server EOF pos with netfslib David Howells
2023-12-07 21:21 ` [PATCH v3 49/59] cifs: Set zero_point in the copy_file_range() and remap_file_range() David Howells
2023-12-07 21:21 ` [PATCH v3 50/59] cifs: Replace cifs_writedata with a wrapper around netfs_io_subrequest David Howells
2023-12-07 21:21 ` [PATCH v3 51/59] cifs: Use more fields from netfs_io_subrequest David Howells
2023-12-07 21:21 ` [PATCH v3 52/59] cifs: Make wait_mtu_credits take size_t args David Howells
2023-12-07 21:22 ` [PATCH v3 53/59] cifs: Implement netfslib hooks David Howells
2023-12-07 21:22 ` [PATCH v3 54/59] cifs: Move cifs_loose_read_iter() and cifs_file_write_iter() to file.c David Howells
2023-12-07 21:22 ` [PATCH v3 55/59] cifs: Cut over to using netfslib David Howells
2023-12-07 21:22 ` [PATCH v3 56/59] cifs: Remove some code that's no longer used, part 1 David Howells
2023-12-07 21:22 ` [PATCH v3 57/59] cifs: Remove some code that's no longer used, part 2 David Howells
2023-12-07 21:22 ` [PATCH v3 58/59] cifs: Remove some code that's no longer used, part 3 David Howells
2023-12-07 21:22 ` [PATCH v3 59/59] netfs: Eliminate PG_fscache by setting folio->private and marking dirty David Howells
2023-12-07 21:38 ` Matthew Wilcox
2023-12-07 21:57 ` David Howells
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231207212206.1379128-15-dhowells@redhat.com \
--to=dhowells@redhat.com \
--cc=asmadeus@codewreck.org \
--cc=ceph-devel@vger.kernel.org \
--cc=christian@brauner.io \
--cc=ericvh@kernel.org \
--cc=idryomov@gmail.com \
--cc=jlayton@kernel.org \
--cc=linux-afs@lists.infradead.org \
--cc=linux-cachefs@redhat.com \
--cc=linux-cifs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nfs@vger.kernel.org \
--cc=marc.dionne@auristor.com \
--cc=netdev@vger.kernel.org \
--cc=pc@manguebit.com \
--cc=smfrench@gmail.com \
--cc=sprasad@microsoft.com \
--cc=tom@talpey.com \
--cc=v9fs@lists.linux.dev \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox