From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0418C83F09 for ; Tue, 8 Jul 2025 16:06:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DF2646B009E; Tue, 8 Jul 2025 12:06:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CDEAE6B009F; Tue, 8 Jul 2025 12:06:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA6DA6B00A0; Tue, 8 Jul 2025 12:06:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A4F856B009E for ; Tue, 8 Jul 2025 12:06:47 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 667AF160696 for ; Tue, 8 Jul 2025 16:06:47 +0000 (UTC) X-FDA: 83641575654.26.A711AD4 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf08.hostedemail.com (Postfix) with ESMTP id BBDBF16001F for ; Tue, 8 Jul 2025 16:06:45 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=rRHDSXZ4; spf=pass (imf08.hostedemail.com: domain of snitzer@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=snitzer@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1751990805; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=llMZ7ew8AO3tztLgTbZ8lhkckHbzdqLG9sRBGsKh3zw=; b=odl5hmoa7wAGfzgang8ZpScUeHrWEL5l9HfP3A9rii+Nv0UuTnFTFUI0AevGJ7fOdxE5cy 6hhkm2dBTNm5wz4z22GeQXjkZ45s7VbVKzy6lARHI9pG4VAevEzxI0t1/dpM24U9cqeyHE FVEmXh823ocvCBGEulWDMpFtXAM/jGw= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=rRHDSXZ4; spf=pass (imf08.hostedemail.com: domain of snitzer@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=snitzer@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1751990805; a=rsa-sha256; cv=none; b=I0Ay5SvoaKv0RCixA2HI55vjKtNSDIGnqoWuCV3sftQnZuYuTrKyQnYyUnQj9hOvxn4c6q guT7fvakEo9lVME3AgOjfTOAX3TBlQB5U5T2kvhfZX9IkQzI3hDDCiPejANH/EDquvof0j Qap/yM7qdcyRwwePJg9zW8bIgasn79k= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 35988A54097; Tue, 8 Jul 2025 16:06:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C3E22C4CEED; Tue, 8 Jul 2025 16:06:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1751990805; bh=qNBR2+GyUuJ6LJTu5QvQVPVfKv/JcsJ9Wf/JZpciUPY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rRHDSXZ4J0DKjRt2fAotsDd1n4RCB4X1CPFx3zGoFfvXkC/cT0k+l2yVIAzYA0emG 6SylHVrfJWAsJwt/PTe9WnMCTvuDsGp9EaEgpF2F3B2rYRSnOt92DRhha6csuCmpMF ehQOt0LAKOu+ej9HHQSRpsoAqFCUJbwzMzH/tF4eD1/V7HnausAXRARuwVlOIUeGub CjbwWaANVqHAxA/va9avig8Q9eySg/7DyJcDbLZG8qqeoJI9YOz/eu/RzDMMTor+2H hJk+lFkqc1mXHewUv/oaeP2S1vZrbJ+y8UvL4bfJYHk/qUFKrirCd/yVsVmMZLyXSh j+/klfRNSCQmQ== From: Mike Snitzer To: Chuck Lever , Jeff Layton , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker Cc: linux-nfs@vger.kernel.org, linus-fsdevel@vger.kernel.org, linux-mm@kvack.org, snitzer@kernel.org Subject: [RFC PATCH v2 6/8] NFSD: add io_cache_read controls to debugfs interface Date: Tue, 8 Jul 2025 12:06:17 -0400 Message-ID: <20250708160619.64800-7-snitzer@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20250708160619.64800-1-snitzer@kernel.org> References: <20250708160619.64800-1-snitzer@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: BBDBF16001F X-Stat-Signature: m64w4hxgfs4zce4stw9epbg18bh5up41 X-Rspam-User: X-HE-Tag: 1751990805-666824 X-HE-Meta: U2FsdGVkX19BuIVi1aM3iLWjc4lKrDNglc+WqYadcfhaCheWox+IXLwnctygKpXQOSMQaOIWxDLrcGDi4KPxc3VCj24KcalZlaaOyXAHv2yok3DFNCZ9buHwYAHjQCedgB/Gw+x7RQi5y1Y5VB2eqzmf9H0qbuiUMNCM8AlxhmirfRzZLEN1nLPXe+vnZtcWC0xXyoD5v4PoIYpY9LEKTsN6GybuXTORV4fIvGyMiPSoW1OywIjUdN9lkINZqkdvNWUwvTjDzoNAX3ZBdOom3jTWUGW4z44zZa2apKxKle8ycmrTWIl8Yp3KgfPpMRBHOpgDYtZwNOuIf4zYTNHGWF35+keq1iDqLMbJ5lgu1YSKo42X1QAB7Ze8PuDYWPT5zytATESjrbdfqOzdeWGaH7sXOXZa+NEPZwlHydcMspj58o6rXWvazdEdaYDkNoTzAom3tJLHJi8E+qE1FLU8iL0jPYD4CHfPuuZlJTUi0M0wuHa+jH5KD8hT6o2tBXMgUeGT08jGKtxfxYX1ckc4nqsWtZ2RtIjQyE3aexqrM+mO39gZQiQeL6qYujCS6JRuRr7N3cz2ht4NwfG4nEK/OKkvSexlksODphSUHGC5z8ZQ3rdOPl/8Y7yFSGfWr2IYkdMpBO5/tiQpjtCOT8Qy3qa30CTvHCCVgAf63RCXR1+2hswSE2nlGlqoNuimpM19kzyd441tlovZ+ChyADbTqDoXfJhRYpc4ouLLSa8XwHviMC9VQdoRQundyXAZcpg1tlAzb1nweDoidZca/RQKKKeVVRxsdvxFKxGokfU06LphElbDcyy16vihkcAb4lTsV84ucTRyKyjvYSWXj5BQ4YkexAXp7sIt2PIy15YrzMzcIdayGqm/Dc9r4WzElAb0f36hMp9wTLyhzVu8jWmJp69ZWIVdIY+UU7W57cU7u/kIIkUTk2zbJDdoPNrVnPbcl3K7NN9cdBVSqXPeWei yVB5/zE/ k1sxoFctHFLRio5KdTT56KWib/bHd7P489GOiQhmHz79fevCvOs1S8kSootcSi3NSdc2FJgbBlq8uSZcV4Hr4rA1sNBa8HGsKgb6noXXP/YDIOIX6Bjrbzes5ZeRiLvvERDdQSxm/aJZQtdwMbv9HNq8kVl7PFTJOLp7Qv6uDAMoc8A6aNQ5r0PxvisfY9tK1yREsweXkXuwnyjlCd/RC+mRY+YjPt++yeqTeFdZ/0Zwf1plhSA6l39IRha/1/dTmoVXCYbJAVg5a/cJApxd0yS6zTq3LhgBkql5yVxr0Cfcil5XbgwIJwCMx6pZgUnE96LPk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add 'io_cache_read' to NFSD's debugfs interface so that: Any data read by NFSD will either be: - cached using page cache (NFSD_IO_BUFFERED=0) - cached but removed from the page cache upon completion (NFSD_IO_DONTCACHE=1). - not cached (NFSD_IO_DIRECT=2) io_cache_read is 0 by default. It may be set by writing to: /sys/kernel/debug/nfsd/io_cache_read If NFSD_IO_DONTCACHE is specified using 1, FOP_DONTCACHE must be advertised as supported by the underlying filesystem (e.g. XFS), otherwise all IO flagged with RWF_DONTCACHE will fail with -EOPNOTSUPP. If NFSD_IO_DIRECT is specified using 2, the IO must be aligned relative to the underlying block device's logical_block_size. Also the memory buffer used to store the read must be aligned relative to the underlying block device's dma_alignment. Signed-off-by: Mike Snitzer --- fs/nfsd/debugfs.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++ fs/nfsd/nfsd.h | 8 +++++++ fs/nfsd/vfs.c | 15 ++++++++++++++ 3 files changed, 76 insertions(+) diff --git a/fs/nfsd/debugfs.c b/fs/nfsd/debugfs.c index 84b0c8b559dc..709646af797a 100644 --- a/fs/nfsd/debugfs.c +++ b/fs/nfsd/debugfs.c @@ -27,11 +27,61 @@ static int nfsd_dsr_get(void *data, u64 *val) static int nfsd_dsr_set(void *data, u64 val) { nfsd_disable_splice_read = (val > 0) ? true : false; + if (!nfsd_disable_splice_read) { + /* + * Cannot use NFSD_IO_DONTCACHE or NFSD_IO_DIRECT + * if splice_read is enabled. + */ + nfsd_io_cache_read = NFSD_IO_BUFFERED; + } return 0; } DEFINE_DEBUGFS_ATTRIBUTE(nfsd_dsr_fops, nfsd_dsr_get, nfsd_dsr_set, "%llu\n"); +/* + * /sys/kernel/debug/nfsd/io_cache_read + * + * Contents: + * %0: NFS READ will use buffered IO (default) + * %1: NFS READ will use dontcache (buffered IO w/ dropbehind) + * %2: NFS READ will use direct IO + * + * The default value of this setting is zero (buffered IO is + * used). This setting takes immediate effect for all NFS + * versions, all exports, and in all NFSD net namespaces. + */ + +static int nfsd_io_cache_read_get(void *data, u64 *val) +{ + *val = nfsd_io_cache_read; + return 0; +} + +static int nfsd_io_cache_read_set(void *data, u64 val) +{ + switch (val) { + case NFSD_IO_DONTCACHE: + case NFSD_IO_DIRECT: + /* + * Must disable splice_read when enabling + * NFSD_IO_DONTCACHE or NFSD_IO_DIRECT. + */ + nfsd_disable_splice_read = true; + nfsd_io_cache_read = val; + break; + case NFSD_IO_BUFFERED: + default: + nfsd_io_cache_read = NFSD_IO_BUFFERED; + break; + } + + return 0; +} + +DEFINE_DEBUGFS_ATTRIBUTE(nfsd_io_cache_read_fops, nfsd_io_cache_read_get, + nfsd_io_cache_read_set, "%llu\n"); + void nfsd_debugfs_exit(void) { debugfs_remove_recursive(nfsd_top_dir); @@ -44,4 +94,7 @@ void nfsd_debugfs_init(void) debugfs_create_file("disable-splice-read", S_IWUSR | S_IRUGO, nfsd_top_dir, NULL, &nfsd_dsr_fops); + + debugfs_create_file("io_cache_read", S_IWUSR | S_IRUGO, + nfsd_top_dir, NULL, &nfsd_io_cache_read_fops); } diff --git a/fs/nfsd/nfsd.h b/fs/nfsd/nfsd.h index 1cd0bed57bc2..4740567f4e7e 100644 --- a/fs/nfsd/nfsd.h +++ b/fs/nfsd/nfsd.h @@ -153,6 +153,14 @@ static inline void nfsd_debugfs_exit(void) {} extern bool nfsd_disable_splice_read __read_mostly; +enum { + NFSD_IO_BUFFERED = 0, + NFSD_IO_DONTCACHE, + NFSD_IO_DIRECT +}; + +extern u64 nfsd_io_cache_read __read_mostly; + extern int nfsd_max_blksize; static inline int nfsd_v4client(struct svc_rqst *rq) diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index 845c212ad10b..632ce417f4ef 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -49,6 +49,7 @@ #define NFSDDBG_FACILITY NFSDDBG_FILEOP bool nfsd_disable_splice_read __read_mostly; +u64 nfsd_io_cache_read __read_mostly; /** * nfserrno - Map Linux errnos to NFS errnos @@ -1107,6 +1108,20 @@ __be32 nfsd_iter_read(struct svc_rqst *rqstp, struct svc_fh *fhp, trace_nfsd_read_vector(rqstp, fhp, offset, *count); iov_iter_bvec(&iter, ITER_DEST, rqstp->rq_bvec, v, *count); + + switch (nsfd_io_cache_read) { + case NFSD_IO_DIRECT: + if (iov_iter_is_aligned(&iter, nf->nf_dio_mem_align - 1, + nf->nf_dio_read_offset_align - 1)) + kiocb.ki_flags = IOCB_DIRECT; + break; + case NFSD_IO_DONTCACHE: + kiocb.ki_flags = IOCB_DONTCACHE; + break; + case NFSD_IO_BUFFERED: + break; + } + host_err = vfs_iocb_iter_read(file, &kiocb, &iter); return nfsd_finish_read(rqstp, fhp, file, offset, count, eof, host_err); } -- 2.44.0