From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 041CEC3DA66 for ; Fri, 25 Aug 2023 15:11:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 958F72800B1; Fri, 25 Aug 2023 11:11:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 92F602800A2; Fri, 25 Aug 2023 11:11:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81DE02800B1; Fri, 25 Aug 2023 11:11:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 740C42800A2 for ; Fri, 25 Aug 2023 11:11:20 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4F27AA074E for ; Fri, 25 Aug 2023 15:11:20 +0000 (UTC) X-FDA: 81162965520.19.4745B7C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf15.hostedemail.com (Postfix) with ESMTP id 737E2A002E for ; Fri, 25 Aug 2023 15:11:17 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NYYVRerq; spf=pass (imf15.hostedemail.com: domain of djwong@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=djwong@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692976277; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RZbkUY9e8Wi7Z5FQEWEttp+LseNlXBERK1HYThFqoFg=; b=mrTvo8HLQQEjNOejTe2y0//dtg4dgo0iz0KSfTljVD5sh30+RYFoX6JhbZRcZhU9BSdcDB XelXZmaqxXljU8kIzCa7cJL4uLC4varM/t6i01YkxMf2cHsvsFyblz2h4tZw13ikalEmtE e76nZ082uTGuzwdxkJ1WmIG7LbbGaHo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692976277; a=rsa-sha256; cv=none; b=td5CxCCH/RtYEYIGSJLOXKcb5+qIEjbT4D6xZmTrsC1FxQlL+2SEWmkjLDousyTb1ICFU4 HbPv5leyirF5oMfF4sCKP1D3oE50sQPw9na/H2oMbZCciUqrjssfPJmCkOvktiHjsQDmnz vW+ao9FMia9OVlfeIcyRyyEH3dGv48w= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NYYVRerq; spf=pass (imf15.hostedemail.com: domain of djwong@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=djwong@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4DC0561D9A; Fri, 25 Aug 2023 15:11:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A51CFC433C7; Fri, 25 Aug 2023 15:11:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1692976275; bh=Yk82zBo/CO5JhPqCebob30ONG4lnlvLUkORIUTY0ODE=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=NYYVRerqYqnGpLF8blJdmxVpMOEPQLv0528BjC5vvBFsZNAz6HSSGB0+0FSTqmw/7 Db7P0mmTsJO1oNY1/ZHWu8NyzK9JrsOCasrcwziq7v3vDEDceQvRopW668kANjqE5f XtN676xYCoJvkIHN5kf251pR26YpJ9SQiBSWWbB3qvKX/kVrAeNfp0cfhulI06kGyS rDA+euytpnrfPPjqUYcEaqjX12Vj0EdMyf1Oo7A6QZMfrIIcEUaHvGZ8srUTvsTFVz en/rpIYHaZ7zIlQRk5TaAyXVnqxQP9imT2Z+zB9eRFWghRFbDct0Qujy/WQpJcsiaU Wx3ea7l8r3I0w== Date: Fri, 25 Aug 2023 08:11:15 -0700 From: "Darrick J. Wong" To: Hao Xu Cc: io-uring@vger.kernel.org, Jens Axboe , Dominique Martinet , Pavel Begunkov , Christian Brauner , Alexander Viro , Stefan Roesch , Clay Harris , Dave Chinner , linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-cachefs@redhat.com, ecryptfs@vger.kernel.org, linux-nfs@vger.kernel.org, linux-unionfs@vger.kernel.org, bpf@vger.kernel.org, netdev@vger.kernel.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-btrfs@vger.kernel.org, codalist@coda.cs.cmu.edu, linux-f2fs-devel@lists.sourceforge.net, cluster-devel@redhat.com, linux-mm@kvack.org, linux-nilfs@vger.kernel.org, devel@lists.orangefs.org, linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, linux-mtd@lists.infradead.org, Wanpeng Li Subject: Re: [PATCH RFC v5 00/29] io_uring getdents Message-ID: <20230825151115.GB17891@frogsfrogsfrogs> References: <20230825135431.1317785-1-hao.xu@linux.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230825135431.1317785-1-hao.xu@linux.dev> X-Stat-Signature: eh674aketgpreow3odnufsna9ke5mt5a X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 737E2A002E X-Rspam-User: X-HE-Tag: 1692976277-397616 X-HE-Meta: U2FsdGVkX1/THx5DNu8/npBGOzVGp+dR3w4aDVF83lsGDiFLUQh7A3tKxcsQBJ/dstJqzavvdioo64cxh3NbsiVqvohYa9zCX3RbOmYqvZWwRbsC54WOw7WJm5+x96v8qliSvvCNWcjk181CE0gF6S81GHcDJ46kSItvkfu3Jop6XJM23NaiQxdqDH+TnrEIdzGs8/NDZ3gyBsc8yrelgSB8TqmBE9YbburKs4Ybyj6D2lRpJIGaxhmE6R9dYyfzBU7fqyxWYSHWzYATU8/xItR3XZMZEDvPLtE5FTBiJFPoMRQxmJOSqp3W4Id2fM+i01v+UHFo0tEIGpirKQsQL71q/BAX0pw6Y1pufv11PnQcUXt8VoIfyl4roy7ffGlPBr+Uoa1na/V/+yw0ZwbOhwgwdefyEiUqIvG3l1GUzH6tqWXf8pA2vWccvQftjSclFbTLHQs2+s+YcPmlm4IxcvIIXR0XkmZ+uok3mEXf814Eiz2GgPS/czqkCQfzQtMjLvjNt1aVWRftBEcgLyYwP8QMn6Bg4eM7vs5UYGVfSx2n9HI6eS8+nwTzR47OIprqbQ5O/uv+6nAeXktpldzKrIHoFuD7e704XU5xge7F3xwdK2ukuZ6Rh+ID65qFEcfv77tOf6D8b0pVeX5AtzOBE6dowfg9tUSGl6ONIyUlUILsCaVFgpRb09djg2gjNG72nh8tS9kbqKDWWyRhLbKR+mKI3+bVTCB+tn/YjanBu4TDG7NDiikF8MGiKxk+xBc1Rd49YDOHGY7gHOMrn9S3xuuHKyl61bDHQZwhYBvh2hgeGWSBKdi/10VlPk+V1pkAufuDxQxhQ/ucjB3dGnCLGA/Wtam7jVicW9NPz316FFh4wALdt1l5reEXFy8iKSuwFG03rsf6rcW+5yKW+j4N+yYdoCpmwqB+PvWpJtIrKvJcBLBRXHPm9BIxqlwgSZcSOMXWfQkTX5PAt6dpYG+ Jjwqnc0e NKe/5oUVWhaXP9h9b33kc/i8V6RqBGoVObWXeROMqxlYHJCRA/5gKC8JMEjXq/Cw7452xZwDeU37ZdRVS1ftbKQJJs+6cK4QvbdUeFcoC1v5D1+6apJ9nPaS+//LyJcYuEc+oFcy3j2fhUK/xl/yWl1tNiJwWwIDIhfam86F6Bk/9l8LEV4Lqv6ADUNpaOUWzg29FwYYgKnNJ+EByqaR5XIdIE5pOcD8KO/xlX3ZbDQQox42uzTic0J1rP/vo8yxrVCQwXFr6VqAAFWNwdjNWiJ3jkbcLMsIjzrKjv/xYT/+kN7ruD7v722xQejaM28Ku4Rx1eIXTcWU7FcL/81LWQWTfi39tlLgo77VOpRJEsjEK8e81qY0996J//fJcwCFfMXugq4sExB24z2R6a/dg8f8ye9Rs1AUHKzVIrcYCE+nY4J5l9kvvE/+6D16IPb1fOjj/fP5kuTylSqtcXyaDQ8ANHd/ZNu5j918NXHzH0uPsAw73+WwnuTcKRyyRsPrPMK9RZwxjyFJEmkG76saqb15vMMZlngLtxiAonFodvvmk7i8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Aug 25, 2023 at 09:54:02PM +0800, Hao Xu wrote: > From: Hao Xu > > This series introduce getdents64 to io_uring, the code logic is similar > with the snychronized version's. It first try nowait issue, and offload > it to io-wq threads if the first try fails. NAK on the entire series until Jens actually writes down what NOWAIT does, so that we can check that the *existing* nowait code branches actually behave how he says it should. https://lore.kernel.org/all/e2d8e5f1-f794-38eb-cecf-ed30c571206b@kernel.dk/ --D > > Patch1 and Patch2 are some preparation > Patch3 supports nowait for xfs getdents code > Patch4-11 are vfs change, include adding helpers and trylock for locks > Patch12-29 supports nowait for involved xfs journal stuff > note, Patch24 and 27 are actually two questions, might be removed later. > an xfs test may come later. > > Tests I've done: > a liburing test case for functional test: > https://github.com/HowHsu/liburing/commit/39dc9a8e19c06a8cebf8c2301b85320eb45c061e?diff=unified > > xfstests: > test/generic: 1 fails and 171 not run > test/xfs: 72 fails and 156 not run > run the code before without this patchset, same result. > I'll try to make the environment more right to run more tests here. > > > Tested it with a liburing performance test: > https://github.com/HowHsu/liburing/blob/getdents/test/getdents2.c > > The test is controlled by the below script[2] which runs getdents2.t 100 > times and calulate the avg. > The result show that io_uring version is about 2.6% faster: > > note: > [1] the number of getdents call/request in io_uring and normal sync version > are made sure to be same beforehand. > > [2] run_getdents.py > > ```python3 > > import subprocess > > N = 100 > sum = 0.0 > args = ["/data/home/howeyxu/tmpdir", "sync"] > > for i in range(N): > output = subprocess.check_output(["./liburing/test/getdents2.t"] + args) > sum += float(output) > > average = sum / N > print("Average of sync:", average) > > sum = 0.0 > args = ["/data/home/howeyxu/tmpdir", "iouring"] > > for i in range(N): > output = subprocess.check_output(["./liburing/test/getdents2.t"] + args) > sum += float(output) > > average = sum / N > print("Average of iouring:", average) > > ``` > > v4->v5: > - move atime update to the beginning of getdents operation > - trylock for i_rwsem > - nowait semantics for involved xfs journal stuff > > v3->v4: > - add Dave's xfs nowait code and fix a deadlock problem, with some code > style tweak. > - disable fixed file to avoid a race problem for now > - add a test program. > > v2->v3: > - removed the kernfs patches > - add f_pos_lock logic > - remove the "reduce last EOF getdents try" optimization since > Dominique reports that doesn't make difference > - remove the rewind logic, I think the right way is to introduce lseek > to io_uring not to patch this logic to getdents. > - add Singed-off-by of Stefan Roesch for patch 1 since checkpatch > complained that Co-developed-by someone should be accompanied with > Signed-off-by same person, I can remove them if Stefan thinks that's > not proper. > > > Dominique Martinet (1): > fs: split off vfs_getdents function of getdents64 syscall > > Hao Xu (28): > xfs: rename XBF_TRYLOCK to XBF_NOWAIT > xfs: add NOWAIT semantics for readdir > vfs: add nowait flag for struct dir_context > vfs: add a vfs helper for io_uring file pos lock > vfs: add file_pos_unlock() for io_uring usage > vfs: add a nowait parameter for touch_atime() > vfs: add nowait parameter for file_accessed() > vfs: move file_accessed() to the beginning of iterate_dir() > vfs: add S_NOWAIT for nowait time update > vfs: trylock inode->i_rwsem in iterate_dir() to support nowait > xfs: enforce GFP_NOIO implicitly during nowait time update > xfs: make xfs_trans_alloc() support nowait semantics > xfs: support nowait for xfs_log_reserve() > xfs: don't wait for free space in xlog_grant_head_check() in nowait > case > xfs: add nowait parameter for xfs_inode_item_init() > xfs: make xfs_trans_ijoin() error out -EAGAIN > xfs: set XBF_NOWAIT for xfs_buf_read_map if necessary > xfs: support nowait memory allocation in _xfs_buf_alloc() > xfs: distinguish error type of memory allocation failure for nowait > case > xfs: return -EAGAIN when bulk memory allocation fails in nowait case > xfs: comment page allocation for nowait case in xfs_buf_find_insert() > xfs: don't print warn info for -EAGAIN error in xfs_buf_get_map() > xfs: support nowait for xfs_buf_read_map() > xfs: support nowait for xfs_buf_item_init() > xfs: return -EAGAIN when nowait meets sync in transaction commit > xfs: add a comment for xlog_kvmalloc() > xfs: support nowait semantics for xc_ctx_lock in xlog_cil_commit() > io_uring: add support for getdents > > arch/s390/hypfs/inode.c | 2 +- > block/fops.c | 2 +- > fs/btrfs/file.c | 2 +- > fs/btrfs/inode.c | 2 +- > fs/cachefiles/namei.c | 2 +- > fs/coda/dir.c | 4 +-- > fs/ecryptfs/file.c | 4 +-- > fs/ext2/file.c | 4 +-- > fs/ext4/file.c | 6 ++-- > fs/f2fs/file.c | 4 +-- > fs/file.c | 13 +++++++ > fs/fuse/dax.c | 2 +- > fs/fuse/file.c | 4 +-- > fs/gfs2/file.c | 2 +- > fs/hugetlbfs/inode.c | 2 +- > fs/inode.c | 10 +++--- > fs/internal.h | 8 +++++ > fs/namei.c | 4 +-- > fs/nfsd/vfs.c | 2 +- > fs/nilfs2/file.c | 2 +- > fs/orangefs/file.c | 2 +- > fs/orangefs/inode.c | 2 +- > fs/overlayfs/file.c | 2 +- > fs/overlayfs/inode.c | 2 +- > fs/pipe.c | 2 +- > fs/ramfs/file-nommu.c | 2 +- > fs/readdir.c | 61 +++++++++++++++++++++++++-------- > fs/smb/client/cifsfs.c | 2 +- > fs/splice.c | 2 +- > fs/stat.c | 2 +- > fs/ubifs/file.c | 2 +- > fs/udf/file.c | 2 +- > fs/xfs/libxfs/xfs_alloc.c | 2 +- > fs/xfs/libxfs/xfs_attr_remote.c | 2 +- > fs/xfs/libxfs/xfs_btree.c | 2 +- > fs/xfs/libxfs/xfs_da_btree.c | 16 +++++++++ > fs/xfs/libxfs/xfs_da_btree.h | 1 + > fs/xfs/libxfs/xfs_dir2_block.c | 7 ++-- > fs/xfs/libxfs/xfs_dir2_priv.h | 2 +- > fs/xfs/libxfs/xfs_shared.h | 2 ++ > fs/xfs/libxfs/xfs_trans_inode.c | 12 +++++-- > fs/xfs/scrub/dir.c | 2 +- > fs/xfs/scrub/readdir.c | 2 +- > fs/xfs/scrub/repair.c | 2 +- > fs/xfs/xfs_buf.c | 43 +++++++++++++++++------ > fs/xfs/xfs_buf.h | 4 +-- > fs/xfs/xfs_buf_item.c | 9 +++-- > fs/xfs/xfs_buf_item.h | 2 +- > fs/xfs/xfs_buf_item_recover.c | 2 +- > fs/xfs/xfs_dir2_readdir.c | 49 ++++++++++++++++++++------ > fs/xfs/xfs_dquot.c | 2 +- > fs/xfs/xfs_file.c | 6 ++-- > fs/xfs/xfs_inode.c | 27 +++++++++++++++ > fs/xfs/xfs_inode.h | 17 +++++---- > fs/xfs/xfs_inode_item.c | 12 ++++--- > fs/xfs/xfs_inode_item.h | 3 +- > fs/xfs/xfs_iops.c | 31 ++++++++++++++--- > fs/xfs/xfs_log.c | 33 ++++++++++++------ > fs/xfs/xfs_log.h | 5 +-- > fs/xfs/xfs_log_cil.c | 17 +++++++-- > fs/xfs/xfs_log_priv.h | 4 +-- > fs/xfs/xfs_trans.c | 44 ++++++++++++++++++++---- > fs/xfs/xfs_trans.h | 2 +- > fs/xfs/xfs_trans_buf.c | 18 ++++++++-- > fs/zonefs/file.c | 4 +-- > include/linux/file.h | 7 ++++ > include/linux/fs.h | 16 +++++++-- > include/uapi/linux/io_uring.h | 1 + > io_uring/fs.c | 53 ++++++++++++++++++++++++++++ > io_uring/fs.h | 3 ++ > io_uring/opdef.c | 8 +++++ > kernel/bpf/inode.c | 4 +-- > mm/filemap.c | 8 ++--- > mm/shmem.c | 6 ++-- > net/unix/af_unix.c | 4 +-- > 75 files changed, 499 insertions(+), 161 deletions(-) > > -- > 2.25.1 >