From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D4DDC27C77 for ; Wed, 12 Jun 2024 14:19:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 89C746B0093; Wed, 12 Jun 2024 10:19:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 84AA06B009B; Wed, 12 Jun 2024 10:19:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7137D6B009D; Wed, 12 Jun 2024 10:19:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 539146B009B for ; Wed, 12 Jun 2024 10:19:23 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C30DD14017D for ; Wed, 12 Jun 2024 14:19:22 +0000 (UTC) X-FDA: 82222444164.26.8A4E2E4 Received: from out-176.mta0.migadu.com (out-176.mta0.migadu.com [91.218.175.176]) by imf23.hostedemail.com (Postfix) with ESMTP id A49FC14000E for ; Wed, 12 Jun 2024 14:19:18 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=JJzYQ8VM; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf23.hostedemail.com: domain of kent.overstreet@linux.dev designates 91.218.175.176 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718201960; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1qyUcDw4VUGrxahyrz6XNaVMu1zSkDs4ZdYN1kGuIwo=; b=mnHZjmc0JfTTXipxZxWBGTuewR5Sajqokl8DRPExr+NgaZ70+yIaz22NQjB/br7mzfz/3o Kae9HsIAu7JRNS7MAmBcg3VLbd7rcb0yvq1Y0nYOVrJ67EYV9LgBBRkoiPYuXr7XiS5kPT 0Z3Ee4o7q08NP4JgEMWQBgkqYuG4aVY= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=JJzYQ8VM; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf23.hostedemail.com: domain of kent.overstreet@linux.dev designates 91.218.175.176 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718201960; a=rsa-sha256; cv=none; b=cVqn5EJ3OTo3gTjnvO45gYub/SX75rN1ILIZe8+O5S8vLCjQROrvlk8I5k5yoY5hmYO56e iseTZChaPrgqLAvInq3KS75UBc4vx9Vv9mrPx3IcW231Eg4ZjuwqN01586v6InUhKRcb5o /is3HJxUqkLuC9Fyt1cKddNII4br95M= X-Envelope-To: bernd.schubert@fastmail.fm DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718201954; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=1qyUcDw4VUGrxahyrz6XNaVMu1zSkDs4ZdYN1kGuIwo=; b=JJzYQ8VM1uLxaZ8wS525J/R7Wmkdn1+cRTDkMVo79K6uogeKOKwB023Pr3vijy4zwQBOJZ gd4a5hwhf5JxwWVmhMRHHFSPicshx6yd/7x0e1gUGn3Lyx7uxc1ktOVccS9O3OLtjQ/LWW abF7CwJ+l66FtnWCMWWlfej2vRAvduo= X-Envelope-To: miklos@szeredi.hu X-Envelope-To: bschubert@ddn.com X-Envelope-To: amir73il@gmail.com X-Envelope-To: linux-fsdevel@vger.kernel.org X-Envelope-To: akpm@linux-foundation.org X-Envelope-To: linux-mm@kvack.org X-Envelope-To: mingo@redhat.com X-Envelope-To: peterz@infradead.org X-Envelope-To: avagin@google.com X-Envelope-To: io-uring@vger.kernel.org Date: Wed, 12 Jun 2024 10:19:07 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Kent Overstreet To: Bernd Schubert Cc: Miklos Szeredi , Bernd Schubert , Amir Goldstein , linux-fsdevel@vger.kernel.org, Andrew Morton , linux-mm@kvack.org, Ingo Molnar , Peter Zijlstra , Andrei Vagin , io-uring@vger.kernel.org Subject: Re: [PATCH RFC v2 00/19] fuse: fuse-over-io-uring Message-ID: References: <20240529-fuse-uring-for-6-9-rfc2-out-v1-0-d149476b1d65@ddn.com> <99d13ae4-8250-4308-b86d-14abd1de2867@fastmail.fm> <62ecc4cf-97c8-43e6-84a1-72feddf07d29@fastmail.fm> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: A49FC14000E X-Stat-Signature: xxgrdepwyjmfrynmywz9zyzmr8k4uihb X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1718201958-871552 X-HE-Meta: U2FsdGVkX19Zx7Pp/c3upJi/7EiwqG1ReBE1MYMGlaedLmQAoBxI0JaxbFi0E9ZbGY41SwS+/mMNtRl0VSzihqav0XBnbxKYp4C6oO1rFLd34wCo59rdQgHWgERZTFcxibja2oaLZfxpQ5O2GmTphCwWc0o2rO6OihvfFM7I0A9G0hfq78n1yc4CtEEqilA9nlVRVhRzVB1V8hc7/Qfi2ZLkZJ1hLR9LAGuKsePZRFW7cWR5J4jHU9RQdrvi1eIWMc2cHdZjMVxBWJVi7OF6aJADRg+dvQ1Pgl8NJCukax1+5dOyHpW/aU045RfOX+1aqAz3BBNNnMij+S+cg3DFMAJ/bChNBj2x43D/69w5UpOftp1Bkd3pxRu9kcz+mKnGAdM7F62uuM9yaxwGrr9dG87Th+e7gxe+cdeZ7dg/jMMawpakS+FNlQDeXs+wRgZNlExgamdOZ34hWjX+TlPY9d+3GGfRNmK54xAatdHSWVK4UKESb/Znfje2UZ6e4ao1xL1Eoidmo8VfLpbxFfusN3LAudO+ZNyTyzM2pcAqdj2zGZvUGbZYGNXDc5zGUnmwlYN9FUcDEJSi1m9R0N4Yp/8fkRsmOhSfwx9MUyOOSD0NBia5Jq2m0zFoq5SnS5NmLVaUpAwf4gVfdwqbiF1pP/4rgcqSDEboTIRTdL95aSn9tlvkXSD8cSYbcdLmDn7Z6gAs/SsZJ65O2Bv3HGuORvn9gpZtNf3Y7UmQEFReRcjxnyKj2DkH5iaaT/2ejfaVxC0Z7fcSwbWV3bm87cbXxJUf8heOUTcoodEPEpDeLINOvlSDbMa8qydZNMNp0L5PEgc5T8h78l+59bNPLQ82boPRj9+dT+IGt+ivrr5OgI/AVbmaRXIfFTD6U8ATIaA+X55g6bHvsHbzbEnGmek2pviHWbkRWFoRJLUC/M1QV2Bsmi5j/BAarZV67rL1EQvXIHjsN98xCBKyLMIs7iz 2A4I31Im HTGShDVwcH08Hc//V0ESUJcmT74QErA0rnQPVfrF+gHb66ax4YqOKuu7L7k0+3t6dpbx5xJjXbR99q2pbLeuZGZkc7Rnso00/knGMz+2KF4vUzh0PYtvBHuj39YSKq0brG1kFA14J9a0bXHomT1xHAzA+hkXuZtdhTCwRwLGrzpNC40A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jun 12, 2024 at 03:53:42PM GMT, Bernd Schubert wrote: > I will definitely look at it this week. Although I don't like the idea > to have a new kthread. We already have an application thread and have > the fuse server thread, why do we need another one? Ok, I hadn't found the fuse server thread - that should be fine. > > > > The next thing I was going to look at is how you guys are using splice, > > we want to get away from that too. > > Well, Ming Lei is working on that for ublk_drv and I guess that new approach > could be adapted as well onto the current way of io-uring. > It _probably_ wouldn't work with IORING_OP_READV/IORING_OP_WRITEV. > > https://lore.gnuweeb.org/io-uring/20240511001214.173711-6-ming.lei@redhat.com/T/ > > > > > Brian was also saying the fuse virtio_fs code may be worth > > investigating, maybe that could be adapted? > > I need to check, but really, the majority of the new additions > is just to set up things, shutdown and to have sanity checks. > Request sending/completing to/from the ring is not that much new lines. What I'm wondering is how read/write requests are handled. Are the data payloads going in the same ringbuffer as the commands? That could work, if the ringbuffer is appropriately sized, but alignment is a an issue. We just looked up the device DMA requirements and with modern NVME only 4 byte alignment is required, but the block layer likely isn't set up to handle that. So - prearranged buffer? Or are you using splice to get pages that userspace has read into into the kernel pagecache?