From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6301FCDB465 for ; Thu, 19 Oct 2023 15:53:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C59DF8009E; Thu, 19 Oct 2023 11:53:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE3118000B; Thu, 19 Oct 2023 11:53:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A83FE8009E; Thu, 19 Oct 2023 11:53:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9557A8000B for ; Thu, 19 Oct 2023 11:53:58 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 66F70120558 for ; Thu, 19 Oct 2023 15:53:58 +0000 (UTC) X-FDA: 81362656956.25.EC53C87 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf11.hostedemail.com (Postfix) with ESMTP id B86EC40028 for ; Thu, 19 Oct 2023 15:53:55 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Bn7c8NCn; dmarc=none; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697730835; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=F6/o8Z2i9JKu0badnkbxfHnQMRLlqY5LsHFDEapT+28=; b=SnkOS8KuAezc2EIDXKYMGD+5xbBaxbDubQyvjGkcshtrSZ1TRy1RFq+Hw8uBROu4OhsXaD +rIPCwlUs7jR3uq+l8Ikh+zWXLdGn9OZYfb2N/NarxxcRotSedD6n38If48GigICQqgMev PaJBhqB2pqo6BLtlKrl331A9FGMIg2I= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Bn7c8NCn; dmarc=none; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697730835; a=rsa-sha256; cv=none; b=JzLcV1nZ3O3FRKr158i+bQ9NBYrZrB/PDZrTOc4Cg0SYRJ7H9M9omFuafuHf4GhGhNh5mz wF5kISGEd5YQaQ+amndkS5nVV1cxLkOw3lKS0y/d1idFpF89p3jxtpewqKaTifaCBrmc1f zCClH3C8OdRaUllqvP3EVdS5O2n2FtY= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=F6/o8Z2i9JKu0badnkbxfHnQMRLlqY5LsHFDEapT+28=; b=Bn7c8NCnrwcQo+noH2OfISTeA8 aqUXazCmB5GvGk0CcmE32SJCRThvwDB1QTSQF5MMjRHD5Yf8g+hTFjkKmFUOuEhkpZuJDOxw2WVps EMxWY4mdYr6Y4asGY7Kea7ytQwabrEXmq6YE/pHQD/lk8GcWRUvNYJHqrOzuAf2gDm2mClSSHBVRJ ydO7nqQpSJcwhjey3LYzd8Y8rxzAyKGrRphz5mpD0X2egoJukybHXdMwnv2PJmyJuLFWdFdrQPeNF YPfM5h0SYw0QnmWab3l03QI4Uupp+Xc9vBWeWCYpMiIo7gHGbJ6hBOozZy5+NkKvqSL6uvXkIJhTZ 7ltrLYTw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qtVLD-007u7P-Tf; Thu, 19 Oct 2023 15:53:43 +0000 Date: Thu, 19 Oct 2023 16:53:43 +0100 From: Matthew Wilcox To: Chuck Lever Cc: Marek Szyprowski , Chuck Lever , Robin Murphy , Alexander Potapenko , linux-mm@kvack.org, linux-rdma@vger.kernel.org, Jens Axboe , kasan-dev@googlegroups.com, David Howells , iommu@lists.linux.dev, Christoph Hellwig Subject: Re: [PATCH RFC 0/9] Exploring biovec support in (R)DMA API Message-ID: References: <169772852492.5232.17148564580779995849.stgit@klimt.1015granger.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <169772852492.5232.17148564580779995849.stgit@klimt.1015granger.net> X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: B86EC40028 X-Stat-Signature: tqxxnns5gfxee4aaqqktggwnacghtjte X-Rspam-User: X-HE-Tag: 1697730835-568496 X-HE-Meta: U2FsdGVkX1/g3ckyg7S29lxCRls1Pf+16iOaLqIjF1/vk/cp3zqC+5x8Q5pbdkxY2gL7MtI+Ze2FV3UMnPMFi0pQmaaVvfpqyNi2K/R1sCUaMR0cQSRUu4vQxdjvecum2nDcYSzGAKi00Y2IDGIblnGanIs0MFUiLSEqBhjFz9JdLgOy5z9SsEmgEaM50ZSNON2qz2TpyCL1v24oDtwJEgNVxTzNkT//vDDkmsEhWq98dd5a7TwExh4VFLg7cCuzYBWvSySR7CCMf9ema0W25KETepRt5Hwn+vyyBcxASN/fgh186ZsuMxECXcN9UprUlEIkFnuKNcGe7bwHYTbPz2oSpdgwNqDIPzdRZFx+/t9lUTJjzNsP0S9tpqB08nfjAEIOty/Qti1eK5km21zerCEIwVhnUl3bn4mdSnrir8WJkyZHWzMmi6dfudPkV1jQHFILL6t5a4H60TuRaYI7EOdSmeAUjzsBmDqNLRCI10ueGapUVbNxNJ/dsqXj95Tv9lwTbB8rBmF9pMPIZp/v5+V5B8CEPRjkPFpSPhv4d8vUUaqc7EkD0TyjLGfPy5/ivyhFNPSBJX3pCwTiW0uze8l9YijX/rh/eGX7gZ5kIj55/9fNR2sGLuj9lLdWylIU67MrFXQnDpJMMTQflWY3l+chJQl491H1ys9DWs6312kvPQ8ze4tpA5lRfPvi/RhY2gi5DpAZykt3OQ7Mth4D2FQ76Os6dYyYHt3shtYVwMPEISg7nn0Q6mQnwiRjBAsvVf4y1vF50yzVVhXoGcE0CrT4FSROjQ7/Ysr/4N+zolAbcT6cs7PbZLuHzoex/sFom+9pAsyRc9JokbWEkOgROwXQzYynN9d9kwN0e1IFsrRs4EVG/4DcoNb4OgFz1F5sHkoqmfe0lo3VLkHlYbCHImIw63t/5QftWoQtmmKHL/x1E+K9vmQw/KK7ptDH+0YLB2tN1kij2zMV3Ckt1CM 4x0A1f8B QoVh/bafB3Z1vLYlsw/j+PfT+wi98IQT+OD6vG15OXbtbl8rtwgfQz9DQME0tYPd29kzoPndgP15sVOaXnSfDWDv4yi6tUFSHK1hQeQq5LM3aCqyR3/sd02hxnkKTvWZLS3AEiaGDV+PI8i8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Oct 19, 2023 at 11:25:31AM -0400, Chuck Lever wrote: > The SunRPC stack manages pages (and eventually, folios) via an > array of struct biovec items within struct xdr_buf. We have not > fully committed to replacing the struct page array in xdr_buf > because, although the socket API supports biovec arrays, the RDMA > stack uses struct scatterlist rather than struct biovec. > > This (incomplete) series explores what it might look like if the > RDMA core API could support struct biovec array arguments. The > series compiles on x86, but I haven't tested it further. I'm posting > early in hopes of starting further discussion. Good call, because I think patch 2/9 is a complete non-starter. The fundamental problem with scatterlist is that it is both input and output for the mapping operation. You're replicating this mistake in a different data structure. My vision for the future is that we have phyr as our input structure. That looks something like: struct phyr { phys_addr_t start; size_t len; }; On 32-bit, that's 8 or 12 bytes; on 64-bit it's 16 bytes. This is better than biovec because biovec is sometimes larger than that, and it allows specifying IO to memory that does not have a struct page. Our output structure can continue being called the scatterlist, but it needs to go on a diet and look more like: struct scatterlist { dma_addr_t dma_address; size_t dma_length; }; Getting to this point is going to be a huge amount of work, and I need to finish folios first. Or somebody else can work on it ;-)