From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8832AC76196 for ; Thu, 6 Apr 2023 10:56:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E83026B0071; Thu, 6 Apr 2023 06:56:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E33DA6B0074; Thu, 6 Apr 2023 06:56:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFD776B0075; Thu, 6 Apr 2023 06:56:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BD11A6B0071 for ; Thu, 6 Apr 2023 06:56:42 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 90C851605EC for ; Thu, 6 Apr 2023 10:56:42 +0000 (UTC) X-FDA: 80650663044.29.593C7C8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf22.hostedemail.com (Postfix) with ESMTP id A2355C0011 for ; Thu, 6 Apr 2023 10:56:40 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=c4iysnDc; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680778600; a=rsa-sha256; cv=none; b=vHdcJJJgHRfrE773Y0jE4BGeP7z23bPg/jsSzcUmOvVj2a3ZtrA30KtXS7wzr/NnaDRuqg +erzZCRffiSRU06GYoasesGJuUvNNhyisCtxFGdJpnaE48rzaSxD85Oj9zeYT8c8l2SjDB GdciL7f9x4ykXx+15o50Qd3IUIeoepo= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=c4iysnDc; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680778600; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zMeQxZQg6MXfV4MGHzz3ouxaZBZa4sXe5adkrdIvgDA=; b=a+Nuz9UQo4epSIqJuPLfkdVH9d0UohcENVHmLcmY7mVuqJEqRg1tKJfspRZLdIKlJjbam5 u8ebOkesMwb5CPLRtSoBr2ZWHQvk3lxYwlth0GUV/JucHR9COYL50C1Tf4yBX4UZ46GvhK SnAopJF0ZNgKZHgxlJVhDlJX0aYU1Co= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680778599; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=zMeQxZQg6MXfV4MGHzz3ouxaZBZa4sXe5adkrdIvgDA=; b=c4iysnDcep2vpPgeXiqeuejsa9roreWiFqlqlCBxJAwT8OkTRW3CNsCtupyIA8m8npN/sA pGNcc60gNVR3vIPRlV53r+hvJgg8IKYFb8ImUGTjjOjfATJanrEOlfBf3RNcKikQTUBPDv pM+/465YM80Fy8cOxFLXqdaN6N85Ing= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-616-dI7Qid53N0-2fcggxMEYxg-1; Thu, 06 Apr 2023 06:56:38 -0400 X-MC-Unique: dI7Qid53N0-2fcggxMEYxg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 01EB9384708E; Thu, 6 Apr 2023 10:56:38 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0B936492C14; Thu, 6 Apr 2023 10:56:35 +0000 (UTC) Organization: Red Hat UK Ltd. Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 From: David Howells In-Reply-To: References: <20230406094245.3633290-1-dhowells@redhat.com> To: Eric Dumazet Cc: dhowells@redhat.com, netdev@vger.kernel.org, "David S. Miller" , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH net-next v5 00/19] splice, net: Replace sendpage with sendmsg(MSG_SPLICE_PAGES), part 1 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <3636417.1680778595.1@warthog.procyon.org.uk> Date: Thu, 06 Apr 2023 11:56:35 +0100 Message-ID: <3636418.1680778595@warthog.procyon.org.uk> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspam-User: X-Rspamd-Queue-Id: A2355C0011 X-Rspamd-Server: rspam01 X-Stat-Signature: 8gc9sre1fp7smj6891wzrqphidin54zw X-HE-Tag: 1680778600-134376 X-HE-Meta: U2FsdGVkX18PaCqcPG6g4LI6e1wD/zr4aXrPV8mCV8WlaKPhE3AsZab6wCP2I+ZNbbDf5kQug1BLa56BTqnQGMS1x+w26oJo87e5m/Z0NzJKPULE+j8VkHEzCN88v0soyji+FL1lEkK+2X3fYo1E5G92nKRJuEga55RaZOU3G4+65v/CuYcyAaNFFXDAKNfhk34wrWhJJgxFs4/B8Ys/vi1bqwuKGG9TbvEiFvdXLfJ6ox+bPIfOzgj6bsVxlHA5472GuhzNEKeF8Wgmc7xXEIv5FxRWSYP39/JkDgFAYlmRcJky/Bqh/3q36/vEa+KZGBxCH0ct3RSCqIrRlAvEpCpGCuJ864JdQtOGP74TpDBzbaNTDb6dIW6tHTk/CQUz9mtj5U1dlpDvhZn9GRxRTDWrwKhTOM/xHPtszE13G8UijSrBhry/BTajNqpNlXUJfI+n7a3kxpJGeHWlEi6ROvKRqIuuvDADGm8EVLr6ax6+lRaAyQzsljSlrhzUoWkie0DZRSUaVEgs7SoVMX7MnOGDyWgh1m8ulpDAdJp7P4Y3c5LyCE05Mln05h1v8cDdZgYVwJRPH6Rh5H1wVGwAXKHV42+oo4uK2Ys0KM9mrTGBKCxtJjPNtcZHbDvSLcAqjiSWslVVKutH2hXkMmNtLtmoxALPE2qvR3dVKNy0XiJXNnOkA9rRxQhYvseZdZMgOCOuis1dzBAAxzPHJELZ46Vc5aXvl7p68W9JsixUlPHwl1wI/WP/JwfTV6roTl8dzBO6RN7kFHW+faRplZ7JR8QjnEztDTNGoyWS9+aN2UbH75m3PuoCr81O2JRjauYjQ29EYjv0iqhQ9j2yOhzSycCbMyWnaLSYENEfQuHg6hQoSZMRb8K2Mjt3xNj/0Yt3cR9z2/wfTN07uipcDazWj5N3fe5nOmhvYRCaisNbHdwrhi3GGjpVsenfTMhcKAoVi19hIPucSzpvIDclfPb 4DI1qF6Q KPsr5mtlQ+8+AVtmYTQ3e1zbTi5+usOenXUmBFkMImhIC/e3oyZswyDPDKrNnbzHVt0QOStBtCmBSJonkCxYyx9NlywBSS6qiAtzExdxgg15702/CjetJXw8WRugHmixtIxMzN3DGGbAimufZroSq4GLuExoQuHm1ggTbP0Ob3nIdzOSmER7zEfGZvkT4aqU4JSpzimBuObFEJ/r0dEcBC5j6bj1a5hn4yF606pAMOvtRZ7lgvhZrL6pUL95h7cOh1Q1yO4fww2JpmHxfeJrCFsvZsoAHpbNnpAJKgpd+9B+0fE2LrfrCUFFjsl1GolsC04/I X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Eric Dumazet wrote: > > Here's the first tranche of patches towards providing a MSG_SPLICE_PAGES > > internal sendmsg flag that is intended to replace the ->sendpage() op with > > calls to sendmsg(). MSG_SPLICE is a hint that tells the protocol that it > > should splice the pages supplied if it can and copy them if not. > > > > I find this patch series quite big/risky for 6.4 If you want me to hold this till after the merge window, that's fine. > Can you spell out why we need "unspliceable pages support" ? > This seems to add quite a lot of code in fast paths. The patches to copy unspliceable pages (patches 6, 14 and 19) only really add to the MSG_SPLICE_PAGES path - I don't know whether you count this as a fast path or not. (Or are you objecting to MSG_SPLICE_PAGES and getting rid of sendpage in general?) What I'm trying to do with this aspect is twofold: Firstly, I'm trying to make it such that the layer above can send each message in a single sendmsg() if possible. This is possible with sunrpc and siw, for example, but currently they make a whole bunch of separate calls into the transport layer - typically at least three for header, body, trailer. Secondly, I'm trying to avoid a double copy. The layer above TCP/UDP/etc (sunrpc[*], siw, etc.) needs to glue protocol bits on either end of the message body and it may have this data in the slab or on the stack - which it would then need to copy into a page fragment so that it can be zero-copied. However, if the device can handle this or we don't have sufficient frags, the network layer may decide to copy it anyway - I'm not sure how the higher layer can determine this. It just seems there are fewer places this is required if it can be done in the network protocol. Note that userspace cannot make use of this since they're not allowed to set MSG_SPLICE_PAGES. However, I have kept these bits separate and discard them if it's considered a bad idea and that MSG_SPLICE_PAGES should, say, give an error in such a case. David [*] sunrpc, at least, seems to store the header and trailer in zerocopyable pages, but has an additional bit on the front that's not.