From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AA84EB64DB for ; Mon, 19 Jun 2023 08:25:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 20FAA8D0003; Mon, 19 Jun 2023 04:25:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1BE408D0001; Mon, 19 Jun 2023 04:25:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0ADD08D0003; Mon, 19 Jun 2023 04:25:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id EFBE08D0001 for ; Mon, 19 Jun 2023 04:25:42 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id BA33A4051B for ; Mon, 19 Jun 2023 08:25:42 +0000 (UTC) X-FDA: 80918813724.10.6504AAB Received: from mail-lf1-f54.google.com (mail-lf1-f54.google.com [209.85.167.54]) by imf11.hostedemail.com (Postfix) with ESMTP id C20234000E for ; Mon, 19 Jun 2023 08:25:39 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of sagigrim@gmail.com designates 209.85.167.54 as permitted sender) smtp.mailfrom=sagigrim@gmail.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687163139; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iX4CLwxuTuO4frE1ZkNfG2gJ9+rD4aADOYrpxzsHz/k=; b=syxn4/TgkwUl0MIzbDZr/D/XATqWc5xVmE37AV/o5fNbOxNBiayH3UkdDkvabuhpyEm7jN 3KZS5kMFAt0GIq2IU3OrY1DxWHSrLGyHrbMlWWxtH+DY6f+8sijh4YkyGYfdPHUYWMnzbc JrtPzHjdbmNoTqFPpgkAmu5Hl2XS0kw= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of sagigrim@gmail.com designates 209.85.167.54 as permitted sender) smtp.mailfrom=sagigrim@gmail.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687163139; a=rsa-sha256; cv=none; b=4WO09LgviDeDJYyIxHF6mCcyypw/Jk/zBQYJDrsR6OcoUPZiIs3avNU8rK+kPC9q5Mcrg4 iGL9WybPW6D6KMdTePKnHI1KKNGUtZ1am4T+7PtAPow9SAVpdRKsGbR98WD4KWxolJMqPS yO/gx/Mb9xnvK8V4cGu84xQvWWOEUwY= Received: by mail-lf1-f54.google.com with SMTP id 2adb3069b0e04-4f843f3f610so705121e87.0 for ; Mon, 19 Jun 2023 01:25:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687163138; x=1689755138; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=iX4CLwxuTuO4frE1ZkNfG2gJ9+rD4aADOYrpxzsHz/k=; b=ZzLesmd50Kmvu/EbPBv+KAAUjzrEldCAo+t/ohOhfOGz3GD+QZNdsMey1oCYKozlTL YDtvkHLMxiK2OxpL0Eq74Fyl1rcMzQm206j9sv0yuxh8pVDsgFVU+SxnMc+nVrYcvf8p HYzx+cd4n6rzalyxYPYnGSgxFWCIjNJDa030ux1xMHBio7z9X1KvhgFX46rUTGAiUz/B 2uUG9v5ig86wA5iH2puwJMznIkw9ArIAyYkkC8/Lu66u+OIqJFHIyQ+Oooqobk1qTz++ 07dlhsjjuOR/ZFXIwhD1YFgYaZ36ZvhXi9bkMer5+4MmQu+VL8LMvPIFCAt5g2AlIFG0 KYfQ== X-Gm-Message-State: AC+VfDzsPxaK+O11Ost00eWna6JX99PU4lyieFJg/wg+Lqkf/sVxJUFV hv5DYsUR8myO7Nn6HDVlAAU= X-Google-Smtp-Source: ACHHUZ44wI+K691se6TLRFSf74cGJkGpEfK4OPsyBLOS0ZdfsAOZc8G8mvYNQR0Th0SgRlbHmYSxDw== X-Received: by 2002:a05:6512:25a:b0:4f7:6a87:f16f with SMTP id b26-20020a056512025a00b004f76a87f16fmr3558941lfo.4.1687163137715; Mon, 19 Jun 2023 01:25:37 -0700 (PDT) Received: from [192.168.64.192] (bzq-219-42-90.isdn.bezeqint.net. [62.219.42.90]) by smtp.gmail.com with ESMTPSA id f13-20020a7bcd0d000000b003f7ba52eeccsm9853963wmj.7.2023.06.19.01.25.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 19 Jun 2023 01:25:37 -0700 (PDT) Message-ID: <55e7058b-07d0-3619-3481-2d70e95875ea@grimberg.me> Date: Mon, 19 Jun 2023 11:25:34 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [PATCH net-next v2 10/17] nvme: Use sendmsg(MSG_SPLICE_PAGES) rather then sendpage Content-Language: en-US To: David Howells , Willem de Bruijn Cc: netdev@vger.kernel.org, Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Keith Busch , Jens Axboe , Christoph Hellwig , Chaitanya Kulkarni , linux-nvme@lists.infradead.org References: <648f353c55ce8_33cfbc29413@willemb.c.googlers.com.notmuch> <20230617121146.716077-1-dhowells@redhat.com> <20230617121146.716077-11-dhowells@redhat.com> <755077.1687109321@warthog.procyon.org.uk> From: Sagi Grimberg In-Reply-To: <755077.1687109321@warthog.procyon.org.uk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: C20234000E X-Rspam-User: X-Stat-Signature: wq61cqbscr9z37shhk5oum1rh98yqkwm X-Rspamd-Server: rspam01 X-HE-Tag: 1687163139-633166 X-HE-Meta: U2FsdGVkX1+VFKa1QcwpoEMjIf/6m2fI9I21V9Barp8IQxG/+WPlLvJo0nAP2ySONo3G4PNlKabgqTP/JU3JINT844VkzUHmev1FS906xf/vvzNygxCi8Y/s+P+Pw/J2Fdf0ztrn4zLTVbtqQ+hip8rKoR1OTlpOD4+e62mZtOQEoKSA1pdYVPKDO9LRwNZvKE0wafho7H3PZRc1SfRp7OGS+tu9jCq82s2MHjGR8S+ywP9W4F0qdRPh7nJvmKr3G6h8SjGhfJthmt6y8HYOQD67u3D5GUBIL8Va4dsV8D8rh8c6VmFX/PPInijP1WQRxFW17sltnQNAUGc7qIt7bvDshjhs2Xrgbej+X17Vj11jysEw2Pj044kfDQOt7HpmVUnk9uvReDd8znynS2kPCNp9KglHqLlqKoXk83hOqZUppO0a/b2BhMUn+fc6qvLnLr4IGjVpdup/Batpr7Vxfgiyg3GiRHWHgL0e3ygi4Y7oCirMVgyxjlDmCgAJkBj/Mupifjtxto/iiUMAChXSEZdcZjMzEVK85TDsc9pnrZdIwUpyir1R0hIudqUuk4Ce2SMxJEllTEpfy+0tStDiuBfRC+am2F4iAlWm347IrCulLgfFnCRWCP2Gn21cfhVJtIVxI+m/DLoeECtMyUDmGZlegzyOR9EwyKE8GFpNqTH2rG0rVOS01BTj94vxB9ZaH8kXAm9N6zYwubZ7sOq0dWQQONBgRYQiT1PFHtQfVCzUfTKtJeUwrIPUnyYSzTfajfBWFzntGA9WOn8XLx6AKn4ScMW+AfxteqiOUSCu+gNfJkJM9lqu78VaVu3oQ18FemP1lsDEbIw+mBtzxWlZZrmY9Awt82KuyvF6x53ew+TRZlqsf91tnEOcZg1I1GxUs5oURgrJ1qTkh7KSmNog9pwRLB1x6VAy0xBiF9exkZ1GkTkRQ6nSon11HiKSajypD212SpBeUoWDIMLoXtt /K8d1nkq vnoBJNv48N7SnyiUZ67JihpwO3khRR2NTA4VMTT0tnwgNnSZvjPC4RiY+XBtNz/8nC7M+JHsWTS8gAkOIFGQiXPv+x+lLSXcfNabB6Jec1IIt44K2N0KF0lE9dgYStSpSvdruY4S5P/8RQgHQRVCSeXX+VzvTPbc8Bh7lU0uQe2pZ7Wr5hnOMGNTEt6qLm4HSnhx5ppgj36N4vg0x+KoR8USb5Ax4chsItO+XQC24uzmrzZQm7G9cq2Y4e1YOtAph9g0zutobh758rcfZ+GCDEUTWvowLZCjacA+O7jT4opN97nLMGSMInwMxUPAeNvKXx9Ft4VqGRXbnGwYvSqolvFYFzAjn7g7yLudJKbrPcGXefYkKUoVLp6mmIlycJV7zRjHNQ0Ks62ra6t2LmDPxuHv2QOMiQ0+CsyJc7l/DSuj3esYYR9vKv9Ysn6KIk2r6WcmqXByHUNARPpqSx0btPWrwY4Rpcn5kmrC5F/IACDncxjTJ0LMSxa9qPKc/l98uig7Xks3FcZG+kLZ6ZfH3o6Sr+UdxYcBbz0RsGaJe53IyFNYAsl6RF/LKHCVO2fo7K7cDvPgIinnRjwwFoi1spIng1DgOhZlMOxEZ78vxy3fxTFJT/1H0TLMEhx9dtvtAc6t6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: >> struct bio_vec bvec; >> struct msghdr msg = { .msg_flags = MSG_SPLICE_PAGES | ... }; >> >> .. >> >> bvec_set_virt >> iov_iter_bvec >> sock_sendmsg >> >> is a frequent pattern. Does it make sense to define a wrapper? Same for bvec_set_page. > > I dunno. I'm trying to move towards aggregating multiple pages in a bvec > before calling sendmsg if possible rather than doing it one page at a time, > but it's easier and more obvious in some places than others. That would be great to do, but nvme needs to calculate a data digest and doing that in a separate scan of the payload is not very cache friendly... There is also the fact that the payload may be sent in portions asynchronously driven by how the controller wants to accept them, so there is some complexity there. But worth looking at for sure. The patch looks good to me, taking it to run some tests (from sendpage-3-frag branch in your kernel.org tree correct?) For now, you can add: Reviewed-by: Sagi Grimberg