From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B54EC761AF for ; Wed, 5 Apr 2023 08:18:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F0F156B0071; Wed, 5 Apr 2023 04:18:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EBF086B0072; Wed, 5 Apr 2023 04:18:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D871E6B0074; Wed, 5 Apr 2023 04:18:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C83FB6B0071 for ; Wed, 5 Apr 2023 04:18:47 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A311D40304 for ; Wed, 5 Apr 2023 08:18:47 +0000 (UTC) X-FDA: 80646636294.12.03D13BB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf22.hostedemail.com (Postfix) with ESMTP id 63718C0003 for ; Wed, 5 Apr 2023 08:18:45 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UbvpM0wX; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680682725; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=L9CXlkWxa5SdW4THJZQP2BBn3FZZ8UI6SlqgD7wH/ew=; b=D941SVxMIlkbM4kXnEVksDxZSZ0ruWqQ7u9CAJpHE0BJgEXmjtz3lsT8R9agdxatccDXG5 IWwJ4YJdMNaZ/PMu4bpPzFz6dvLWbmuPVZtPFo76jOgkhBJksKMiW+IRf26bJdJURH/oB6 p1Bk+7QSLAD4F+7Y0uq8UBQzKWjGntE= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UbvpM0wX; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680682725; a=rsa-sha256; cv=none; b=Nvro7FP5LlVmHn4vihB71WGcPyaTPOShouQXm0bUhI8d/RroRPg7iUuFFCRDGzgjnXVMOL ltYL52BrOm9e+sd9mpmEVS7UyUJlw45qX1PK05hGAIMp5p3gOzQ0WSKSOYXDcRO6BqRDrz qg+/wlmm3xAjm9b3OZgiidQ7mvW/hKQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680682724; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L9CXlkWxa5SdW4THJZQP2BBn3FZZ8UI6SlqgD7wH/ew=; b=UbvpM0wXW1DzXZXzYdh0UX3KOe6v9N+v0DRxgpgeUKxKM1mYYOf+4BJLtUWuZX1OEBZx+f aOnDsIVsz3oKjbpi8kcHEHs2pM0Zq5Ug8aO0JP931fbAoEIyXdPrMEHB8nqXpbMXWUygYg q9aNLM5PPo1DyidgC8GOGdTzKAzWW+A= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-368-nUYnXXrKNLKOkXBWn5--hQ-1; Wed, 05 Apr 2023 04:18:41 -0400 X-MC-Unique: nUYnXXrKNLKOkXBWn5--hQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C2C1B3813F40; Wed, 5 Apr 2023 08:18:39 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2DC842027061; Wed, 5 Apr 2023 08:18:37 +0000 (UTC) Organization: Red Hat UK Ltd. Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 From: David Howells In-Reply-To: References: <20230331160914.1608208-1-dhowells@redhat.com> <20230331160914.1608208-39-dhowells@redhat.com> To: Bernard Metzler Cc: dhowells@redhat.com, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , "netdev@vger.kernel.org" , "linux-fsdevel@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , Tom Talpey , "linux-rdma@vger.kernel.org" Subject: Re: [PATCH v3 38/55] siw: Use sendmsg(MSG_SPLICE_PAGES) rather than sendpage to transmit MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <2904968.1680682716.1@warthog.procyon.org.uk> Content-Transfer-Encoding: quoted-printable Date: Wed, 05 Apr 2023 09:18:36 +0100 Message-ID: <2904969.1680682716@warthog.procyon.org.uk> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 63718C0003 X-Rspam-User: X-Stat-Signature: tcgps4nr4rz3ea41x6fjq36qeu7er77e X-HE-Tag: 1680682725-418591 X-HE-Meta: U2FsdGVkX1/VHm5QeFsz6NDCzB2W6CkPArKrZ2SwkUDumig3u1klGeMHhGYTAsV50s8VNp3Q08f2IoGU8xYVtcME/wBWcOt3uAk2XjFPEnGQZ+q3+fSziB26hNqEKylYuqFSdijaFpsWQ9Fn9QpC3PrvglO7907Coc+2v3x5bp4FjkksNJPt0gkk8q0Ubo16cjXQ9rAhg+1NMQVZGUo/2mqISB+d4SDpfW03lUGDI+b10zpEfGY0U4xTTvZQywK1Otb98hiCh0wuRBLpQ4D18UiSlGzwMkiqE1DpAAZbb8sVCHeYJMoaORHhGoRc7xU0SmcYZwwSd/tk6sGWkU2um1nQuxx7S5FsTJkmMmqnyq12ccpmMimn/wzhjPiQAOP3ALmec0xEQYE/uBZXuc8FFaps5v+YlCLI6Gjl2QQkyERi3QkNMyJya1mCVxbe8mdc8aeoMQC69DiwzmbStT7ZBmitMo/L5PfP5J0POOny3C4p/iCIXuSnuBJyjB/rVXTrIVr8iagCpEHPJSS69Rtfu+Mr/d7BMJwVbvHpdnZn7iGcMRkFlkHNuHpMwFUbC3mlzrqwcad8emmZ8I8emsiLRMPduKkNmhjPWbfPPhTEM3f+ZA3pyf4BpCww9BewxyuNNX8EXd+snmR9enCm4ieJ1E2rRfLWgt4hRG+74PKBbscAKLD78vmUcWAr+oOOjB+RtjBaBxxBbJ3DztoSnzrkFQ4oHu34PbkcZAFWkF1A8H1LGh1GlxteX8SXMq+BqTveShnkp2VekvH0IeKKIqH0GB8pJ+hDAzSBW+RvZr7xn1+aow2XyY8YsEzTJG6fpWj69X0OIrBmqG+Oy6QzvBYgfLCrgdHxloh2mMaEHJiH1hSs2wL8YcTQOZ3Dha3n/XiUgk4AatGK/DgUg28o/sIFa2gIE5A6zD2UAdl1aS05YzeZhxKvQEenWn+k4gTbnRakeQDr30bhaFs+HTJTwbd Aqzacvjf BJqRBytlu6wfN/xgrizkcLL/hJQYeQo8D5g+7rjb6e/Ek8a1MaPrhjqDV9IoZ/71hMEJBk39z4C87ao7DI+/xS0cQJko9iowPGKW/E/NPNV8zaR+9ccfzfCampy4AnwudxTzAs6BidmRXkVchokI9AZr5Tr4MpPNuYxKBDVIw8XpYDYLhhdN39x2Vz+7qbRUTWo908o9KlmK08RB/ha6FpKgixKCPyESifXQfqdQ3LNubqoUm7Uo/TLvgj3qbpl2vLHkEMq4iH0V3Lo/THmocjBQPAkPduapU4Nd5BlwriIDZS2hbv7Suncw+KBNr0saW0Cc2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Bernard Metzler wrote: > > if (c_tx->state =3D=3D SIW_SEND_HDR) { > > if (c_tx->use_sendpage) { > > @@ -457,10 +350,15 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, > > struct socket *s) > > = > = > Couldn't we now collapse the two header handling paths > into one, avoiding extra = > 'if (c_tx->use_sendpage) {} else {}' conditions? Okay, see the attached incremental change. Note that the calls to page_frag_memdup() I previously added are probably = not going to be necessary as copying unspliceable data is now done in the protocols (TCP, IP/UDP, UNIX, etc.). See patch 08 for the TCP version. David --- diff --git a/drivers/infiniband/sw/siw/siw_qp_tx.c b/drivers/infiniband/sw= /siw/siw_qp_tx.c index 28076832da20..edf66a97cf5f 100644 --- a/drivers/infiniband/sw/siw/siw_qp_tx.c +++ b/drivers/infiniband/sw/siw/siw_qp_tx.c @@ -335,7 +335,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struc= t socket *s) struct siw_sge *sge =3D &wqe->sqe.sge[c_tx->sge_idx]; struct bio_vec bvec[MAX_ARRAY]; struct msghdr msg =3D { .msg_flags =3D MSG_DONTWAIT | MSG_EOR }; - void *trl, *t; + void *trl; = int seg =3D 0, do_crc =3D c_tx->do_crc, is_kva =3D 0, rv; unsigned int data_len =3D c_tx->bytes_unsent, hdr_len =3D 0, trl_len =3D= 0, @@ -343,25 +343,11 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, str= uct socket *s) pbl_idx =3D c_tx->pbl_idx; = if (c_tx->state =3D=3D SIW_SEND_HDR) { - if (c_tx->use_sendpage) { - rv =3D siw_tx_ctrl(c_tx, s, MSG_DONTWAIT | MSG_MORE); - if (rv) - goto done; + void *hdr =3D &c_tx->pkt.ctrl + c_tx->ctrl_sent; = - c_tx->state =3D SIW_SEND_DATA; - } else { - const void *hdr =3D &c_tx->pkt.ctrl + c_tx->ctrl_sent; - void *h; - - rv =3D -ENOMEM; - hdr_len =3D c_tx->ctrl_len - c_tx->ctrl_sent; - h =3D page_frag_memdup(NULL, hdr, hdr_len, GFP_NOFS, - ULONG_MAX); - if (!h) - goto done; - bvec_set_virt(&bvec[0], h, hdr_len); - seg =3D 1; - } + hdr_len =3D c_tx->ctrl_len - c_tx->ctrl_sent; + bvec_set_virt(&bvec[0], hdr, hdr_len); + seg =3D 1; } = wqe->processed +=3D data_len; @@ -466,12 +452,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, stru= ct socket *s) trl =3D &c_tx->trailer.pad[c_tx->ctrl_sent]; trl_len =3D MAX_TRAILER - c_tx->ctrl_sent; } - - rv =3D -ENOMEM; - t =3D page_frag_memdup(NULL, trl, trl_len, GFP_NOFS, ULONG_MAX); - if (!t) - goto done_crc; - bvec_set_virt(&bvec[seg], t, trl_len); + bvec_set_virt(&bvec[seg], trl, trl_len); = data_len =3D c_tx->bytes_unsent; = @@ -480,7 +461,6 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struc= t socket *s) iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, bvec, seg + 1, hdr_len + data_len + trl_len); rv =3D sock_sendmsg(s, &msg); - if (rv < (int)hdr_len) { /* Not even complete hdr pushed or negative rv */ wqe->processed -=3D data_len; @@ -541,10 +521,6 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, stru= ct socket *s) } done_crc: c_tx->do_crc =3D 0; - if (c_tx->state =3D=3D SIW_SEND_HDR) - folio_put(page_folio(bvec[0].bv_page)); - folio_put(page_folio(bvec[seg].bv_page)); -done: return rv; } =