From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D4BCC369D5 for ; Mon, 28 Apr 2025 17:30:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 158E36B0007; Mon, 28 Apr 2025 13:30:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 108356B000A; Mon, 28 Apr 2025 13:30:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EEAF96B000C; Mon, 28 Apr 2025 13:30:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CE0116B0007 for ; Mon, 28 Apr 2025 13:30:40 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 1839BC0F07 for ; Mon, 28 Apr 2025 17:30:42 +0000 (UTC) X-FDA: 83384142324.08.6B383CB Received: from sea.source.kernel.org (sea.source.kernel.org [172.234.252.31]) by imf04.hostedemail.com (Postfix) with ESMTP id 546B740004 for ; Mon, 28 Apr 2025 17:30:40 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=EFNCyztK; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf04.hostedemail.com: domain of kbusch@kernel.org designates 172.234.252.31 as permitted sender) smtp.mailfrom=kbusch@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745861440; a=rsa-sha256; cv=none; b=VZybS+kurmsjnSrHPCCity1sv4SVvXeLEIX0Kg62+Rd0QMo8r+M+CafJ3Re9l6O6H1rsCo 5laBBb8rHvseEBVmHVCgmmqSAUmnP3KPqnmlK3lt61kgKsq2BOxBt2DYAviPiJCO6PsX2n Opgh6v99E0iTFDzOqHZjpih9df20g3s= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=EFNCyztK; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf04.hostedemail.com: domain of kbusch@kernel.org designates 172.234.252.31 as permitted sender) smtp.mailfrom=kbusch@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745861440; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Pwh48c4CvWu4Mh6c0UxMsCYmgmmJH/KAsrSENlzdaiY=; b=x6STFxraB3WoPLvVPNtadOvxzjrmmWiwc+QaCEeU2O7fNMTCd9XV5MZ9a4adN6Cmr1Uiet hnufLUiUcRdEVG1l9lxXdH8LpxcMCXOdTjvmZieLcKY/NegdksGBIfNgneNc9VQXGAMyTb J8A1OpmFVotE1rG/kL3IwIgrZiMxcsw= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sea.source.kernel.org (Postfix) with ESMTP id D646D49AFD; Mon, 28 Apr 2025 17:30:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 78575C4CEE4; Mon, 28 Apr 2025 17:30:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745861438; bh=ab2ae8e/65ivTF8ecHo7+deewxqwGOGmR1WVeufmqHc=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=EFNCyztKW7+/Evx7uEJGPtK2SefAgYmqtED4ANsvGmvDwBuQlhn5S/nOmqu3WBR1z +VFQyVwyP+0O2cMp8wyAXepn0PBELCi0ksp0PKLJBKlMwsEdig7AFEj4wICYu3fAJQ /zZeikIV4GSdae/Y2/DwH4bks++gRwmwp9dpfRTMH4Z1ZCpPgQcQmi4pOO08SmuMpb PTiv7J0eh/bS37eMFbiD7C8K8hWZiVIArkSUuuY+DzvZ8SoAg2vgkigVBG4ZUgJkdE 7Vrm2LUXRrJhMi9MPNfGblL+7IObYzGv0HZAEJnZAxGsJP7ubWBtZWh1DwAjICEoUP 0fglvd90xB7zw== Date: Mon, 28 Apr 2025 11:30:34 -0600 From: Keith Busch To: Leon Romanovsky Cc: Marek Szyprowski , Jens Axboe , Christoph Hellwig , Jake Edge , Jonathan Corbet , Jason Gunthorpe , Zhu Yanjun , Robin Murphy , Joerg Roedel , Will Deacon , Sagi Grimberg , Bjorn Helgaas , Logan Gunthorpe , Yishai Hadas , Shameer Kolothum , Kevin Tian , Alex Williamson , =?iso-8859-1?B?Suly9G1l?= Glisse , Andrew Morton , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux.dev, linux-nvme@lists.infradead.org, linux-pci@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, Niklas Schnelle , Chuck Lever , Luis Chamberlain , Matthew Wilcox , Dan Williams , Kanchan Joshi , Chaitanya Kulkarni , Nitesh Shetty Subject: Re: [PATCH v10 23/24] nvme-pci: convert to blk_rq_dma_map Message-ID: References: <007e00134d49160d5edab94a72c35b7b91429b09.1745831017.git.leon@kernel.org> <20250428172225.GG5848@unreal> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250428172225.GG5848@unreal> X-Rspamd-Queue-Id: 546B740004 X-Stat-Signature: 9yiokcpo9e4hup9juyrpmoza3i5xirqb X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1745861440-297220 X-HE-Meta: U2FsdGVkX19n1Elgmh+b73Rvq8JFt/5kJtDKtOda1HiYCF6xtyMnpwwa2qRO5HmVUMuvYPALkUVqvKQfmHLKni0SrHmYJ8gPvy9a17+qwLeHrPQSVNtMGA9MCwpAGzJwsd7NLEzV/6bNKlue9tRh+tgJHg6wWbjP5KrpfTE2iJwiAKjL0S80jHB32zxV5LLqNISAPUFet1oezpz483P2ZRM3FlIHdBVh3IDrmrJr3HdfYa6LGBSh6kcZMcII2O60y80qCbDTseHbTgJIbxDPIfSVHWBB6eEPRi+FHpr6Xhd23lFDxkAt8kWvgJEQgWXZb4JYP9tEUni9ohnkAG9f0kaYUB+2QL2ce+9vjfnIj2aQYW3wiXw7AkojDAaQMVkaJi5KeXAHZMZf1fuyRMoFHdff5+vJdoz9BylsasU2Y6la/SH+nuyA4PBtMwQ86+s2+nm7p/LSlfZXyzMqvr9Gn8/mRgknnaeiflHs4rqHXuxjbSBfxLgvIN5Ij79BUfvZyWEBnXv1kEjb9DKhOtyvgGLsp32fu+f17ir26IACs53n4z7EINt4aQz/LnrhetgmTLXmnTS5buNkQ1xilr6HggjuhG10zOLrysLxkClOwqiYsSAD5rB5ntwKcSUS/CpCZTH8v2yq/MzlNcXo1gG7IskTAW/4eXhyvQWRDmgZsarz+zs+9cMJGBANBUfVqQGSKnXbhozLUynlw3ySfg+yf4YGy1y44xzeFZ7CpkQ4AHq6lVuACuprqj5W2FnoTmCLqRyh1fdBA4dUEwZ5hty2jFz4kUM/tsuxPnBMVYGSSQCVxhO8jeBjeD/PeEp+KiItQWzZ1ZMw+5eDykW5WMEcMg0Ib08ZbKFwsjIMpiw2a2juNXUIMUGWgctrUyQs6UZUmJrTcuNDZ0iGdRmHza4glcOUUwsFit8DphfbcikYAbE4uM32vhgqE/pkEYnW3iox9gdeOutAnqPf5A/roae DlV0ExIO PVlqi5pL5l3iGvZckufKh7UdNhuC+kzEfnK6ny62HcYs1n6kMR82iA1xFiy0Sml+Ir/tgK+VmDixSL30fsIIh0HaaAdxQKjKKGQgMr9Dv/0EfxadY4bmIpmE5KcTPTg7U7xaqLQtAeRJCA6DChtIHCbPPfdkQN9BCqA38YEuoMaH2r7e/Ykffd1MicUpH96ROeRvhP74BxRrJX+KMK8b38dyAYHFB12ybEfYUVB937B9WD3wvcpPdoH4P4/QWTPVXRGG/ZiT56NjYAA0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Apr 28, 2025 at 08:22:25PM +0300, Leon Romanovsky wrote: > On Mon, Apr 28, 2025 at 10:46:19AM -0600, Keith Busch wrote: > > On Mon, Apr 28, 2025 at 12:22:29PM +0300, Leon Romanovsky wrote: > > > + do { > > > + if (WARN_ON_ONCE(mapped == entries)) { > > > + iter.status = BLK_STS_IOERR; > > > + break; > > > + } > > > + nvme_pci_sgl_set_data(&sg_list[mapped++], &iter); > > > > I think this should say "++mapped" so that the data blocks start at > > index 1 (continued below...) > > > > > + iod->total_len += iter.len; > > > + } while (blk_rq_dma_map_iter_next(req, dev->dev, &iod->dma_meta_state, > > > + &iter)); > > > > > > -out_unmap_sg: > > > - dma_unmap_sgtable(dev->dev, &iod->meta_sgt, rq_dma_dir(req), 0); > > > -out_free_sg: > > > - mempool_free(iod->meta_sgt.sgl, dev->iod_meta_mempool); > > > - return BLK_STS_RESOURCE; > > > + nvme_pci_sgl_set_seg(sg_list, sgl_dma, mapped); > > > > because this here is setting sg_list index 0 to be the segment > > descriptor. > > > > And you also need to increment sgl_dma to point to the element after > > sg_list, otherwise it's pointing right back to itself, creating a looped > > list. > > Thanks for pointing to the difference between data_map and metadata_map, Yeah, They're different because the SQE has 16 bytes available for the data map, so the command can fit an SGL descriptor directly in its submission entry. The metadata field only has 8 bytes in the SQE, so we have to set it to the address of an external SGL segment descriptor.