From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6869C28B30 for ; Thu, 20 Mar 2025 16:13:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A6127280004; Thu, 20 Mar 2025 12:13:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A10FF280003; Thu, 20 Mar 2025 12:13:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B0D4280004; Thu, 20 Mar 2025 12:13:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6C309280003 for ; Thu, 20 Mar 2025 12:13:45 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 9ED46121743 for ; Thu, 20 Mar 2025 16:13:45 +0000 (UTC) X-FDA: 83242425210.23.DB2AD44 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) by imf25.hostedemail.com (Postfix) with ESMTP id 8E58AA0017 for ; Thu, 20 Mar 2025 16:13:42 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=samsung.com header.s=mail20170921 header.b=mKTXmvAW; dmarc=pass (policy=none) header.from=samsung.com; spf=pass (imf25.hostedemail.com: domain of joshi.k@samsung.com designates 203.254.224.34 as permitted sender) smtp.mailfrom=joshi.k@samsung.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742487223; a=rsa-sha256; cv=none; b=rIBovyuz+SaXE/h/pfeHLqBOoX8kZ4wyw3KQqDkcKwNPpCEkpT9hVYQLl+A4UCciQfhaQv R+f3geH0PcL4ZQr7BePCIjMKPZVDgcCBcp36mkmgzY4dE42OT1/MvKmYQNQPK2PBYot+Wt vnnWSqRRAVRr0emyn3pFpIChOoQS7QM= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=samsung.com header.s=mail20170921 header.b=mKTXmvAW; dmarc=pass (policy=none) header.from=samsung.com; spf=pass (imf25.hostedemail.com: domain of joshi.k@samsung.com designates 203.254.224.34 as permitted sender) smtp.mailfrom=joshi.k@samsung.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742487223; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1QhXPUc+UhsbA3QNDxXoX2eKXdOw0cgYCugrDd2js8Y=; b=kcTFkMtFZgLSDMM+Mz5OoHrxwQjWeuOy4xJVAo6a9PXYNQnkKIXzBo5qUBUgWRRA19vH8z W9GNtdmfd24lv32s/ckSvGRgtHd5Y5y5jWPUHBkmDoKj0B9C1fEAE7I2KqF98h+i/Z3NkX Of6r/1JpRdJh/n4sZpyjimSR2/GX314= Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20250320161339epoutp0434dd2b6c7a7faddbef6c5c0f61895c02~ujuTortgF0543705437epoutp04S for ; Thu, 20 Mar 2025 16:13:39 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20250320161339epoutp0434dd2b6c7a7faddbef6c5c0f61895c02~ujuTortgF0543705437epoutp04S DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1742487219; bh=1QhXPUc+UhsbA3QNDxXoX2eKXdOw0cgYCugrDd2js8Y=; h=Date:Subject:To:Cc:From:In-Reply-To:References:From; b=mKTXmvAW2l6BMRPv6hVoOS7GlBcBv3c58LsOpkCGvuVHLLusUxnpRooDLf1EYqvSD HuLEanTr/hEErOJnpJT4t3PEGueerPXNxe7OiR8gOruUZrSg1/6cWy6rBS4rNwCVp4 Iv9g5yRFW/xz0AroyEnSvL2/X6bCi0/DcEqLES30= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20250320161338epcas5p3300497d14102a14142b36f46515e39a6~ujuSqt-RI0817008170epcas5p3l; Thu, 20 Mar 2025 16:13:38 +0000 (GMT) Received: from epsmgec5p1new.samsung.com (unknown [182.195.38.174]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4ZJVxX4mxsz4x9Pq; Thu, 20 Mar 2025 16:13:36 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmgec5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 34.3B.19710.0BE3CD76; Fri, 21 Mar 2025 01:13:36 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20250320161335epcas5p2709afc3ff29ffec350164003b511f92d~ujuQEy8Ch0265102651epcas5p2M; Thu, 20 Mar 2025 16:13:35 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20250320161335epsmtrp241cc853355b1a879f392060c68c9cbd8~ujuQDsIZq3146031460epsmtrp2o; Thu, 20 Mar 2025 16:13:35 +0000 (GMT) X-AuditID: b6c32a44-36bdd70000004cfe-8e-67dc3eb0b364 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 99.43.18729.FAE3CD76; Fri, 21 Mar 2025 01:13:35 +0900 (KST) Received: from [107.122.11.51] (unknown [107.122.11.51]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20250320161332epsmtip13339e115f8abf0a704f8b543d976a2d5~ujuNIqF5u1448214482epsmtip1d; Thu, 20 Mar 2025 16:13:32 +0000 (GMT) Message-ID: <796adeb4-2b7e-4d01-866b-1f47b6eb10e5@samsung.com> Date: Thu, 20 Mar 2025 21:43:31 +0530 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [LSF/MM/BPF TOPIC] breaking the 512 KiB IO boundary on x86_64 To: Keith Busch , Bart Van Assche Cc: Christoph Hellwig , Luis Chamberlain , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, lsf-pc@lists.linux-foundation.org, david@fromorbit.com, leon@kernel.org, sagi@grimberg.me, axboe@kernel.dk, joro@8bytes.org, brauner@kernel.org, hare@suse.de, willy@infradead.org, djwong@kernel.org, john.g.garry@oracle.com, ritesh.list@gmail.com, p.raghav@samsung.com, gost.dev@samsung.com, da.gomez@samsung.com Content-Language: en-US From: Kanchan Joshi In-Reply-To: Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA01TWVBTVxjm5N4sMAWvEeXIQ4mZWgYsmGCgBwuhM2XayzI2M07t8tA0E24D Q0hiFitahgyboIWwaWkUtSClQKsYGAQhZVFphYqUFIS0QZDwUBYR6JQCbWk2W96+8/3f969z WBjbwAxkpSm0lFohkXMZPnjbnZCQsGbhrzLextmXUJPNwEDzd1YAuvBsHUNLljYaau2fBMhi 90MTPR001FVTTkMNTfdoaHijHaCii81MVN43BlDlRh2GzNYDqMt8H0eT32zR0XfzZgyNl84C tFqXx0S9A4VMdH1+CUebf15ivL6HtPdeppGWn5PIgRpIdhhtTPLh5E2cbPk6lLQ80JGmxiIG aVopZ5IV4/WA7JzQM8jlWStOlrQ2ArJl8DS5anpRtOOD9JhUSpJCqTmUQqpMSVPIYrlJR8Vv iCOjePwwfjR6lctRSDKoWG58sijszTS5Y3gu54RErnNQIolGwz0ojFErdVqKk6rUaGO5lCpF rhKowjWSDI1OIQtXUNrDfB4vItIh/Cg9daN6mK4q9T7ZvKXWg2vMs8CbBQkB3DTZHdiHxSY6 AXxa00VzP1YANCzn0JwqNvEHgGsFp547nlU+YLh5M4CN1TK3YRHAHz6bAM6ALyGEU0UGlxkn 9sPf6to8/E54/ws77sS7iSD42FrlamMXkQhHlrpdSf2JBDhx/nOGMylG2DDYPmZ0BTAiAFrt VxxJWSwGEQKHK3RO2puIg+sr/R5JELy1eAlzeiFxyxvaegY9c8bDheLbdDfeBee+b/XwgXD1 qZnhxulw6skU7safwvaWEo8+Dur/Gqc762KOujduH3TX8oPFm3ZXO5DwhYUFbLd6H5wsn/U4 A+B01TUPJuHgeAPu3lUZDdZZG/BSwDFuW4tx25TGbeMY/698FeCNYC+l0mTIKGmkiq+gPvnv 3FJlhgm4fkZofDsYv/JPeB+gsUAfgCyM6+/rX/iLjO2bIsk8RamVYrVOTmn6QKTjPmVY4G6p 0vG1FFoxXxDNE0RFRQmiD0XxuQG+uR15MjYhk2ipdIpSUernPhrLO1BPe/u99+dDE4ePliRn q/Yc95qeDlurb5qdGUX7Z6bezQ0ePSSViRNLbEf8KmQDaWcsq1vFufvMP5q+xLUvJH21UHLM z8JMWV8z/TRU2FMVcr2GOl2SMK+tjRtQj2gr946Kk7O2kvINd4948YRNj8/bOBF3O+nczpz8 LP9HAoZ16BXGWztu5L7sN/f3OcPv5Qn9tctPgmNeuynN9lm5ly/uvjBTsMotG8iJGDrcLQpo 0l82aKz5+vCR8AUChB4vzTv54VxQjf0YZ5SdJZ66esZWtHFAeO5EptejwrHa1I+rlY3dskyR fKctuHYxuLR2hJ5fUfmw91uzaFnX8M5Fq7BeIsrm4ppUCT8UU2sk/wKQrvlbogQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA01Sa0hTYRjuO+d4drZaHpfpl9ptUHihlSXxhWUXiI5alFlEQpeRp1XqlM1p WcGsEGeXqU3LaSss081VtnmlTFslalTSbDVzdmFRZqUtArWL5Vbgv4fnfW4/XgoXfCYCqP3S dFYmFScLSR5Rf084a8GNqF7Jorq6MFTtUJNo4J4LoOKhERwNWusxVNvWB5DVORXZW5swdLu8 EEP66gcY6hptBEhVWsNBhRYbQJrRChw194Sh280dBOozjnmhOwPNOHqR/w6gbxUnOOhuZy4H XR8YJNCP4TJylR/jvKvDGGt3LNNZDpkmrYPDPOm7STDmqlDG+kjBmAwqkjG5CjnM2ReVgLll V5LM13c9BHOm1gAY88PDzDfTrE3eCbzliWzy/gxWtjBqN2/f6IUur7R87sGaMZkSXOHkAS4F 6Qg4pHlE5gEeJaBvAXhsqBd4Dv7wuG34n2ga1P9+z/GIBgB8XnXGLeLTUfC1So2NY4KeB/sr 6v/xPrCjxEmM4+n0bPiq57w7aBodA58OtpDj2JeOhvaic+5mnHbg0G5wEJ6GAgwatGq3G/87 o8d58W8DRZF0COw6qxinufRKOOJqIz2SpTCvLg948GzY8LkMzwcC7YQd2glJ2gkW7QTLJUAY wAw2TZ4iSZGHp4VL2UyRXJwiV0gloj2pKSbg/ojQkEbQYBgSWQBGAQuAFC705fvmvpQI+Ini Q1msLHWXTJHMyi0gkCKE/nz/D6cTBbREnM4msWwaK/t/xShugBITqURv5pQGvsnNiinxrlow PS4us9Gp10fsCihQFIflt228tjaoLjgzNGOzjctV2x8e6HiZ2YLV2HRrlWa1wCeeyjqnB5Fq jaV92dWA4tb+x8YtJ3f4roteUVqyl9nmCBmNm1xr+tQfZAkyrl5h//nlY2dgzMaMZ92BcxNO 8S7nGBtcpV5v7yjnz0zlpSt/FmwfWemzxng/tiyn/COxQTcWuzc5J3sJX3fJ3L0t+H78jqQo v8U3NdGFP37Ft2eQp13V+iO6JRxnhKNp+ffr6VMqW9tbZ5zK3jMpyft75FTpVrReg/sdshGD TS0l2qNFsb3mnerhyGxVTeTj4MqYyX0JViEh3ycOD8VlcvEfOW2zHoADAAA= X-CMS-MailID: 20250320161335epcas5p2709afc3ff29ffec350164003b511f92d X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20250320155856eucas1p1e98451f76fb93feebc7e4bb497e8155f References: <20250320141846.GA11512@lst.de> X-Rspamd-Server: rspam07 X-Rspam-User: X-Stat-Signature: srhp6fucbu96zpaqte1hm1kuc6c5ubrd X-Rspamd-Queue-Id: 8E58AA0017 X-HE-Tag: 1742487222-66196 X-HE-Meta: U2FsdGVkX1/jBT/rICrh2m8FgfLGV+bEx4McLk9aekUaViDSc/koqCrmoQQQQSjLxtcmI3gcW/4QgIpOIjrwODvnNeM7dLCRXbVCv1Ui7TU6t5EwlDe5y3PcrcXV1BCG9lTEIxKfqiWx90P2F9aF0lEhkyoXrLiz7rm90X05uh8ZRRbz0D5ma+zw5tKw+wwFzMNlaEaOzI+byawKO1Gyg9nLbhUX3ToPO5GEcZJUXxDH3z88eUrh7YDWqTRk434BoB8z5l9y4F3vp4W73gFxvtCb13xU1oZyUYTpq1DkMOud0wVS2HyKHNmArYosEMqNYnINBK+U96Y3BeZf5TDLazSFipmIeSN4Fuwim4KlvffnzuRmMS8gyasfYER7cSWL0MCX+0nMswAQEIyMiuEj+4x+gIp53W35ekj+TpeMFMmNLWnBVUv+UToqOL+Tx3MyCFadq4fdJ06OAd+kSLCYcnL5Msh0FRLeYBPgyEmKQmoUCfHsVpEOe9sul6F+ja9evbmoqfgOYzZU/vGVbqmAG85x70HozTw8YESiuquczSstLZo9+4e8HnnkHCjUpeKyA3GZ2u87pp8LL/5NQLC/D6kVPCYEpsEiztZTlQ2rpsWYmRyACgzF0VQHthlP6W5FXkm3xvfrZJE2GsggoYQkTZTpR1lBtYrGEpEK7lb+e3di9fAmG3mEDeG/WtL4AvPEM5r+CIXfBbMPnQCOoUJzV65FQ4TY/hOj9pfNW07G1I/RDq2eQV0ihwwK4QmFWZkd59C20BQ23ywdb4v7yX9GkgFZFxnRBa/e68lK+4wyQdmXz8K0NYowhvNmW7BWftrhB64gBB1S6IRU0B0yP6CkHBCxuiHNSdOMrkvswJ6uA6/eZ1zkWuDiuNDKgdmpuwO2Rip5Z+HrSnf46xgKx71IeAewFJd6bt/DoXDFETyrmPbHcu4kZFDEETUJfoX6jbTpioirWoFjHc0EJ0J0ve5 jP8wLZtb 6ZhUESesh3bcsEEotHvNMdNjrh49lq8YxCGQlzRJ51eu7NTqP67kAAZiMvaTnBbnemDsB4EgmoxLBgGalMRmpuIgmHoT9V81Yvqf/iu6ggv7m9EpBhfSZ1ihnIK6Bs7cA3CAESmbMLuRmdBWCLFz4cT2yLIG7SEHkUZt87lMTrliZXQyerW8Z2j36YendrylOL7RX4cfAAE6rnPgDyYZYDohb0/SIuRfSXzn2LvIWZ6yPA10AeDF5fHRtFOiC3CtqYmRY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 3/20/2025 9:28 PM, Keith Busch wrote: > On Thu, Mar 20, 2025 at 08:37:05AM -0700, Bart Van Assche wrote: >> On 3/20/25 7:18 AM, Christoph Hellwig wrote: >>> On Thu, Mar 20, 2025 at 04:41:11AM -0700, Luis Chamberlain wrote: >>>> We've been constrained to a max single 512 KiB IO for a while now on x86_64. >>> No, we absolutely haven't. I'm regularly seeing multi-MB I/O on both >>> SCSI and NVMe setup. >> Is NVME_MAX_KB_SZ the current maximum I/O size for PCIe NVMe >> controllers? From drivers/nvme/host/pci.c: > Yes, this is the driver's limit. The device's limit may be lower or > higher. > > I allocate out of hugetlbfs to reliably send direct IO at this size > because the nvme driver's segment count is limited to 128. The driver > doesn't impose a segment size limit, though. If each segment is only 4k > (a common occurance), I guess that's where Luis is getting the 512K > limit? Even if we hit that segment count limit (128), the I/O can go fine as block layer will split that, while application still thinks it's single I/O. But if we don't want this internal split (for LBS) or using passthrough path, we will see failure.