From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5834C28B30 for ; Thu, 20 Mar 2025 21:46:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34FA0280004; Thu, 20 Mar 2025 17:46:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2D743280001; Thu, 20 Mar 2025 17:46:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 19F27280004; Thu, 20 Mar 2025 17:46:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id ECD6A280001 for ; Thu, 20 Mar 2025 17:46:24 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7B1A8585C9 for ; Thu, 20 Mar 2025 21:46:25 +0000 (UTC) X-FDA: 83243263530.14.0AC1391 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf25.hostedemail.com (Postfix) with ESMTP id C9B5FA0019 for ; Thu, 20 Mar 2025 21:46:23 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="Is/D9gNb"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf25.hostedemail.com: domain of mcgrof@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=mcgrof@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742507183; a=rsa-sha256; cv=none; b=Ef4DeULeMVWHteNBorH6td3sK51X4asBDMNs4GRZomL2Jk9szuslPNpnXtmSUMmH2eAoMr l8Vmjzf3GB0agGw877A2+bF/tYhoD8Yyt62JiTTPn0S9EeKIuM988rIssl/Awqo1ASqdhX GJrZleyVLc6MLQuahXRUU+ZhXzZIRIY= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="Is/D9gNb"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf25.hostedemail.com: domain of mcgrof@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=mcgrof@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742507183; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fcGIRqdxhkSS+YNANhd15iNbIQZ08ZQ1giXWKhBmbtc=; b=wRoDUoxAvtfeyaJjvHdnRriGGXi+B41+JSZkiKd0IUy4QiLrlDm02W2ytNeJUq7jU4BQcM +aUYgSOwSfDI/VZJY9LigQpE3iQFQh8swsumWQs+VlSBzDEHvMdwlfG0mTs0KbOdAnZ1bR vbP4u6947Xbj7diqbaZbUxiQIQh8Yu0= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id B90BEA4970E; Thu, 20 Mar 2025 21:40:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 44C43C4CEDD; Thu, 20 Mar 2025 21:46:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1742507182; bh=tIioghicpQr3pmw5Sb/QIdJmci3qZXGbXH7x0CysHYU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=Is/D9gNbxFZ9rjohQBIz3DUAHsyZTatAqRba6SOq67JBFVhtI44bCIJM+54Mk5Uzk E2/uBjG5cW9uLJeK/QNESgQRnN//O9W7mdzP1TYXuQxkeLUUAy8Kz8w1Ur/4Dd6V7t 8EfexWOd4Y+IRPtaz4jxxHFm1ICPw3CIBKpJwRPx3i1FkNR1h7up5F/JBfSa2WEG9P BAm5YzncD8+msKa1XebPoEkziuL4sKIbS5qqtuNIp6UVWWRdgb3rtLfsyTARJVjWN1 L/+jjJB4QZPt2HBtCxwaTsT7ASt63fFZC0PJaHwvKgXHC4ItOTD9i909QpEWvLdlCn IE5NanQCaqAbA== Date: Thu, 20 Mar 2025 14:46:20 -0700 From: Luis Chamberlain To: Keith Busch Cc: Bart Van Assche , Christoph Hellwig , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, lsf-pc@lists.linux-foundation.org, david@fromorbit.com, leon@kernel.org, sagi@grimberg.me, axboe@kernel.dk, joro@8bytes.org, brauner@kernel.org, hare@suse.de, willy@infradead.org, djwong@kernel.org, john.g.garry@oracle.com, ritesh.list@gmail.com, p.raghav@samsung.com, gost.dev@samsung.com, da.gomez@samsung.com Subject: Re: [LSF/MM/BPF TOPIC] breaking the 512 KiB IO boundary on x86_64 Message-ID: References: <20250320141846.GA11512@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: C9B5FA0019 X-Stat-Signature: mqgj8b1ujbemohmfo69wjy6n6mmuq68k X-HE-Tag: 1742507183-259071 X-HE-Meta: U2FsdGVkX1/9SYPxr1HU5AF2d0PFrrltIMFk1dsTsdoMTIDnmaqMvco2BAH7S6uHY4Y+tCNER1SXo4izLgt0TRkXEXgcNnRzJdsa4CP/EJzQBvRKcCbVAWKbpahXMa5IYTTy5jfxuc0Xk3eBMZWVkWlZcA6zGqKqX2FkMoY6I9VE46HzskAoWmGYtjuty6LxgRYA4DrSirC1MQu8b/TpD2UKH/zLJAvNDtKGCI8GAy5gPhimkeIq/OCJPh3pXs0gLSJt/KflHeE711YuVXgIrdrLRP7RKIw+4kyzb9lU9fIRUzUCvOqW1bh2BALAg+cDBz8bwDCOPzGCPY3jPSovUKl3gx3EVRUsghH2I9F6NxoSWidmE+PLfa8GZqD5eGvxzegmY9RX5O9pyf457A613DBFa1/5/T0hjvbKdM6SOIpizPKwj1b3+csY6T5sFlZcS28BxJEcOqfRIUhFn4zTMg/azAWAolzQk4hASCPGgk4JIEfN964kbSdqN3lPB3c+E98XzE7LD6W4DsdhChLyrLWD/64jM2QOOfQZ8qOStK4NhEpq/tBEV0fpMQbMbQOMkL15wlzLvEcsl902+JqOY5NSgWqA6Btl5H2ac47ONTC3vicoz4QoGY4cGUcKrddiG16VrC+9QtfGLfGq8R5y5CuxHpfucwrJq0w36D36mW5/t/EQ+Qr/cxHcOTW9+/gwSn+I4ezZT3GHUGr0mosdI/1YycYrHbthpLrqTcwO2mIh69e8cQxqXak5cg5Z9d4fw7hrh9/4K6/CNi30a+1if6IWk3DsHWOk1rtsKFIIAM0JAKQQpqZqa92pLY2olb6KoTNJJuwtEg0uHamsgi6YlkFg8sNMYxaTqCpqO4RByKgmmNNdokXjFbkEtampuGvM4Y4c2pBggEO0tf6/bgnKDdVH0xFx1ONa2z56dmZL13iQfFSpLmSMnatMyjfAMQJSnjSAdieu57FQ5dB8tIY ICGPq6uD Z+HvlJl3IvZsphx65LOWDEeoQ0UMymPlYok// X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Mar 20, 2025 at 09:58:47AM -0600, Keith Busch wrote: > I allocate out of hugetlbfs to reliably send direct IO at this size > because the nvme driver's segment count is limited to 128. The driver > doesn't impose a segment size limit, though. If each segment is only 4k > (a common occurance), I guess that's where Luis is getting the 512K > limit? Right, for direct IO we are not getting the large folios we can take fruit from with buffered IO, so in effect we can end up with large IOs possible with buffered but not direct IO. Yes, direct IO with huge pages can help you overcome this as you noted. But also mTHP can be enabled and used for direct IO, or io-uring cmd but mTHP is not deterministic for your allocations even if you have a min order filesystem. The min order requirement is only useful to use in the buffered IO case. Luis