From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14AF7ECE57A for ; Mon, 9 Sep 2024 14:51:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 95CCD6B00C4; Mon, 9 Sep 2024 10:51:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 64ED46B00CB; Mon, 9 Sep 2024 10:51:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4EDE16B00C4; Mon, 9 Sep 2024 10:51:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 317766B00C2 for ; Mon, 9 Sep 2024 10:51:49 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id D04CC1203B2 for ; Mon, 9 Sep 2024 14:51:48 +0000 (UTC) X-FDA: 82545489096.20.655FF67 Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by imf24.hostedemail.com (Postfix) with ESMTP id B7E3D18001D for ; Mon, 9 Sep 2024 14:51:46 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=F3+gZHkB; spf=pass (imf24.hostedemail.com: domain of axboe@kernel.dk designates 209.85.215.170 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725893405; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=f9rWAsIGzZ6u/QYJf5tnAdUimUu+PCvUeW7OogHaSuI=; b=ubrdTC/Iyl8PfMSPCbMxFsdBrhfaRzQeWqCH+TaMbh4O5NOfT3cSCzCSqpnhyBzWyiECVk EP7KBLwfG3V9suLNO3yT2TBkg1FXf5pjqfFp3us5IwDYicHl1qKlOzHuC0/4c6RMLrTTzI jTKaTDcbxWIa9boyc6Vjr5xJCRlf7AQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725893405; a=rsa-sha256; cv=none; b=emXO/PiQi94zm9He2wVlSGOjvlq+SIB4hn1VHtDhjdOl+ks0E5FIGObCVYwqz3x3uZLL8g zge0DD/dqUUGDIesw+gt0vqbI9ecKaMUg2aW43tAguQ8sIJS+QEHBa+BXyHUufShL0jNkg vk2S7lgDhaKy964WOheZPsyXPCKSh/8= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=F3+gZHkB; spf=pass (imf24.hostedemail.com: domain of axboe@kernel.dk designates 209.85.215.170 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none Received: by mail-pg1-f170.google.com with SMTP id 41be03b00d2f7-6e7b121be30so2840931a12.1 for ; Mon, 09 Sep 2024 07:51:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1725893505; x=1726498305; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=f9rWAsIGzZ6u/QYJf5tnAdUimUu+PCvUeW7OogHaSuI=; b=F3+gZHkBjEGnH8GK48UPNOqDLn+/3UqpXbErKZ9etSPHPUqobddYfMCiiAupG+tR7o R3FbBCJei2cytpaZvUPpqUsk7v/vto7poTjR5+ScknrlFF/pgRps3Os8Q4pNFpSzUb2q yaurnGKGWgqUWMPn8tWEvmalOjvNfBunJ9oIr1Q+gKZPonw/Hne3o1oLuCyrMcmHoL1d bHblnfgaiO3CjXJNhW6aEJuKTRm9BxOcR16mJ0NLfZOz//+F1N2mPp/8+BK0ADQU1zlW QBfPEI4CL2n51ckxH4K7yELejXjv90lwf+uPi9AYKmdMhLOuV8YWBKtYB4FSMVfZvXoE hQZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725893505; x=1726498305; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=f9rWAsIGzZ6u/QYJf5tnAdUimUu+PCvUeW7OogHaSuI=; b=FbHbe20TTUKH3KQriCwTc4cHX7QWV9C/WELLURNra0eurIJDORycCleEpCDPQJOy0l ArphTWnnBWqc/9r/jUvUy8dWaIOfMpB5AfmhOPHR86XpL+IerYgnIccV37LpLDm+9j1V 5VeQudhsQSzmX1mILfdzI8aI4TItVihj53pjcEjHMa71ATWIQyIwrbncJnk87IAGbr39 rzRAoMguZlfyVw1xYOjsSQKCP2qdEvoOiYv5d1r56nr/A174K42BfL+trjI/O7P7HwlK FneOIc025QziMxgDWihiJVxvvRabmoIs0ROhRzWYoIvMM8p0FhiAoyIZ4Dn6uigzfMlA LPyg== X-Forwarded-Encrypted: i=1; AJvYcCU+dGUbV2uuwG4did4Sp2hCc8WGfuK+0RQQ2YcBI5lZDD6G+0fvPoScMMnx1DroqFPwsLQ1Urqv8Q==@kvack.org X-Gm-Message-State: AOJu0YzfTrtE6WA/DsSrBwW2RJJdr6KpsA863mKdKLd/MlQEcmU4+PJR Lo8RFe0B+tC7tdKf3o8wM6S4Q3O/KH4CL6J3Ir8fsuxMpAZfFBKA3x4KlwNa5pc= X-Google-Smtp-Source: AGHT+IGXmoYoNnbLnmlnvdjNnKtCotyC6KrIkej+yws/iMo0+HjJYgBJ0A65h3AE+feie3K02nBLtw== X-Received: by 2002:a05:6a20:891e:b0:1cf:2939:a0de with SMTP id adf61e73a8af0-1cf2939a1e8mr5746386637.14.1725893505231; Mon, 09 Sep 2024 07:51:45 -0700 (PDT) Received: from [192.168.1.150] ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20710f31235sm34960025ad.253.2024.09.09.07.51.44 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 09 Sep 2024 07:51:44 -0700 (PDT) Message-ID: <29245c2e-d536-4a98-88ed-d1757795b3cd@kernel.dk> Date: Mon, 9 Sep 2024 08:51:43 -0600 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 0/8] implement async block discards and other ops via io_uring To: Pavel Begunkov , io-uring@vger.kernel.org Cc: Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig References: Content-Language: en-US From: Jens Axboe In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B7E3D18001D X-Stat-Signature: 8k8qxmryub19ft3tenff4qfjmbnoksbk X-HE-Tag: 1725893506-259651 X-HE-Meta: U2FsdGVkX1+DdKh+awEb4kiqMWq+h+zfuuIYhGMYarlv4A2/J0tAJ3q+GJCz2jXGpjpn6gRfoxhtb0fRcax/vQ4viaeG/7OieiEN0Pio8KW283mnev1bG8o9FVFr0GnOpzl5/AQzmmB81SbErHO5TRWsn6vVTi+M77z1JfiFu2JKAI0XDsFsiTqqQWCXZi2kvADVDi6LRLZ/PmxyRzCwb/ygMd/3o3k2y1pXjorzPH8AlkhA3nfeI1TjFM+HDvRMR6+xdRGhsBiND88MFFVaegLV+G3NVNlt+/KWxUx7NuHnLAaPfZN8ogPNpEc1JCHcqTeBcS3Ch4X+g89FEkElNqmIYDteAehWh9B5ywozBmZo/1uLhDnIp7ZqIjUBcQqd+ix/evFywrZdsVYr0/nPlSRlWMRhXGWOF8p03oFGNJGemyfZcGhBaXUNFzV/TjEuoyGjhkjzgiAdaPrx5pmZG6hXbpzBf/ccT9NQKnQon2TaEj9yC9WP/WJWXC5FN8poG1iOv4t9ss3uhgsxjZEiCdvMwT2t3ZTpC5H3vGkStofDENukY3ZPL+TLLovfpYZvjyjvyf7NUpSJzi2+fFiPrgIEprzcxYCiIIyCYmof2IVYJIU//y86DeUslBxfaULjb3PkSb4aGo8k34TUvqfbTr501ibkbf60LRksVS7drCuns0tT+kCQAHR649wnJHZmiIn+zSY0CGqbiOIf9QGbmV85hNLTlk5atlTNkgR8aX0lB9OI98JnlDlzkAgPrU3jBaXcUCkMdVqCDif8TLTYC2OpSe/1hqiBYDm8FtVu4e02tI2YzuwS/qKcK15y+/559GQG0BMXv0bcd+1UpbT6J1veoDnS5H4OiNTOHp1ybGjUcetLyd9O6SH3IHgFRfU/4NV12thIKiOEzjsNBmWx0ItzgzAcqn2I49wuJvuy6iCbvfYS2P2rLpCF0iBCxvMhL93h2pQEdK61ltt4a66 Xt2bxcEP mqtzoX2t667aaeqv8PEQU3UDhb5IaH1QpVlnK46naE5rUvWg2XNxEAbHFiMsBsQDqwIlOi42TIldg6W+eaTvrAGUh/iWBpPfkNbrpQAwIn87EYtGC6bpE5C3Fqzj+krN9llulz9V8ycy42bsylO0eOC7X5wlVAVm2YuIk53iFkrDuxbOu2VDx7EizYq+QiC9nb9JV5mtq+oejNwPBEM+gm2V1ZjTdHj6WKQKbk4tTahhpMe/a6kkP0vMvM64XT98/Zq9M7BlDPE5Mcdmdzywg60YTRu2cHCJKaZ1E8PY4OVL1YCcfFmQ2WE2Pizg4SNBBomCyS+CEraMocHlGJDWwPglq9+At8jRDgqptHODhELRX+V6Fi1NMlaUlRx/HYHbU7SwrIVnd2CgNC4R2LfspkHTt2bBGDupv0NGkNFCSeAIp5ae0cjikZsPdqfV6Pu/q8dZuDUp7R6gegD7kmdWAvkG3Ew== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000086, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 9/6/24 4:57 PM, Pavel Begunkov wrote: > There is an interest in having asynchronous block operations like > discard and write zeroes. The series implements that as io_uring commands, > which is an io_uring request type allowing to implement custom file > specific operations. > > First 4 are preparation patches. Patch 5 introduces the main chunk of > cmd infrastructure and discard commands. Patches 6-8 implement > write zeroes variants. Sitting in for-6.12/io_uring-discard for now, as there's a hidden dependency with the end/len patch in for-6.12/block. Ran a quick test - have 64 4k discards inflight. Here's the current performance, with 64 threads with sync discard: qd64 sync discard: 21K IOPS, lat avg 3 msec (max 21 msec) and using io_uring with async discard, otherwise same test case: qd64 async discard: 76K IOPS, lat avg 845 usec (max 2.2 msec) If we switch to doing 1M discards, then we get: qd64 sync discard: 14K IOPS, lat avg 5 msec (max 25 msec) and using io_uring with async discard, otherwise same test case: qd64 async discard: 56K IOPS, lat avg 1153 usec (max 3.6 msec) This is on a: Samsung Electronics Co Ltd NVMe SSD Controller PM174X nvme device. It doesn't have the fastest discard, but still nicely shows the improvement over a purely sync discard. -- Jens Axboe