From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92A2BECE57A for ; Mon, 9 Sep 2024 15:33:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF8846B00C1; Mon, 9 Sep 2024 11:33:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CA8686B00C3; Mon, 9 Sep 2024 11:33:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AFBF06B00C5; Mon, 9 Sep 2024 11:33:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 91AA16B00C1 for ; Mon, 9 Sep 2024 11:33:53 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E1EA8160967 for ; Mon, 9 Sep 2024 15:33:52 +0000 (UTC) X-FDA: 82545595104.27.4D536DF Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by imf14.hostedemail.com (Postfix) with ESMTP id DBD9F100014 for ; Mon, 9 Sep 2024 15:33:50 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=J8Fok55I; dmarc=none; spf=pass (imf14.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.172 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725896004; a=rsa-sha256; cv=none; b=Dw6Oe+WzoP8lcitVb3IuQECVTqpxcPCzOyaujAdNNhxrqVuOIRiYeGyo8wyPYO0EurqttO VkvUaMAUfDfrIO4ZuIMs5AiH1n0sjGVxGvQqbksQGrbUzyc05QV03gxoNGlvj3DkiD5SRH E84OXw8rYJocWrIBdmJ42jz3D/5vzSU= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=J8Fok55I; dmarc=none; spf=pass (imf14.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.172 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725896004; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Jz+0OHvumrwbT49/ZLbJGnoGzf7RLRObyJvUXiont54=; b=kowlo3DuO/jZR2uH2JUAT+tUm1kE6u37WIQ41HfAp8SeFrBNRBZ9Nyiu3fuCtcHWXWyPJL noTSuC+7BgGaIF9MozvPdujmm80+qi4xqbajWFlLMDdDRcbCWbhKe/ioFzkpyWWVHEA7MG MvmVWyg+nikg7fOe5xu2F4MkB/CB/OI= Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-71788bfe60eso3353098b3a.1 for ; Mon, 09 Sep 2024 08:33:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1725896029; x=1726500829; darn=kvack.org; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id:from:to :cc:subject:date:message-id:reply-to; bh=Jz+0OHvumrwbT49/ZLbJGnoGzf7RLRObyJvUXiont54=; b=J8Fok55Ih+iNIux+kR32aLXN/j+ivjqNmumujSLJbggGUas6km24icZYJ18rGlP5+M cyzaiq2zofGE2H6ZoNIUKLzh6KDXfx56IXOqBIW4gf65OLN+Sk16zwRmIZMqy31onods m65GM6m8R0nJ/A4Debbbq8eJeU5TGDGkx2K0OTD7QmJaG8UCmlwwSyAMJhEXFEwjoheG Ayk5kk9Az+VS+qeTFbDerzbXr6OQYutE3X5D8+++c9hW1b2+p7scdnfzZ6miGEZCbrhh qK/TuFwm3tUbapbXstnFYSDaruuXlpSQCPkNQdshfYIhrM3BRmoaGgQuXXvWrTVdSQPC N+iA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725896029; x=1726500829; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Jz+0OHvumrwbT49/ZLbJGnoGzf7RLRObyJvUXiont54=; b=wcBahteCaeUmExXHnJ6zWO0psh76hIPzEakEaFRliv5MVHIPjyyY15H0+z4kcPkl6c PwSTrZHWyIDD+8pNwomZqiQk9VxbNrgRBCWynjwjM+DmpGGqEoW8qFCjGezN+V3drvpC hoImozJLKQ86OA5gk6gLIShEFPjpQb1D212GdSvDsb4D6aXZyNS0JbF5HTjJqjWXqPpy rDePg0IPNdlOrQUC+0kz7/wtU3QZewd7aHW3CH2t34fe4Ts6A32GgluexHMcSVKSuS9t +M6F63+xoN6miaLKK6U0HX8p7in38ieLrEfJnfODTOc8GqRABPvd0bDcR/9wA3XffNcp zimg== X-Forwarded-Encrypted: i=1; AJvYcCWbl5bbRnWRqWGCSIdE+0kYFiK3JagpQpHuFx++9nxl2A7ByMUZzm2WV7JcqzjjKa4naGQ3KCxszw==@kvack.org X-Gm-Message-State: AOJu0YxP0cSoAzqhNjK2wtxx/HkRjgXeFwe/OJFH+ajzfTH549Q+z9sR AIgSg4AaHCoViLSRh/LMX4bqmfKhH1Qsb0olMesvj+uI2cbGfPf1CjI7HBB522M= X-Google-Smtp-Source: AGHT+IF5m99KU1sY/y43nzvWbYbkiNLlQQoKCC/b30rJ4NsSCugz4Kj7izv+FOPnC2J7LbiXlyDv2A== X-Received: by 2002:a05:6a20:4909:b0:1c4:9f31:ac9e with SMTP id adf61e73a8af0-1cf2a0b7277mr5830465637.42.1725896029458; Mon, 09 Sep 2024 08:33:49 -0700 (PDT) Received: from [192.168.1.150] ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-718e58b2a88sm3638567b3a.46.2024.09.09.08.33.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 09 Sep 2024 08:33:48 -0700 (PDT) Message-ID: <14a6ec2f-dbb8-4372-bccd-8e91c9527ea3@kernel.dk> Date: Mon, 9 Sep 2024 09:33:47 -0600 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 0/8] implement async block discards and other ops via io_uring From: Jens Axboe To: Pavel Begunkov , io-uring@vger.kernel.org Cc: Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig References: <29245c2e-d536-4a98-88ed-d1757795b3cd@kernel.dk> Content-Language: en-US In-Reply-To: <29245c2e-d536-4a98-88ed-d1757795b3cd@kernel.dk> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Queue-Id: DBD9F100014 X-Rspamd-Server: rspam01 X-Stat-Signature: 8zyffc3trk3p84yy9oe91khkb9yhmnfj X-HE-Tag: 1725896030-493866 X-HE-Meta: U2FsdGVkX18SCVuwlgWFdDTZgiFCBD+IFyJZ+njBqoEmD+nes2GYloH2k/cfBzO+2K0OKdQ0PvMRSlizB+rIWpmLw+Nc1Z6qjyE+ZIm5f6fIqVlqRXPPGYorkSZ3zhy8MfCCJvVowULfkh0gmz4wa94KjBt1GAztSzBw28AQMz+tKCPEOedjlCB5QSIjZ8SyKuPRNcpUHt7Xt1tbbY0AgQT/ZXaG4mnVykvjJ0X+9i5scbMe29CJHMKTWf9MVBDPHz+4kbm+rVeWGIewhf3/63VdMD4Qa8uMIDPzjXP6dCB2SV8nVnuqEAJzpIXkd0lZiVKcA546tH99RydCzbY9Hgms80qbKONTKU98AHnOwahQDYxhISA8D0FHwCx7GwPJTyrW2z4EKuApDZBNuGBFLyTYKH9UwKtzjcgkoRkExji84XpvCSPJSZKg+esNt1g7JzKElRWMjKI1C36c/VbJXqL5F00Hek/dyeuBOGlVq7MPThXKFKHRV4l/yjqZ2clhnDLOZS2h+s2erwli5LYsSvMXcPU5LHJXF1d/s//i4nCzHI65KfdO/ZvmhTWmpc4nK1E4QZvj/aiIYl1EikjcBt9yNEIrI9j8rFQstoNpN55PgddbMVytcvjXu02r9Yxbbt6vs9Eoyv53McUphkWU2OcrejcPC8/d5ievDWSKxlkMtjumYqxgX+VtQeCCzSEIwXSBEVjC3xBSbMOfFHK6i4dGccZU0vXeu54zP3Oug2KO8W2/IWTzG+DmT8LXzabFtM5ypmuTW4Kj5Q/JvjXT9oRImqsu5/Fn2xZk9rmjXF/Y5sqJO16Hz9kSgoodLxu2Glts1ZS7wFQKi40SSp5/0kKFikNF1ZHoIiXMhi7uNkNIMbN8aNZHjEDKUWjsiKrPF/TW5UNb8j0OhNxk18UaN0l2QoMtoDOMolElcSevILgriYAmdsfWADF4S+yCTOKMMK2OOhnPt02PMugbOq3 gyCiszRT A9fdmpiETBcEnobkApgrPyB5pmWZejOOrF9Cb2GnNHOtX2VrpQ8qBPhvaCVy5rnnbvYcd3Pjp7azGczepmTfaDiRqYyUzzRv36vbyhg182mqLnzrsX1sVc7qggrt+zFZm5RdtUwqbQG0Rx+BVxXH82igsRybmy8V67rSE9yrBkew01t4Txy92F5k/CVJMUR99DU7d3pyke3aXeS4UKLIHH2+tjGaHRN1LHwFXWw3JjvFVkzo/krV6wZ/qBkpnrvdwBohnxY1oIUbxTmaUJZ209W7KnRm/BFz5w8z+5p5kmqrXhy5qL2c4nmHXu+E4K11rAnm/u2ciDSoLJMM7e22zSbamoopmAQ+Wsy+5uTXZUiDyYglOhAzplp2/3OvkU/WZ17LeQSqflUwMLVxHmDr1afo+N0QtMc5/RhOv5n5ZGWaC74aslF6DEaAq9mgiTGy4KKqY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000320, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 9/9/24 8:51 AM, Jens Axboe wrote: > On 9/6/24 4:57 PM, Pavel Begunkov wrote: >> There is an interest in having asynchronous block operations like >> discard and write zeroes. The series implements that as io_uring commands, >> which is an io_uring request type allowing to implement custom file >> specific operations. >> >> First 4 are preparation patches. Patch 5 introduces the main chunk of >> cmd infrastructure and discard commands. Patches 6-8 implement >> write zeroes variants. > > Sitting in for-6.12/io_uring-discard for now, as there's a hidden > dependency with the end/len patch in for-6.12/block. > > Ran a quick test - have 64 4k discards inflight. Here's the current > performance, with 64 threads with sync discard: > > qd64 sync discard: 21K IOPS, lat avg 3 msec (max 21 msec) > > and using io_uring with async discard, otherwise same test case: > > qd64 async discard: 76K IOPS, lat avg 845 usec (max 2.2 msec) > > If we switch to doing 1M discards, then we get: > > qd64 sync discard: 14K IOPS, lat avg 5 msec (max 25 msec) > > and using io_uring with async discard, otherwise same test case: > > qd64 async discard: 56K IOPS, lat avg 1153 usec (max 3.6 msec) > > This is on a: > > Samsung Electronics Co Ltd NVMe SSD Controller PM174X > > nvme device. It doesn't have the fastest discard, but still nicely shows > the improvement over a purely sync discard. Did some basic testing with null_blk just to get a better idea of what it'd look like on a faster devices. Same test cases as above (qd=64, 4k and 1M random trims): Type Trim size IOPS Lat avg (usec) Lat Max (usec) ============================================================== sync 4k 144K 444 20314 async 4k 1353K 47 595 sync 1M 56K 1136 21031 async 1M 94K 680 760 -- Jens Axboe