From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70046C3DA7F for ; Thu, 15 Aug 2024 15:50:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E33506B0158; Thu, 15 Aug 2024 11:50:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DE2C06B0159; Thu, 15 Aug 2024 11:50:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CAC226B015A; Thu, 15 Aug 2024 11:50:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AB5DB6B0158 for ; Thu, 15 Aug 2024 11:50:18 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 51CF1A7635 for ; Thu, 15 Aug 2024 15:50:18 +0000 (UTC) X-FDA: 82454916516.04.0C311D0 Received: from mail-il1-f169.google.com (mail-il1-f169.google.com [209.85.166.169]) by imf30.hostedemail.com (Postfix) with ESMTP id 0F56580017 for ; Thu, 15 Aug 2024 15:50:15 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=RntB1LIJ; dmarc=none; spf=pass (imf30.hostedemail.com: domain of axboe@kernel.dk designates 209.85.166.169 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723737003; a=rsa-sha256; cv=none; b=un9QiDAh26OWLAT8uicx3ZORePvhdnj3WGqChENzSLGEbHOqOyH6phxGcgadaNbuczgINq MqVE8hRqwc1Qd/cf0NczOgAWzeBKxVXq+UCuaXf0e3vQUOVUsPUbnF4xX8I37SAiHgZZsO Ly4Wh+q0dYoJzhCHlsHcocoyJwjUO/U= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=RntB1LIJ; dmarc=none; spf=pass (imf30.hostedemail.com: domain of axboe@kernel.dk designates 209.85.166.169 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723737003; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VdOckHrkEyAP+PfY5R6862py6dLLc14uZGSEwR4R1DI=; b=QAGO8IBktOOtY2PqyRWHowHCpGONcnhPtSjoQwbIiZ+oZizwHSR5mbgaU9s2toUVyDNsBS UXHSEtBT03CBEGdsSU6IA86CZ0S0oBuXcpiHIyvjq4XOeIQ7OY4JIhsX9wfblIF5UwpaSu 670v6V4WDRTZ85u9lSE0s5kZW20FS44= Received: by mail-il1-f169.google.com with SMTP id e9e14a558f8ab-39b051817e0so641605ab.0 for ; Thu, 15 Aug 2024 08:50:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1723737015; x=1724341815; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=VdOckHrkEyAP+PfY5R6862py6dLLc14uZGSEwR4R1DI=; b=RntB1LIJr+Opq9Zqc0MhpYy8bQ0NfOZk+dkFMEF0c4y+UbmVesLDAYVOSu9xNkaE3S 3Mpw9atl0IE5I5P+H8UujpcwwNlD6D42fS7ZtNxsInNveC9WPcu72ZuY+cvAfDhtpKUO W5ujRiG0mTRjdpHoq7DiXBIcqMfCSOOko3QmngXHzY8/MInf6hm7AzGqhnxti/9075oQ dys8s2+cLY9yZU/Rl6hvFFFoAHNBksYdzbQxUrg83PN+GgZ8i9OgJ5XJUacrFbTpI8V0 9dtl+Z7pvuq9RIPUqCPd9FJfrwr85V42FcunRxX9jZHd6qVt5v/zQtMgCviMazMxDWG7 2MIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723737015; x=1724341815; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VdOckHrkEyAP+PfY5R6862py6dLLc14uZGSEwR4R1DI=; b=Bxbylf/pfLtP21X/sB5S1uXul7kpAt8NWQ4xv9wuPW4JBlcvZkfwoBOyS6h+yweJfb U4DMPX6e2MJ2jrCy2EIxDJ3H8SzFZgIJnfGqhX+NdfP+kUALkmElqckcnDlg/OgLeIgg 8FIJBw/xcs2x1FkccMlDIVn68G3qpM2Yso0MHIlPPl6bFbzovDYE23l0I1iCQJfqqqQZ dPqIziMMZt4cwYLlCumPG7el6/xeFGk99n5c7wce55Q8xGm3evAl6OPVxQ+YhiWweL8w H/ZanSv1e4gP3FpHKoxfVHMm1z3EebfDV7qvyRwFV8Mgpgs71wQT9ZPGJWK6MJa4n8Tt 4VLw== X-Forwarded-Encrypted: i=1; AJvYcCX0ANeVAcRp8Gf9eLdbqFZCY6cYSgM6OkKBJueeAWStbPkvU8pQuEOFQiwwachFlD9Lz8zIZGqp8A==@kvack.org X-Gm-Message-State: AOJu0Yzb5DnzAnNX1nC7fX8cj2wOUfPqCa4VCzovh5zBps7X10qjD8U0 rVHPOUMzm2U2dZVyqw+U5tIDIv8NshTe/DbykS4BApIiLMibHpqJ7MAuLapwdXE= X-Google-Smtp-Source: AGHT+IE2zOP+lY1YyGAZT/nsjWDqkSGP1A3erhshqNBaTpi9hPdWu0zNuzQIz05cD0DHnR8VUOkj9A== X-Received: by 2002:a05:6602:42cc:b0:7f9:3fd9:cbb with SMTP id ca18e2360f4ac-824f25e4952mr14867739f.1.1723737014687; Thu, 15 Aug 2024 08:50:14 -0700 (PDT) Received: from [192.168.1.116] ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4ccd6e93f89sm557006173.57.2024.08.15.08.50.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 15 Aug 2024 08:50:14 -0700 (PDT) Message-ID: <9eac5571-a330-40b1-92ac-c6983be3619c@kernel.dk> Date: Thu, 15 Aug 2024 09:50:13 -0600 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC 0/5] implement asynchronous BLKDISCARD via io_uring To: Pavel Begunkov , io-uring@vger.kernel.org Cc: Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org References: Content-Language: en-US From: Jens Axboe In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Queue-Id: 0F56580017 X-Rspamd-Server: rspam01 X-Stat-Signature: q6jxjxqph6ojmig8d6mpt6ttsmf8wjia X-HE-Tag: 1723737015-860636 X-HE-Meta: U2FsdGVkX19ZxMqTk88nGnd4V8RKEsmP7d/ORlzC66+r/mjiqYru2W70njPTA1kurQwbYrnSSswojubRWdIH15f+Cc+tqjrhbyURrDy4KfLOGrhi1sgCkZJ30Sao/PkDkHT5X+xYNVxgOhTDRMt0ZC/PhOg7deJxkKnLHH5JPQ+u5m7RG5+NjBJ6tk4WMXea8BNhNOs8mzMGZI+WnWI25+6/UoVFvrla2voAdnWtxoB5iyTgO7eTI+tibjrT5bixNPZ/LHYi0N3/wnNc/O229Byy++VxPl2dKzwuqLaJPniMUfsW9dVMF3aFhpQhWCrItMPdvNt2Xi8Hy8l81+O5slNfNbc+MIX7+1ig552cr/NyPTZnMRegLs9/0S/Fwlf+8FNYlmP458lTnb8HnsNkrbv1s/vc1+qdgOIUe54THrR74DLJpch60G6VxFMME6Tl3rzLe46/8SgazLfuoXcEgvb9Ok2+3fnTCqvZyX5flVdjUlw6ApXd3vb3+0tfpDHxQJilKnLCq3xkySSXxzTD9LAIPUPTnbEv1K45YafB+trLsS857ZenxqFHvfzResTL9kFu5RvxDTctbVbms8FmeGVdi2b7+LaGH37fzD462QULh+zqO3Se+Yj5KnX1xyx6+gLjlr52rJwqWJbT5jqQnfZ6QS+A4KwWhGPxsnA8bwLEQYrJVrgsVWhxQJ42ew+fyBxtvCCmsclxOJKOOJx2ePn4WmiO7ADO7+kizh3HbQrAByodkixksoDfwlPJqXH9I1/4l+Oq6dr1+yFywDN782i3trse0cFLmXb0aIJHBIdUfJ2t+zOyximUAmCuI3Z2WsLIK+SBVXDEKVkMMSUFfdp/kwSsGlzbSlBj7qIkhJ4A3YaVVoOdxe3yKFfhrbiWMGlduwOJmB00McbQOPQO5FF3pRbzJt10cs07uY2IlwfrHcS7g/XxcUJkZLdTt0dgnmmQg6IwMNxpEZiWELr nVNNCMNs sjEJkBr4y7b//J0VEja5rMa4zVgHpXPPqsCNS+MhWHLtmmSJOKIA8lL3ym7mV6eMo1MlgQFmlmResGjDlxjz8UZqB+lJgNVzSdJLeI6lxHkCnDP4qiwJGEt1+i/vFt1v85PpyK7gq87yN2v8n/C6pqf6Me2vAHBb9P5PVuAC8mRFoNs1JaEDx/H1WuktH4aiALTcD8lH7zEkWaHU2AFmsnToPdW53YaqpXaJIfIKcly6WSrcZHP4KRIJv1vPteaTEMlfbgQKqeq7I/1MMhIo8cXa7uEN2ojldCIsXBCxeyzOSssaWCaKzq1qqDc+ZptJYyCFYKPIIk9fPL8tj2VOKd8riPGOPLF+u7HUZAS9akyKEqzpuGsjlTj8vZzeaUIrbO8yq+bTl6Q7TMdFsJEo2Xf4ZP/08Inqws/w/ILOThL5vMR7SldV9n+qqVatncKHGpNnaN9yGgBE/U3REYdqk+b15WIehQ0PKhYCQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.001367, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 8/14/24 4:45 AM, Pavel Begunkov wrote: > There is an interest in having asynchronous block operations like > discard. The patch set implements that as io_uring commands, which is > an io_uring request type allowing to implement custom file specific > operations. > > First 4 patches are simple preps, and the main part is in Patch 5. > Not tested with a real drive yet, hence sending as an RFC. > > I'm also going to add BLKDISCARDZEROES and BLKSECDISCARD, which should > reuse structures and helpers from Patch 5. > > liburing tests for reference: > > https://github.com/isilence/liburing.git discard-cmd-test FWIW, did a quick patch to wire it up for fio. Using this job file: [trim] filename=/dev/nvme31n1 ioengine=io_uring iodepth=64 rw=randtrim norandommap bs=4k the stock kernel gets: trim: IOPS=21.6k, BW=84.4MiB/s (88.5MB/s)(847MiB/10036msec); 0 zone resets using ~5% CPU, and with the process predictably stuck in D state all of the time, waiting on a sync trim. With the patches: trim: IOPS=75.8k, BW=296MiB/s (310MB/s)(2653MiB/8961msec); 0 zone resets using ~11% CPU. Didn't verify actual functionality, but did check trims are going up and down. Drive used is: Dell NVMe PM1743 RI E3.S 7.68TB Outside of async trim being useful for actual workloads rather than the sync trim we have now, it'll also help characterizing real world mixed workloads that have trims with reads/writes. -- Jens Axboe