From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A1EACE7AA5 for ; Thu, 5 Sep 2024 23:35:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 969EB6B0085; Thu, 5 Sep 2024 19:35:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F2B26B0088; Thu, 5 Sep 2024 19:35:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7BAC56B0089; Thu, 5 Sep 2024 19:35:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 590CF6B0085 for ; Thu, 5 Sep 2024 19:35:46 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 103D6A9E10 for ; Thu, 5 Sep 2024 23:35:46 +0000 (UTC) X-FDA: 82532294292.13.786A27D Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf25.hostedemail.com (Postfix) with ESMTP id 658D3A0012 for ; Thu, 5 Sep 2024 23:35:44 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=U1jajGzU; spf=pass (imf25.hostedemail.com: domain of snitzer@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=snitzer@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725579215; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uPAgCHnO8brcwtIXQzleJIo3jouzWCj4RjvufSLN2As=; b=BLaYDggOhz5L9dNYmbeIHQEk0vVLP+1ot6Chqr4VcSoNE1ML6yhdb1W1VMMDYI81hsxn4i X7/ubk3CP+I7WSdhmcXh0Igl8YyY3YQ51l54X+ik0c2hC/9xyE3ik4yz/NRORYKGqg9czh NYCp2a7ss1D8qPgKd2pEVpVGsAuVqjo= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=U1jajGzU; spf=pass (imf25.hostedemail.com: domain of snitzer@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=snitzer@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725579215; a=rsa-sha256; cv=none; b=ESpuaAPsIYRpIwo6vgxBsfOW3PlyYVivVVOBjPh0I3tGFoPg3snIfcMFMDib5d+TYx2Jxe RWl6735aehTa0i9yubeB5ksHWpgw4ZqEtYAUWwm9MZ3oUFs+QJle3FKHO/B47cwlhPocZU qzwnh4tn/UK44b6fJvpVNy5x+m1RiCE= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id EE5E35C5464; Thu, 5 Sep 2024 23:35:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3FA6C4CEC3; Thu, 5 Sep 2024 23:35:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725579343; bh=kdJBJm07vSWsMujAWNUkpnWYrq/ajC+MWp90UGPtRbc=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=U1jajGzUrXBq4LkcyHu9Gawoocw7falpk/v+3iLXUWwe0zfR6FB9ddMFQ5lHxbzVF NybCr2auPZbMJ6AjlNFbPuH4I3ustsyovf4+UZ3UAG9X9IDlYgdsDMfjYLcIeDTikx UVGePWLQ/7mJo7Yn8EVL6x0o7GEXLQ5/weIy4GYj4pLoAZEw9J0Bqc+mFKuKq2lAl2 gwN3KPjfat0dhVFHYjpyMusXWkyxxwEsXjJG7cdCuC8hGG4lJZpSoJZHw6v8FQpPfQ uKgv07v5u1309uNVSwj6OJLDH0hBmcQt+VeofpTBj0l4uENtngmrt7fwBpheI3NQ5d 17hw0avXtMsmQ== Date: Thu, 5 Sep 2024 19:35:41 -0400 From: Mike Snitzer To: Eric Biggers Cc: Mikulas Patocka , dm-devel@lists.linux.dev, Alasdair Kergon , Tejun Heo , Lai Jiangshan , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sami Tolvanen , linux-nfs@vger.kernel.org Subject: sharing rescuer threads when WQ_MEM_RECLAIM needed? [was: Re: dm verity: don't use WQ_MEM_RECLAIM] Message-ID: References: <20240904040444.56070-1-ebiggers@kernel.org> <086a76c4-98da-d9d1-9f2f-6249c3d55fe9@redhat.com> <20240905223555.GA1512@sol.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240905223555.GA1512@sol.localdomain> X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 658D3A0012 X-Stat-Signature: 43hqtozubj9n56ec57qw35a8mpozsm83 X-Rspam-User: X-HE-Tag: 1725579344-464712 X-HE-Meta: U2FsdGVkX1+Xa0eJw3w/BUfMkU2KVuXuVifCRNV7iesA2xMuzb+lX3jqbIt9S+K0o51VoOJhDAoXc0wje5s7o39f2xf934iXXpkGW/k/D6KdeR32Q2gN6FGPEczleMrd7/hQ15s6iF4waAhxhkhFt5d20T50bcbOkG6ct2f1aAhTp0jzqIfB07kUqJToq+oLrFFcLum2wj7Cv6ZRrDPfZUOnN8uF7JK4PqolLU0N1W9nFj67KqqgFxjp7NmPSzR4lExtaro0sdTbAoXERLyCfHlJUYc5oOIG/miFOKUui3j30HM7K5b2QElMW6T7cBz4HWceSoMbANi8haFtknKxk+gUElWFi68lIC8fvE2bA6w1CVNhCJ/p308QTBezK0baubl0plpQhHq9Co8Nc+xzgodExd0apbWxTZJmZo7ymRwyQadLj7Ke80FTCUb3f3cmPt7p5MRx+EaMsV8YXLR4+pH8xQpJ/e9WBhEmHJMpE5K0n1raa2pfIB5PI3gT635Nd3PWaKWe3oH7gyJwgcm7e6/LM4TPfbNlTe8E6p4I7G+hlhELOqCdPSbCZq1vJWe3mj93uK1UQzg90WN+Y5EkD/hy4bpiZDOlp5V9rDsFHHwiRPeRccR5cNUxDz77GbyqDyhJaghXGZ569/FhONx7/4SI92VtLwIB0dkrWDTQJRuiVQy0kSAQnKdIOeQdjKh1CxnHAHs9jEavDDrmZhYwYZgRI8wK+Wg+RzOM1NKoINGmLsKioIfAey4zTl6TbKXvAu9xmZOJCMkKpyFbKDJORLatrDtL3OjObDVi4Rb2FNM9MmzR8xJoIblq4KuhtWxJqCxGmxNDuBN+1Vi/rznnsmLCP0g1YQ0sTFpaQhKz7iR0SGPdYUB2cPxUpBfjDiJ0vsGCxXJ19B6Z4eNgwq4ZCKzAvSOlu+h51EbhdqW0paDfBD2Oyz5PcKIQH8GHymiHCA8MXz55j/UxbBUHrh2 /CfL8Ktl FrstXoGnSSY2D3d9qZMGX/i9/wIKHF9QZQTEYzdhuolVnDuJmMbt42tjaX1bVwBtC516yrWeWfL0lmnI3Mnp/tpEImtd/7x1p9ri5h01eW2F8yE45K69hc5t4h1rFb9F+XtSOTn9jHu8K6lJHJUrs6aCmWjNyvvQUZEbOxnxT7w7T7+TVVjTT90McF/4pX2RhX4gpRHoTiC/WLR1NLRjqE8JChDjYiMId47JKYcWSZfjVJ2/sZKl6kD9a80mwiW6mSYwmEJNkrcgDVKmsLBNUp/9FF/92KxXLV0XhNFEL7WAbFcYw0hS9IcAz9RDGTFuCov5dlrQOe/S6hzvgHRuuv7WcSY2OC/kcebCzWnbAsMLSGfoM6JXeDBZ20NV1Y/6rNBPPycvGaPA0cOY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Sep 05, 2024 at 03:35:55PM -0700, Eric Biggers wrote: > On Thu, Sep 05, 2024 at 08:21:46PM +0200, Mikulas Patocka wrote: > > > > > > On Tue, 3 Sep 2024, Eric Biggers wrote: > > > > > From: Eric Biggers > > > > > > Since dm-verity doesn't support writes, the kernel's memory reclaim code > > > will never wait on dm-verity work. That makes the use of WQ_MEM_RECLAIM > > > in dm-verity unnecessary. WQ_MEM_RECLAIM has been present from the > > > beginning of dm-verity, but I could not find a justification for it; > > > I suspect it was just copied from dm-crypt which does support writes. > > > > > > Therefore, remove WQ_MEM_RECLAIM from dm-verity. This eliminates the > > > creation of an unnecessary rescuer thread per dm-verity device. > > > > > > Signed-off-by: Eric Biggers > > > > Hmm. I can think about a case where you have read-only dm-verity device, > > on the top of that you have dm-snapshot device and on the top of that you > > have a writable filesystem. > > > > When the filesystem needs to write data, it submits some write bios. When > > dm-snapshot receives these write bios, it will read from the dm-verity > > device and write to the snapshot's exception store device. So, dm-verity > > needs WQ_MEM_RECLAIM in this case. > > > > Mikulas > > > > Yes, unfortunately that sounds correct. > > This means that any workqueue involved in fulfilling block device I/O, > regardless of whether that I/O is read or write, has to use WQ_MEM_RECLAIM. > > I wonder if there's any way to safely share the rescuer threads. Oh, I like that idea, yes please! (would be surprised if it exists, but I love being surprised!). Like Mikulas pointed out, we have had to deal with fundamental deadlocks due to resource sharing in DM. Hence the need for guaranteed forward progress that only WQ_MEM_RECLAIM can provide. All said, I'd like the same for NFS LOCALIO, we unfortunately have to enable WQ_MEM_RECLAIM for LOCALIO writes: https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/commit/?h=nfs-localio-for-next&id=85cdb98067c1c784c2744a6624608efea2b561e7 But in general LOCALIO's write path is a prime candidate for further optimization -- I look forward to continue that line of development once LOCALIO lands upstream. Mike