From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D816C5320E for ; Tue, 27 Aug 2024 03:46:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F8EC6B0093; Mon, 26 Aug 2024 23:46:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A9606B0095; Mon, 26 Aug 2024 23:46:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 097946B0096; Mon, 26 Aug 2024 23:46:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DFBFE6B0093 for ; Mon, 26 Aug 2024 23:46:29 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7BFB11A0EA4 for ; Tue, 27 Aug 2024 03:46:29 +0000 (UTC) X-FDA: 82496638098.10.06034DF Received: from out-178.mta0.migadu.com (out-178.mta0.migadu.com [91.218.175.178]) by imf10.hostedemail.com (Postfix) with ESMTP id 832D9C000B for ; Tue, 27 Aug 2024 03:46:27 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=KzMh5uQM; spf=pass (imf10.hostedemail.com: domain of kent.overstreet@linux.dev designates 91.218.175.178 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724730368; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sZOYCYtMYjTlpd/hIkR0T2CyO/7x5lAIgRzfcbKmRfY=; b=77mDOWemLCbGFaeJfdfl4G7GbYHwTjv79M4+FIjKNtt2ImXfj09aULFjOKMHTcXP6VnnfN RQ4Kca1G3kY7q2ORkncnKEy2xQleJqwW+9ri8HyQQ0abDUqBe9YlTdprXDssFuysiCrBGO VraSZJ+XGUF1t+xwMZdhl935/LsncVs= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=KzMh5uQM; spf=pass (imf10.hostedemail.com: domain of kent.overstreet@linux.dev designates 91.218.175.178 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724730368; a=rsa-sha256; cv=none; b=lbhk8HDm9n/+xZ/CHeabaIsIRrKIE5aTzDG8Z+EjpS9orEpVf9MCdWmDGpH+UsN/XZzwdC PBN9YZFzchZgmKxxDG5/0l1DVlVPfNiSHQSdt272Jsil1tsU6hq7bg7+4ZKhlEGlfT7PWP 8JP6XEJ5q9CvxMRR33CXAIUyuPN4ukQ= Date: Mon, 26 Aug 2024 23:46:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1724730385; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=sZOYCYtMYjTlpd/hIkR0T2CyO/7x5lAIgRzfcbKmRfY=; b=KzMh5uQMoLAIG/Nm7YPcPe5UW2H8NLRCzNP1KDsTo9XWYTnsaIFDwUnJg/AHOlthHCG/WU 5khKuAI34KlgJCqmea/h9FaMBgNgpkL+K4u1l5eMfHrcEyNS+zflq6NllRqzx1xnft4QXE MuuTQtU1Q16SxisYWDDiRoXeHCP3VMs= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Kent Overstreet To: Matthew Wilcox Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-bcachefs@vger.kernel.org Subject: Re: bcachefs dropped writes with lockless buffered io path, COMPACTION/MIGRATION=y Message-ID: <7qttwffpydvnzqm7mjflqui7ywjb5gvhmvwebxktoth33ffpnc@llewszyqw42z> References: <2iroae47robod2vijalby64iczk2emltrshmztlwyrxmkeiydd@4lxo55nlgpxo> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2iroae47robod2vijalby64iczk2emltrshmztlwyrxmkeiydd@4lxo55nlgpxo> X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Stat-Signature: qpzh669iyknfk4e3jjojqsxsh173e51e X-Rspamd-Queue-Id: 832D9C000B X-Rspamd-Server: rspam11 X-HE-Tag: 1724730387-832807 X-HE-Meta: U2FsdGVkX19/a8TfD4uqZXbhGTw+96YATgk2reuLuCeCEw8ZwrbOJqzqsv2NYWO3NJ0Jf813N4UxBSshuSv4em2eBqTxH9FO9P6SDWEcWo6cl3cymvHYTIB+eadkpGQEpPBz4a2IYc4fxKSaFhavr+s8aOjhsUe7w0j9gTu5uwhcYLRcpZvHR35MCYvsF53OzsikYqyAbkS/RMrHDIiUEpOJNim0PDkMzyM3ARECloFt2gbh8wn6wbDg9fAX/P/OofO9EF4t1qCvKDMyscVHZW/t293MnrjuRZWbCF0h98jpk7Bs8I8AAJDIAcKTmbs3c6a6x2pR+QlmplJMKJiLubDF4NHE+0hH2bnZ6HS6kNz1BQscNzWJrlI58jJUTuRrWbeUT0UbzhPsI3XMLJLLFVPMrY0CnUtjVzi1w1dGpqBppeEeDfdcvlbIrnLTsbDpbXyysTDpxbxzZpZHC+wsBntsy8Gz0+1T713DItTsPDGPSQN+LpXkW9I+g6c8sZoWnwNpcyqTwk3yYrvtQZnYxIwMrzFZqmjFZccsEXWQcZu1GbOpqYKc11AwKTBJoFGX6D1JBmVZQUPnW9L6mUb3YDln0Y1wDGkZy8Cu97hshk3P1mjjujDuG26yfK8Cap8ZrWCXysuT3p6muF4WQV2P16cBR4WaeRMsWNgdmhVJGah8ABgumchfwHJLyWAg7fPvufes2q/wRbiiMCG9SNNUghzCPBlju8vo2xRyQP3ofb0hlnYZ7ie1lJF0zsbSL6bNPFoVNNFq0BsI+BCjxYnp0VI4F4pq5DnXceW4vQMnU3GZJ8oB3xGxdua4fdjxnDVmWodY+sIAneMeJR0tsEDocLrzMcPnJ1BPENOPuW+KdLiv08Z/Q1oE+x0HfiEtPgmJbUmGPndThNMLtdc+SLJuD3ATNYDWoO17yHW8MevxADYfUKDscHqEpkmWDdPH+ir9oTB0C7D/ZRPTnJJgsrD JD7gvhIW 1eSGgo1FKDMq5/kgWLmktCd70SgIq1WjkGixGvJlhg+IsYwSB744tjt1c9OpVU6wlCkKQ6Oi0SypbKGLPWnU43mf9OFyypJTOsR2vszO/4YhBCYpphPplgUP/q4Qe2AZfTL9gWCuWukP3CIayGlngcVpKCxPwt/bCIbTV9KLXg8lisxrfGZxnMbN0/Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Aug 26, 2024 at 11:40:57PM GMT, Kent Overstreet wrote: > On Tue, Aug 27, 2024 at 04:36:33AM GMT, Matthew Wilcox wrote: > > On Mon, Aug 26, 2024 at 11:29:52PM -0400, Kent Overstreet wrote: > > > We had a report of corruption on nixos, on tests that build a system > > > image, it bisected to the patch that enabled buffered writes without > > > taking the inode lock: > > > > > > https://evilpiepirate.org/git/bcachefs.git/commit/?id=7e64c86cdc6c > > > > > > It appears that dirty folios are being dropped somehow; corrupt files, > > > when checked against good copies, have ranges of 0s that are 4k aligned > > > (modulo 2k, likely a misaligned partition). > > > > > > Interestingly, it only triggers for QEMU - the test fails pretty > > > consistently and we have a lot of nixos users, we'd notice (via nix > > > store verifies) if the corruption was more widespread. We believe it > > > only triggers with QEMU's snapshots mode (but don't quote me on that). > > > > Just to be crystal clear here, the corruption happens while running > > bcachefs in the qemu guest, and it doesn't matter what the host > > filesystem is? > > > > Or did I misunderstand, and it occurs while running anything inside qemu > > on top of a bcachefs host? > > The host is running bcachefs, backing qemu's disk image. > > (And I'm using nested virtualization for bisecting, it's been a lot to > keep straight). Also, the size of the missing data is not a power of two - it's not a single folio.