From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B509C7EE31 for ; Fri, 27 Jun 2025 11:51:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 995626B00A3; Fri, 27 Jun 2025 07:51:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9453C6B00BE; Fri, 27 Jun 2025 07:51:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 834666B00BF; Fri, 27 Jun 2025 07:51:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6F4946B00A3 for ; Fri, 27 Jun 2025 07:51:25 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id F056F1A033F for ; Fri, 27 Jun 2025 11:51:24 +0000 (UTC) X-FDA: 83601015288.03.92C41BF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id E8EB94000F for ; Fri, 27 Jun 2025 11:51:22 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=FP9lq+o+; spf=pass (imf11.hostedemail.com: domain of bfoster@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=bfoster@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1751025083; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9uBNunHvRRBuDidf3xDCcu/JTINehRAr/oGm6RoL/Uo=; b=WG+B6FEgYcsMQxZ9Xv22xMY6yfYg6QuQm2PVFi2PcmT/7Q6ZRhpqwcgZwKTRi41Eew3zAY Tu+acPL0fRdD0nclyhilo/8RZOtpsxp4RujCX/p45kxRXhWfwxaYag9/UWwmVFhZ61B/C1 02uzRiHzvnc6hwdZiNJaRnXbzvEnXfo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1751025083; a=rsa-sha256; cv=none; b=OxVMiTXuuPJrBK4fE7pwiVsEqJsnrYJBAxC2rt+W0ePv193bB0s6GEsxAFspUc174b2dqF q8/d88HWEGyw+dcwRc0XDXjau7gYwPOqSMDMoKX4X7OLZOVN7eLRVxdzaq40lUsxM9VR9q VqLcDr4t/UPzGW58kTGBZmtrCgWo+KU= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=FP9lq+o+; spf=pass (imf11.hostedemail.com: domain of bfoster@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=bfoster@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751025082; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=9uBNunHvRRBuDidf3xDCcu/JTINehRAr/oGm6RoL/Uo=; b=FP9lq+o+39wKquuLRLJ66bW2EHQ2x+fjg6/2dAC/tFuYJfiIEZ/68kGVJUV5OgySyy5+3f l5TyiANxjcI/L2VtwxAD1+wrYsB6rTDlhwujY45o3mlkxtXA1F3+qPQdj37fOadjBRwGda HXqZiHHNUBuly1SMPSi449Lb0YsAGgw= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-608-uoK2ZjBYO321nAut1aNS_w-1; Fri, 27 Jun 2025 07:51:16 -0400 X-MC-Unique: uoK2ZjBYO321nAut1aNS_w-1 X-Mimecast-MFC-AGG-ID: uoK2ZjBYO321nAut1aNS_w_1751025075 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7C35419373D7; Fri, 27 Jun 2025 11:51:15 +0000 (UTC) Received: from bfoster (unknown [10.22.64.142]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5ADC519560A7; Fri, 27 Jun 2025 11:51:14 +0000 (UTC) Date: Fri, 27 Jun 2025 07:54:52 -0400 From: Brian Foster To: Baolin Wang Cc: Hugh Dickins , Matthew Wilcox , linux-mm@kvack.org Subject: Re: [PATCH] tmpfs: zero post-eof folio range on file extension Message-ID: References: <20250625184930.269727-1-bfoster@redhat.com> MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Ezi3ZX3Jt71z9FFisQCaLgH6hSmOB2xI4Yrq7VqbME4_1751025075 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Rspamd-Queue-Id: E8EB94000F X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: wy8sa1jiwow76853797bd84f5raaw6e6 X-HE-Tag: 1751025082-178086 X-HE-Meta: U2FsdGVkX1/nzn0QuCugBXJ1+trEQFIsIabVdNluOTut+9usY/R4FHgg6OO1pei249B0GdxdZbj3GcNaIWLbNpYFupqV8FHJKkk3M+ALpkd913DOa/ffIe65ujy2V7tL8ZtolovgXjz41k14rAR9W1C8zk842K+qh7SmhJw4Ry70KoPyQY+s18Gr+/CSzTXmtE37r61PwOiML47jEni48S4xE99Xu3ufPZ0lK7ZxbXuCjFdAr/cI5RXHswGs7kpgMijUc9TqdoHG1z+iD+fD6mkJDCbm+z+Fi4NqgIFhzjaXggflaoJtZhw1mls9zptgErs583iJ2nl/WgluGk9b+UL/IZlT7DVUtfnT9bneIA4Lb33yzNwvJokHhrWIkYIJjrrePuvFiht2Fz5xoLI3sHEZpJbV5sJT9guLQQZ51kvms0upfw+hewKDmouqBmBzMncU4NnHtuak5dsC/u3mY1YwSqpkiMjdGR/awMERhOX9gayi8M5Hf/CQl4J4PmTCdPWwHckthAQr0JuidpQWSnqDdieezaQ/v+IbsDX23JyzasQTabhRlwyVifMgLiMLzG598UsGTgxcLQQgtWZPH7tdeMZ+h2OSHVNyHolpCl0+1ec+819ZnD9OlWzcnwvTBD3hy7llWSsDmTRoJG82OBcVfcnBPuTOOZc5HH/vN45HCJlSl8CHvj/ZrgLWCOSqroY46DMig6ThGq+GhDX31pxGRb7/Y+kqLaQrV22Fwb3NLAdjLVa+8+YO+qGZY6K2eiIP7K9uYAztKbAihqUyq9/KaUHY9m15eSRYIj6+Qe54A3XjGrYyMgpgmQVtS+X9I15hLDB4Jfc2jEggahNhX/rVTITaN+g/SaGAugzEkqwZDvWxhkF0go59kxCGwlljBhuEDyTCa3Fh0e+FtkCiQqHtHbe6q6FQA9AjUA+cDbFc8ynfkKF8MgPmALtC+/C2iQ0M7Yw3BU/r/k/XiUi Y8ITimMX pniNsMCRaQ0zzRUThLK4V2rWKv3X9GOum92o9pmm71owF8yvKMarP4jCNhr5d0iHs0kx5/7Z+qcUvf2kWjxmjGio47MCog8vWfLNBQhWPTNVmjcCuik2/jNK/LRsaeGbxJq+5cEAxenR2B0Xw6MBoZkAY/mBtnjo6/zSkbok5imBob8r9yx2hRNggBICngt5gpItztoU7PeEt9lVMAdXy+x8/UXFqX4tQV9wU/unS/3lrOwHqxiHBUjN2ttHsurmKJe8cRafsKrhTdLnP0Vv7zFugHv7atCV60IOcFXeKHm3M9h17KxoNLgg73PJXna39cqNvtnFNKMzkqIQmC+NvVrC5uDtKRv9eEzYp1h5u5VylsL2Q4GAUrFYCurgxKIuQEEyBAousj8TG2NBKJq3QkIJKiemvoOPhmLbIQCY1dG64XcZ+SB2Uk81TT+8eawozeQr2HldsUixvsu/sA4ZreX/eyO4Hpan7i3PR6PEy6ygA9xU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jun 27, 2025 at 11:21:56AM +0800, Baolin Wang wrote: > > > On 2025/6/26 20:55, Brian Foster wrote: > > On Wed, Jun 25, 2025 at 10:35:44PM -0700, Hugh Dickins wrote: > > > On Wed, 25 Jun 2025, Matthew Wilcox wrote: > > > > On Wed, Jun 25, 2025 at 02:49:30PM -0400, Brian Foster wrote: > > > > > Most traditional filesystems zero the post-eof portion of the eof > > > > > folio at writeback time, or when the file size is extended by > > > > > truncate or extending writes. This ensures that the previously > > > > > post-eof range of the folio is zeroed before it is exposed to the > > > > > file. > > > > > > > > > > tmpfs doesn't implement the writeback path the way a traditional > > > > > filesystem does, so zeroing behavior won't be exactly the same. > > > > > However, it can still perform explicit zeroing from the various > > > > > operations that extend a file and expose a post-eof portion of the > > > > > eof folio. The current lack of zeroing is observed via failure of > > > > > fstests test generic/363 on tmpfs. This test injects post-eof mapped > > > > > writes in certain situations to detect gaps in zeroing behavior. > > > > > > > > > > Add a new eof zeroing helper for file extending operations. Look up > > > > > the current eof folio, and if one exists, zero the range about to be > > > > > exposed. This allows generic/363 to pass on tmpfs. > > > > > > > > This seems like what I did here? > > > > > > > > https://lore.kernel.org/linux-fsdevel/20230202204428.3267832-4-willy@infradead.org/ > > > > > > > > Which fix should we prefer? > > > > > > > Quite similar, indeed. This is actually about the same as my initial > > prototype when I was just trying to confirm the problem for truncate. As > > Hugh notes, we still need to cover other size extension paths > > (fallocate, buffered write). > > > > > Thank you Brian, thank you Matthew. > > > > > > Of course my immediate reaction is to prefer the smaller patch, > > > Matthew's, but generic/363 still fails with that one: I need to look > > > into what each of you is doing (and that mail thread from 2023) and > > > make up my own mind. > > > > > > > FWIW, I started off with something trying to use shmem_undo_range() and > > eventually moved toward the current patch because I couldn't get it > > working quite right. Explicit zeroing seemed like less complexity than > > calling through undo_range() on various operations, primarily due to > > less risk of unintentional side effect. It's possible (likely :P) that > > was just user error on my part, so now that I have a functional patch I > > can revisit that approach if it is preferred. > > I also tried using shmem_truncate_range() to fix this issue, but I failed. > Ultimately, at least for me, I think the fix is simple and works. > Ok, thanks. > > However one thing I wasn't aware of at the time was the additional > > zeroing needed into the target range on fallocate, so that might require > > care to avoid not immediately punching out the folios that were > > fallocated just prior. I suspect this would mean we'd need a helper or > > something to restrict the range to undo to just the eof folio. That > > seems like a plausible approach, I'm just not so sure the final result > > will end up much smaller or simpler than this patch. > > > > > (I'm worried why I had no copy of Matthew's 2023 patch: it's sadly > > > common for me to bury patches for long periods of time, but not > > > usually to lose them completely. But that is my worry, not yours.) > > > > > > I haven't been much concerned by generic/383 failing on tmpfs: > > > but promise to respect your efforts in due course. > > > > > > > It's certainly not the bug of the century. ;) I added the test somewhat > > recently because we had bigger zeroing issues on other filesystems and I > > noticed we had no decent test coverage. > > > > > I procrastinate "in due course" because (a) the full correct answwer > > > will depend on what happens with large folios, and (b) large folio > > > work in 6.14 changed (I'll say broke) the behaviour of huge=always > > > at eof (I expected a danger of that, and thought I checked before > > > 6.14-rc settled, but must have messed up my checking somehow). > > > > > > > Interesting.. I assume huge=always refers to a mount option. I need to > > give that a test. > > I tested your patch by adding the 'transparent_hugepage_tmpfs=always' > command line parameter, which will change the default huge policy to > 'huge=always' for tmpfs mounts. Your patch also passed the generic/363 test > with the tmpfs 'huge=always' mount option. > Thanks. I ran a full auto fstests run with huge=always and an uptodate tweak yesterday and also didn't see any regressions. My change was pretty much the same as yours below other than I inverted the logic. I do see two new test failures with huge=always in generic/285 and generic/436, but that appears to be related to the mount option and not the patch. Both of those seem to be seek related, FWIW. > > I'm also curious if either of you have any thoughts on the uptodate > > question. Does filtering zeroing on uptodate folios ensure we'd zero > > only folios that were previously written to? > > Yes, I think so. Caller will handle the !uptodate folios. So I change your > patch to: > > static void shmem_zero_eof(struct inode *inode, loff_t pos) > { > struct folio *folio; > loff_t i_size = i_size_read(inode); > size_t from, len; > > folio = shmem_get_partial_folio(inode, i_size >> PAGE_SHIFT); > if (!folio) > return; > > if (folio_test_uptodate(folio)) { > /* zero to the end of the folio or start of extending > operation */ > from = offset_in_folio(folio, i_size); > len = min_t(loff_t, folio_size(folio) - from, pos - i_size); > folio_zero_range(folio, from, len); > } > > folio_unlock(folio); > folio_put(folio); > } > > > > So there's more (and more urgent) to sort out before settling on > > > the right generic/383 fix. > > However, let's still wait to see if Hugh has any better ideas. > Ack.. I'll give it a bit and then follow up with a v2 later unless I hear otherwise. Thanks again, appreciate the feedback. Brian