From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F4A7C19F32 for ; Fri, 7 Mar 2025 15:46:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5566B280001; Fri, 7 Mar 2025 10:46:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 505ED6B0083; Fri, 7 Mar 2025 10:46:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 38129280001; Fri, 7 Mar 2025 10:46:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 12C546B0082 for ; Fri, 7 Mar 2025 10:46:20 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id CF5D9A9294 for ; Fri, 7 Mar 2025 15:46:19 +0000 (UTC) X-FDA: 83195181678.26.B89EB7A Received: from mail-yw1-f173.google.com (mail-yw1-f173.google.com [209.85.128.173]) by imf22.hostedemail.com (Postfix) with ESMTP id 1DEB5C0015 for ; Fri, 7 Mar 2025 15:46:16 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=toxicpanda-com.20230601.gappssmtp.com header.s=20230601 header.b=O5GykKau; spf=none (imf22.hostedemail.com: domain of josef@toxicpanda.com has no SPF policy when checking 209.85.128.173) smtp.mailfrom=josef@toxicpanda.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741362377; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=24J3CynJ4fNToC40YYg0INXois46zfdfDDswNEU4WCw=; b=fnV8HfSiySPcM7r+MkrhXlTQexwSUxd23TR5x16NDUPBxa8zyrPczIUewjPdbjI8YVN+B8 7VOmwmcO+Ayu9TORPS5VQ3n70dsMCAfGssQ/XMJpsueBOySGDXkhAkTDe5QE4deW0orMp0 pTMQk8wDMnpEGB32y1WOZeHpLLaaYWU= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=toxicpanda-com.20230601.gappssmtp.com header.s=20230601 header.b=O5GykKau; spf=none (imf22.hostedemail.com: domain of josef@toxicpanda.com has no SPF policy when checking 209.85.128.173) smtp.mailfrom=josef@toxicpanda.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741362377; a=rsa-sha256; cv=none; b=DqDCmXKw5Zfa34Clh5Q4rOnN/kuTrUqvxH5dcD+p/NRFhA+V5Qcuz1Ms66/K7jK5ireEoT Bkf64wOGh+KuR4wh+cEthHzuqsVZSFuCUUBBRimO766mMG5F3EWCi7O9MDDN9Lgo4NjJT4 btfW+2Vkof+SnjP2vUaRoFieujABIYE= Received: by mail-yw1-f173.google.com with SMTP id 00721157ae682-6feafc707d3so19108197b3.1 for ; Fri, 07 Mar 2025 07:46:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20230601.gappssmtp.com; s=20230601; t=1741362376; x=1741967176; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=24J3CynJ4fNToC40YYg0INXois46zfdfDDswNEU4WCw=; b=O5GykKaut9zzKJ6CBO0qdgBA3eMBX7uHE2nyBNe3XXt4D3qxyhDuOL/MOg2SxQx531 FdoIqh1qZHHi2C41aOD7zKaSyooV0gW7AIZeEYB5uzHHCvrgCwiTEvzUEZYY8w/pORN1 +nxrwgYJLpFZYy6ADKinVvqy3kRGa/mnj7zIPsQZbC2+6LClbhCCW5L3FOtUNpUEeKw+ VpqRFwZwE1IiHdI36uRq4EL2PV1XXJUnjWoJ/1uXd+GzZx079SE1P80qogeFxBbBoJ3V jKsDiSPG4DxZmMIOe6yoTJOy5GMyDVPZhdoWMZY23lqmyv3q0MJxyPiHZclCpWELtieV ryXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741362376; x=1741967176; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=24J3CynJ4fNToC40YYg0INXois46zfdfDDswNEU4WCw=; b=nMapl2crRwRxcu/2zQn1h9xG1zjcl/ZpsQUai8vMN0n+qs7i3Y10FJpXKPifIIdVLM ODEmb2iyD7kd4XHSoxm/6NncyyN69v77BQ1py3GpuqJ+iq8WIsMagtA2Q2MnVgonsgcq 0RpqItykBs1zA4Xb/WFU70DKvsjsj3cuMAOoF8peaUNl6fSFPxBTQs3pqIg4GW2zaGrL fP8NbS9EX4tMlfLzbrBT9dTOKMMeMEgAprg0YXuJyQQWz9MoOFk9X9hNSc9VcpV5BNp0 XZaMQ11DaVx0AvPqsIsHKbRKQeVyEgqH/6qeIKMqazdYu4uZX0KVhelmx3aUVDah1Gz4 HEsQ== X-Forwarded-Encrypted: i=1; AJvYcCVeuKNMriyiiY5tFK7ftaVgyhpHXpHVFHF7Hcb6Zy74fPFk947oovfabfqlh/gDbFlHZ8zjBMeOpg==@kvack.org X-Gm-Message-State: AOJu0YxSPtXjUXJUGZotQwXqbZhZDx9DHYna7L+ds8PULKbUn85uNSA+ UhHR1opFC3xqXJHeqVqlSJnaF194KUHeW5iux8ITWfGW+Ste/GV04pJrHp+gvPE= X-Gm-Gg: ASbGncseQBkvxCKkqaGV9z1cH1O6xLgtXctd6LzlKH3elO9r27RA4t/11o8RepGrxx0 KbKSDSQ2bdR9bOv8PYFXPMsHqZjsP2ulSbXuGGVpH/9D+2RddllWLChIZnYnXzh1KTNDuijvdKL nZtlrn+DTXogmBYwStKw5ESGVIPc48BNFuzO25yzKM6NgohKrBmKgeofSDr7oFRrTEZzQ8C1wOC mhPgN4Mma+9rqpAbxJ/GSHPPd8/kl5rjrEt3JYgjdNqWpZiksdiaF3U7YeYrRU5oFAoZCMe0ZOi vc1kjUw3sMzf+X2duW8BZh9cVNqixRhCdEsZDGcwK0vv9ylwl6gSHz4gY+8h+lE87u/QHor1dCr JMxFexw== X-Google-Smtp-Source: AGHT+IFbO7B7I5QW8p7YyFhEoZvye5dOa0V+vkMhRA5dzshNa6dL/BbXoDUHXepaAKDlZqjdHG+27w== X-Received: by 2002:a05:690c:3386:b0:6fb:9474:7b5f with SMTP id 00721157ae682-6febf2eb32dmr60526787b3.14.1741362375930; Fri, 07 Mar 2025 07:46:15 -0800 (PST) Received: from localhost (syn-076-182-020-124.res.spectrum.com. [76.182.20.124]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6feb2c4676fsm7827907b3.103.2025.03.07.07.46.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Mar 2025 07:46:14 -0800 (PST) Date: Fri, 7 Mar 2025 10:46:14 -0500 From: Josef Bacik To: Amir Goldstein Cc: Jan Kara , syzbot , akpm@linux-foundation.org, axboe@kernel.dk, brauner@kernel.org, cem@kernel.org, chandan.babu@oracle.com, djwong@kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-xfs@vger.kernel.org, syzkaller-bugs@googlegroups.com Subject: Re: [syzbot] [xfs?] WARNING in fsnotify_file_area_perm Message-ID: <20250307154614.GA59451@perftesting> References: <67a487f7.050a0220.19061f.05fc.GAE@google.com> <67c4881e.050a0220.1dee4d.0054.GAE@google.com> <7ehxrhbvehlrjwvrduoxsao5k3x4aw275patsb3krkwuq573yv@o2hskrfawbnc> <20250304161509.GA4047943@perftesting> <20250304203657.GA4063187@perftesting> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 1DEB5C0015 X-Stat-Signature: ibgkwpc6reoct1a98onrhrxkcarw8a8x X-HE-Tag: 1741362376-984917 X-HE-Meta: U2FsdGVkX1+5S0V0t7Rz/rrZPqXHnCFMn26gm377S4IW4xbftN2GQOHB4fPyILt1ylTKe7y9kDZxcziGLCr1wrOPp1bKl45V+KKjwqdnm/BVeP5ZIePRmT0bX74MRnYS3nr8ocUzRpKOZFsPdGmkRvK1FZQM/RJ2WY8aFW1QJOsJDxxJuflyjRSPWbque+wvr+e115Tmrn00oNf2R5W+qk6btHYdunsNF0+Fi4Iaf1mMS0DM+29qOYUcFxOD0xBSaVHgo2aLD9pkYNlu8DYKW2ToFFM0B+xkU9TOVBFgSBU2OvwTzXqUuO55nnRl+R06ssa/lA4m+Y92EXhOjQ3a9gvvZWY8fmK0HqTM1RRPVecF9SnOeVVBhdbhM6B7bGnHMNN3QG3kBgCrxnJ4lEtB63V+jW04JdN8IUfJaoeBytUrThHdCNGMPon03phoUsMVsLyzS/vpDiz182VqRXGRyALhwQ4J1UzgCm/0bzvv7gCfIhMdrTtRarou1ymL48VWZfHb+xxBuKsp2UP116JKIFPg15rzIccguFt5VcykygdjxtRB7798iiYYk204FyI9ARx7jDqcZkmGGry8Q8H/b1jwVHm/68pG+lNcLs371XcJpfIxwm3lK7ZFaGeUtJNYPaVoEii/KaU2L9/drv5t5PW8x0a6/WHvDW0+L2Ct2YSvelEdjIYyRKFFYfWlrOFBKqGsO3MQrqGypk2EKObLC2bGQeqtOOfAXybusRaLCCSmGirlU9nXAw0Tqa9gOU72j4W1Gxn5oRP8LtMX2ncK5xPismrIZpWXHMgPxBmnP3Zy8vCkZogdthg+4PUCVIk2PNizwattYJHfRMriEg4tOo+AZC1SnhMBwDuaUE0AsP/AdDBcTh7OER4UDl+klF1nJz7Jhvk8gluneU7vvUkYk30CWLdiJNgAGObSwaoOWCzbrMIBWV+a8hXTQLgFFHI+UOHQgmIypMei6iCRSYU Ss57eDgA 16X4st9JJ6p3SmrsYau3THPOYa633NW7tPBW3PaurF4oPRvB1yqF75yFx6014Ui9/na7ybH43NrIkuipEesZNkeHJ+AFylZCrIgQlMTtHw8TsLOS9Ta9+CLoLA7GoVY2yQPiHsxjYasgkSCq6ISXiC2mVrON+WdZQzHp4Hb+O86K2AWx/qZs+EGyNYRbgZgqZXctdRQ9oNAcwzI3z5em43U4tzXXQiitP+JeTNyQ8qj/16C8dPnNIu3i8Ao3GDS9rdC9BWvS3rp5JrdGpn0ap4oVHNM3E9YQE4feiSelseGTgtzQ/cEs+vDZpVlq+Pv8YQmav6zR9ndxViqUVjLpoc29DywssCb6/7Zj43AuYcS7kycq1g0U57VyzLYLvffyIYo9hKQxSCiH65K1RcHQlHW75XwjZ6Ea0hJNrjy+Il63EiKt6phHwYJctEsnumrx8LrrxE9ZrP0hzStc2m8P/4wwpdqLvHI5rgb6MZ+PcDjfCfNEiW1Bna0asfhLW94c9/Iei4Qh8RSdveiVf55NnRcIw9+w4YM1PwUgXBXG2dO3ekGnQEBr63cAppck8r7oAZlAbfF0fr5CYdY1DPHNuTvMgs7JwIQIMNGV4Jv2Cu6t5Yom+OiUrSM4HnR9khJ9rFXlmfOIOw0r0LP05MPLFz1DY/I317HkCswOyCRYt1kO5/s8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000008, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Mar 04, 2025 at 10:13:39PM +0100, Amir Goldstein wrote: > On Tue, Mar 4, 2025 at 9:37 PM Josef Bacik wrote: > > > > On Tue, Mar 04, 2025 at 09:27:20PM +0100, Amir Goldstein wrote: > > > On Tue, Mar 4, 2025 at 5:15 PM Josef Bacik wrote: > > > > > > > > On Tue, Mar 04, 2025 at 04:09:16PM +0100, Amir Goldstein wrote: > > > > > On Tue, Mar 4, 2025 at 12:06 PM Jan Kara wrote: > > > > > > > > > > > > Josef, Amir, > > > > > > > > > > > > this is indeed an interesting case: > > > > > > > > > > > > On Sun 02-03-25 08:32:30, syzbot wrote: > > > > > > > syzbot has found a reproducer for the following issue on: > > > > > > ... > > > > > > > ------------[ cut here ]------------ > > > > > > > WARNING: CPU: 1 PID: 6440 at ./include/linux/fsnotify.h:145 fsnotify_file_area_perm+0x20c/0x25c include/linux/fsnotify.h:145 > > > > > > > Modules linked in: > > > > > > > CPU: 1 UID: 0 PID: 6440 Comm: syz-executor370 Not tainted 6.14.0-rc4-syzkaller-ge056da87c780 #0 > > > > > > > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 12/27/2024 > > > > > > > pstate: 80400005 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) > > > > > > > pc : fsnotify_file_area_perm+0x20c/0x25c include/linux/fsnotify.h:145 > > > > > > > lr : fsnotify_file_area_perm+0x20c/0x25c include/linux/fsnotify.h:145 > > > > > > > sp : ffff8000a42569d0 > > > > > > > x29: ffff8000a42569d0 x28: ffff0000dcec1b48 x27: ffff0000d68a1708 > > > > > > > x26: ffff0000d68a16c0 x25: dfff800000000000 x24: 0000000000008000 > > > > > > > x23: 0000000000000001 x22: ffff8000a4256b00 x21: 0000000000001000 > > > > > > > x20: 0000000000000010 x19: ffff0000d68a16c0 x18: ffff8000a42566e0 > > > > > > > x17: 000000000000e388 x16: ffff800080466c24 x15: 0000000000000001 > > > > > > > x14: 1fffe0001b31513c x13: 0000000000000000 x12: 0000000000000000 > > > > > > > x11: 0000000000000001 x10: 0000000000ff0100 x9 : 0000000000000000 > > > > > > > x8 : ffff0000c6d98000 x7 : 0000000000000000 x6 : 0000000000000000 > > > > > > > x5 : 0000000000000020 x4 : 0000000000000000 x3 : 0000000000001000 > > > > > > > x2 : ffff8000a4256b00 x1 : 0000000000000001 x0 : 0000000000000000 > > > > > > > Call trace: > > > > > > > fsnotify_file_area_perm+0x20c/0x25c include/linux/fsnotify.h:145 (P) > > > > > > > filemap_fault+0x12b0/0x1518 mm/filemap.c:3509 > > > > > > > xfs_filemap_fault+0xc4/0x194 fs/xfs/xfs_file.c:1543 > > > > > > > __do_fault+0xf8/0x498 mm/memory.c:4988 > > > > > > > do_read_fault mm/memory.c:5403 [inline] > > > > > > > do_fault mm/memory.c:5537 [inline] > > > > > > > do_pte_missing mm/memory.c:4058 [inline] > > > > > > > handle_pte_fault+0x3504/0x57b0 mm/memory.c:5900 > > > > > > > __handle_mm_fault mm/memory.c:6043 [inline] > > > > > > > handle_mm_fault+0xfa8/0x188c mm/memory.c:6212 > > > > > > > do_page_fault+0x570/0x10a8 arch/arm64/mm/fault.c:690 > > > > > > > do_translation_fault+0xc4/0x114 arch/arm64/mm/fault.c:783 > > > > > > > do_mem_abort+0x74/0x200 arch/arm64/mm/fault.c:919 > > > > > > > el1_abort+0x3c/0x5c arch/arm64/kernel/entry-common.c:432 > > > > > > > el1h_64_sync_handler+0x60/0xcc arch/arm64/kernel/entry-common.c:510 > > > > > > > el1h_64_sync+0x6c/0x70 arch/arm64/kernel/entry.S:595 > > > > > > > __uaccess_mask_ptr arch/arm64/include/asm/uaccess.h:169 [inline] (P) > > > > > > > fault_in_readable+0x168/0x310 mm/gup.c:2234 (P) > > > > > > > fault_in_iov_iter_readable+0x1dc/0x22c lib/iov_iter.c:94 > > > > > > > iomap_write_iter fs/iomap/buffered-io.c:950 [inline] > > > > > > > iomap_file_buffered_write+0x490/0xd54 fs/iomap/buffered-io.c:1039 > > > > > > > xfs_file_buffered_write+0x2dc/0xac8 fs/xfs/xfs_file.c:792 > > > > > > > xfs_file_write_iter+0x2c4/0x6ac fs/xfs/xfs_file.c:881 > > > > > > > new_sync_write fs/read_write.c:586 [inline] > > > > > > > vfs_write+0x704/0xa9c fs/read_write.c:679 > > > > > > > > > > > > The backtrace actually explains it all. We had a buffered write whose > > > > > > buffer was mmapped file on a filesystem with an HSM mark. Now the prefaulting > > > > > > of the buffer happens already (quite deep) under the filesystem freeze > > > > > > protection (obtained in vfs_write()) which breaks assumptions of HSM code > > > > > > and introduces potential deadlock of HSM handler in userspace with filesystem > > > > > > freezing. So we need to think how to deal with this case... > > > > > > > > > > Ouch. It's like the splice mess all over again. > > > > > Except we do not really care to make this use case work with HSM > > > > > in the sense that we do not care to have to fill in the mmaped file content > > > > > in this corner case - we just need to let HSM fail the access if content is > > > > > not available. > > > > > > > > > > If you remember, in one of my very early version of pre-content events, > > > > > the pre-content event (or maybe it was FAN_ACCESS_PERM itself) > > > > > carried a flag (I think it was called FAN_PRE_VFS) to communicate to > > > > > HSM service if it was safe to write to fs in the context of event handling. > > > > > > > > > > At the moment, I cannot think of any elegant way out of this use case > > > > > except annotating the event from fault_in_readable() as "unsafe-for-write". > > > > > This will relax the debugging code assertion and notify the HSM service > > > > > (via an event flag) that it can ALLOW/DENY, but it cannot fill the file. > > > > > Maybe we can reuse the FAN_ACCESS_PERM event to communicate > > > > > this case to HSM service. > > > > > > > > > > WDYT? > > > > > > > > I think that mmap was a mistake. > > > > > > What do you mean? > > > Isn't the fault hook required for your large executables use case? > > > > I mean the mmap syscall was a mistake ;). > > > > ah :) > > > > > > > > > > > > Is there a way to tell if we're currently in a path that is under fsfreeze > > > > protection? > > > > > > Not at the moment. > > > At the moment, file_write_not_started() is not a reliable check > > > (has false positives) without CONFIG_LOCKDEP. > > > > > One very ugly solution is to require CONFIG_LOCKDEP for > pre-content events. > > > > > Just denying this case would be a simpler short term solution while > > > > we come up with a long term solution. I think your solution is fine, but I'd be > > > > just as happy with a simpler "this isn't allowed" solution. Thanks, > > > > > > Yeh, I don't mind that, but it's a bit of an overkill considering that > > > file with no content may in fact be rare. > > > > Agreed, I'm fine with your solution. > > Well, my "solution" was quite hand-wavy - it did not really say how to > propagate the fact that faults initiated from fault_in_readable(). > Do you guys have any ideas for a simple solution? Sorry I've been elbow deep in helping getting our machine replacements working faster. I've been thnking about this, it's not like we can carry context from the reason we are faulting in, at least not simply, so I think the best thing to do is either 1) Emit a precontent event at mmap() time for the whole file, since really all I care about is faulting at exec time, and then we can just skip the precontent event if we're not exec. 2) Revert the page fault stuff, put back your thing to fault the whole file, and wait until we think of a better way to deal with this. Obviously I'd prefer not #2, but I'd really, really rather not chuck all of HSM because my page fault thing is silly. I'll carry what I need internally while we figure out what to do upstream. #1 doesn't seem bad, but I haven't thought about it that hard. Thanks, Josef