From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59E83C02199 for ; Sat, 8 Feb 2025 00:23:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF4AD280004; Fri, 7 Feb 2025 19:23:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C7DDB280001; Fri, 7 Feb 2025 19:23:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B1E4A280004; Fri, 7 Feb 2025 19:23:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 90CE4280001 for ; Fri, 7 Feb 2025 19:23:10 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 46C0D121155 for ; Sat, 8 Feb 2025 00:23:10 +0000 (UTC) X-FDA: 83094877740.21.B6AC1D0 Received: from mail-qt1-f182.google.com (mail-qt1-f182.google.com [209.85.160.182]) by imf24.hostedemail.com (Postfix) with ESMTP id 4043E180010 for ; Sat, 8 Feb 2025 00:23:08 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=LhWDfucX; spf=pass (imf24.hostedemail.com: domain of joannelkoong@gmail.com designates 209.85.160.182 as permitted sender) smtp.mailfrom=joannelkoong@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738974188; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wMMA64lDvybDlSuA0rt3djW23m9gTKqcOUi9JQbLAKk=; b=ciWq8vfOh27OncyFiwUc/hn7kOvSXmzJU4RCQ7mIyJDFWK5egrxwJCfpsT/QA5kgGIWHwt bXmwtkvwNFB/vpdDIUvkfFNZRO/4ptRZPQiAFa4UNef/RCwsNnV3RidjSsJPtL335n4W6e khMRuY2L9D3nRAwXMgkIJRFSEiTDMlU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738974188; a=rsa-sha256; cv=none; b=dCtFHasHLo8kePcuAOukLKuESsDPw/9pYZx7dPLtRdRyVhOfWFcVaVFKGVtDr2M/wAFlDw ZwKcAUdk0M4ZKLLdUNvdnG8AZ3+/MOXmzej0fhdLPTkin0vsNfxlGSN2oPvurFjYPcJAHf A5FMbe3VARSXnowxByOA25FrQacZvgQ= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=LhWDfucX; spf=pass (imf24.hostedemail.com: domain of joannelkoong@gmail.com designates 209.85.160.182 as permitted sender) smtp.mailfrom=joannelkoong@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-qt1-f182.google.com with SMTP id d75a77b69052e-467a3c85e11so17014591cf.2 for ; Fri, 07 Feb 2025 16:23:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738974187; x=1739578987; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=wMMA64lDvybDlSuA0rt3djW23m9gTKqcOUi9JQbLAKk=; b=LhWDfucXEN6AsHSt7g4Kaqb1teVlH1wZhCIqJkGnryx0YlD3ce8RLfLtgqVJ2vA9W3 lHwRE8k1Oj0qlvUFcA3X7m2COecSfk4dFWJ88Mi4YMJEgx+HR2nVBDDEySBmzRtuVlpm D3YQ+9OFMxofy4OUxzGCzg+XJ+ZmP/5O4Kz5tZQMJIHQz708BfEatnhg96Cbk9LXB08i F7TOWyHqszYhV6h3FDTPM4aN0kxDIszM3XQfALBl4wcDTvE2FPAb5Mo1ggqv5upmfkby k/AY7NcBNiUpwHFF28tXhfnfA0O5IVDUuh02xC4FL9jQfpzEGLESc8Bnwzn5K/Pfo2Dz ur8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738974187; x=1739578987; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wMMA64lDvybDlSuA0rt3djW23m9gTKqcOUi9JQbLAKk=; b=tAVYpbZovTxKlGnbXIbKPpr/W6CisCZrvLMbfvdth6bY/BAhcG7c8JKceDhBhb35lE UsiTfZSf9iLCyF6/ElflVPEFwhJlUXesJFGn5ZjVkmr/4GW3DEO7ZFm97ihkUZcxA523 hH1QhP8NcbNp1iSnRhE9Su+kk16JuoyXYkzkkOX1v+ZpxvoOzwZ33a9aGGMpDQKrAAHH Ml/pF1vXBdSn+vxDt4jEUmVAuAr7eaPDFP3fMiMsEk+hK5W5hE7VUm8KynD+7njlT3A5 gFlVhxABsJPr/jlnIGFZFvQ7AU8B+Kvi8Zva2hycliMVKgkpDcG0fpn/H+KoEo3/8oIs yv2g== X-Forwarded-Encrypted: i=1; AJvYcCV4HIHjCSVJXq81vMYoE2FtlqbirmjZt81tWWfWNPeiItpobV/dHjyxbK1zmP2EX4azry2MGVg2Kw==@kvack.org X-Gm-Message-State: AOJu0YwlnJGlzNalrN7GblU9MTD4raQo9DUFNeDAQy9Y+YaYYWo+VJVo vUb8/mTlYntHFMMg/fdFZCbmPh9Jbvw2BzNdiI7FQoyDhAaEvOdA4ytM93hm8wwEPR1+uayD6ww srzgsijDzg5ryu1iNJHuWqHCFr2w= X-Gm-Gg: ASbGncvi+m9c0jAN0qIaAa+34Hcx+LOQj21QUgy4WiiK/Aobj8Px8GizH9yRKV4VDV6 MAWiDSrEtEfe+omfGVszQclX82Iz/19zW5InVMk7HCj2enX7dTiCRyLf87UF9U1ka4vW9+1X4zg == X-Google-Smtp-Source: AGHT+IFynXnfoPvMksQZzaLkrmUUnGS4bSe4CamOZn8jJ/vYzyH5WcI7y0Y0eR9P7OnCEJANNBRyyGlLcn/pqDWXcug= X-Received: by 2002:a05:622a:1802:b0:468:fb3c:5e75 with SMTP id d75a77b69052e-47167b31c04mr90216311cf.38.1738974187204; Fri, 07 Feb 2025 16:23:07 -0800 (PST) MIME-Version: 1.0 References: <2f681f48-00f5-4e09-8431-2b3dbfaa881e@heusel.eu> <9cd88643-daa8-4379-be0a-bd31de277658@suse.cz> <20250207172917.GA2072771@perftesting> <8f7333f2-1ba9-4df4-bc54-44fd768b3d5b@suse.cz> In-Reply-To: <8f7333f2-1ba9-4df4-bc54-44fd768b3d5b@suse.cz> From: Joanne Koong Date: Fri, 7 Feb 2025 16:22:56 -0800 X-Gm-Features: AWEUYZnFp_ULZNnOUQ3w9lhp7jHrWUkkYz1PD0CpILo4xK1xPJS4vjBA4OF3cJg Message-ID: Subject: Re: [REGRESSION][BISECTED] Crash with Bad page state for FUSE/Flatpak related applications since v6.13 To: Vlastimil Babka Cc: Josef Bacik , Miklos Szeredi , Christian Heusel , Miklos Szeredi , regressions@lists.linux.dev, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Matthew Wilcox , linux-mm , =?UTF-8?Q?Mantas_Mikul=C4=97nas?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Queue-Id: 4043E180010 X-Rspamd-Server: rspam07 X-Stat-Signature: zh1w3e6gn1s88z9kx75998b9xti71dd8 X-HE-Tag: 1738974188-137635 X-HE-Meta: U2FsdGVkX1+XjSMZ316fYWX/3wdiUz1iAmyxsXwaZTO7q91p/PFx0UiG57srOUF2qx0Vh6U6UEMb7876urhzoLxtQyHeKBJdTyj3MmiCvMPb/IAD5UPg4QExo3TPBMu2/qcCUvuinGfVZaNDO2CCnqaXe9fzzzA+OIdl+koq66gMilUvFi4zr5yaCQJzewHEI6T/JeeSzZomw6C/2X/1a0l9zHCyBnrwAF96cz/xnCy2zpbfIXkTvaabO1P/TFcdtHnHCENrvSRhi6Fthu8zsMe7fe6FM/uLwXanNR0D6mh+WnlKz1seY3HhxSs3Yt1x7p9g/ocWct9qT0vD/ZqmyaIdSBaj/0pKw4M85z+Hi8WzWVHjk/MSlKFZ2njbynCj9w+VI9yM1RQMP50Xm/c4iQI2DQRgBA9tFoi92U+NdSKOO40bzCXebh2AKgaLhQD5G4BA7890z9DBVJzecyGqVBg7Osgq1dBp1Ol8sK1bwmKp8Aif1GzRCtxGZ7xqfoaIj3j47UE3+xMcZBUXtFGO7JboeLxpfJcvrQ0OgnXuSv5xnGxk+0oBDFrpgyuE6Ocj27c2MdmBlKgBextsexo9o5LBoLPj3mNjCnow5NN7Kz7aJK7Q2357QjYDLWxwh4h0UvcL5jZhE7EFVUsOds/EuhcveUGINxdrhUespelFLLYeVrxDoQaxH+ttlm4m+WFONy/2n6wCbYqf/DO33qu3DSHYollEQ5iwIk3D0g+LFJdiVYvb3PmuwQJXH56AwXllwMlN1BDBz5F+dtRNXAfCvy0BG0xyna6xZPbgXfzTXqU55wRxhHD5AVxHjg+n2uYHmUYIB6h9iHWHzyXNA+G7UGcrhD4ApIFZa/gu2aX1B4fHzQxjdYSr/0qLG6KU5HOFL1sRlVFHPGOiBL8qOcIYZ5LuKkA+w+k4WEJX7Lv2whbg1DXmyr7I2MfV/NHGPL5GkT5xNbxc1Bx53djOS4D bJm0JCZz KN2eIGrUd1NhKFgZbKgIjXwPlxgMOlFIyUuSUcN/DGTgYUVu+P22EssAsS5YkUmaiNuckcSgYsTDZuoF0BJFyEJP6TJEz/+L4l6FaK2LjoMbcOakz3Wqtnl+zygwIlZMxec/VZ/uk9oIMKWl6bQqWuiR/DZYkd+C0hA8aizT97+89QrXNIjtuAfFr+14jArtVC/UeS+wk3NfDJl6cCr3iaPKQFdlw8zX5nqG4GXEq3YqIbIyQaR5WlS1JvcIez43spU6lgTVY8ZRJGEuRmr+4pRicBHYynDe5weuHEflr3VoEA4T+lQv7z6oA7lDoh5FQyfuiteS2Ms3P1Ogwi9zSnanruhkiJpguQdEtbV55iUuDDn69V9tNVbJb0dG3xyybkg0q X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Feb 7, 2025 at 10:39=E2=80=AFAM Vlastimil Babka wr= ote: > > On 2/7/25 18:29, Josef Bacik wrote: > > On Fri, Feb 07, 2025 at 05:49:34PM +0100, Vlastimil Babka wrote: > >> On 2/7/25 10:34, Miklos Szeredi wrote: > >> > [Adding Joanne, Willy and linux-mm]. > >> > > >> > > >> > On Thu, 6 Feb 2025 at 11:54, Christian Heusel = wrote: > >> >> > >> >> Hello everyone, > >> >> > >> >> we have recently received [a report][0] on the Arch Linux Gitlab ab= out > >> >> multiple users having system crashes when using Flatpak programs an= d > >> >> related FUSE errors in their dmesg logs. > >> >> > >> >> We have subsequently bisected the issue within the mainline kernel = tree > >> >> to the following commit: > >> >> > >> >> 3eab9d7bc2f4 ("fuse: convert readahead to use folios") > >> > >> I see that commit removes folio_put() from fuse_readpages_end(). Also = it now > >> uses readahead_folio() in fuse_readahead() which does folio_put(). So = that's > >> suspicious to me. It might be storing pointers to pages to ap->pages w= ithout > >> pinning them with a refcount. > >> > >> But I don't understand the code enough to know what's the proper fix. = A > >> probably stupid fix would be to use __readahead_folio() instead and ke= ep the > >> folio_put() in fuse_readpages_end(). > > > > Agreed, I'm also confused as to what the right thing is here. It appea= rs the > > rules are "if the folio is locked, nobody messes with it", so it's not = "correct" > > to hold a reference on the folio while it's being read. I don't love t= his way > > of dealing with folios, but that seems to be the way it's always worked= . > > > > I went and looked at a few of the other file systems and we have NFS wh= ich holds > > it's own reference to the folio while the IO is outstanding, which FUSE= is most > > similar to NFS so this would make sense to do. > > > > Btrfs however doesn't do this, BUT we do set_folio_private (or whatever= it's > > called) so that keeps us from being reclaimed since we'll try to lock t= he folio > > before we do the reclaim. > > > > So perhaps that's the issue here? We need to have a private on the fol= io + the > > folio locked to make sure it doesn't get reclaimed while it's out being= read? > > > > I'm knee deep in other things, if we want a quick fix then I think your > > suggestion is correct Vlastimil. But I definitely want to know what Wi= lly > > expects to be the proper order of operations here, and if this is exact= ly what > > we're supposed to be doing then something else is going wrong and we sh= ould try > > to reproduce locally and figure out what's happening. Thanks, > > Thanks, Josef. I guess we can at least try to confirm we're on the right = track. > Can anyone affected see if this (only compile tested) patch fixes the iss= ue? > Created on top of 6.13.1. This fixes the crash for me on 6.14.0-rc1. I ran the repro using Mantas's instructions for Obfuscate. I was able to trigger the crash on a clean build and then with this patch, I'm not seeing the crash anymore. > > ----8<---- > From c0fdf9174f6c17c93a709606384efe2877a3a596 Mon Sep 17 00:00:00 2001 > From: Vlastimil Babka > Date: Fri, 7 Feb 2025 19:35:25 +0100 > Subject: [PATCH] fuse: prevent folio use-after-free in readahead > > Signed-off-by: Vlastimil Babka > --- > fs/fuse/file.c | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/fs/fuse/file.c b/fs/fuse/file.c > index 7d92a5479998..a40d65ffb94d 100644 > --- a/fs/fuse/file.c > +++ b/fs/fuse/file.c > @@ -955,8 +955,10 @@ static void fuse_readpages_end(struct fuse_mount *fm= , struct fuse_args *args, > fuse_invalidate_atime(inode); > } > > - for (i =3D 0; i < ap->num_folios; i++) > + for (i =3D 0; i < ap->num_folios; i++) { > folio_end_read(ap->folios[i], !err); > + folio_put(ap->folios[i]); > + } > if (ia->ff) > fuse_file_put(ia->ff, false); > > @@ -1048,7 +1050,7 @@ static void fuse_readahead(struct readahead_control= *rac) > ap =3D &ia->ap; > > while (ap->num_folios < cur_pages) { > - folio =3D readahead_folio(rac); > + folio =3D __readahead_folio(rac); > ap->folios[ap->num_folios] =3D folio; > ap->descs[ap->num_folios].length =3D folio_size(f= olio); > ap->num_folios++; > -- > 2.48.1 > >