From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE12EC02198 for ; Mon, 10 Feb 2025 19:12:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4DFE16B0088; Mon, 10 Feb 2025 14:12:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 49046280001; Mon, 10 Feb 2025 14:12:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3573C6B0092; Mon, 10 Feb 2025 14:12:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 18A786B0088 for ; Mon, 10 Feb 2025 14:12:50 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 227D414057D for ; Mon, 10 Feb 2025 19:12:40 +0000 (UTC) X-FDA: 83104981680.10.410519E Received: from mail-yb1-f180.google.com (mail-yb1-f180.google.com [209.85.219.180]) by imf10.hostedemail.com (Postfix) with ESMTP id 0E2EBC0006 for ; Mon, 10 Feb 2025 19:12:37 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=toxicpanda-com.20230601.gappssmtp.com header.s=20230601 header.b=aJQFIZq4; spf=none (imf10.hostedemail.com: domain of josef@toxicpanda.com has no SPF policy when checking 209.85.219.180) smtp.mailfrom=josef@toxicpanda.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739214758; a=rsa-sha256; cv=none; b=QNVnstrChcKAMLsTIw8Gy1OfkwnmOUE2MshHkCq6w/iwgDusFeS80m4mST+nSs72TsEDCI 2kxmtfW1b4nF2lELmPuYFDKDDQ771iK7icvwmMhPppfSoqHoidqGn9a7LWS7BbaQyAdqWg 5I5jJV6QxviJ03Cc+aJcPqiVKh0jYY8= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=toxicpanda-com.20230601.gappssmtp.com header.s=20230601 header.b=aJQFIZq4; spf=none (imf10.hostedemail.com: domain of josef@toxicpanda.com has no SPF policy when checking 209.85.219.180) smtp.mailfrom=josef@toxicpanda.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739214758; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7X6uNuQlMRodZ9pzqm1wHqWFoOKmQViZcOnpPZv5eDA=; b=eJ70TicjJ6PJ+J7RFoc+wHYQFEJz4joYvVTLnw/tCMJ+d1YkV1f27GxqRrtYtZ0a77ECB7 2dJrfKCgZPbI/3Rn127INetQewMX4lYxajuX9fzdoAeFXRCNMNdYbwa3VA9VirGfcnc/MX zZhhvHYY0u5vx71BIxe8X7IJXbtLYzc= Received: by mail-yb1-f180.google.com with SMTP id 3f1490d57ef6-e5b3391b033so3687570276.1 for ; Mon, 10 Feb 2025 11:12:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20230601.gappssmtp.com; s=20230601; t=1739214757; x=1739819557; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=7X6uNuQlMRodZ9pzqm1wHqWFoOKmQViZcOnpPZv5eDA=; b=aJQFIZq4NxBw8Iui1Yn9HLtN2O8W1DSMeub/mV/7DHI2PTk/KdSZq34kSiVcGktP20 BZT3/Ph7uChTzHRbVL0GyCx/mDBoKHSdwXnwyxS5C8mfDGJvCOjKNgEQT8mNj0aDq0Yo jH6dA8rb7uAC6lXyXzBUVGKRG2f7qmYDik+zODq5Sf19uOeE2DnNJjEMgfN2wFR9G9rS MW/+uKmGdYUfi3XC+j3BOr2pEwRNPsHGH9dAhC52hvXI1PSMJKAnHjOaXMHrwe6xaHyW l29jQ1BWKfpgkHGyYltVHaylSMEGPMg/IM/vRmZxAWmSZz1cxJY815am9owNwuMaQBfA 80/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739214757; x=1739819557; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7X6uNuQlMRodZ9pzqm1wHqWFoOKmQViZcOnpPZv5eDA=; b=azpqGSx52gx8JpsWvmVH3Wq1IIBLInRMFpfOtHbcUIXh8DzkT6BdLMgsUqfsSGRblj 0Q/26OJlgDj4N93iYbewmcaT0SQ3/PMWKgIlhJSKs3Az5vNF4cPZgLHavrZv8Wys8ksn DPqrCIPmW3mgW1sjy8ylOj7PQ/JGsYRjq0ggHrSPailUvub/S2mbHaku3gSR0hz23CNW FDHoQv1qPFvn7MLDuOyECn+5lGBmpmzYz/cC2BBgv+RtQW0N4EIpdKbiP+CngNvEYZnW AHG3a+7Gb1PvOdFL5EY5eTEXn26rD+p+rbOh7MGg7xJ0sd9ziJ98XbOiVO0bZJ7q0Wht fPqQ== X-Forwarded-Encrypted: i=1; AJvYcCX0TQEAKNdLT3ytwO5B2+ClTjAIdFlUPyZ1nHMlr0KflSeUXb0JRCjUX0Hxflybe8R9tB16lMNaZg==@kvack.org X-Gm-Message-State: AOJu0YxDJa3GgkIxGJW4uLxnYM/zuoGbLQvYR7Aq9Y9OyL5lFxikZUJI OZsb9D1LQvzpyggnpovoNDs/kHXX2NRH0LhrFGOe3y72yeru9rDWnfRmULKxSxY= X-Gm-Gg: ASbGncsI0pCIb5ukHgDP+Q4VU2uB4sItddmOKt9eWESfMFz63Cb9zZwlDOK2gW+3n3s HBZQUsz5skhNWb/1rPxPi4zph9rdEYvJkNzSfqNmpLT2uD5E9bYti1VXH/zB/EV3GK1H+dxpkNU GfL+MxjQxOS8eStPUKI7U25X7+22Xt1xv1+iNlTi7VP7v7i2UABC0ENwvJsD1HLoIXcyW5YvWv2 l7TYLiLmFktbNrgATsRYrzr6RtugUc1eiAXNwyZSfVBHSOSSiQuD5dQ4Lt0VrhY1LzBjjqwUc6g u4MIGKjT4XvJlE3uIxhv1spcNajOjHv3aUtSwjZ+lEROxs3KzRJL X-Google-Smtp-Source: AGHT+IEyRIKhmgO/4HoHaCjVUun69PAKcV1oSQIHgk5rqDLOXak/j3xysFVCh8NWH5SOGRmd1WM8qQ== X-Received: by 2002:a05:690c:7301:b0:6f9:7b99:8a29 with SMTP id 00721157ae682-6f9b2a07011mr124242147b3.34.1739214756954; Mon, 10 Feb 2025 11:12:36 -0800 (PST) Received: from localhost (syn-076-182-020-124.res.spectrum.com. [76.182.20.124]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f99ff6a8e3sm17839597b3.90.2025.02.10.11.12.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 11:12:36 -0800 (PST) Date: Mon, 10 Feb 2025 14:12:35 -0500 From: Josef Bacik To: Joanne Koong Cc: Vlastimil Babka , Matthew Wilcox , Miklos Szeredi , Christian Heusel , Miklos Szeredi , regressions@lists.linux.dev, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm , Mantas =?utf-8?Q?Mikul=C4=97nas?= Subject: Re: [REGRESSION][BISECTED] Crash with Bad page state for FUSE/Flatpak related applications since v6.13 Message-ID: <20250210191235.GA2256827@perftesting> References: <2f681f48-00f5-4e09-8431-2b3dbfaa881e@heusel.eu> <9cd88643-daa8-4379-be0a-bd31de277658@suse.cz> <20250207172917.GA2072771@perftesting> <8f7333f2-1ba9-4df4-bc54-44fd768b3d5b@suse.cz> <81298bd1-e630-4940-ae5b-7882576b6bf4@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Stat-Signature: kihuykh44oh9bb5nt8rqzpstoyfua4sr X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 0E2EBC0006 X-Rspam-User: X-HE-Tag: 1739214757-236065 X-HE-Meta: U2FsdGVkX19twR45tdRF62dlJM3Ggu7kjiRlKVwxmCpKfQRyGsg350tdvGyBWZqnC/vvVRouXgu22kFe2Nb4vB/9tpT1epOiFUdTGqrIlM5YrOaoo5Nq29BeJcj+Zgqccee1xAK7xoYs9k0FMPxDUlZmWgqSZ77/PCwhONb3v1lpjqsJ/jXu+uo3bd1662PkXEk2W569bG53m/IG/cqDIwOPIOY+pctgISxQ4d7Zz38H5fCVWKN3akxFGag5jR3iZy7gMzzqRCzULOqWFaDnkwnLt/ZH8QixrlY05JinD9IwmeZrbqctv2z7FMtnoT3x0oJOkHRbfk/67qb8myGnOsU2EPYkHnlbdA1CAYK7Zd3N3LWdSWLA6dqi/Hk8SBOSMKiqW17TGT60ZM0t5e2z5YgfOJ88EyzlYi8gxDS/JOV9L5TrY0J05SAoAOmyqJdPLlBn9MockW8YzuC9NrlxS+hq50O6NIjPZ0TIiV4Gwf7/yw3dR7Fh0eGbKbEG9liO4fmU6YitOxav32JbmL9uCE+ATaHYTCmlyqLmyArvipNpTLDeRwFcbgsRDcTFVWs4Soq+nlvmcTBgjSFbvq8eUU4st7cJ4u3FHM1Xh4shBQxZXHXMEmX//+RHaw9H9eKwN+aYwhCZBQHHYTOYEP85oH8gTCNRo/DCxnjj5TkAGFxPe74kErg0LprYhhqB7hA9SwsbUQaFpcgTyg6Fc6M8elMvJ7iCkCeykevuv/xFm8nLGe5PdemOyi2+K1dijMcBVGXAVGMLIFy9qLxCRTpqm7eRQP7XaZp0KegLxOxlkuoHpMeONTsRy93ZM3vC1qEUYcvfSUJnmcgdecvIYcVwZjFByPzGVcvFfr9+/H8vUj9o/pFp4kg3gOjccjtOnEdpXRzjgUq4zfHX9aD/PL31pjfArLfFYIcVtnUH62pLLhsVuaJ0a3QXDqm5SbbrCucZ6AUf83C+Rz/OA/Su2h5 PI3URW7y iIcCZpYW74O+FxMCkts6FmJ3OEY+dhi54l7zuF3xr2nnF5y9YH6U28Nstr/ZIDAPu/OwOYp/p3zZuRq0IgvMqHd+epFnKhQauWRZIg7/i37LuMQ+ihuS/wxkyDOW8S8UuK4cjbzU18yXGhOSrUXRq+vw8Ch1SQgsSxoGr5y13x6vnlisDx7baGnsXKGqfShNTiDHSHmR6xhvxS+QkZG6nc2GtDwokgpKb1waYfYbrOCs+3K6mVpPFvbm6HcjtWXqHQECyXERlJOGxBXG6Yvoddd9fqlgDj5eejCV0ThI8b6l6Bp/EYifsOFZN7hBlZ2ftL1KmAuiYHLlwXgNhTe127URK5xbyI0/aMX7eBnJC+XX18L0gl99UW/ITmvH/Xy+tTcPLrkPOtJAXIj7xzqGk/fJOfU2jtvRQ55iS0m/SZPEpm5Ko46en4qMucGcda//ORBSas4837uDL9wWIN2qwihOfsmkaTtlLSS+ay4XH85yCLm4W5P/+mrN9oFMnJToL3/oJ8tELopOwuGCvy+gnJVkW77THeR/9yemjykx5rmPzygJJeSDJGfAG+H7lUKDQo7JNkvKVdi+M5EexY2d8JeYs7P1rkRRWfv6ZAJ2i1lJw1TD7RxvetMHf6w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Feb 10, 2025 at 10:13:51AM -0800, Joanne Koong wrote: > On Mon, Feb 10, 2025 at 12:27 AM Vlastimil Babka wrote: > > > > On 2/8/25 16:46, Joanne Koong wrote: > > > On Sat, Feb 8, 2025 at 2:11 AM Matthew Wilcox wrote: > > >> > > >> On Fri, Feb 07, 2025 at 04:22:56PM -0800, Joanne Koong wrote: > > >> > > Thanks, Josef. I guess we can at least try to confirm we're on the right track. > > >> > > Can anyone affected see if this (only compile tested) patch fixes the issue? > > >> > > Created on top of 6.13.1. > > >> > > > >> > This fixes the crash for me on 6.14.0-rc1. I ran the repro using > > >> > Mantas's instructions for Obfuscate. I was able to trigger the crash > > >> > on a clean build and then with this patch, I'm not seeing the crash > > >> > anymore. > > >> > > >> Since this patch fixes the bug, we're looking for one call to folio_put() > > >> too many. Is it possibly in fuse_try_move_page()? In particular, this > > >> one: > > >> > > >> /* Drop ref for ap->pages[] array */ > > >> folio_put(oldfolio); > > >> > > >> I don't know fuse very well. Maybe this isn't it. > > > > > > Yeah, this looks it to me. We don't grab a folio reference for the > > > ap->pages[] array for readahead and it tracks with Mantas's > > > fuse_dev_splice_write() dmesg. this patch fixed the crash for me when > > > I tested it yesterday: > > > > > > diff --git a/fs/fuse/file.c b/fs/fuse/file.c > > > index 7d92a5479998..172cab8e2caf 100644 > > > --- a/fs/fuse/file.c > > > +++ b/fs/fuse/file.c > > > @@ -955,8 +955,10 @@ static void fuse_readpages_end(struct fuse_mount > > > *fm, struct fuse_args *args, > > > fuse_invalidate_atime(inode); > > > } > > > > > > - for (i = 0; i < ap->num_folios; i++) > > > + for (i = 0; i < ap->num_folios; i++) { > > > folio_end_read(ap->folios[i], !err); > > > + folio_put(ap->folios[i]); > > > + } > > > if (ia->ff) > > > fuse_file_put(ia->ff, false); > > > > > > @@ -1049,6 +1051,7 @@ static void fuse_readahead(struct readahead_control *rac) > > > > > > while (ap->num_folios < cur_pages) { > > > folio = readahead_folio(rac); > > > + folio_get(folio); > > > > This is almost the same as my patch, but balances the folio_put() in > > readahead_folio() with another folio_get(), while mine uses > > __readahead_folio() that does not do folio_put() in the first place. > > > > But I think neither patch proves the extraneous folio_put() comes from > > fuse_try_move_page(). > > > > > ap->folios[ap->num_folios] = folio; > > > ap->descs[ap->num_folios].length = folio_size(folio); > > > ap->num_folios++; > > > > > > > > > I reran it just now with a printk by that ref drop in > > > fuse_try_move_page() and I'm indeed seeing that path get hit. > > > > It might get hit, but is it hit in the readahead paths? One way to test > > would be to instead of yours above or mine change, to stop doing the > > foio_put() in fuse_try_move_page(). But maybe it's called also from other > > contexts that do expect it, and will leak memory otherwise. > > When I tested it a few days ago, I printk-ed the address of the folio > and it matched in fuse_readahead() and try_move_page(). I think that > proves that the extra folio_put() came from fuse_try_move_page() > through the readahead path. This patch should fix the problem, let me know if it's stil happening >From 964c798ee9e8f2e8e2c37cfd060c76a772cc45b7 Mon Sep 17 00:00:00 2001 Message-ID: <964c798ee9e8f2e8e2c37cfd060c76a772cc45b7.1739214698.git.josef@toxicpanda.com> From: Josef Bacik Date: Mon, 10 Feb 2025 14:06:40 -0500 Subject: [PATCH] fuse: drop extra put of folio when using pipe splice In 3eab9d7bc2f4 ("fuse: convert readahead to use folios"), I converted us to using the new folio readahead code, which drops the reference on the folio once it is locked, using an inferred reference on the folio. Previously we held a reference on the folio for the entire duration of the readpages call. This is fine, however I failed to catch the case for splice pipe responses where we will remove the old folio and splice in the new folio. Here we assumed that there is a reference held on the folio for ap->folios, which is no longer the case. To fix this, simply drop the extra put to keep us consistent with the non-splice variation. This will fix the UAF bug that was reported. Link: https://lore.kernel.org/linux-fsdevel/2f681f48-00f5-4e09-8431-2b3dbfaa881e@heusel.eu/ Fixes: 3eab9d7bc2f4 ("fuse: convert readahead to use folios") Signed-off-by: Josef Bacik --- fs/fuse/dev.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c index 5b5f789b37eb..5bd6e2e184c0 100644 --- a/fs/fuse/dev.c +++ b/fs/fuse/dev.c @@ -918,8 +918,6 @@ static int fuse_try_move_page(struct fuse_copy_state *cs, struct page **pagep) } folio_unlock(oldfolio); - /* Drop ref for ap->pages[] array */ - folio_put(oldfolio); cs->len = 0; err = 0; -- 2.43.0