From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2DE8C761A6 for ; Thu, 6 Apr 2023 01:50:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0B1556B0075; Wed, 5 Apr 2023 21:50:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 061E66B0078; Wed, 5 Apr 2023 21:50:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E44396B007B; Wed, 5 Apr 2023 21:50:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D0D1A6B0075 for ; Wed, 5 Apr 2023 21:50:08 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 95CCD1C6BC2 for ; Thu, 6 Apr 2023 01:50:08 +0000 (UTC) X-FDA: 80649285696.28.2C80CAD Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf03.hostedemail.com (Postfix) with ESMTP id BDB2D2000D for ; Thu, 6 Apr 2023 01:50:06 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=aAton7TQ; spf=pass (imf03.hostedemail.com: domain of chao@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=chao@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680745806; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XmQvIiYi/4i07AFgDk5IgQ9ctrFqbVOjaV5M6jy/SzE=; b=w0amarDMIXW6qmqmXtK+dbizpXDcysdarMzB0dueZusL8Jyn89tu3Wm5XhsvA34g3dm93Q fpkRiZlrqaG3vfzBDJl31zjyJIFuLbCZZVsMXCGUXnZvHYH1XLUScB9CQps/WkNgbM4ma3 HwFJOtrpp6wdNa/rSAee5goOE/mTlbo= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=aAton7TQ; spf=pass (imf03.hostedemail.com: domain of chao@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=chao@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680745806; a=rsa-sha256; cv=none; b=gvaw67KXhcQ4Y5xh0kL9pfxdTKUo641HAJt6V8mISnIWlIter/X6pwcnJHdLm/t9tziY8d kclu/KJXcwBQy1HoOR9qpBT81xrkc6bPLDD8HXwSoAI9bKitUDLtTYT3MAtkuAMt1k6Ur5 bwGIWtVRaMZkeNLIbGsWhaR2lEKKMq0= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id B163464185; Thu, 6 Apr 2023 01:50:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 88A1FC433D2; Thu, 6 Apr 2023 01:50:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1680745805; bh=xgpkfNiyM2OsqZ9AjY4VjyDtc1aONna/URR/VaLtHlg=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=aAton7TQlCgNxi38nrF0Di/+BG0JaOPRYgDRjd1hEVudb3rurcj+JPE0hrpS3vP5m kLdFzQLeoFO56ngekzX4tlDDvgneNXs2IdyLX8wM8TFOAwo+fbg0nBJL5GzOJutRQQ jR9xmxItGDDCvR+PCEOPfCVEIW7/CfpSfqQ5o5J/qsIjAvT1OJq14igoRsFBs4EOAV exhqmAZpsf8eXQXoSl2iwZxvW4CTjZ4TOYr4/soe3ULjuM2g70H68PflMyx4rqbtpr 0LrcBmGc+VK7K+n3Zwr78O4tpUkonUXd8mhRnN/Dk4pOKux0Gu6DZ2FWpazv9NS2cW hg3kSO5Ef2qJQ== Message-ID: <9dc4ba32-5be5-26d8-5dd2-9bd48d6b0af4@kernel.org> Date: Thu, 6 Apr 2023 09:50:01 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.9.1 Subject: Re: [f2fs-dev] [PATCH] f2fs: get out of a repeat loop when getting a locked data page Content-Language: en-US To: Jaegeuk Kim , Matthew Wilcox Cc: linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, stable@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org References: <20230323213919.1876157-1-jaegeuk@kernel.org> <8aea02b0-86f9-539a-02e9-27b381e68b66@kernel.org> From: Chao Yu In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: BDB2D2000D X-Rspam-User: X-Stat-Signature: k7m1ihag7ocqh9noigkimfhsgz5ymkns X-HE-Tag: 1680745806-404724 X-HE-Meta: U2FsdGVkX18qYpgeS8zt7WJDhAzWATvPyO2CWN3/RhwYb+Kfhbk/+SGcX37dmxPHcLB+oYsxIUompA+9ICgm3nAaOIemdHIG/QOlxYBoZbndafPrqRfJoq2Wwpeh1QOhogbsnnVbXP0ml9zBhOI8+chGHu2stQNMNDl6LhPOYg8PkFA4l5/INsi/bdgbz08p4Di7wl9U+4yQlSklBsWCMzCOFdi3yqTjCddDL1GtBBG2em0ctvLdlu2GWApqF3EeARsLqLx0PKcZNfLFb1DccRVeyskbeiJqyVx0kRORZVCww3T/yYf3zTR8HA7vzcecUpuiKR04st98bAdDciZMs8+ChEMdnEDn89J1o1CQZrE9nz/hsu9kx5yNn8dpgvhTNAZWYgkuCEmoPRzvuPDN3J+eFgg086THRRUsocZQLCRWFvjKuw5J1sXUVHI0ZLP7d6kaHEM3apB+jbdp2p0tvWoyvTZlpApiRdKqORAe3h34x+1K5z2shhhMHMmbcvTLgw+jQVGDQBf/QKLq6RoQdWrn8eKv4Jvpd1wnmmNXXwiKOlxsTkLQSKmJELqkKND20fc+SzX55BK7t/BoyZslDpW05jKK8cs0EagX6vjO+4YVuK6fdpreqsnbwVsjH/20cRT6WgHZtBAVmmw2fDszfDfgGFBSfc08kMtd03hMyHAr+w3fVn9hmN9OfsX4R7Y4OzI1ASYyylj2qCCXNvRjYv23mTpPfKuNadbVrmQvaJ1OrTCSOshZpPw33Y5UTDa4mADiOYhqY9sB25jRE3oDAO8T1tATkUsS0uZuESxitihC9U+t1ERYNW0h8P56dRy1TN1ZnuaysW56e9+fhOjrWf0yRd6i5frwA70LtWuw/kwvviF9cL8W4lPnY3QJlJnMcPNk4i4gemwA0z6tGDsmAM73Cjs5+V0Ixet6u9PNkvxzP1IhzAPTTmmaBkyq/PAUWEFRCPf4bJKyV4t0ygX OLXRmzMA I8CYid+DL43xlpUqPWb1Cg5YI4U1xqJ+GwWD5PRARvZHZGTc6R0Rx0sxDzcnMGSdcx7IpjSHKjd1lc1boFRFmlkH6cSrixI1Lyc7M3W2h/xLCyo5tQJD5qB7EbzbY4fiDlsghaPV+oywmZ4QgZ+uf+tfSwRo7GfKFH8/vaUphz/icbAR/Srz2G+HMSM1wLMlqVTT2zKHClEx90O9P++jTuJfCSpC9DH9QGITNbeKZoi12H+eZ9GYSiG4WpOkSAcU+I+0mcfkMDBkwOdhmNaXMZbNdwOysf7gWFjm09uda5M+Ga2HFMgEbv2NMMWZlAZH7m37h7yVMv+sX2hDlNkp1J9YmwjlZ1D9krIBq0Yot6LE//SYXM9g5R9hq1k/ts7Oyu7aV9rab9cz4zWUWK31M2GCNSiInaX6PrTV+LPNgmgqpmAIl0frKOmU6yQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2023/4/6 0:39, Jaegeuk Kim wrote: > On 03/27, Matthew Wilcox wrote: >> On Mon, Mar 27, 2023 at 08:30:33AM -0700, Jaegeuk Kim wrote: >>> On 03/26, Chao Yu wrote: >>>> On 2023/3/24 5:39, Jaegeuk Kim wrote: >>>>> https://bugzilla.kernel.org/show_bug.cgi?id=216050 >>>>> >>>>> Somehow we're getting a page which has a different mapping. >>>>> Let's avoid the infinite loop. >>>>> >>>>> Cc: >>>>> Signed-off-by: Jaegeuk Kim >>>>> --- >>>>> fs/f2fs/data.c | 8 ++------ >>>>> 1 file changed, 2 insertions(+), 6 deletions(-) >>>>> >>>>> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c >>>>> index bf51e6e4eb64..80702c93e885 100644 >>>>> --- a/fs/f2fs/data.c >>>>> +++ b/fs/f2fs/data.c >>>>> @@ -1329,18 +1329,14 @@ struct page *f2fs_get_lock_data_page(struct inode *inode, pgoff_t index, >>>>> { >>>>> struct address_space *mapping = inode->i_mapping; >>>>> struct page *page; >>>>> -repeat: >>>>> + >>>>> page = f2fs_get_read_data_page(inode, index, 0, for_write, NULL); >>>>> if (IS_ERR(page)) >>>>> return page; >>>>> /* wait for read completion */ >>>>> lock_page(page); >>>>> - if (unlikely(page->mapping != mapping)) { >>>> >>>> How about using such logic only for move_data_page() to limit affect for >>>> other paths? >>> >>> Why move_data_page() only? If this happens, we'll fall into a loop in anywhere? >>> >>>> >>>> Jaegeuk, any thoughts about why mapping is mismatch in between page's one and >>>> inode->i_mapping? >>> >>>> >>>> After several times code review, I didn't get any clue about why f2fs always >>>> get the different mapping in a loop. >>> >>> I couldn't find the path to happen this. So weird. Please check the history in the >>> bug. >>> >>>> >>>> Maybe we can loop MM guys to check whether below folio_file_page() may return >>>> page which has different mapping? >>> >>> Matthew may have some idea on this? >> >> There's a lot of comments in the bug ... hard to come into this one >> cold. >> >> I did notice this one (#119): >> : Interestingly, ref count is 514, which looks suspiciously as a binary >> : flag 1000000010. Is it possible that during 5.17/5.18 implementation >> : of a "pin", somehow binary flag was written to ref count, or something >> : like '1 << ...' happens? >> >> That indicates to me that somehow you've got hold of a THP that is in >> the page cache. Probably shmem/tmpfs. That indicate to me a refcount >> problem that looks something like this: >> >> f2fs allocates a page >> f2fs adds the page to the page cache >> f2fs puts the reference to the page without removing it from the >> page cache (how?) > > Is it somewhat related to setting a bit in private field? IIUC, it looks the page reference is added/removed as pair. > > When we migrate the blocks, we do: > 1) get_lock_page() - f2fs_grab_cache_page - pagecache_get_page - __filemap_get_folio - no_page -> filemap_alloc_folio page_ref = 1 (referenced by caller) - filemap_add_folio page_ref = 2 (referenced by radix tree) > 2) submit read > 3) lock_page() > 3) set_page_dirty() > 4) set_page_private_gcing(page) page_ref = 3 (reference by private data) > > --- in fs/f2fs/f2fs.h > 1409 #define PAGE_PRIVATE_SET_FUNC(name, flagname) \ > 1410 static inline void set_page_private_##name(struct page *page) \ > 1411 { \ > 1412 if (!PagePrivate(page)) { \ > 1413 get_page(page); \ > 1414 SetPagePrivate(page); \ > 1415 set_page_private(page, 0); \ > 1416 } \ > 1417 set_bit(PAGE_PRIVATE_NOT_POINTER, &page_private(page)); \ > 1418 set_bit(PAGE_PRIVATE_##flagname, &page_private(page)); \ > 1419 } > > > 5) set_page_writebac() > 6) submit write > 7) unlock_page() > 8) put_page(page) page_ref = 2 (ref by caller was removed) > > Later, f2fs_invalidate_folio will do put_page again by: > clear_page_private_gcing(&folio->page); page_ref = 1 (ref by private was removed, and the last left ref is hold by radix tree) > > --- in fs/f2fs/f2fs.h > 1421 #define PAGE_PRIVATE_CLEAR_FUNC(name, flagname) \ > 1422 static inline void clear_page_private_##name(struct page *page) \ > 1423 { \ > 1424 clear_bit(PAGE_PRIVATE_##flagname, &page_private(page)); \ > 1425 if (page_private(page) == BIT(PAGE_PRIVATE_NOT_POINTER)) { \ > 1426 set_page_private(page, 0); \ > 1427 if (PagePrivate(page)) { \ > 1428 ClearPagePrivate(page); \ Since PagePrivate was cleared, so folio_detach_private in f2fs_invalidate_folio()/f2fs_release_folio will just skip drop reference. static inline void *folio_detach_private(struct folio *folio) { void *data = folio_get_private(folio); if (!folio_test_private(folio)) return NULL; folio_clear_private(folio); folio->private = NULL; folio_put(folio); return data; } Or am I missing something? Thanks, > 1429 put_page(page); \ > 1430 }\ > 1431 } \ > 1432 } > >> page is now free, gets reallocated into a THP >> lookup from the f2fs file finds the new THP >> things explode messily >> >> Checking page->mapping is going to avoid the messy explosion, but >> you'll still have a page in the page cache which doesn't actually >> belong to you, and that's going to lead to subtle data corruption. >> >> This should be caught by page_expected_state(), called from >> free_page_is_bad(), called from free_pages_prepare(). Do your testers >> have CONFIG_DEBUG_VM enabled? That might give you a fighting chance at >> finding the last place which called put_page(). It won't necessarily be >> the _wrong_ place to call put_page() (that may have happened earlier), >> but it may give you a clue. >> >>>> >>>> struct page *pagecache_get_page(struct address_space *mapping, pgoff_t index, >>>> int fgp_flags, gfp_t gfp) >>>> { >>>> struct folio *folio; >>>> >>>> folio = __filemap_get_folio(mapping, index, fgp_flags, gfp); >>>> if (IS_ERR(folio)) >>>> return NULL; >>>> return folio_file_page(folio, index); >>>> } >>>> >>>> Thanks, >>>> >>>>> - f2fs_put_page(page, 1); >>>>> - goto repeat; >>>>> - } >>>>> - if (unlikely(!PageUptodate(page))) { >>>>> + if (unlikely(page->mapping != mapping || !PageUptodate(page))) { >>>>> f2fs_put_page(page, 1); >>>>> return ERR_PTR(-EIO); >>>>> }