From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C3A4EB64D7 for ; Tue, 20 Jun 2023 21:41:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 68E578D0002; Tue, 20 Jun 2023 17:41:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 63E8A8D0001; Tue, 20 Jun 2023 17:41:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 505F28D0002; Tue, 20 Jun 2023 17:41:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 40CB28D0001 for ; Tue, 20 Jun 2023 17:41:12 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 095FC1A0694 for ; Tue, 20 Jun 2023 21:41:12 +0000 (UTC) X-FDA: 80924447184.03.7FE48E1 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by imf10.hostedemail.com (Postfix) with ESMTP id A86BFC0004 for ; Tue, 20 Jun 2023 21:41:09 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=anarazel.de header.s=fm2 header.b=h1fsw2NW; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=eQVIyGYP; dmarc=none; spf=pass (imf10.hostedemail.com: domain of andres@anarazel.de designates 64.147.123.19 as permitted sender) smtp.mailfrom=andres@anarazel.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687297270; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IFjjY5roZiwIkKPSUbI5fcD7pRZU5oiVbQVG2wmNIyk=; b=2QEsd4nk39Vwjs3Qz6LslWxcveAlQxlqsNCbsCEAeISqfvfagjzZrHPceCAjGTJ4pR3avg UTmLjs71uypriYe1KeHrWJmCAQ7EL7TpMtZCR4Ez8aTNjdm+7v0ePJXV5vv9E1XNh6NGFN MoaD30hJLWz0QRSQeWqXXZM9Lq2NY54= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=anarazel.de header.s=fm2 header.b=h1fsw2NW; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=eQVIyGYP; dmarc=none; spf=pass (imf10.hostedemail.com: domain of andres@anarazel.de designates 64.147.123.19 as permitted sender) smtp.mailfrom=andres@anarazel.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687297270; a=rsa-sha256; cv=none; b=JVvrs1ywsGhdeTd6U6Ic4mDJ5aX3vhIfJFq8SzImu7NF5gv4TnHtBw6TaDho1z9yTYzZkS rDDzdDxzJ9uO0M97Ix1e+pYAZI9YYA8HHFEdEM0l0Aunj7GrUJRqYzYX7DjY8vvjhKXogJ yb5+Xk9S/SQnguWK/ffJ+tDa3rVYi6o= Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 59608320091D; Tue, 20 Jun 2023 17:41:07 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Tue, 20 Jun 2023 17:41:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=anarazel.de; h= cc:cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm2; t=1687297266; x=1687383666; bh=IF jjY5roZiwIkKPSUbI5fcD7pRZU5oiVbQVG2wmNIyk=; b=h1fsw2NWh5JvfJotED pWAHMvSeuUWwgN6ejG3ulwtxY01wU4Bx3GlAnbfz1blo++XmLfnxHXu5q4uhEg7F +eYLNbG6BoJQ5pRcU5Cj6mdwbCYP6KGF63IR/OZua+A9/TlfNI4tbViujV44QOEs eJ+A1QV2WW2PF9/QZBLhiyrDLnJH0+LJN/vsQvpePaCrxNO8t76TL+UBrhZhmktH LinlWnVdLjxKPSGAd7nPewPGq8+2GADYL66Qi56cApOipg3zouDIsdaZpNISoKEN XKNQ/rz1bMqc641hFOnMTVQRDKuHqKqFDD9aVLSTVA3YCQLv6O544XkAEMdj0kPF JFow== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; t=1687297266; x=1687383666; bh=IFjjY5roZiwIk KPSUbI5fcD7pRZU5oiVbQVG2wmNIyk=; b=eQVIyGYPd5rS5NNsI653/DGuL5SUK zfEVfjFjKFYyoEX42Mj9taTFKf9Hj1miWpJEAxCHc2kgcdzXvIxgmxVJLC6QpMBh FQF+ow+RfBVHLJC5YdQNGgYpR0eCQWSUq6ZMPVcvI2QIQrS1/H+qUCnwwl7HRAXG H3XCAvT+yEJP1E7MiQdz92pc6oiR2EsO5weP6hyZd1eeh8n6Iovo58AgkEWezbHB P7IUXc/0YuuMuCipInZ/A9Q7IDPtEk43/ap3+XudS15BIho0Yp+tlS/8SOgd7hPh zzhYNjV89pOm0nKV5RD4DidRR/lNXZ8lqsMc+ytjAq1pOo9O7XQcZWrtA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrgeefiedgtddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesthdtredttddtvdenucfhrhhomheptehnughr vghsucfhrhgvuhhnugcuoegrnhgurhgvshesrghnrghrrgiivghlrdguvgeqnecuggftrf grthhtvghrnhepueelhedvkeehvddvgeevleektefhteefueefhefhteeigffgtdegkeek geeigeeunecuffhomhgrihhnpehkvghrnhgvlhdrohhrghenucevlhhushhtvghrufhiii gvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegrnhgurhgvshesrghnrghrrgiivghl rdguvg X-ME-Proxy: Feedback-ID: id4a34324:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 20 Jun 2023 17:41:06 -0400 (EDT) Date: Tue, 20 Jun 2023 14:41:05 -0700 From: Andres Freund To: Peter Xu Cc: David Stevens , linux-mm@kvack.org, Hugh Dickins , Andrew Morton , Matthew Wilcox , "Kirill A . Shutemov" , Yang Shi , David Hildenbrand , Jiaqi Yan , linux-kernel@vger.kernel.org Subject: Re: [PATCH v6 4/4] mm/khugepaged: maintain page cache uptodate flag Message-ID: <20230620214105.3d4tm54gzodcx3uk@awork3.anarazel.de> References: <20230404120117.2562166-1-stevensd@google.com> <20230404120117.2562166-5-stevensd@google.com> <20230620205547.qzmivkjox2hkpzmm@awork3.anarazel.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: A86BFC0004 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: mtk1xypmop7d3kopgah83munsp6a5xjm X-HE-Tag: 1687297269-585161 X-HE-Meta: U2FsdGVkX1/mevlmKAVaB0cFNmauRn4KvmQiA1Jq8uybaDd4Ee9PIznFSxXgPcRBpL5ctJX/x99QLMbQn1bFRr7EBLoUm2EjnaiNLmcv/soaiKqr2NN3RR9VJefqmJ2CnMlPP533lHsru2u1Ne4pzsKM5Ni2gjiWVzfb8O81yPu2bc0IHM/7eELkBL3s2kRNLMpWxw8d8vUK93vjrKgNYvynAC8DNTsyna1HzNS32hgYsM0CvrRHVl4TSiQ9MnnTErMdMY6oUuNyu+RK1z6irwK8fiRXqh0xXOUx1ObLB2YXEKVvHelv5eNzCtNjFI+qv8jfEEwyZdzKUnN8b81by0vLk00jfY0RjnsRX8HtQ77l83sFao1jUi1I6eTNLbTm8DFCiF5GB87Tuy9fhpvoZ1TaeLzZXbgGVu23KcXZyfclGOa3ixSK9TjZ04auAhWcykOewlNCWZKYvqa9lLFVqjn3b7+sqk9MQW/1zoh+1chGT9diiRq25Y7P2kocE23lY6hro64oZEYi7AKnHb78aZcAFt7UO8L4SMjb0TUBM7lvlDFaUSLKcD99nztIVsOCn/Ww7y8H+ZHYNhB/Oph5zUTTbLgesqrn9naBC56goqMSU76pAnDflcUOIZowr5rAyUgQeR0nq93DOrR0JcO34yQnwdjCnr9xHtYOEla78MWweQgOO9qKQKX/0TJkfj541H+T4VNJGa7BKL6OOsG+Dpf+UmIiFKFx9MphBVdE8mF+Ps9veKMFuQRfHRpySxWxb2w9p3dL8nbsuF6eIIUwbeaWfVAumZKfaTqJ5LbcfMzckMtiIqluFnQ++fl/6Y+6MEiQUkAHrjDNzKGJ/iS2HalfTOFiPcVQ1mfILQJmTeeqh47zhVV1pvPYuE0ojsJN+fBrkw0Cu5U7F69jG3OpjsNwZu2mIGDk48hUkuoHjFuRonhXjS/oCE9pKqmxtJq6+cNCnbXQjqn4K+UcdQ8 1qtrEm2s 3E10Jvw2Tg7rEuHv9cRRb9fEFCVcU/EM/l2sAvnIObKShtoV5eVPyW/ZjGi0HJ3L4OkGAoZ1srcmiQk0qdkdaSifrmLhpn/T3O2Elhgfuei/qzUhQ7oGOzhOjVqduamLHUWKdymHKVecnGWw/Uylohzu607R/lbWdNPXzBzlK9YmMECCSxn50UiYAQ9Ok0UtORWwAN7htn3Cc6DcGPlD6tru1Izxvv2iXYAdhxVs/Qvhrv+L7KM6rYgVgNp6R2b7wDK/bAgbNw9ihDJqCC2rgEWrhixgj06rCg8yuGREGqeTCnL/BWfKQdtJ8xz5QXvOxb1GRMEHaNXBYrJMutRcVUktPY+A9XoqntjfXpnPbLqqbEXNGgeNP+DarHbitSvQVERlK8btdxd/6clZX9VD9wJ9NZoe6QKhffdkl X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, On 2023-06-20 17:11:30 -0400, Peter Xu wrote: > On Tue, Jun 20, 2023 at 01:55:47PM -0700, Andres Freund wrote: > > On 2023-04-04 21:01:17 +0900, David Stevens wrote: > > > From: David Stevens > > > > > > Make sure that collapse_file doesn't interfere with checking the > > > uptodate flag in the page cache by only inserting hpage into the page > > > cache after it has been updated and marked uptodate. This is achieved by > > > simply not replacing present pages with hpage when iterating over the > > > target range. > > > > > > The present pages are already locked, so replacing them with the locked > > > hpage before the collapse is finalized is unnecessary. However, it is > > > necessary to stop freezing the present pages after validating them, > > > since leaving long-term frozen pages in the page cache can lead to > > > deadlocks. Simply checking the reference count is sufficient to ensure > > > that there are no long-term references hanging around that would the > > > collapse would break. Similar to hpage, there is no reason that the > > > present pages actually need to be frozen in addition to being locked. > > > > > > This fixes a race where folio_seek_hole_data would mistake hpage for > > > an fallocated but unwritten page. This race is visible to userspace via > > > data temporarily disappearing from SEEK_DATA/SEEK_HOLE. This also fixes > > > a similar race where pages could temporarily disappear from mincore. > > > > > > Fixes: f3f0e1d2150b ("khugepaged: add support of collapse for tmpfs/shmem pages") > > > Signed-off-by: David Stevens > > > > I noticed that recently MADV_COLLAPSE stopped being able to collapse a > > binary's executable code, always failing with EAGAIN. I bisected it down to > > a2e17cc2efc7 - this commit. > > > > Using perf trace -e 'huge_memory:*' -a I see > > > > 1000.433 postgres.2/1872144 huge_memory:mm_khugepaged_collapse_file(mm: 0xffff889e800bdf00, hpfn: 46720000, index: 1537, is_shmem: 1, filename: "postgres.2", result: 17) > > 1000.445 postgres.2/1872144 huge_memory:mm_khugepaged_scan_file(mm: 0xffff889e800bdf00, pfn: -1, filename: "postgres.2", present: 512, result: 17) > > 1000.485 postgres.2/1872144 huge_memory:mm_khugepaged_collapse_file(mm: 0xffff889e800bdf00, hpfn: 46720000, index: 2049, is_shmem: 1, filename: "postgres.2", result: 17) > > 1000.489 postgres.2/1872144 huge_memory:mm_khugepaged_scan_file(mm: 0xffff889e800bdf00, pfn: -1, filename: "postgres.2", present: 512, result: 17) > > 1000.526 postgres.2/1872144 huge_memory:mm_khugepaged_collapse_file(mm: 0xffff889e800bdf00, hpfn: 46720000, index: 2561, is_shmem: 1, filename: "postgres.2", result: 17) > > 1000.532 postgres.2/1872144 huge_memory:mm_khugepaged_scan_file(mm: 0xffff889e800bdf00, pfn: -1, filename: "postgres.2", present: 512, result: 17) > > 1000.570 postgres.2/1872144 huge_memory:mm_khugepaged_collapse_file(mm: 0xffff889e800bdf00, hpfn: 46720000, index: 3073, is_shmem: 1, filename: "postgres.2", result: 17) > > 1000.575 postgres.2/1872144 huge_memory:mm_khugepaged_scan_file(mm: 0xffff889e800bdf00, pfn: -1, filename: "postgres.2", present: 512, result: 17) > > > > for every attempt at doing madvise(MADV_COLLAPSE). > > > > > > I'm sad about that, because MADV_COLLAPSE was the first thing that allowed > > using huge pages for executable code that wasn't entirely completely gross. > > > > > > I don't yet have a standalone repro, but can write one if that's helpful. > > There's a fix: > > https://lore.kernel.org/all/20230607053135.2087354-1-stevensd@google.com/ > > Already in today's Andrew's pull for rc7: > > https://lore.kernel.org/all/20230620123828.813b1140d9c13af900e8edb3@linux-foundation.org/ Ah, great! I can confirm that the fix unbreaks our use of MADV_COLLAPSE for executable code... Greetings, Andres Freund