From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91BF1EB64D7 for ; Tue, 20 Jun 2023 20:55:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0ACD88D0003; Tue, 20 Jun 2023 16:55:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 05D458D0001; Tue, 20 Jun 2023 16:55:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E67088D0003; Tue, 20 Jun 2023 16:55:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D71128D0001 for ; Tue, 20 Jun 2023 16:55:54 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A05DD40637 for ; Tue, 20 Jun 2023 20:55:54 +0000 (UTC) X-FDA: 80924333028.24.C55D3F6 Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) by imf03.hostedemail.com (Postfix) with ESMTP id 3B3242001A for ; Tue, 20 Jun 2023 20:55:51 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=anarazel.de header.s=fm2 header.b=kzzsNaPO; dkim=pass header.d=messagingengine.com header.s=fm2 header.b="e0ziG89/"; dmarc=none; spf=pass (imf03.hostedemail.com: domain of andres@anarazel.de designates 64.147.123.20 as permitted sender) smtp.mailfrom=andres@anarazel.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687294552; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jLwjxXidYR03NQqMGEOwIDdRP2UTQNWbZ7zItOdShrw=; b=JYSwEcKk5xiT4hZty7bDuK38h4j6UGpNvE4PZmeSl3J9G4r9MTaRIr4AqegtW5YmCclN/3 XI/BBngegi4q8TibZCOCLV8Wj/+PydD/x/IlDE/DAnLPIdSsBLLEULLmHg4eRkDyPIz/9o YtBB9UyAIqBUaYClPvm53FDrWbqXtEs= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=anarazel.de header.s=fm2 header.b=kzzsNaPO; dkim=pass header.d=messagingengine.com header.s=fm2 header.b="e0ziG89/"; dmarc=none; spf=pass (imf03.hostedemail.com: domain of andres@anarazel.de designates 64.147.123.20 as permitted sender) smtp.mailfrom=andres@anarazel.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687294552; a=rsa-sha256; cv=none; b=nYvTfyUnmI4fuEJUg6b9Md40OJYW9UEJr4l75NNNo8Iie/87jUxdplhTgTdZJRq0yzzmJk XeNmhHSLgQ3PnbuhkLEtVn30ypaPrdCwItaMYB7C/7yAEXiwHPOo8UI0otTB6tXh/+Jl8Q BWkXXTeZKKbvLObm+J9ydqtc7jxnPJQ= Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id C76E0320092D; Tue, 20 Jun 2023 16:55:49 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Tue, 20 Jun 2023 16:55:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=anarazel.de; h= cc:cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm2; t=1687294549; x=1687380949; bh=jL wjxXidYR03NQqMGEOwIDdRP2UTQNWbZ7zItOdShrw=; b=kzzsNaPOa0XFhhBycA nWaMtSnqQ2DvpJ4s5KDSKzQwhuOmszKazGGgeWxgvQv/h+atmCnR105Dba7pOpEr qyw+mYIAvvX0oSLnhpWuWvU1OuEwc/dIDUtx+hNsd4tYulYIyUtc9gvcjMjb3qXe qlgV/hXaRg/dJWnzFTDl4u8WJpxuUAWBwfatNNnMqC1Ain9KegGGPrKQnmH4zWkG H9I7cnIxgSgFeZgR3a1T8jVSTIJXusCHWae1VN+iKEdO19lKDdBsMK/t94BJ0ILw lWqNDrvWfT9JTSK04wJ6Ky7f2FSViN3O7Qe6kwViGrxwj9WQLcrwhkQw8VTyUymF QwUw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; t=1687294549; x=1687380949; bh=jLwjxXidYR03N QqMGEOwIDdRP2UTQNWbZ7zItOdShrw=; b=e0ziG89/qBaJj7UwzLjoc27dXQsOZ BVr0/4VI1Js1J8jF9H/Avt19GDmxYUn1t3bK+d19Rp525X9ySAxy+psiPxcC/XhO KBDOzPJ7SYGAli+KMkUsqCCtNi9Bvj4JJ22EpYZ1eiuf+EDpYxzu5vgLXQk8pEfK KfPkGxSUEVm3E7nTzG7BhwfDILGb+YLPo/lzI7pOxHEQi1mQqXqNIA6ewGO6Gn94 RMWUu8mWvdeeKw1icLdncsDIfizCCgJNTxERiy1M7wiDQ6rDomO8fvoahr4CYpdo dtUFgUCHrojWVwb8/Q6zFQfpCcyBVLGKeusE8EdcxuvUnzN7+kk2JJ5ag== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrgeefhedgudefhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehttdertddttddvnecuhfhrohhmpeetnhgu rhgvshcuhfhrvghunhguuceorghnughrvghssegrnhgrrhgriigvlhdruggvqeenucggtf frrghtthgvrhhnpedvffefvefhteevffegieetfefhtddvffejvefhueetgeeludehteev udeitedtudenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrnhgurhgvshesrghnrghrrgiivghlrdguvg X-ME-Proxy: Feedback-ID: id4a34324:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 20 Jun 2023 16:55:48 -0400 (EDT) Date: Tue, 20 Jun 2023 13:55:47 -0700 From: Andres Freund To: David Stevens Cc: linux-mm@kvack.org, Peter Xu , Hugh Dickins , Andrew Morton , Matthew Wilcox , "Kirill A . Shutemov" , Yang Shi , David Hildenbrand , Jiaqi Yan , linux-kernel@vger.kernel.org Subject: Re: [PATCH v6 4/4] mm/khugepaged: maintain page cache uptodate flag Message-ID: <20230620205547.qzmivkjox2hkpzmm@awork3.anarazel.de> References: <20230404120117.2562166-1-stevensd@google.com> <20230404120117.2562166-5-stevensd@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230404120117.2562166-5-stevensd@google.com> X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 3B3242001A X-Stat-Signature: ijrwr9j6pdpkffixgmzaaufohd9p686h X-Rspam-User: X-HE-Tag: 1687294551-257407 X-HE-Meta: U2FsdGVkX1+Ghv9sHZu41zNTZ1L5LZWrJrgTrR14wjv8292p7HApWpVEOQ46/2KI6tH5zmvVA4MfQT2SzGw7wsvlG8hbCAVgOiuoxwt6kAfoT8WULS1AWwp+GFVMqcCA2sk4dnJI1DxE8Jdz8stHgNlGUwtt7sKscJJ3vwuqSrgPhuJYr9ubMmHD/U8j24KeGzcdrP5y7tH4fPKnpj0D4ozZjEr6qf3TBac0Lv+FrOV05dB/dX8ZUJKgkdU7FzcEhLEjc/E/yw3tkV0WL2cfqimBdVWi4QCXPrAqBnS4moiIU2e+YIeUUp3D9WfkPfqbmFiqNYOvAYcj/3UP0h8PoAYxOEKVxXmTyU6ENteRKOIikABAnq6gIZ+znOJZ36P552mNGqSZ+fFF0w7wUh0IdZhttmWcr56BqQrkwdbv5+Fcu7rQP3b1vlox95C6XhXE3cU9vT6+PI4Y8DsiOIRYsJRFT88SsJKGFCuGQzG+DgQ6zttodxYXdIWKAf4giYNHwbcCuOIYFTkOZ7uUIR43wFvHk20PTBV6K7jPUx2bNLC4GhbJxxeIbx0mkKQ44FDAVxhbbVwbMLZu7EP6oaZgZS1uiyw/LUUjGcc1oW80mVo7HOb10vmfHsAbkBu4BEtdXXWOmS34cA9P6LZHNrZsnJbDlZH8YLIXhfxwGqUBsZxKKj8LObMpG09g67Dxj2fwqTbIK3b9NKzEU6wDRMM//ugxMG2paer9YTlmxaBjUo6m5x8DRxpXhgLFCK6DuKBKp90sINSn9fh30vgLrkM+xNe8IQUmWw6bU9fF5kn1c37ZHnu9K5nIT89tmHKmkJogb62Zk/hl6Bh7dD0woPmS+s1yOKNKY7gG8QXn0YMnOV9wl66SjcodQ+faKBhjQdEqDjP395h2Jn6edriJP/YKKs6N+0NqAtngeQtwv32FEtMjGInB3dP69rYdrjC99T4OeVPuxruEsQZXzZLfjjq t/mC7Pf1 iAQftA8iJ0kL8G/pGVUtOHYhU5NFTuTmx2y5C0O7G1efeK9n+7qXfUT0haTtliQJR4Js/TWNj8cQW6Od7y9o39VDtcMt8c9Owm3c+/HKQbpKtm33QngpC5qP8AEtaJ+YB7/zLLL1ab/8hKSa0BCWlxEmLR6PrpKvChFzkOG1BzgFT63POtS3GJ0Vc+j9xRJCwCF5QXmAXEXPuwog5tHjNW8p6fTqud3DjcZ2gI/wzVsfd9Ja2R1iSZxeLvJoOHbGgvqIK9ezpJ5DLPwf+gg9FzNSy+Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, On 2023-04-04 21:01:17 +0900, David Stevens wrote: > From: David Stevens > > Make sure that collapse_file doesn't interfere with checking the > uptodate flag in the page cache by only inserting hpage into the page > cache after it has been updated and marked uptodate. This is achieved by > simply not replacing present pages with hpage when iterating over the > target range. > > The present pages are already locked, so replacing them with the locked > hpage before the collapse is finalized is unnecessary. However, it is > necessary to stop freezing the present pages after validating them, > since leaving long-term frozen pages in the page cache can lead to > deadlocks. Simply checking the reference count is sufficient to ensure > that there are no long-term references hanging around that would the > collapse would break. Similar to hpage, there is no reason that the > present pages actually need to be frozen in addition to being locked. > > This fixes a race where folio_seek_hole_data would mistake hpage for > an fallocated but unwritten page. This race is visible to userspace via > data temporarily disappearing from SEEK_DATA/SEEK_HOLE. This also fixes > a similar race where pages could temporarily disappear from mincore. > > Fixes: f3f0e1d2150b ("khugepaged: add support of collapse for tmpfs/shmem pages") > Signed-off-by: David Stevens I noticed that recently MADV_COLLAPSE stopped being able to collapse a binary's executable code, always failing with EAGAIN. I bisected it down to a2e17cc2efc7 - this commit. Using perf trace -e 'huge_memory:*' -a I see 1000.433 postgres.2/1872144 huge_memory:mm_khugepaged_collapse_file(mm: 0xffff889e800bdf00, hpfn: 46720000, index: 1537, is_shmem: 1, filename: "postgres.2", result: 17) 1000.445 postgres.2/1872144 huge_memory:mm_khugepaged_scan_file(mm: 0xffff889e800bdf00, pfn: -1, filename: "postgres.2", present: 512, result: 17) 1000.485 postgres.2/1872144 huge_memory:mm_khugepaged_collapse_file(mm: 0xffff889e800bdf00, hpfn: 46720000, index: 2049, is_shmem: 1, filename: "postgres.2", result: 17) 1000.489 postgres.2/1872144 huge_memory:mm_khugepaged_scan_file(mm: 0xffff889e800bdf00, pfn: -1, filename: "postgres.2", present: 512, result: 17) 1000.526 postgres.2/1872144 huge_memory:mm_khugepaged_collapse_file(mm: 0xffff889e800bdf00, hpfn: 46720000, index: 2561, is_shmem: 1, filename: "postgres.2", result: 17) 1000.532 postgres.2/1872144 huge_memory:mm_khugepaged_scan_file(mm: 0xffff889e800bdf00, pfn: -1, filename: "postgres.2", present: 512, result: 17) 1000.570 postgres.2/1872144 huge_memory:mm_khugepaged_collapse_file(mm: 0xffff889e800bdf00, hpfn: 46720000, index: 3073, is_shmem: 1, filename: "postgres.2", result: 17) 1000.575 postgres.2/1872144 huge_memory:mm_khugepaged_scan_file(mm: 0xffff889e800bdf00, pfn: -1, filename: "postgres.2", present: 512, result: 17) for every attempt at doing madvise(MADV_COLLAPSE). I'm sad about that, because MADV_COLLAPSE was the first thing that allowed using huge pages for executable code that wasn't entirely completely gross. I don't yet have a standalone repro, but can write one if that's helpful. Greetings, Andres Freund