From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6C80EB64D7 for ; Tue, 20 Jun 2023 21:11:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5FE498D0002; Tue, 20 Jun 2023 17:11:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5AD378D0001; Tue, 20 Jun 2023 17:11:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 475418D0002; Tue, 20 Jun 2023 17:11:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 36D2F8D0001 for ; Tue, 20 Jun 2023 17:11:39 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1297B4063C for ; Tue, 20 Jun 2023 21:11:39 +0000 (UTC) X-FDA: 80924372718.13.04CC0C5 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf08.hostedemail.com (Postfix) with ESMTP id 5012D16001F for ; Tue, 20 Jun 2023 21:11:35 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="cm/PCufU"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687295497; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gU8UWWJWFgoQ9BWqYzFjtCCMQSFbRe/QWEx3Oetwzeo=; b=rsw3xgj64e6FR1D9Xxtq3V2CCd4bt6RSTDQlZn//fla9i3TA0HxSe3VI14stFDhe0oe+KK dk501gGj3VCmFK5Q6PEhN/BPJl62lS/hi8YLw2EU5sz64jU6xidEdcLHgJbuFHmzXjZo+q vR+BePln1XeUvUpBnNDOXzEkUPK4zt4= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="cm/PCufU"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687295497; a=rsa-sha256; cv=none; b=dIeWKKTdsOdwMT5Clmc8v1YnyMHc8TYGxekRGmkGTWwicPlEqY/qUeQ6irG8rPF1jw5Mbw kW9C7shTVau73qebYFbq3wJjkINrbel3LcHhk1HU2YCLwA2ognsZfJGaf2WBa4e7Q/PlLc Fx2m7agebcAJTNpPZb/iOZhpxBamMLk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687295494; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gU8UWWJWFgoQ9BWqYzFjtCCMQSFbRe/QWEx3Oetwzeo=; b=cm/PCufUuDwaEwMZA635tkAMwtyc9UrV6EcS+BpidE5BIhbyRVmpyYJo8qZtx+ljMzouM+ tTSFeloPx55nXZ0kFS9BtIZspx6WPBTLgimbukLDiL6yFKUlRA979gIAiV2WSr+9HCbL6l W/NJHGW1QmbSmboDCMuDXT5vk3z6Avo= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-423-2iUCvLUIODODi9VxdJbFmA-1; Tue, 20 Jun 2023 17:11:33 -0400 X-MC-Unique: 2iUCvLUIODODi9VxdJbFmA-1 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-62ff893f03bso12463926d6.0 for ; Tue, 20 Jun 2023 14:11:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687295492; x=1689887492; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=gU8UWWJWFgoQ9BWqYzFjtCCMQSFbRe/QWEx3Oetwzeo=; b=TXnGmhSmlSochCTiWKOdSzQsNqdZwBk8hYOCgqfWfqaf23UUugBogI830x6TfjwTaS XaV6bYKAnccg8Ehc0wdrQT+2PQKhNqKtpfzSiH1R3F8fRt1eMYlzEcVNvCcWReuIbEp2 ggepgK1AsLVh3yRWZs4DZWSsh4gnj0izaLcAuKzSujsbFp8Tb5wpDt+Wq6c7fCacfqKg t9RkhUeNv0w/uelziqTTl6x7Vtysn+kliE/+0KRqV48V+4pHjeTx9aomcSBzcUL29p8q RkRRta4fZFYiR+JHUsNI6s8sOuUEFJU1AJa3mGrCvoUNgG8Pe0NVOpflmjqQPn19nXAo 6obA== X-Gm-Message-State: AC+VfDwI7Vqt2mfI7uhGryyUZFnMwDIl3tnyXuCEi7Tb6m1GDeB1129+ 5pj0HmCioZG0inxeRdrCW7zqWm/RWpm/N0f9GnWdDJFkxH6uzyVxr5X2YJAqDjaVgIRbS5gtXHh PGDQt5Yib5wk= X-Received: by 2002:a05:622a:292:b0:3ff:3151:a8a7 with SMTP id z18-20020a05622a029200b003ff3151a8a7mr2091060qtw.3.1687295492488; Tue, 20 Jun 2023 14:11:32 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ67KxXxa/fr98eOy/24gQ4FhGS1F/cuhF47oElajda/uk0rdsXxswcKRGR+v+VjpN2/lzbTOQ== X-Received: by 2002:a05:622a:292:b0:3ff:3151:a8a7 with SMTP id z18-20020a05622a029200b003ff3151a8a7mr2091035qtw.3.1687295492139; Tue, 20 Jun 2023 14:11:32 -0700 (PDT) Received: from x1n (cpe5c7695f3aee0-cm5c7695f3aede.cpe.net.cable.rogers.com. [99.254.144.39]) by smtp.gmail.com with ESMTPSA id fc5-20020a05622a488500b003ff243b2509sm1539586qtb.71.2023.06.20.14.11.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 14:11:31 -0700 (PDT) Date: Tue, 20 Jun 2023 17:11:30 -0400 From: Peter Xu To: Andres Freund Cc: David Stevens , linux-mm@kvack.org, Hugh Dickins , Andrew Morton , Matthew Wilcox , "Kirill A . Shutemov" , Yang Shi , David Hildenbrand , Jiaqi Yan , linux-kernel@vger.kernel.org Subject: Re: [PATCH v6 4/4] mm/khugepaged: maintain page cache uptodate flag Message-ID: References: <20230404120117.2562166-1-stevensd@google.com> <20230404120117.2562166-5-stevensd@google.com> <20230620205547.qzmivkjox2hkpzmm@awork3.anarazel.de> MIME-Version: 1.0 In-Reply-To: <20230620205547.qzmivkjox2hkpzmm@awork3.anarazel.de> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Rspamd-Queue-Id: 5012D16001F X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: xn6yxij4oezodgkeph5uyif41gtx3xk7 X-HE-Tag: 1687295495-244116 X-HE-Meta: U2FsdGVkX1+fIZkzXkwdVcRWT8Jramhx3FrqImVAoX3IsnhvwunwcjtgZy+6wZCcWCwXaUErER5V3C86MMuHHmBnmRyKQ69NYpy4uWowz6wKIDqzA/xB2UlUCPnKitI/4x/2XQPdW8wn6g3YRpRNecqI5XrGC5ntlRtEWjpducbJy+JdWrXTdCM8PwFX1NiFIKi6nfk5bScyXfzraZqaIx8tFf9ZVi5BR6GGKqmC1gBYtN7uRZOESpaOJjxtxPnLtUxR7BfTsVHcMrHOD1abhjnsd9TKw9Dm8AmqhLJDXwjbjwq/YKML0I78HMZ4D9rER8pI82gUl+mj1Jf1T2cuBA1+rvSypuyuRHGvubkIh7jwQMu9UsCFumh9b4TvS4caaglAf3toDCFA8G5IazMn95C4MELHv9KQZLMN4Z7nnkbhcAUbJyBIf7C+HnvAfEOdQhC8sbyz/GN7iBUU3QHZLckNaw+IN8uDKTTiyT4UIdZGzqWt8iWMJdUdL+QgiBcD156oKXCfwWp2WAxdN/cw5aHQzWmEaOviW7n2vT9PIB4lD11GinWEpSClcX2E1rg1lz9QkEDipsAIOR3nJvPwaoUo09qq6sd50ROGMHl8GfVSCyMjr7r9sbX04bSLFHV53yZChv6+44bnQmWFP5mibt1vzIyMzL9TWEF+zHrqcuCCMRVnkGtqAo2m9CK6lX5/4GaXClQtRLg1DrS8gHl2e7zIubNwjmfv4+ubm+0kifJipKuWkHOvP0wDOwNwi+L+tFDMI9oIKO1Of1+8NwAaIoFFyziYX4GhthPoHcAIoHE8iywIa4nM7UKJOxptlEbeM5hLOLHhYtNxRYT+ajUXNS3ikYQ3pU9G0ytNHrdJPcuRCsTg5n67TRiIBnzYb10x4Cnx14C6NUK31kTdp9tW4tHu3xxoXoA9C7FwsZv6ZkROdtANCR/GCQo1sariZiz8loC1adiXSzdW7Mfu/e/ Usy9as2O lP2pVFjDr5BwiQWJrlZQYv7RVnfxWN/lx5ZVYNSuwPY1AtXbQ99lEySYNWurA+QJ8C4F5r8v7rKnSR8NJqnHJjW74JBPRcXIz8WtoRltzHpEz6gZj6oKOExJhFnYeq6j0KO3G5p3iE68Ik6EQlNM4QwqNqnNXD4MoWoqRlsGXnIuy/LitfNSmr0izHOsJmbp9OZdedGpdBFTq1rO2byEyuLk5oMCc7rEH+b6l7DUvqtGHJ9yDuUc12QPhXFtPlgzhOUVqOT1g2T2BzK6r9xiXE0ds5PBebMr2t94s7pyHkdmGq4LraaO9sg0V43yh2H0PMH3Hh3PFGb/OiE0q7GurlD54jLx+ddn9V7eb2eZXXudTvcaa1GC35eciAHMzAhHVK/x6zrJXdR290RDySvYwG2fiz8E2b3S/+0nSxdq9RDLh0jac3xWWfVpl5v2jAxVsexAwLzV7uFnEvJKdvs+xYa7DDIPQnbBLjqHtWeJ0Vmuh1PA3jZ6KbJwVfgEBUrok3HWfHfaPfzyzpo3/2lz0jJUuuuTvF+fS3reiRHDR4yVMEi9a5YWb/HDmew== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jun 20, 2023 at 01:55:47PM -0700, Andres Freund wrote: > Hi, Hi, Andres, > > On 2023-04-04 21:01:17 +0900, David Stevens wrote: > > From: David Stevens > > > > Make sure that collapse_file doesn't interfere with checking the > > uptodate flag in the page cache by only inserting hpage into the page > > cache after it has been updated and marked uptodate. This is achieved by > > simply not replacing present pages with hpage when iterating over the > > target range. > > > > The present pages are already locked, so replacing them with the locked > > hpage before the collapse is finalized is unnecessary. However, it is > > necessary to stop freezing the present pages after validating them, > > since leaving long-term frozen pages in the page cache can lead to > > deadlocks. Simply checking the reference count is sufficient to ensure > > that there are no long-term references hanging around that would the > > collapse would break. Similar to hpage, there is no reason that the > > present pages actually need to be frozen in addition to being locked. > > > > This fixes a race where folio_seek_hole_data would mistake hpage for > > an fallocated but unwritten page. This race is visible to userspace via > > data temporarily disappearing from SEEK_DATA/SEEK_HOLE. This also fixes > > a similar race where pages could temporarily disappear from mincore. > > > > Fixes: f3f0e1d2150b ("khugepaged: add support of collapse for tmpfs/shmem pages") > > Signed-off-by: David Stevens > > I noticed that recently MADV_COLLAPSE stopped being able to collapse a > binary's executable code, always failing with EAGAIN. I bisected it down to > a2e17cc2efc7 - this commit. > > Using perf trace -e 'huge_memory:*' -a I see > > 1000.433 postgres.2/1872144 huge_memory:mm_khugepaged_collapse_file(mm: 0xffff889e800bdf00, hpfn: 46720000, index: 1537, is_shmem: 1, filename: "postgres.2", result: 17) > 1000.445 postgres.2/1872144 huge_memory:mm_khugepaged_scan_file(mm: 0xffff889e800bdf00, pfn: -1, filename: "postgres.2", present: 512, result: 17) > 1000.485 postgres.2/1872144 huge_memory:mm_khugepaged_collapse_file(mm: 0xffff889e800bdf00, hpfn: 46720000, index: 2049, is_shmem: 1, filename: "postgres.2", result: 17) > 1000.489 postgres.2/1872144 huge_memory:mm_khugepaged_scan_file(mm: 0xffff889e800bdf00, pfn: -1, filename: "postgres.2", present: 512, result: 17) > 1000.526 postgres.2/1872144 huge_memory:mm_khugepaged_collapse_file(mm: 0xffff889e800bdf00, hpfn: 46720000, index: 2561, is_shmem: 1, filename: "postgres.2", result: 17) > 1000.532 postgres.2/1872144 huge_memory:mm_khugepaged_scan_file(mm: 0xffff889e800bdf00, pfn: -1, filename: "postgres.2", present: 512, result: 17) > 1000.570 postgres.2/1872144 huge_memory:mm_khugepaged_collapse_file(mm: 0xffff889e800bdf00, hpfn: 46720000, index: 3073, is_shmem: 1, filename: "postgres.2", result: 17) > 1000.575 postgres.2/1872144 huge_memory:mm_khugepaged_scan_file(mm: 0xffff889e800bdf00, pfn: -1, filename: "postgres.2", present: 512, result: 17) > > for every attempt at doing madvise(MADV_COLLAPSE). > > > I'm sad about that, because MADV_COLLAPSE was the first thing that allowed > using huge pages for executable code that wasn't entirely completely gross. > > > I don't yet have a standalone repro, but can write one if that's helpful. There's a fix: https://lore.kernel.org/all/20230607053135.2087354-1-stevensd@google.com/ Already in today's Andrew's pull for rc7: https://lore.kernel.org/all/20230620123828.813b1140d9c13af900e8edb3@linux-foundation.org/ -- Peter Xu