From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBAEDC30653 for ; Thu, 4 Jul 2024 11:41:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5F4DF6B00DD; Thu, 4 Jul 2024 07:41:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A5006B00DE; Thu, 4 Jul 2024 07:41:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46C436B00E0; Thu, 4 Jul 2024 07:41:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 231926B00DD for ; Thu, 4 Jul 2024 07:41:58 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id B3934A402F for ; Thu, 4 Jul 2024 11:41:57 +0000 (UTC) X-FDA: 82301881074.10.1E7E55E Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.21]) by imf22.hostedemail.com (Postfix) with ESMTP id 4029AC0015 for ; Thu, 4 Jul 2024 11:41:55 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=CpHFzQeK; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf22.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 198.175.65.21) smtp.mailfrom=kirill.shutemov@linux.intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720093297; a=rsa-sha256; cv=none; b=salRltRGV47+7TMBahEdzWd4TPgZ7+noJUectWzCa57qpKY3iE38tJjaLjWR8bgls9kOEe 8zvqJg/DhCMelYBDdXEj7wYZ7hpKUZAwVdUK8nRyjdhJiajRy23UUq70pkkiaqeEh+UIc1 jwGGf16qtlY9LTckLsDwJS5GwvRXn6A= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=CpHFzQeK; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf22.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 198.175.65.21) smtp.mailfrom=kirill.shutemov@linux.intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720093297; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1+l53xG3CD6shhYVrSHctfBZy9zerkunkSKbnLMhl0A=; b=C9nmIaapE40Qm8CDV8j2NZRNpd8G2LX6PBqgb6dV6JFe1re1L+Yp5PQUc7/cLeaX0x9wMJ StiefqkhKSl8AsKi/fmrT7/812s8MB7GkxR6IAMPEDix/68WUvyiC1mKjKV0miX6zdmBZR qOn4aoS2kCLHFOgFsFYTgJ8vyarT3zU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1720093315; x=1751629315; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=C5eYtAI3l7ENonq5tzLecy1Q8Boih2aXF951n8Ey2yo=; b=CpHFzQeKOeh8+4Nvq8pAg8SbH5/rsajAnajQz0eCaeK4bIjp13+wdWJq +zgNd3hkEp+gA669Bf/JHD7zi+jG2Aqx763BobUKsUPWuREeadWEXFpY1 29pBMHxciRRmVipB8mIfZ38/lUfycPtw7s/ZpjmbEUuzWjdsGVgcFXmIa hZRpMbJsmQcPJU7eLzsrsmKAvNphN2TlhvClj/jl17XhUBsGi/eJqSj2H uNUDjJAG1xT86RjpfT7IljyZsFHLKhZeuzBMnYJuRUHO/r0TGw4/ieP7J um3X2Z5G3L3SFAhP4ImlSb6ce9/45KitEufFflGrmrS6ffbr09CD4b+Mn g==; X-CSE-ConnectionGUID: Eglmz8FgTmyUXOZgVRQdEg== X-CSE-MsgGUID: nNt05RFlQoCTDuFvDUP/0w== X-IronPort-AV: E=McAfee;i="6700,10204,11122"; a="17319485" X-IronPort-AV: E=Sophos;i="6.09,183,1716274800"; d="scan'208";a="17319485" Received: from fmviesa010.fm.intel.com ([10.60.135.150]) by orvoesa113.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2024 04:41:53 -0700 X-CSE-ConnectionGUID: kpfzoIK+Q8WHilb7NIXUWw== X-CSE-MsgGUID: gde/OwcnQtCkfYJLfGEXiw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,183,1716274800"; d="scan'208";a="46646553" Received: from black.fi.intel.com ([10.237.72.28]) by fmviesa010.fm.intel.com with ESMTP; 04 Jul 2024 04:41:52 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 4E2BA22B; Thu, 04 Jul 2024 14:41:50 +0300 (EEST) Date: Thu, 4 Jul 2024 14:41:50 +0300 From: "Kirill A. Shutemov" To: Ryan Roberts Cc: Hugh Dickins , Mel Gorman , Linux-MM , Catalin Marinas , David Hildenbrand , Matthew Wilcox Subject: Re: huge zero page confusion Message-ID: References: <1cfae0c0-96a2-4308-9c62-f7a640520242@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1cfae0c0-96a2-4308-9c62-f7a640520242@arm.com> X-Rspamd-Queue-Id: 4029AC0015 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ms8kpamqzd61u3jafbk5gueti5941hoe X-HE-Tag: 1720093315-957589 X-HE-Meta: U2FsdGVkX1+lc+iGLVxSRh0fjEYAwl9+jLk5n2/R3XgGWreNK8Emei3DA5785ze88qNg0XLEpUEmGBfm6UDQLyt3sL8Msi/FodLQoJqUQBMZhRodim50WyirzjhgcF4L2PAkjr4IEtQxLU2yJr0dz/OE21v6SaLeOv53zM0RHSN2iPsmn9/l04n0bprQmGVdtX9bl4m4sa8zUvt6sUu9oxa+yauGM4UVtjQp5emiJwmfKxcgCrcAlI4Ou8Z+xQe3zh5S8aoF1hTha4uBJELlqMsMz6DsOgpUwYtoXmGfiD8aLqbmQQ+nuztWmfa6c2h5bCk7lUICVijm4GbgZqgURez9JUP46ES1z20RRqrkM90+ynRnXFEbPqMx6w6WWRRKm5feo0Ahxqamh64eKSQvrng3QDbnRpcJagl1BS/KW3K+hguw6QR4uOGIFfxfXZAvfBcZKQtFjdPcwzsik934w6j6nOfJXdugTJTgqbACzjz4iPZYXbQEWsGRN3h4DqxPXTAgeZXHLZM/qTLgsc8sxZoEZS9xlyyZtJqgBOqENIOHjyMasZH8EboLvbAjOops+h/JGtVyhHcLCbfGUhfnwY30H69dLjyjAPnSI2NfcVj0ZiXwJ1Mko2L7EhZmwyzljm+RPchBUIyEmmo72CPGrMNJySfsQ6geQ7uAJ0DH9FeEPxEBZE9CpTmdKDiZ3FUqo9dI/+AsvPcISewtm4yfLC74Bk9Ms7gNDf0v0by2mnpJDyIlNjB2ieZv4zETP0MPyLVjWpzXuaW9yECeYwMiTIkVzNECBJ2uQhVMAMeISJ5EgglQglk+mS0jWusEIpL90dzuN8Du8PhIUyPaoqpSp3XMr5gN9t+of6Aqu7OJ/bImOYGR1rCd1nMtMEctxPNyALHjSw9xN0jy4i62qSwasxRCogNJs3qlQ0P1NCPVZTWnJfx3IoBlcdQmkjqVC0/vEv5JRtExrRIa9/+x+E6 LsAo5bf+ 0BYnU3XGdMzEJzl1nlaOGHeSLXTpNSXJTU1lKFHtvryqbxK0TMYL8C8TnJ5rB2yLEfahQ2zduOPNt7l/s8IKXBJWu7++i/JnsizxMq4OrOP7+wiE7CMX+U5Pt3lTXRNCCMpkAa+rO56iHRoKQqclXLRI+ROvVrAppuaCD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jul 03, 2024 at 06:37:48PM +0100, Ryan Roberts wrote: > Hi Kirill, Hugh, Mel, > > We recently had a problem reported at [1] that due to aarch64 arch requiring > that atomic RMW instructions raise a read fault, followed by a write fault, this > causes a huge zero page to be faulted in during the read fault, then the write > fault shatters the huge zero page, installing small zero pages for every PTE in > the PMD region, except the faulting address which gets a writable private page. > > A number of ways were discussed to solve that problem. But it got me wondering > why we have this behaviour in general for huge zero page? This seems like odd > behaviour to me. Surely it would be less effort and more aligned with the app's > expectations to notice the huge zero page in the PMD, remove it, and install a > THP, as would have been done if pmd_none() was true? Or if there is a reason to > shatter on write, why not do away with the huge zero page and save some memory, > and just install a PMD's worth of small zero pages on fault? > > Perhaps replacing the huge zero page with a huge THP on write fault would have > been a better behavior at the time, but perhaps changing that behaviour now > risks a memory bloat regression in some workloads? Yeah, I agree that WP fault on zero page page should give THP. I think treating zero page as none PMD on write page fault should be safe and reasonable. Untested patch is below. diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 2aa986a5cd1b..04c252303951 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -552,6 +552,11 @@ static inline bool thp_migration_supported(void) } #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +static inline bool pmd_none_or_zero_folio(pmd_t pmd) +{ + return pmd_none(pmd) || is_huge_zero_pmd(pmd); +} + static inline int split_folio_to_list_to_order(struct folio *folio, struct list_head *list, int new_order) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 89932fd0f62e..fdd5236004bc 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -951,7 +951,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, __folio_mark_uptodate(folio); vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); - if (unlikely(!pmd_none(*vmf->pmd))) { + if (unlikely(!pmd_none_or_zero_folio(*vmf->pmd))) { goto unlock_release; } else { pmd_t entry; @@ -1536,8 +1536,7 @@ vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf) vmf->ptl = pmd_lockptr(vma->vm_mm, vmf->pmd); VM_BUG_ON_VMA(!vma->anon_vma, vma); - if (is_huge_zero_pmd(orig_pmd)) - goto fallback; + VM_BUG_ON(is_huge_zero_pmd(orig_pmd)); spin_lock(vmf->ptl); @@ -1606,7 +1605,6 @@ vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf) unlock_fallback: folio_unlock(folio); spin_unlock(vmf->ptl); -fallback: __split_huge_pmd(vma, vmf->pmd, vmf->address, false, NULL); return VM_FAULT_FALLBACK; } diff --git a/mm/memory.c b/mm/memory.c index 0f47a533014e..cc12deeb0593 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5488,15 +5488,15 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, if (pud_trans_unstable(vmf.pud)) goto retry_pud; - if (pmd_none(*vmf.pmd) && + vmf.orig_pmd = pmdp_get_lockless(vmf.pmd); + + if (pmd_none_or_zero_folio(vmf.orig_pmd) && thp_vma_allowable_order(vma, vm_flags, TVA_IN_PF | TVA_ENFORCE_SYSFS, PMD_ORDER)) { ret = create_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; } else { - vmf.orig_pmd = pmdp_get_lockless(vmf.pmd); - if (unlikely(is_swap_pmd(vmf.orig_pmd))) { VM_BUG_ON(thp_migration_supported() && !is_pmd_migration_entry(vmf.orig_pmd)); -- Kiryl Shutsemau / Kirill A. Shutemov