From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D048EE4998 for ; Wed, 11 Sep 2024 12:10:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7A7C694002C; Wed, 11 Sep 2024 08:10:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 731E8940021; Wed, 11 Sep 2024 08:10:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D21494002C; Wed, 11 Sep 2024 08:10:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3F68A940021 for ; Wed, 11 Sep 2024 08:10:35 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E9331141333 for ; Wed, 11 Sep 2024 12:10:34 +0000 (UTC) X-FDA: 82552340388.24.517B623 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf16.hostedemail.com (Postfix) with ESMTP id 1A624180016 for ; Wed, 11 Sep 2024 12:10:32 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; spf=pass (imf16.hostedemail.com: domain of dev.jain@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=dev.jain@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726056581; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HV/E+heeEbOeTiaUT49IJEpXkJhjFnE7RMTcnI1E6Lw=; b=t8Spqe3kMiNbsRUVbGr1kXQK0qRocSjRojrBNmo42A7RGYQfCYPPRST5eKOtuFALcnNxwZ 7YeF4u+aAfpThtU07J9csB8dEdMByzdjlgtMV/uiKW+XeT/SEO/huZm3GL/bmjYI1yOtyU uPO1ShSceUQ7dNboXinggxt5CouWsXE= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; spf=pass (imf16.hostedemail.com: domain of dev.jain@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=dev.jain@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726056581; a=rsa-sha256; cv=none; b=zFJjKiXeoy/y+VDs+iVtff9Xx8eMD2tnoUC+K8IY4UMDYyvLrILeqDYw5pmFpvSm1YA/sw FikTuDq6fGsGZbaKSEEJKJ6aZxT3QfDhvRQye89AfLDKCXhWB8Hrsf+DDFNXR9ZiwGbp7T 0tD8qfPhK+rxPzB6J59nE5960+4yE8s= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DB9491007; Wed, 11 Sep 2024 05:11:01 -0700 (PDT) Received: from [10.162.40.31] (e116581.arm.com [10.162.40.31]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F27653F66E; Wed, 11 Sep 2024 05:10:23 -0700 (PDT) Message-ID: Date: Wed, 11 Sep 2024 17:40:20 +0530 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 2/2] mm: Allocate THP on hugezeropage wp-fault To: David Hildenbrand , akpm@linux-foundation.org, willy@infradead.org, kirill.shutemov@linux.intel.com Cc: ryan.roberts@arm.com, anshuman.khandual@arm.com, catalin.marinas@arm.com, cl@gentwo.org, vbabka@suse.cz, mhocko@suse.com, apopple@nvidia.com, dave.hansen@linux.intel.com, will@kernel.org, baohua@kernel.org, jack@suse.cz, mark.rutland@arm.com, hughd@google.com, aneesh.kumar@kernel.org, yang@os.amperecomputing.com, peterx@redhat.com, ioworker0@gmail.com, jglisse@google.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20240911065600.1002644-1-dev.jain@arm.com> <20240911065600.1002644-3-dev.jain@arm.com> <783a0d91-2910-4446-a979-c681dde402ec@redhat.com> Content-Language: en-US From: Dev Jain In-Reply-To: <783a0d91-2910-4446-a979-c681dde402ec@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 1A624180016 X-Stat-Signature: ei145kkdekq1wztfbr8nonjntfhsm9pk X-HE-Tag: 1726056632-183304 X-HE-Meta: U2FsdGVkX18zoD/ZFn7zjrIM7XI0gdcnLfAogV1OvPLpR1Ov+pAIoMJgi7JRJD6YWeC0GIj19fF3uU+qBGlLPEIOsHRtOJqbdzVe05Tdx18xWStLcZx2c9eH5ZcGFrLbPQBPMDdS4lFRmjG4/SvizX5EoCt1+dZnrVeTG0FI9dZkM04koV6y7RZZg/r4LLGJRa7zVGatew5ETRZ2PeYpTdwvxBg2YrXYmHB7RCrgCAOz03Tz1W1latKTrWxwLtXpKMv7W8OL/vacWvRWurlVl95R8yEP8OZR8+3HvPr0aBuyXdVRvMpW4hM5nrKKK6crqO/+Zo7mcOSsO5evKdCzawiMGOlTzigIihyFcuCTQq07Aj/IiDaqSoqzbzh1nixHM/ksf6lllNMJrynfHSnmBbwUlxlx0lFqh14Rv9vJdIJJMASPlJ3EqNVOpxEXVQZDDIfpgWiMqOCUe3N92TxFtDVxagJLht1y3wWG4LaGz7pbyTysvHcMTSVk3TDbtjmKaRREO1bruAcHK8SiSTTtTvYiY9/Gwlh/xchQu8aiGM2SZR9Dj/1Y+DXc2kA/B/5TC778uHZbVrkqEbmgzZvYmfhtwz3jK+7CdWuAA9yNpNBXV16Zn9doYxzzgfz0Z9U778Mjso2WXLFmFMQVAPqTSn5j0nhnnAqkCKE3c3NsbrLt7AbTRs0jW/AZNgeWU1R4o5zz3PCk4V3sPm4hC7VldAwh01ihBstIMl9clybX9n2g/ArhjEh0hpwcbN9zZZ2u+DwQEg9wiLNISfIHpsiDi9tY/7Bo4+03Ylm+LGcOedmJyoO44EdlMV7nLj9eIpI/M4Q/KBdrXNhj/SZV9Ev6UFVstIl7uoqfx0FVphaUc/udvKw9Dq2qUGv9kK/otzshHzYQ5+E5QiWRrIgez8oBL4CQ4r3wfr4En7mlbbacIrGBCPC3ypzfKGnNomRu5Q1u+n3OwtAy6zgvd2zw06Z XACYOOQ8 r40XJwDbzJYCGCohIkFMpMu18kmiER8iY3JnSgKUJcRACQBrhmlseC6pVaCKbP/CH+jSYtHYFXaDIJZ0FOk2HWW1uGb6K1hLSXwQRib9sBLR0uzv8/a9Uj6KBm7jer06N9odAIamprUTOUBWT0Q3FUgJg59eBB01DqlRsYrC7VKM18SUdsBXC7y8sL6OAVyinFmOUNHbhye4qkggAt56JmWVzil8xT6H0P0baDxDe/Zejbp4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 9/11/24 15:06, David Hildenbrand wrote: > On 11.09.24 08:56, Dev Jain wrote: >> Introduce do_huge_zero_wp_pmd() to handle wp-fault on a hugezeropage and >> replace it with a PMD-mapped THP. Change the helper introduced in the >> previous patch to flush TLB entry corresponding to the hugezeropage. >> In case of failure, fallback to splitting the PMD. >> >> Signed-off-by: Dev Jain >> --- >>   mm/huge_memory.c | 52 +++++++++++++++++++++++++++++++++++++++++++++--- >>   1 file changed, 49 insertions(+), 3 deletions(-) >> >> diff --git a/mm/huge_memory.c b/mm/huge_memory.c >> index b96a1ff2bf40..3e28946a805f 100644 >> --- a/mm/huge_memory.c >> +++ b/mm/huge_memory.c >> @@ -987,16 +987,20 @@ static void >> __pmd_thp_fault_success_stats(struct vm_area_struct *vma) >>   static void map_pmd_thp(struct folio *folio, struct vm_fault *vmf, >>               struct vm_area_struct *vma, unsigned long haddr) >>   { >> -    pmd_t entry; >> +    pmd_t entry, old_pmd; >> +    bool is_pmd_none = pmd_none(*vmf->pmd); >>         entry = mk_huge_pmd(&folio->page, vma->vm_page_prot); >>       entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); >>       folio_add_new_anon_rmap(folio, vma, haddr, RMAP_EXCLUSIVE); >>       folio_add_lru_vma(folio, vma); >> +    if (!is_pmd_none) >> +        old_pmd = pmdp_huge_clear_flush(vma, haddr, vmf->pmd); > > This should likely be done in the caller. > >>       set_pmd_at(vma->vm_mm, haddr, vmf->pmd, entry); >>       update_mmu_cache_pmd(vma, vmf->address, vmf->pmd); >>       add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR); >> -    mm_inc_nr_ptes(vma->vm_mm); >> +    if (is_pmd_none) >> +        mm_inc_nr_ptes(vma->vm_mm); > > And this as well. > > No need to make this function deal with this if the callers exactly > know what they are doing. Sure, thanks. > >>   } >>     static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf) >> @@ -1576,6 +1580,41 @@ void huge_pmd_set_accessed(struct vm_fault *vmf) >>       spin_unlock(vmf->ptl); >>   } >>   +static vm_fault_t do_huge_zero_wp_pmd(struct vm_fault *vmf, >> unsigned long haddr) > > Is there a need to pass in "haddr" if we have the vmf? Was passing it because it was getting used many times. But nowhere do vmf and haddr get both passed in the codebase, so I'll drop it for cleanliness and consistency. > >> +{ >> +    struct vm_area_struct *vma = vmf->vma; >> +    gfp_t gfp = vma_thp_gfp_mask(vma); >> +    struct mmu_notifier_range range; >> +    struct folio *folio; >> +    vm_fault_t ret = 0; >> + >> +    folio = pmd_thp_fault_alloc(gfp, vma, haddr, vmf->address); >> +    if (unlikely(!folio)) { >> +        ret = VM_FAULT_FALLBACK; >> +        goto out; >> +    } >> + >> +    mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma->vm_mm, >> haddr, >> +                haddr + HPAGE_PMD_SIZE); >> +    mmu_notifier_invalidate_range_start(&range); >> +    vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); >> +    if (unlikely(!pmd_same(pmdp_get(vmf->pmd), vmf->orig_pmd))) >> +        goto release; >> +    ret = check_stable_address_space(vma->vm_mm); >> +    if (ret) >> +        goto release; > > The clear+flush really belongs here. > >> +    map_pmd_thp(folio, vmf, vma, haddr); >> +    __pmd_thp_fault_success_stats(vma); >> +    goto unlock; >> +release: >> +    folio_put(folio); >> +unlock: >> +    spin_unlock(vmf->ptl); >> +    mmu_notifier_invalidate_range_end(&range); >> +out: >> +    return ret; >> +} >> + >