From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59C82C369AB for ; Thu, 24 Apr 2025 09:40:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6472A6B00AE; Thu, 24 Apr 2025 05:40:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F2B56B00B1; Thu, 24 Apr 2025 05:40:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 492386B00B2; Thu, 24 Apr 2025 05:40:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 27DDA6B00AE for ; Thu, 24 Apr 2025 05:40:56 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 1A4CAB9A64 for ; Thu, 24 Apr 2025 09:40:57 +0000 (UTC) X-FDA: 83368443354.14.F7A2221 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf20.hostedemail.com (Postfix) with ESMTP id 4A9AF1C0006 for ; Thu, 24 Apr 2025 09:40:55 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf20.hostedemail.com: domain of anshuman.khandual@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745487655; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=np4v8rvRiTqO7Lw4N2PMb/7JYdrjHe9Ek1nHrQDN81E=; b=DcGqnEam6C4lBH2qHwqxJSUq6dpJKHF1CFBe3Yb+f6bJRPWU6ypU4UiZYdb/+pHGKdIDhH H+tR1C0rS9N3ByKsFum9olhjON7Fu1mOWRMGK/47yOMhCiMkQOQVWJTjn1rSR9TVJXvbwL 8CImSAK8XldvMpKFxYRhIs/9PTBhgyg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745487655; a=rsa-sha256; cv=none; b=3MexYobz+LjL3ikZl1ZRjGuJV5xT25cuvcrCx8AAM9DDvFoXELV5iR2Emzl13y+UWKPNCd /9821OXoBO+50DZ9nr8sfJ4ikpdcIkyUhX+IM6VnBfFD0Pj2TZsdSCEJqW8by7WTSApIOu XZ1S+wzuM5elU8vQyGY8j/SghROVDcs= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf20.hostedemail.com: domain of anshuman.khandual@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8AEEA1063; Thu, 24 Apr 2025 02:40:49 -0700 (PDT) Received: from [10.163.49.106] (unknown [10.163.49.106]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9FC063F66E; Thu, 24 Apr 2025 02:40:49 -0700 (PDT) Message-ID: <1e148263-1155-411b-9b1d-16599bd875cd@arm.com> Date: Thu, 24 Apr 2025 15:10:46 +0530 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 05/11] arm64: hugetlb: Use __set_ptes_anysz() and __ptep_get_and_clear_anysz() To: Ryan Roberts , Catalin Marinas , Will Deacon , Pasha Tatashin , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , David Hildenbrand , "Matthew Wilcox (Oracle)" , Mark Rutland , Alexandre Ghiti , Kevin Brodsky Cc: linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20250422081822.1836315-1-ryan.roberts@arm.com> <20250422081822.1836315-6-ryan.roberts@arm.com> Content-Language: en-US From: Anshuman Khandual In-Reply-To: <20250422081822.1836315-6-ryan.roberts@arm.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 4A9AF1C0006 X-Rspam-User: X-Stat-Signature: 4u5gpco75ey94okfqmjek4bxf37tiytn X-HE-Tag: 1745487655-19831 X-HE-Meta: U2FsdGVkX1/3TMPucGLz39IpakCtfS4uOsGM40nis0ZWclWxSId0gdstDrzsNdUykwZYdsBdf5BXAMIrpe5VI+Cq3IDLAZ3L/QZlQuFEDWu77SjNEYwnZk4+sbqPltZYJnCU5wk6k9cDGd7k+BznWbqyWyzDeD4yE3rCzl9PqL65d11ZDD0M9wzZpz3fe5+Fu71n8Mgmr4k1/zEBfVY9ek1kfSSvZHNMTerTXqwDnY2oVu/HVO6DHozhjs3seF+mS0hxHj97l2pqG7Re0Isa8Ukj8RsI0G8GulsvVufBWe6strWDdfiMa/1TIfvqjO6HMqyUWWd89RGHDTxCww5DDzlTKhsl8UfXs/tCIl5NZ7OJee7GB6eSaAqiYs0D353+uzzA2Njin6uVqK0FWyQNfRlyoKjr59C4dMH0kgKAXEygPm+GdKuvtxeLQ+f5NeF3sQKe/1sTlTOc/FHAsYNkH6ZxSGASAYWyGE6AzBhRitM/YC5W5I0X4+85TeYoL6y9ZRH8JbgzhEcqw0sgkcc2Eq0C5LcfE3L+/V8Q+almUsTUYjKtE9Vh/7VLFOKFO07ASU4bQFf4bLfX/OybmzKkZ49mbA2wj1fSM555zIKq/vO5Fmh/JmMO6Om454OfmCkPUEzAgA51CsYP/bpMTDIc8m6cvJyvuQmz0B7oYbHqzUKtFmPIYkB/cR2PM1Qy8nGG0l8Cs7I+9ZEjI6VD20OUZv18w0JbKeU1I8lRr/wb5+1GjuQL8FtTdQk1ZeOPcebjAqktB5K6kbMh6UZiLl/pN31HrT5i7qGMbEBuoSifYBIYS5rWgj4u27s/3CU+BkdszhWVU/W+2E+hM+Ijhj/etEL5ydPTkBK3EHLQPM18UTO0JJyGGkMV73Bea02d0utB/cetGuUzhiEELADmKUILfONq0AiSoPdr7AzftQdVndahWZg5zKA7atLAcqu/v4+RG40hKaPclWZtZBzwkJb lAPBHMKv KFEWvewfS+K1Zpt0ehjdwaHLW6ZJThq5eNY5Z+Sm1N0i+z2xDKjcOAN6rVgVpt3PerQdZOVAJ/YK8vioF/4mibfaWZikUXwJxJ6xukBcY67cyMFMvZNIwIU5KYEaqZfLO+jqJKgkp0maLRsizVef+rw8TJGvQArdcPyU9sBso+0PZuOXek2RqARo8/CgADaiDOwXdAhDaC7NWcOYyq7u3VRl1M3VdtJ9tLMmFXdG7Ff/zxplfHMJjG+g++g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 4/22/25 13:48, Ryan Roberts wrote: > Refactor the huge_pte helpers to use the new common __set_ptes_anysz() > and __ptep_get_and_clear_anysz() APIs. > > This provides 2 benefits; First, when page_table_check=on, hugetlb is > now properly/fully checked. Previously only the first page of a hugetlb > folio was checked. Second, instead of having to call __set_ptes(nr=1) > for each pte in a loop, the whole contiguous batch can now be set in one > go, which enables some efficiencies and cleans up the code. > > One detail to note is that huge_ptep_clear_flush() was previously > calling ptep_clear_flush() for a non-contiguous pte (i.e. a pud or pmd > block mapping). This has a couple of disadvantages; first > ptep_clear_flush() calls ptep_get_and_clear() which transparently > handles contpte. Given we only call for non-contiguous ptes, it would be > safe, but a waste of effort. It's preferable to go straight to the layer > below. However, more problematic is that ptep_get_and_clear() is for > PAGE_SIZE entries so it calls page_table_check_pte_clear() and would not > clear the whole hugetlb folio. So let's stop special-casing the non-cont > case and just rely on get_clear_contig_flush() to do the right thing for > non-cont entries. > > Reviewed-by: Catalin Marinas > Signed-off-by: Ryan Roberts > --- > arch/arm64/mm/hugetlbpage.c | 53 +++++++------------------------------ > 1 file changed, 10 insertions(+), 43 deletions(-) > > diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c > index 087fc43381c6..d34703846ef4 100644 > --- a/arch/arm64/mm/hugetlbpage.c > +++ b/arch/arm64/mm/hugetlbpage.c > @@ -159,12 +159,11 @@ static pte_t get_clear_contig(struct mm_struct *mm, > pte_t pte, tmp_pte; > bool present; > > - pte = __ptep_get_and_clear(mm, addr, ptep); > + pte = __ptep_get_and_clear_anysz(mm, ptep, pgsize); > present = pte_present(pte); > while (--ncontig) { > ptep++; > - addr += pgsize; > - tmp_pte = __ptep_get_and_clear(mm, addr, ptep); > + tmp_pte = __ptep_get_and_clear_anysz(mm, ptep, pgsize); > if (present) { > if (pte_dirty(tmp_pte)) > pte = pte_mkdirty(pte); > @@ -208,7 +207,7 @@ static void clear_flush(struct mm_struct *mm, > unsigned long i, saddr = addr; > > for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) > - __ptep_get_and_clear(mm, addr, ptep); > + __ptep_get_and_clear_anysz(mm, ptep, pgsize); > > __flush_hugetlb_tlb_range(&vma, saddr, addr, pgsize, true); > } > @@ -219,32 +218,20 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, > size_t pgsize; > int i; > int ncontig; > - unsigned long pfn, dpfn; > - pgprot_t hugeprot; > > ncontig = num_contig_ptes(sz, &pgsize); > > if (!pte_present(pte)) { > for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) > - __set_ptes(mm, addr, ptep, pte, 1); > + __set_ptes_anysz(mm, ptep, pte, 1, pgsize); > return; > } > > - if (!pte_cont(pte)) { > - __set_ptes(mm, addr, ptep, pte, 1); > - return; > - } > - > - pfn = pte_pfn(pte); > - dpfn = pgsize >> PAGE_SHIFT; > - hugeprot = pte_pgprot(pte); > - > /* Only need to "break" if transitioning valid -> valid. */ > - if (pte_valid(__ptep_get(ptep))) > + if (pte_cont(pte) && pte_valid(__ptep_get(ptep))) > clear_flush(mm, addr, ptep, pgsize, ncontig); > > - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) > - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); > + __set_ptes_anysz(mm, ptep, pte, ncontig, pgsize); > } > > pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, > @@ -434,11 +421,9 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, > unsigned long addr, pte_t *ptep, > pte_t pte, int dirty) > { > - int ncontig, i; > + int ncontig; > size_t pgsize = 0; > - unsigned long pfn = pte_pfn(pte), dpfn; > struct mm_struct *mm = vma->vm_mm; > - pgprot_t hugeprot; > pte_t orig_pte; > > VM_WARN_ON(!pte_present(pte)); > @@ -447,7 +432,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, > return __ptep_set_access_flags(vma, addr, ptep, pte, dirty); > > ncontig = num_contig_ptes(huge_page_size(hstate_vma(vma)), &pgsize); > - dpfn = pgsize >> PAGE_SHIFT; > > if (!__cont_access_flags_changed(ptep, pte, ncontig)) > return 0; > @@ -462,19 +446,14 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, > if (pte_young(orig_pte)) > pte = pte_mkyoung(pte); > > - hugeprot = pte_pgprot(pte); > - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) > - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); > - > + __set_ptes_anysz(mm, ptep, pte, ncontig, pgsize); > return 1; > } > > void huge_ptep_set_wrprotect(struct mm_struct *mm, > unsigned long addr, pte_t *ptep) > { > - unsigned long pfn, dpfn; > - pgprot_t hugeprot; > - int ncontig, i; > + int ncontig; > size_t pgsize; > pte_t pte; > > @@ -487,16 +466,11 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, > } > > ncontig = find_num_contig(mm, addr, ptep, &pgsize); > - dpfn = pgsize >> PAGE_SHIFT; > > pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); > pte = pte_wrprotect(pte); > > - hugeprot = pte_pgprot(pte); > - pfn = pte_pfn(pte); > - > - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) > - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); > + __set_ptes_anysz(mm, ptep, pte, ncontig, pgsize); > } > > pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, > @@ -505,13 +479,6 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, > struct mm_struct *mm = vma->vm_mm; > size_t pgsize; > int ncontig; > - pte_t pte; > - > - pte = __ptep_get(ptep); > - VM_WARN_ON(!pte_present(pte)); > - > - if (!pte_cont(pte)) > - return ptep_clear_flush(vma, addr, ptep); > > ncontig = num_contig_ptes(huge_page_size(hstate_vma(vma)), &pgsize); > return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); Reviewed-by: Anshuman Khandual