From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1817C07545 for ; Tue, 24 Oct 2023 17:40:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97FD76B02D6; Tue, 24 Oct 2023 13:40:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9305B6B02D7; Tue, 24 Oct 2023 13:40:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F81D6B02D8; Tue, 24 Oct 2023 13:40:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 719D76B02D6 for ; Tue, 24 Oct 2023 13:40:31 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2847BB5A3A for ; Tue, 24 Oct 2023 17:40:31 +0000 (UTC) X-FDA: 81381069462.26.37E9C63 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf19.hostedemail.com (Postfix) with ESMTP id 610BD1A0015 for ; Tue, 24 Oct 2023 17:40:29 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=clLagogh; spf=pass (imf19.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698169229; a=rsa-sha256; cv=none; b=Fdyen0dFW3eP0Ot9ZpUzPImUtiZLv7FSo3kYZ5NyMLfaoQhpE2GAiJmxP3fOG+c25vi5nm 3PmCwwWyota8zpLd86PFv9BRM/U7c8XGMhZsRymE96zhPBL5lFKuPWan8OBaMNHODomN1p kreLxlIzRr9p7AZz7Fi7Am83xbtmoyw= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=clLagogh; spf=pass (imf19.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698169229; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0xWGwGBYtGQcESxLDi2O/elWlIJtx15pyBUk6hkFzL4=; b=KS8DKP8jWc0Ix9z5wcDEUuhS4NtaWOmom/am0MCJBr8/a8DywJbcK0dyvzqGtaSRip4rx1 0ITqUAftIr1iQIqHRVkYCLS32JC7JOTjmIPSGW5XYoF2A5c9M+5WAhZsg7AJQuY0LVsKaa vvuB416iYdtV79F/xesayU+dpVKVPA4= Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-27d425a2dd0so4131569a91.2 for ; Tue, 24 Oct 2023 10:40:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698169228; x=1698774028; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=0xWGwGBYtGQcESxLDi2O/elWlIJtx15pyBUk6hkFzL4=; b=clLagoghU/ge3raC2TRt7dlKjq8zNcO8a6G25a7cW9FDJofCnroRi4z9kViSWe98aX LmhaTcsPerX9hLYb+QMv0dKDXOTAwWYSeb7yETHCN/3JRBmXF+mGsczG6dYkjdxbEuom 9NqXYRjom+R2Mi+58AbUzR2yKiT1Cp0dMHiVK2I6z6Qb+EAsw5z2ErdnTqQNBtLDALoM VFzNaO0nE8VNfgGs6Zu8NmFToFwI10G/S2mkhSMTAJ79+sSSsc/NhrlBXZQIEetEhd0X V6rHITc39lxpEZPVrAbEtDYo7WpdWMhjgtD2Q+iO3znceK2Dtmzfon3lMlA+s+deKMoC hDbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698169228; x=1698774028; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0xWGwGBYtGQcESxLDi2O/elWlIJtx15pyBUk6hkFzL4=; b=nHa/Seya2FK0Y6cVNLpaN28QdQ7ZDT/Uja8UdLW+F6YYu6aqCHETs9dAFBPUyCAKIG qS76JIR0Sl5lHDqIKprXF4ftMAPz/AqR3Q/EwyKp3wG/odTz2paMal7ZRsQinItKHZ7R q9FhDfuCneXLNBEvEl4GHhp67KDjOzqRqxdJe/0wCyUc3hcjO7G3ErQiukDMXFNhDpTv Ntsdi4SCCS9MxKkVJWKxN9Ydnu4obl70fIZE3T5KnWImQLfkePRYSA9eTgI1m1xcaSnC k9OYU+ixba24E0mLRPEN3tL2qc/pr48sYyxna0ahcnrL+/cGdQrgbGUt/Tel3xA/OaYa N1MA== X-Gm-Message-State: AOJu0YySvTRPBDSs8QZdzyXowpiJnLmwdnx9mv7L9FC8zVi3B1cxH0RG YJSIyE15HR2D0O3l9f/yYTzRBBCoaBHySXice8fHdzS+ X-Google-Smtp-Source: AGHT+IHIes8+KZbr5QFSv8mkeyL/tKYrsJTKUM8ciSJZzTy3Cni0SQN69j/7pYt+DueUNi+xQbN8hBmQTffZGeAIEdw= X-Received: by 2002:a17:90a:ac02:b0:27d:b3d:5c33 with SMTP id o2-20020a17090aac0200b0027d0b3d5c33mr12739037pjq.28.1698169228336; Tue, 24 Oct 2023 10:40:28 -0700 (PDT) MIME-Version: 1.0 References: <20231020183331.10770-1-vishal.moola@gmail.com> <20231020183331.10770-6-vishal.moola@gmail.com> In-Reply-To: <20231020183331.10770-6-vishal.moola@gmail.com> From: Yang Shi Date: Tue, 24 Oct 2023 10:40:16 -0700 Message-ID: Subject: Re: [PATCH v3 5/5] mm/khugepaged: Convert collapse_pte_mapped_thp() to use folios To: "Vishal Moola (Oracle)" Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 610BD1A0015 X-Stat-Signature: 37b5oxrfn3d5xhtmpwefhqmijpd88pxw X-Rspam-User: X-HE-Tag: 1698169229-532870 X-HE-Meta: U2FsdGVkX18qgPWgaKwOr5B8foKX7UYVfo9C/rHQtnvYA9EILj0dDz6Mh45DzIFa5B2WOPcZpWdT056OExu/htZ1eUCuCoNkFksLtYI5S4/fW/bT0OpSMS1+zkW9e6SRJC+1npblCu6iqGtU8/oL+Z5jbLtKTTR/oHtjwZPqsMNoyYgAUcnGDVippTNHKKeBbRz6POPLnIJze6i3+UHdAoF+2RdZrQVGnagsgT6djdB/VnfA7Tdyou+30ZaolMnTYjLIX0wncxhq53EXyf5uMfQuG0vhBC8BEpZX/ib1M4/cEzO8v7Lgu9Ec9nKOfSyr6oI1FauclK1mNw5Fx9Y08h4ul77y+IZelXqJJg6KSxluDQfIPhFbrWvu2wT6hZ87buv3j+EaK5G8A1VpAhxAHdNTvr3ndooSdkhUiucpecWEL1YKKyQRPpErmQJpONpKRLoUW1q9KqWRijg+gYDk94k4B/XN9Xajff0Bmp1BSl+kYmCjY5n23OEnbpSLdhsSdNYDjyKbQH/4AExYognqRcZTqJ6+iEFF/lyNVtjJ04WQnkqj36rr897+TCuhT1bcuqC2f13UNSdwIJdoBwVkS2z8nzVPOFykfdfe8o+HcpkLI5IDUI7BZ6T3fJyHpqycJDY0WA5R587QqBxPTDxdcWDAHMQhewHfWmpCfLMrlHKNHcLoStV+cFQCWRLweOMfG3MsQIS1Q8tbVIVUy6viVApXpCLVBYbMVTsPSPSnZ0Efs5Dymj/kpn5KbAdGGZ9irMO9A0ErnHu0keRVHFJJ86N2hUE/8uA3f75JKNhQfFe++SZJfsKJ0r0H2ai+j6ZdKW2Cjjkutwb6Jk+qdbCD/CtWvRNA9TlcKMW3wh2Z4vX4gvvgkbmrxK91y3QzN0cZUt4eUo2DFDVqtcwpYQDUO5JxsX0cnj6Gprmpi70VUfUjI+SeQqerLovD7P27p5VR9PpEeUYRE/mgKrV5iH4 IMMY6REZ sf8ykuuLVvVOcKYCK+QQzgWigrcHD0AE18kn2jVfFktKKyAvjJEqaN7HEOUzirzqBpI5B+9CkutTTS7KywapcJ+YDXyHcGeT827k/wkGT4enWyGQHrqI4I+FzKKKgVEaaM1xp28a5/+OD8+Ql8Ap4RdjUbulF9uukmx3/aXaJDbzG8DMCx/agV8sLWumzYQc7Zi45SH7CNMC94nxxGn3+QAVVEHGRnfUZgbcex/947kPeBi4GIHzPkuJGfQgQ/9kOjE2+4fBNdrpFw5hXIeCyt8OYsZW9MLjCa3AZpNKic3jgJDsH3sb3bXNbrxhSouSfZGkqrlOOsjJMYrNcBIG34FYzSXsYxYStw813YL3Tuxu0f5VIXJh8dRpM2DMJ6ijr1XeV22K/e+XrHw0ZRRb+hymijjPH554yqM/nhXhAC3M4WjIVzaEMnszRS5oyE3KKi093StqtkcfkBnU8To7JWyq0ArkW9vl4KXiPFx9JYBddmWA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Oct 20, 2023 at 11:34=E2=80=AFAM Vishal Moola (Oracle) wrote: > > This removes 2 calls to compound_head() and helps convert khugepaged to > use folios throughout. > > Previously, if the address passed to collapse_pte_mapped_thp() > corresponded to a tail page, the scan would fail immediately. Using > filemap_lock_folio() we get the corresponding folio back and try to > operate on the folio instead. > > Signed-off-by: Vishal Moola (Oracle) Reviewed-by: Yang Shi > --- > mm/khugepaged.c | 45 ++++++++++++++++++++------------------------- > 1 file changed, 20 insertions(+), 25 deletions(-) > > diff --git a/mm/khugepaged.c b/mm/khugepaged.c > index 6a7184cd291b..bc2d8ff269c7 100644 > --- a/mm/khugepaged.c > +++ b/mm/khugepaged.c > @@ -1477,7 +1477,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, u= nsigned long addr, > bool notified =3D false; > unsigned long haddr =3D addr & HPAGE_PMD_MASK; > struct vm_area_struct *vma =3D vma_lookup(mm, haddr); > - struct page *hpage; > + struct folio *folio; > pte_t *start_pte, *pte; > pmd_t *pmd, pgt_pmd; > spinlock_t *pml =3D NULL, *ptl; > @@ -1510,19 +1510,14 @@ int collapse_pte_mapped_thp(struct mm_struct *mm,= unsigned long addr, > if (userfaultfd_wp(vma)) > return SCAN_PTE_UFFD_WP; > > - hpage =3D find_lock_page(vma->vm_file->f_mapping, > + folio =3D filemap_lock_folio(vma->vm_file->f_mapping, > linear_page_index(vma, haddr)); > - if (!hpage) > + if (IS_ERR(folio)) > return SCAN_PAGE_NULL; > > - if (!PageHead(hpage)) { > - result =3D SCAN_FAIL; > - goto drop_hpage; > - } > - > - if (compound_order(hpage) !=3D HPAGE_PMD_ORDER) { > + if (folio_order(folio) !=3D HPAGE_PMD_ORDER) { > result =3D SCAN_PAGE_COMPOUND; > - goto drop_hpage; > + goto drop_folio; > } > > result =3D find_pmd_or_thp_or_none(mm, haddr, &pmd); > @@ -1536,13 +1531,13 @@ int collapse_pte_mapped_thp(struct mm_struct *mm,= unsigned long addr, > */ > goto maybe_install_pmd; > default: > - goto drop_hpage; > + goto drop_folio; > } > > result =3D SCAN_FAIL; > start_pte =3D pte_offset_map_lock(mm, pmd, haddr, &ptl); > if (!start_pte) /* mmap_lock + page lock should prevent t= his */ > - goto drop_hpage; > + goto drop_folio; > > /* step 1: check all mapped PTEs are to the right huge page */ > for (i =3D 0, addr =3D haddr, pte =3D start_pte; > @@ -1567,7 +1562,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, u= nsigned long addr, > * Note that uprobe, debugger, or MAP_PRIVATE may change = the > * page table, but the new page will not be a subpage of = hpage. > */ > - if (hpage + i !=3D page) > + if (folio_page(folio, i) !=3D page) > goto abort; > } > > @@ -1582,7 +1577,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, u= nsigned long addr, > * page_table_lock) ptl nests inside pml. The less time we hold p= ml, > * the better; but userfaultfd's mfill_atomic_pte() on a private = VMA > * inserts a valid as-if-COWed PTE without even looking up page c= ache. > - * So page lock of hpage does not protect from it, so we must not= drop > + * So page lock of folio does not protect from it, so we must not= drop > * ptl before pgt_pmd is removed, so uffd private needs pml taken= now. > */ > if (userfaultfd_armed(vma) && !(vma->vm_flags & VM_SHARED)) > @@ -1606,7 +1601,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, u= nsigned long addr, > continue; > /* > * We dropped ptl after the first scan, to do the mmu_not= ifier: > - * page lock stops more PTEs of the hpage being faulted i= n, but > + * page lock stops more PTEs of the folio being faulted i= n, but > * does not stop write faults COWing anon copies from exi= sting > * PTEs; and does not stop those being swapped out or mig= rated. > */ > @@ -1615,7 +1610,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, u= nsigned long addr, > goto abort; > } > page =3D vm_normal_page(vma, addr, ptent); > - if (hpage + i !=3D page) > + if (folio_page(folio, i) !=3D page) > goto abort; > > /* > @@ -1634,8 +1629,8 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, u= nsigned long addr, > > /* step 3: set proper refcount and mm_counters. */ > if (nr_ptes) { > - page_ref_sub(hpage, nr_ptes); > - add_mm_counter(mm, mm_counter_file(hpage), -nr_ptes); > + folio_ref_sub(folio, nr_ptes); > + add_mm_counter(mm, mm_counter_file(&folio->page), -nr_pte= s); > } > > /* step 4: remove empty page table */ > @@ -1659,14 +1654,14 @@ int collapse_pte_mapped_thp(struct mm_struct *mm,= unsigned long addr, > maybe_install_pmd: > /* step 5: install pmd entry */ > result =3D install_pmd > - ? set_huge_pmd(vma, haddr, pmd, hpage) > + ? set_huge_pmd(vma, haddr, pmd, &folio->page) > : SCAN_SUCCEED; > - goto drop_hpage; > + goto drop_folio; > abort: > if (nr_ptes) { > flush_tlb_mm(mm); > - page_ref_sub(hpage, nr_ptes); > - add_mm_counter(mm, mm_counter_file(hpage), -nr_ptes); > + folio_ref_sub(folio, nr_ptes); > + add_mm_counter(mm, mm_counter_file(&folio->page), -nr_pte= s); > } > if (start_pte) > pte_unmap_unlock(start_pte, ptl); > @@ -1674,9 +1669,9 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, u= nsigned long addr, > spin_unlock(pml); > if (notified) > mmu_notifier_invalidate_range_end(&range); > -drop_hpage: > - unlock_page(hpage); > - put_page(hpage); > +drop_folio: > + folio_unlock(folio); > + folio_put(folio); > return result; > } > > -- > 2.40.1 >