From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 661CDC77B73 for ; Mon, 22 May 2023 23:35:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6FD9C6B0074; Mon, 22 May 2023 19:35:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 65F656B0075; Mon, 22 May 2023 19:35:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4B1C9900002; Mon, 22 May 2023 19:35:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 368126B0074 for ; Mon, 22 May 2023 19:35:53 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 03FCB160560 for ; Mon, 22 May 2023 23:35:52 +0000 (UTC) X-FDA: 80819500986.05.D0EAA00 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf04.hostedemail.com (Postfix) with ESMTP id 347C14000D for ; Mon, 22 May 2023 23:35:49 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=fI5qJBcI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf04.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684798550; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mLmCuy21+rPe8wgn0OHZOYds1k+J9dWSGBzctzYhcE8=; b=rwpCllZL8PhBQOPEaanJ58/0zlNAOF+OhsjphAQbppsXMnaEw9YQ84NAMYlT1urNIZIFUC 4x+09rtPpfX5q1WPJgTY+JbBDtbc6bsMP5w3l+jvTlpMMzs+anActaBT7ZpGoEcDeQwJZV 0LVP7cYJSsZiPBONr6ksfY6o3QGAlCE= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=fI5qJBcI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf04.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684798550; a=rsa-sha256; cv=none; b=li6wBfnIYaZMRwQXF/ne0k85xhK+8s0LY1fS4BGXyFg1UR4o/lWxXXLBH8tg1BX6ZgXBmD 5w3kZ6wo2qTjTlFqK4hzEb8FErXuBML9C3tpMTYHN44FCvizDY2WM9NLPA1zECB/iA0eGv sEnZs5fC8rc4OHv+GVf8dWkOA8k/PWE= Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-25332422531so3074458a91.0 for ; Mon, 22 May 2023 16:35:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684798549; x=1687390549; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=mLmCuy21+rPe8wgn0OHZOYds1k+J9dWSGBzctzYhcE8=; b=fI5qJBcIvptknKxPI4Cv5bhukMTzxZ5aQAX2MqkNIZz6Ru9j6nE9Pyvlp24mE2ESe/ ZqHBMZRDNknvNtNdh9novi9SzVZiAuvx6JjDqwOIe+DVWQqmJubCoNA+lv56GgjVijH1 MDRZ1MpFuupCh7XK9IlZii1TzWNHbdRCgUopsygbqcmQvC7RLtAMCgSYiXqL+fIaYQJm CCDizhRp94Om1/zNF8pgdxG/EzoJzrGAX9j9K7MvaBgbgIjXL4K42Lp/UD2eiybKo+KK fgwGkpMtvUOQpOoqCav6oPmSPrplUIt0kN9w/8TYLyiIgI/6+eCbdPUj40kpB4TKEgx2 3OmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684798549; x=1687390549; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mLmCuy21+rPe8wgn0OHZOYds1k+J9dWSGBzctzYhcE8=; b=cy7jbnb8Agv4CX9ylTE0fAmvWjS6g9NFtADR81TpAoxAY6A/S4g4eL/kbez33RF4BO r3vi34QR4mZaUYOzD1s4g3QyT8snNbaqr4kxaExJrRUehXr8iqAdR+0dYM2MMXWHwX1C +W4Jq2XKDQ6tTIlY8AjAZSlY+rDFXQhmBlCwiT8DvkqrvbmoySqKOx/Eaww2i+uH3WYg U4tm2UpRiW/HfQr2lx0WLwKGO4kYM2pjp3n9EAMqyvN9Vwe4Ny262usR0pH6BxKfCli4 +H9vxMn/SQpqsPKSWWIzgDeokd9gbS5IZowcmZRgoNsCbV2Zf0YRYzXelB7gdKilfJMZ Z7fw== X-Gm-Message-State: AC+VfDzOJliGZ6QQ2rOvxxU4mBI+/+yDK6aKUKwbpG/+MQNsQfMxz7nv hiST0duftg7PFzzBJwaORE7morfoxwH088sY7IA= X-Google-Smtp-Source: ACHHUZ55j6Ipp0XXkA7Zs+JztD8+N+YYfcUbdHbU0Cr1sBy7SfiDFrQodWiXZqkiIF1hMY2qu9vwoFg6sq5hdPkJpwA= X-Received: by 2002:a17:90b:160e:b0:253:4f8f:52fb with SMTP id la14-20020a17090b160e00b002534f8f52fbmr18320957pjb.10.1684798548695; Mon, 22 May 2023 16:35:48 -0700 (PDT) MIME-Version: 1.0 References: <68a97fbe-5c1e-7ac6-72c-7b9c6290b370@google.com> <3f442a9c-af6d-573d-1ad1-f6f413b1abc9@google.com> In-Reply-To: <3f442a9c-af6d-573d-1ad1-f6f413b1abc9@google.com> From: Yang Shi Date: Mon, 22 May 2023 16:35:36 -0700 Message-ID: Subject: Re: [PATCH 26/31] mm/huge_memory: split huge pmd under one pte_offset_map() To: Hugh Dickins Cc: Andrew Morton , Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 347C14000D X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: rro4tx5jzyfuznnworis1j8mgza6txrn X-HE-Tag: 1684798549-198293 X-HE-Meta: U2FsdGVkX1/loeS0p9dd2zdx0RfOGEMtl3AH9+PK/Zt0xEb1pl0Na0vXcSuNsAkYN7XaFJwpoqwzhjMwwUMz2LMOZXIN2FBVaqhAjEEqnKzAJlEM4uZBT1FopwmrsDHG/C+QFlj6YlQ+JWrbPjLa9hMWBO28G/umlu7QlI69j0ehVGC3yY2XqN4YnEj6GQIOLXQ1IXeswNuHKJNYuCML7he5OswCpHDCTLJtZLrlOdMxgmxe0f886X0BsAoJWI9hZtGpY880XvlahLMteyLN/BX3v5QJtGca5MHSXVQeveFqlZkO+pd73uFwiRSKlyZIJ52hq5E0whJW1zkek7DM8j/iexTrVp4At3N7hlyZUFPvWJCURVatCNZGnOezH1ZCQ6sLVNaUF3itQ/DGWebdaDgSYqTHUpVosQxxuI+3zzm9OY4fr4VKic6nJOK0DnbEAnpiChNoM8aus6P32C84DUFbTt1izdLiiOE5Jci6LX0I8wMuHX3kam0W5VG+n+JPXVDLa39jJDXlNjw3mr/QLwVENbnUW3DdEb+WaqdOcnreXchc8vURq+vBrRLOvZzAajjO6nUKwCj0b5ggrbv8XOOQjT8cxP2i+NUKM6hgu6a3dcmT1fYmDJnOkpWdI2kxFUay3DQtrlk9j9m02wmlZCkznidHAuJ5SwH8Z0ZgOLFBjN5Lj3XFjLPD5iO+Im5mitI06+AFLZxsPU3aTN7VeH+OgndpN2k0i/zvr+XoLmNS56P1wR6WqLJobeGw/OEIdxhGaNLxJv6SpQL+9jZwAU1b1uE3JE47TZkRvtIHhrHlp+UJ/8kP0UIjaNp37LXtnL7KJib19DudMNrBoHCxsVJPj1vvAv5HnMUN6P2U85/uXzvnth+NKlybbEl0IHixf+5USt0WqWO48VEfQwWgPJgdnxCcJ4SfoQ+qdXVNypZEyaL6jDpgvsEPMoEZEA323+N6f/WIUnZxcneykWM aME0eL8D Jb+2MOUfvwDh7W+3Aax/2Jd/foFa9I72mztGCT9MXDiTHljS5f5AfjBsBEka69m+MXq7cQ+PELKlIPiSqQ55XDLT8gFSxtMxzAXyiLLxlRLs4uYLhx0Zvk9pKu6FqGyI1ybZq2FYkkDkcmMIn9fIFCmxlVza5QjA1A1sal88PL17GCd+SmNzmKtdNep8FMDdn/7BkF3+yxZcw1p9NJgMLzNpwlXs7WNtmkAmewY4FHOydXhejRIlQOgRynt6tu9OogFi74QXpNeyBEz61Lc73Se1QjLUt+6aLfe2vN3O32EHnKw1NvUPxytQvTKR7AInn2J99PJrACQGKI6hHX1S4i9GjFt1TQeoxRaP8wI9xD1mFgYDKVG739XHQATNUORMHrohdc5Nno+5EKBkQG5IAjlOQSvxxTSMsyvA0IfZ3eOogEJrejLfozqJsD+poErwqseT+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, May 21, 2023 at 10:23=E2=80=AFPM Hugh Dickins wr= ote: > > __split_huge_zero_page_pmd() use a single pte_offset_map() to sweep the > extent: it's already under pmd_lock(), so this is no worse for latency; > and since it's supposed to have full control of the just-withdrawn page > table, here choose to VM_BUG_ON if it were to fail. And please don't > increment haddr by PAGE_SIZE, that should remain huge aligned: declare > a separate addr (not a bugfix, but it was deceptive). > > __split_huge_pmd_locked() likewise (but it had declared a separate addr); > and change its BUG_ON(!pte_none) to VM_BUG_ON, for consistency with zero > (those deposited page tables are sometimes victims of random corruption). > > Signed-off-by: Hugh Dickins Reviewed-by: Yang Shi > --- > mm/huge_memory.c | 28 ++++++++++++++++++---------- > 1 file changed, 18 insertions(+), 10 deletions(-) > > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > index d4bd5fa7c823..839c13fa0bbe 100644 > --- a/mm/huge_memory.c > +++ b/mm/huge_memory.c > @@ -2037,6 +2037,8 @@ static void __split_huge_zero_page_pmd(struct vm_ar= ea_struct *vma, > struct mm_struct *mm =3D vma->vm_mm; > pgtable_t pgtable; > pmd_t _pmd, old_pmd; > + unsigned long addr; > + pte_t *pte; > int i; > > /* > @@ -2052,17 +2054,20 @@ static void __split_huge_zero_page_pmd(struct vm_= area_struct *vma, > pgtable =3D pgtable_trans_huge_withdraw(mm, pmd); > pmd_populate(mm, &_pmd, pgtable); > > - for (i =3D 0; i < HPAGE_PMD_NR; i++, haddr +=3D PAGE_SIZE) { > - pte_t *pte, entry; > - entry =3D pfn_pte(my_zero_pfn(haddr), vma->vm_page_prot); > + pte =3D pte_offset_map(&_pmd, haddr); > + VM_BUG_ON(!pte); > + for (i =3D 0, addr =3D haddr; i < HPAGE_PMD_NR; i++, addr +=3D PA= GE_SIZE) { > + pte_t entry; > + > + entry =3D pfn_pte(my_zero_pfn(addr), vma->vm_page_prot); > entry =3D pte_mkspecial(entry); > if (pmd_uffd_wp(old_pmd)) > entry =3D pte_mkuffd_wp(entry); > - pte =3D pte_offset_map(&_pmd, haddr); > VM_BUG_ON(!pte_none(*pte)); > - set_pte_at(mm, haddr, pte, entry); > - pte_unmap(pte); > + set_pte_at(mm, addr, pte, entry); > + pte++; > } > + pte_unmap(pte - 1); > smp_wmb(); /* make pte visible before pmd */ > pmd_populate(mm, pmd, pgtable); > } > @@ -2077,6 +2082,7 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, > bool young, write, soft_dirty, pmd_migration =3D false, uffd_wp = =3D false; > bool anon_exclusive =3D false, dirty =3D false; > unsigned long addr; > + pte_t *pte; > int i; > > VM_BUG_ON(haddr & ~HPAGE_PMD_MASK); > @@ -2205,8 +2211,10 @@ static void __split_huge_pmd_locked(struct vm_area= _struct *vma, pmd_t *pmd, > pgtable =3D pgtable_trans_huge_withdraw(mm, pmd); > pmd_populate(mm, &_pmd, pgtable); > > + pte =3D pte_offset_map(&_pmd, haddr); > + VM_BUG_ON(!pte); > for (i =3D 0, addr =3D haddr; i < HPAGE_PMD_NR; i++, addr +=3D PA= GE_SIZE) { > - pte_t entry, *pte; > + pte_t entry; > /* > * Note that NUMA hinting access restrictions are not > * transferred to avoid any possibility of altering > @@ -2249,11 +2257,11 @@ static void __split_huge_pmd_locked(struct vm_are= a_struct *vma, pmd_t *pmd, > entry =3D pte_mkuffd_wp(entry); > page_add_anon_rmap(page + i, vma, addr, false); > } > - pte =3D pte_offset_map(&_pmd, addr); > - BUG_ON(!pte_none(*pte)); > + VM_BUG_ON(!pte_none(*pte)); > set_pte_at(mm, addr, pte, entry); > - pte_unmap(pte); > + pte++; > } > + pte_unmap(pte - 1); > > if (!pmd_migration) > page_remove_rmap(page, vma, true); > -- > 2.35.3 >