From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4159EC2BA18 for ; Tue, 18 Jun 2024 01:57:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C2C886B02BB; Mon, 17 Jun 2024 21:57:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BDC1D6B02BD; Mon, 17 Jun 2024 21:57:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA44E6B02BE; Mon, 17 Jun 2024 21:57:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8D3C66B02BB for ; Mon, 17 Jun 2024 21:57:06 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 049F41602CE for ; Tue, 18 Jun 2024 01:57:05 +0000 (UTC) X-FDA: 82242346452.09.F2947D4 Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) by imf15.hostedemail.com (Postfix) with ESMTP id 2A327A0003 for ; Tue, 18 Jun 2024 01:57:03 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=LDoukxv1; spf=pass (imf15.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.208.48 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718675821; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Wae8M8t+Wn0YykTgf7N/e/0Ifm2hpdV8ttPW81oP2Ak=; b=HD1KL2mXi+ev+uzp5obRgW4WsXgytQteWgP/omZYaTfrYrldCRnOno7EyCblhLpb1uRzkH NjPEsQGbddyhJyLMOm+HQxeFtWkF8WmE/FNZdcpjDCbYjtq2HPmbhLcFa5hqiNAtMD0KM8 Ei5COsZ04BgFSFv/AZ0XWvhaVyMIR+A= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=LDoukxv1; spf=pass (imf15.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.208.48 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718675821; a=rsa-sha256; cv=none; b=1aVGFx44pAf/qD/0zLrBXvVaafMkG6iF/ExqKcCxcRGocicTmkoEm28d+xTvzWO8N5WM7v /TpxPP/q4956/QhHrNY0QXYOOYYFtqvhJ0sR7HITOHCVjc+5dsu3RMqflwCEfqAhHT7o3o I5c6qdrSvvZpmfah8L8d6CldQ8H/2PE= Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-57c6011d75dso6042235a12.3 for ; Mon, 17 Jun 2024 18:57:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718675823; x=1719280623; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Wae8M8t+Wn0YykTgf7N/e/0Ifm2hpdV8ttPW81oP2Ak=; b=LDoukxv1bnegb2GgZmwEyIffxTpq2C16pMc7C89VAjABA+LZpE1erVES1KlTjxIN6q K/KFkoHlS2hf7jUUk10HwSPy0sDyU9UQDn4MzLTPXD3AJbmwFInyukfO6F6Klugcnv0M BywpxOmiwRhvFaXC0WTNq4r4UN/GPy5oTJOiUo+SWVwbgv5gXFWa1DI8ldZIHVag7rNr r+m68Bf1zYNFw53vt7xDbfnwH+ckyUleiTH5hY1or/SDvbmA5c2VbNN8nXxpYLrtefdO 6dsNsg7Xd1cqI2KgP9Z4CwI6ri5SdhWtMuHWcbK9HE+yc3FtuqcAqIaCKxUPz5NZsJFa EZ/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718675823; x=1719280623; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wae8M8t+Wn0YykTgf7N/e/0Ifm2hpdV8ttPW81oP2Ak=; b=kOOsOO7gAJKvDoUDCBJDYW6JBxdFc4ObppLqRaKKCo3OH9awLCXzV7fEOG4KPsfbE0 kDpXSjyd1mRvT4C7ihTXV5hMkO73W7IeSNChIvRHN1iaWwFauBSfKAkzbZFQMIgrnju+ SI6PjOiBqJ67J8rzy8eRWiaabj/UdmJM2I31BQtp/vR7M2akU2CWIUkE1HU0B8SxL+i2 eLDGXrOcmxgTalv2AE+Wq/IQ12VrGRa/mXm3UVJW27jYd3QQtUQGuwA9LvWPLsiwDvVh pDd7aD9ldA1xXRTUAjg66GdfzPnZ3Y+K6B6oXemCiJgO1mXtf5Q+4ZinBBCs688MMQc8 NJCw== X-Forwarded-Encrypted: i=1; AJvYcCXmHiRe9vkhu+8CYY/SuMpqJ0FGXm+uUvPFztpsLncJdD9rNYF41fYapusLFCdUvARcE8OGIcW2jWCoU2qiHymq6m8= X-Gm-Message-State: AOJu0Yxj5jA92iUQcy4rcZODciO4AAw/Cnz6P/t16BEPTyeiAZKPNKWZ ia37dAwYUKlt4lWxFczrg9gdcJF9vfNhYrlfzCV4uTLyJPNgvgbc2eGLFQGMIv+DKyi8z4VOIE1 iOQUpx1Fl9q7a/hCHxAOlRCcbD44= X-Google-Smtp-Source: AGHT+IF75sPalUePJwA4s9srnEf6OaVwVv/+ZRszkXP2keldbLUTIdhmHDwB7fEMt/PwKK1/GGubep5KLtiyfb8sFkI= X-Received: by 2002:a50:8d45:0:b0:57c:713c:cdbf with SMTP id 4fb4d7f45d1cf-57cbd8b9828mr7320014a12.38.1718675822477; Mon, 17 Jun 2024 18:57:02 -0700 (PDT) MIME-Version: 1.0 References: <20240614015138.31461-1-ioworker0@gmail.com> <20240614015138.31461-4-ioworker0@gmail.com> In-Reply-To: From: Lance Yang Date: Tue, 18 Jun 2024 09:56:51 +0800 Message-ID: Subject: Re: [PATCH v8 3/3] mm/vmscan: avoid split lazyfree THP during shrink_folio_list() To: David Hildenbrand Cc: akpm@linux-foundation.org, willy@infradead.org, sj@kernel.org, baolin.wang@linux.alibaba.com, maskray@google.com, ziy@nvidia.com, ryan.roberts@arm.com, 21cnbao@gmail.com, mhocko@suse.com, fengwei.yin@intel.com, zokeefe@google.com, shy828301@gmail.com, xiehuan09@gmail.com, libang.li@antgroup.com, wangkefeng.wang@huawei.com, songmuchun@bytedance.com, peterx@redhat.com, minchan@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 2A327A0003 X-Stat-Signature: 5ncjcmis5989kpxonfmicrjme5rgtiof X-HE-Tag: 1718675823-645424 X-HE-Meta: U2FsdGVkX1+HXGOItZ0KhjosuH3vFRAbTiCLcLD6n2fNpDsUOaQx/fLa4ZrIceJyRMKVw2Y2bwPSgZXMXahomX8JVV2jEyHwPuUvABN9DbOJwTwDgKxtlA++DIlPR/ljGFZi9Jsp6VVcQetqNYIneAxTOJ/KMNseJoKRtg7wlUxvBE91OpllUofOLeFDqDj4jEbSYTNlcUDT41Kk1FrjXy/NO3ay5FnxlnFX7sjMfVEcdciPBJeOEcxHo2Y+d6Rn0WEbkzWyyzW56rCIrarWaVhw0aJWyEKgIEt7rpPxqJi3Xmc/x9aYu4zRaZ72+d+EDtkmQXvkojcYYJnPrQpQIby35xbFhb6csuYueT9+T9zTKkX6zu30rd1QhBWt9WuF0gxhdAQHEUk3EDtOGD3hptB4iUhyxoeIrkhs8RPXxMCFuFVoF150Gx6QNewEw4jRcsDtJJTSaQ+xPzE0eUnpNYDRQHq5XsAQ21PD6i+86L/w7ja1WtyFbPx1KiLEt3lYjQk99VHJChigMXS5U8BmXXZdpK6GQHk6TxyYplRA8cnDKPehCVAL8MZxurrY43S8349BqrnFysOwSDehly2DqmYcxM2K5srB824CciDcQf+E5rfYHZf0gcPnC5PseulzW+eVpFNZfXGuET48ZhETMFD4kRwzE2DtgMOKfVenIY90sPUgn5IhNergg86MbNkmB8lx2f74Ye+JknfUdqOPmX6jqWrD/ym7m5P2WZqP8lOPn9GKJdKgxFMNX3NxCxwRw8BeqSUSk4ejmPoPKcx1I3cusTO/BJ2U+chNmbcT+DQrmOCtnmXWHoHUwUG7OYB3vTk8vGjGtVdq2ngTi8aoPg0jLGkmv4xRNxQnxuZ4xRvR/7k6RVSAwUEPwAiXlKbdoHqPe30PL4S7TGzYOEHV+MCiem3h5SeeCXPf87ZypallgSFTfRHC2CoOyMK3tIymuVPDNAEd9YiALFq0d4e JVOUslNY fR6Ds/IQh7GVhnDg/qTO3lcqbrNYXx2TXLed8HMrvg++u0IiA/lYUE0mznyliMhSuyxaXxMicDpEdLVKwafm3a7wnmGlajo666D9sYe0AUyiWqUcsn1uya41P4melML9POBPVE8TuD/QASgy/s6QQzl4TekBY2H4vF1auBL1uUtDFW5o4O1jf0aHiEZ5TL5DuhwJHz/efGBPT7QAaAD+gElszeOy9mfMPzn/IO/vuo/8BVw6Nsj1Ytji/XTgpLjxc+6aI1kYvrh9EWpH6uNzehoIKqZqzVcXBJn4cvNe35/76H5kADQuXyawjTSDo9kpX1q2oYyVNwwJwuEh2EpcL9Ho2BYczxkgj66QngBElDYpMmgOvGHwH9Tdu5g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jun 18, 2024 at 2:04=E2=80=AFAM David Hildenbrand wrote: > > Sorry for taking so long to review ... getting there. Mostly nits. No worries at all :) Thanks for taking time to review! > > > @@ -497,6 +499,13 @@ static inline void split_huge_pmd_locked(struct vm= _area_struct *vma, > > unsigned long address, pmd_t *pm= d, > > bool freeze, struct folio *folio= ) {} > > > > +static inline bool unmap_huge_pmd_locked(struct vm_area_struct *vma, > > + unsigned long addr, pmd_t *pmdp, > > + struct folio *folio) > > +{ > > + return false; > > +} > > + > > #define split_huge_pud(__vma, __pmd, __address) \ > > do { } while (0) > > > > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > > index e766d3f3a302..425374ae06ed 100644 > > --- a/mm/huge_memory.c > > +++ b/mm/huge_memory.c > > @@ -2688,6 +2688,82 @@ static void unmap_folio(struct folio *folio) > > try_to_unmap_flush(); > > } > > > > +static bool __discard_anon_folio_pmd_locked(struct vm_area_struct *vma= , > > + unsigned long addr, pmd_t *pm= dp, > > + struct folio *folio) > > +{ > > + VM_WARN_ON_FOLIO(folio_test_swapbacked(folio), folio); > > + VM_WARN_ON_FOLIO(!folio_test_anon(folio), folio); > > I would drop these (that's exactly what the single caller checks). In Agreed. I will drop these. > any case don't place them above the variable declaration ;) Yep, I see. > > > + > > + struct mm_struct *mm =3D vma->vm_mm; > > + int ref_count, map_count; > > + pmd_t orig_pmd =3D *pmdp; > > + struct page *page; > > + > > + if (unlikely(!pmd_present(orig_pmd) || !pmd_trans_huge(orig_pmd))= ) > > + return false; > > + > > + page =3D pmd_page(orig_pmd); > > + if (unlikely(page_folio(page) !=3D folio)) > > + return false; > > I'm curious, how could that happen? And how could it happen that we have > !pmd_trans_huge() ? Didn't rmap walking code make sure that this PMD > maps the folio already, and we are holding the PTL? Makes sense to me. I was adding these just in case, but it's probably too m= uch. Let's drop them ;) > > > + > > + if (folio_test_dirty(folio) || pmd_dirty(orig_pmd)) > > + return false; > > + > > + orig_pmd =3D pmdp_huge_clear_flush(vma, addr, pmdp); > > + > > + /* > > + * Syncing against concurrent GUP-fast: > > + * - clear PMD; barrier; read refcount > > + * - inc refcount; barrier; read PMD > > + */ > > + smp_mb(); > > + > > + ref_count =3D folio_ref_count(folio); > > + map_count =3D folio_mapcount(folio); > > + > > + /* > > + * Order reads for folio refcount and dirty flag > > + * (see comments in __remove_mapping()). > > + */ > > + smp_rmb(); > > + > > + /* > > + * If the folio or its PMD is redirtied at this point, or if ther= e > > + * are unexpected references, we will give up to discard this fol= io > > + * and remap it. > > + * > > + * The only folio refs must be one from isolation plus the rmap(s= ). > > + */ > > + if (folio_test_dirty(folio) || pmd_dirty(orig_pmd) || > > + ref_count !=3D map_count + 1) { > > + set_pmd_at(mm, addr, pmdp, orig_pmd); > > + return false; > > + } > > + > > + folio_remove_rmap_pmd(folio, page, vma); > > + zap_deposited_table(mm, pmdp); > > + add_mm_counter(mm, MM_ANONPAGES, -HPAGE_PMD_NR); > > + if (vma->vm_flags & VM_LOCKED) > > + mlock_drain_local(); > > + folio_put(folio); > > + > > + return true; > > +} > > + > > +bool unmap_huge_pmd_locked(struct vm_area_struct *vma, unsigned long a= ddr, > > + pmd_t *pmdp, struct folio *folio) > > +{ > > + VM_WARN_ON_FOLIO(!folio_test_pmd_mappable(folio), folio); > > + VM_WARN_ON_FOLIO(!folio_test_locked(folio), folio); > > + VM_WARN_ON_ONCE(!IS_ALIGNED(addr, HPAGE_PMD_SIZE)); > > + > > + if (folio_test_anon(folio) && !folio_test_swapbacked(folio)) > > + return __discard_anon_folio_pmd_locked(vma, addr, pmdp, f= olio); > > + > > + return false; > > +} > > + > > static void remap_page(struct folio *folio, unsigned long nr) > > { > > int i =3D 0; > > diff --git a/mm/rmap.c b/mm/rmap.c > > index dacf24bc82f0..7d97806f74cd 100644 > > --- a/mm/rmap.c > > +++ b/mm/rmap.c > > @@ -1678,16 +1678,23 @@ static bool try_to_unmap_one(struct folio *foli= o, struct vm_area_struct *vma, > > goto walk_abort; > > } > > > > - if (!pvmw.pte && (flags & TTU_SPLIT_HUGE_PMD)) { > > - /* > > - * We temporarily have to drop the PTL and start = once > > - * again from that now-PTE-mapped page table. > > - */ > > - split_huge_pmd_locked(vma, pvmw.address, pvmw.pmd= , > > - false, folio); > > - flags &=3D ~TTU_SPLIT_HUGE_PMD; > > - page_vma_mapped_walk_restart(&pvmw); > > - continue; > > + if (!pvmw.pte) { > > + if (unmap_huge_pmd_locked(vma, pvmw.address, pvmw= .pmd, > > + folio)) > > + goto walk_done; > > + > > + if (flags & TTU_SPLIT_HUGE_PMD) { > > + /* > > + * We temporarily have to drop the PTL an= d start > > + * once again from that now-PTE-mapped pa= ge > > + * table. > > Nit: it's not a PTE-mapped page table. > > Maybe > > "... restart so we can process the PTE-mapped THP." Nice. Will adjust as you suggested. > > > > > } > > > > /* Unexpected PMD-mapped THP? */ > > Nothing else jumped at me :) Thanks again for your time! Lance > > -- > Cheers, > > David / dhildenb >