From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F121BC433F5 for ; Thu, 10 Mar 2022 01:10:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7690F8D0002; Wed, 9 Mar 2022 20:10:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7176E8D0001; Wed, 9 Mar 2022 20:10:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B8B68D0002; Wed, 9 Mar 2022 20:10:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id 4BAF88D0001 for ; Wed, 9 Mar 2022 20:10:04 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2CBC6605CC for ; Thu, 10 Mar 2022 01:10:04 +0000 (UTC) X-FDA: 79226695128.02.3856463 Received: from mail-lj1-f178.google.com (mail-lj1-f178.google.com [209.85.208.178]) by imf09.hostedemail.com (Postfix) with ESMTP id A9410140014 for ; Thu, 10 Mar 2022 01:10:03 +0000 (UTC) Received: by mail-lj1-f178.google.com with SMTP id u3so5686111ljd.0 for ; Wed, 09 Mar 2022 17:10:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=CO1x8wR9UloeAQZMTJ9QYtj6Y5afewKCBrXfFXHKexI=; b=LMNYc543td3nbv3cQXFN/ARySGS2A1+J15zHpQz4uz9SMJ8qKkFvfWblYFHPkH2s/4 2LgAJqJJdUF3uQcA2bZo7R1MPd2XmGooS843xjxJ6rkI6lEkF/PYrWxADCjNtLPeXMVt X6sXje2hVTScGO+iGcWXAWcxi9IfEHwmKEyNXwb2w5Rb9pMAt/WmhfTg6tcbI9VXIA71 RhzMC6VaIVfcwDna9j36MPlSAm3LcIQKDGzdGwX8+3YtIwEHNI4gtm/b5p4DY5WMuWUB e9rTxYayUlPxMikMwjrAs2k0z6rJdYpmRfICwyb1RuMUR43b5Sy8Ufszuxc73MoO1LML RjXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=CO1x8wR9UloeAQZMTJ9QYtj6Y5afewKCBrXfFXHKexI=; b=YV8Gl2OBOkg81qUKuWxv14YVw6rA76sUA1yVv+yPMOqUICCO8clEF7IWjf1lhKTI4i pp0nuidRl/rFOVsToBG321jdFivDH7z8vjpX6DDazNgqQwtHmpkCLC9zz+Qim3uXelPw /Jk5JzkDjlCTGNZZIOEve9LCnWCViXPIqdA0rq+LFcokcNEkI9HxCLWND0cl8OzPciX8 /X7tc4bTe96IteE/R3weuQRhhoT2ValBxZOpI/4yp0e7woLq8lD2sL10Ty7sMTetLAAP 2yb2OUvYAf0oPqQGj5vl5/ULSZ4YY2pAvxrx4Y2sVP2BKQZt0prn6sAGkhNdcKUuvg9U devQ== X-Gm-Message-State: AOAM531yFLSKoaxtcCx7/NgCqK53IMvq7sjEM7TbCP5yjyrSF5LqiIFU rTun+LoRwNZX4gb62RIVqcmgdUcxUCexx1d1Odvopw== X-Google-Smtp-Source: ABdhPJwY2PuoxQ70WBp+/F2kolBQGooHwklXn8jqKsAnC5tbzoLHRVbjUyGMK/8KDKpgVAP1bK7X7VFnVrvLXVXWXw4= X-Received: by 2002:a2e:9203:0:b0:247:ebe3:7d0c with SMTP id k3-20020a2e9203000000b00247ebe37d0cmr1376960ljg.278.1646874601733; Wed, 09 Mar 2022 17:10:01 -0800 (PST) MIME-Version: 1.0 References: <20220308213417.1407042-1-zokeefe@google.com> <20220308213417.1407042-8-zokeefe@google.com> In-Reply-To: From: "Zach O'Keefe" Date: Wed, 9 Mar 2022 17:09:24 -0800 Message-ID: Subject: Re: [RFC PATCH 07/14] mm/khugepaged: add vm_flags_ignore to hugepage_vma_revalidate_pmd_count() To: Yang Shi Cc: Alex Shi , David Hildenbrand , David Rientjes , Michal Hocko , Pasha Tatashin , SeongJae Park , Song Liu , Vlastimil Babka , Zi Yan , Linux MM , Andrea Arcangeli , Andrew Morton , Arnd Bergmann , Axel Rasmussen , Chris Kennelly , Chris Zankel , Helge Deller , Hugh Dickins , Ivan Kokshaysky , "James E.J. Bottomley" , Jens Axboe , "Kirill A. Shutemov" , Matthew Wilcox , Matt Turner , Max Filippov , Miaohe Lin , Minchan Kim , Patrick Xia , Pavel Begunkov , Peter Xu , Thomas Bogendoerfer Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: A9410140014 X-Stat-Signature: 5c7wwiyuyb46zkw3omczdasxeapjedin X-Rspam-User: Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=LMNYc543; spf=pass (imf09.hostedemail.com: domain of zokeefe@google.com designates 209.85.208.178 as permitted sender) smtp.mailfrom=zokeefe@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam07 X-HE-Tag: 1646874603-900325 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Mar 9, 2022 at 4:41 PM Yang Shi wrote: > > On Wed, Mar 9, 2022 at 4:01 PM Zach O'Keefe wrote: > > > > > On Tue, Mar 8, 2022 at 1:35 PM Zach O'Keefe wrote: > > > > > > > > In madvise collapse context, we optionally want to be able to ignore > > > > advice from MADV_NOHUGEPAGE-marked regions. > > > > > > Could you please elaborate why this usecase is valid? Typically > > > MADV_NOHUGEPAGE is set when the users really don't want to have THP > > > for this area. So it doesn't make too much sense to ignore it IMHO. > > > > > > > Hey Yang, thanks for taking time to review and comment. > > > > Semantically, the way I see it, is that MADV_NOHUGEPAGE is a way for > > the user to say "I don't want hugepages here", so that the kernel > > knows not to do so when faulting memory, and khugepaged can stay away. > > However, in MADV_COLLAPSE, the user is explicitly requesting this be > > backed by hugepages - so presumably that is exactly what they want. > > > > IOW, if the user didn't want this memory to be backed by hugepages, > > they wouldn't be MADV_COLLAPSE'ing it. If there was a range of memory > > the user wanted collapsed, but that had some sub-areas marked > > MADV_NOHUGEPAGE, they could always issue multiple MADV_COLLAPSE > > operations around the excluded regions. > > > > In terms of use cases, I don't have a concrete example, but a user > > could hypothetically choose to exclude regions from management from > > khugepaged, but still be able to collapse the memory themselves, > > when/if they deem appropriate. > > I see. It seems you thought MADV_COLLAPSE actually unsets > VM_NOHUGEPAGE, and is kind of equal to MADV_HUGEPAGE + doing collapse > right away, right? To some degree, it makes some sense. Currently, MADV_COLLAPSE doesn't alter the vma flags at all - it just ignores VM_NOHUGEPAGE, and so it's not really the same as MADV_HUGEPAGE + MADV_COLLAPSE (which would set VM_HUGEPAGE in addition to clearing VM_NOHUGEPAGE). If my use case has any merit (and I'm not sure it does) then we don't want to be altering the vma flags since we don't want to touch khugepaged behavior. > If this is the > behavior you'd like to achieve, I'd suggest making it more explicit, > for example, setting VM_HUGEPAGE for the MADV_COLLAPSE area rather > than ignore or change vm flags silently. When using madvise mode, but > not having VM_HUGEPAGE set, the vma check should fail in the current > code (I didn't look hard if you already covered this or not). > You're correct, this will fail, since it's following the same semantics as the fault path. I see what you're saying though; that perhaps this is inconsistent with my above reasoning that "the user asked to collapse this memory, and so we should do it". If so, then perhaps MADV_COLLAPSE just ignores madise mode and VM_[NO]HUGEPAGE entirely for the purposes of eligibility, and only uses it for the purposes of determining gfp flags for compaction/reclaim. Pushing that further, compaction/reclaim could entirely be specified by the user using a process_madvise(2) flag (later in the series, we do something like this). > > > > > > > > > > Add a vm_flags_ignore argument to hugepage_vma_revalidate_pmd_count() > > > > which can be used to ignore vm flags used when considering thp > > > > eligibility. > > > > > > > > Signed-off-by: Zach O'Keefe > > > > --- > > > > mm/khugepaged.c | 18 ++++++++++++------ > > > > 1 file changed, 12 insertions(+), 6 deletions(-) > > > > > > > > diff --git a/mm/khugepaged.c b/mm/khugepaged.c > > > > index 1d20be47bcea..ecbd3fc41c80 100644 > > > > --- a/mm/khugepaged.c > > > > +++ b/mm/khugepaged.c > > > > @@ -964,10 +964,14 @@ khugepaged_alloc_page(struct page **hpage, gfp_t gfp, int node) > > > > #endif > > > > > > > > /* > > > > - * Revalidate a vma's eligibility to collapse nr hugepages. > > > > + * Revalidate a vma's eligibility to collapse nr hugepages. vm_flags_ignore > > > > + * can be used to ignore certain vma_flags that would otherwise be checked - > > > > + * the principal example being VM_NOHUGEPAGE which is ignored in madvise > > > > + * collapse context. > > > > */ > > > > static int hugepage_vma_revalidate_pmd_count(struct mm_struct *mm, > > > > unsigned long address, int nr, > > > > + unsigned long vm_flags_ignore, > > > > struct vm_area_struct **vmap) > > > > { > > > > struct vm_area_struct *vma; > > > > @@ -986,7 +990,7 @@ static int hugepage_vma_revalidate_pmd_count(struct mm_struct *mm, > > > > hend = vma->vm_end & HPAGE_PMD_MASK; > > > > if (address < hstart || (address + nr * HPAGE_PMD_SIZE) > hend) > > > > return SCAN_ADDRESS_RANGE; > > > > - if (!hugepage_vma_check(vma, vma->vm_flags)) > > > > + if (!hugepage_vma_check(vma, vma->vm_flags & ~vm_flags_ignore)) > > > > return SCAN_VMA_CHECK; > > > > /* Anon VMA expected */ > > > > if (!vma->anon_vma || vma->vm_ops) > > > > @@ -1000,9 +1004,11 @@ static int hugepage_vma_revalidate_pmd_count(struct mm_struct *mm, > > > > */ > > > > > > > > static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, > > > > + unsigned long vm_flags_ignore, > > > > struct vm_area_struct **vmap) > > > > { > > > > - return hugepage_vma_revalidate_pmd_count(mm, address, 1, vmap); > > > > + return hugepage_vma_revalidate_pmd_count(mm, address, 1, > > > > + vm_flags_ignore, vmap); > > > > } > > > > > > > > /* > > > > @@ -1043,7 +1049,7 @@ static bool __collapse_huge_page_swapin(struct mm_struct *mm, > > > > /* do_swap_page returns VM_FAULT_RETRY with released mmap_lock */ > > > > if (ret & VM_FAULT_RETRY) { > > > > mmap_read_lock(mm); > > > > - if (hugepage_vma_revalidate(mm, haddr, &vma)) { > > > > + if (hugepage_vma_revalidate(mm, haddr, VM_NONE, &vma)) { > > > > /* vma is no longer available, don't continue to swapin */ > > > > trace_mm_collapse_huge_page_swapin(mm, swapped_in, referenced, 0); > > > > return false; > > > > @@ -1200,7 +1206,7 @@ static void collapse_huge_page(struct mm_struct *mm, > > > > count_memcg_page_event(new_page, THP_COLLAPSE_ALLOC); > > > > > > > > mmap_read_lock(mm); > > > > - result = hugepage_vma_revalidate(mm, address, &vma); > > > > + result = hugepage_vma_revalidate(mm, address, VM_NONE, &vma); > > > > if (result) { > > > > mmap_read_unlock(mm); > > > > goto out_nolock; > > > > @@ -1232,7 +1238,7 @@ static void collapse_huge_page(struct mm_struct *mm, > > > > */ > > > > mmap_write_lock(mm); > > > > > > > > - result = hugepage_vma_revalidate(mm, address, &vma); > > > > + result = hugepage_vma_revalidate(mm, address, VM_NONE, &vma); > > > > if (result) > > > > goto out_up_write; > > > > /* check if the pmd is still valid */ > > > > -- > > > > 2.35.1.616.g0bdcbb4464-goog > > > >