From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B51DEC47DB3 for ; Mon, 29 Jan 2024 19:04:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 15BFB6B0075; Mon, 29 Jan 2024 14:04:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 10C566B0088; Mon, 29 Jan 2024 14:04:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F14A36B0092; Mon, 29 Jan 2024 14:04:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E167C6B0075 for ; Mon, 29 Jan 2024 14:04:20 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7B524C085C for ; Mon, 29 Jan 2024 19:04:20 +0000 (UTC) X-FDA: 81733274280.14.3982979 Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) by imf18.hostedemail.com (Postfix) with ESMTP id 8D7401C0018 for ; Mon, 29 Jan 2024 19:04:18 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=UFlAAqzh; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of zokeefe@google.com designates 209.85.208.54 as permitted sender) smtp.mailfrom=zokeefe@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706555058; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Zfk63n86B5JSqicAWX9Q/j/lAYaF2C1u83gX8kyOotg=; b=diuk5iRd3Jd9J2tua3vvXZc2A4iHsHEZS19RH/QxNp599MA7iVayU7QXmbazI2gX8BkCd4 aFXF3F/0dUTrpVy+50owYqqVnfYOPKL6nCN1Gfkb6+O2el6N2LLBpxuDAhgNGFicfTKRCh zPFupFoVkF/ZMjruq1//XuQm1PKBS4U= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=UFlAAqzh; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of zokeefe@google.com designates 209.85.208.54 as permitted sender) smtp.mailfrom=zokeefe@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706555058; a=rsa-sha256; cv=none; b=bJkUk1l/VquSKQwmiqq5Ag/stsRXDiTbc54Xdyl1SMxfB39Ncsv6fXau9aovD1OKPxcT6V fCGlp7Bh25vz7S/qGRXJQGCQ2fQDb9dcwb72rCLz4v1VAnFzUK3B6WuDYk+impMaVobiwR Em606k8a/d3wyYzVSq77xp5gzDIu7tE= Received: by mail-ed1-f54.google.com with SMTP id 4fb4d7f45d1cf-55c24a32bf4so1233a12.0 for ; Mon, 29 Jan 2024 11:04:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706555057; x=1707159857; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Zfk63n86B5JSqicAWX9Q/j/lAYaF2C1u83gX8kyOotg=; b=UFlAAqzhtJn/ACIW0hIWOCfD3dV/C5zGMP83H1ssLm0cWFnbZbVQZME/ViGbzJN8R2 efTRXNPMJRccfCB97ZOH3Q5zkW9RHkCKyKgT2yV6kaZVu3mHiHRQUrTlTak/rM80ks6c zca2Z3JmUSSbKZb+V7GnXRzGuX49dokzTjU58yYA4pzAeL673TBMhYpsLjPGElybhS7D lkDYogXbCia4olNXxjTdTqtrBAXvyVnOi35Yq7Y5Z4AY7khrSUZOjsAd7Lzf1vDh6og5 2jWgr6rwXh9TPWlOsDNav54f7yKt4dirx/qIMugZN9Oralk78fCA118LHKhTZBrgnxF6 YTag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706555057; x=1707159857; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Zfk63n86B5JSqicAWX9Q/j/lAYaF2C1u83gX8kyOotg=; b=b6VBWQ8QwWd4csCOJcHh/DSMWfLoP9riiNw71NgyzA7nJk4WWN3UgVylPdAjyBOl4t ZQLUovhV2/OAv7OIzoJlH6Lim58y1ywikbV2zjoSnrsTUroEg2/m3CKLomRCergHiT59 qmv9lr796LubSGvh/a9c+6mjvy0S1KO+o9GmzL45OihdfjkdPXDYmfvH7O3IRHx2L3QG jpc2JmmSdwds1BO7zoRcYalp7XGGDGT2Pm131F2hYiAk6VZbUM6PqqaDxPfqaf4IGcCZ soJ+7nwOiuk6X3aeIHpmNoj42eieWGcdjZzHttG26UJM1XAHwxVFxktjd78T/o1ew5EN LbVg== X-Gm-Message-State: AOJu0YzjzMhlPYlTRwKIIb+Tyg92JzhTrHhxYf25jEMFKOkCK1vAvWJh kyiGJSTGYtbjx1RgaWTfNaPbiaaaoAQUfj28iF1lcIjkXUNUZxuVZg6ozVLrNQdAYl1XTltvTeH ESgVHyboA4VbVdNP2TOuBcgynVrYPZ38l7FMw X-Google-Smtp-Source: AGHT+IEvbaU/dgjjI1icFJRHFIY5hi4PapHDFTvsY7KqHsSoeG7OlrWxGUjRRkk4L9vrUQjZbK4MODRkMXA+gD+Yutw= X-Received: by 2002:a05:6402:37a:b0:55f:368:246f with SMTP id s26-20020a056402037a00b0055f0368246fmr10881edw.4.1706555056885; Mon, 29 Jan 2024 11:04:16 -0800 (PST) MIME-Version: 1.0 References: <20240129054551.57728-1-ioworker0@gmail.com> In-Reply-To: From: "Zach O'Keefe" Date: Mon, 29 Jan 2024 11:03:40 -0800 Message-ID: Subject: Re: [PATCH 1/1] mm/khugepaged: bypassing unnecessary scans with MMF_DISABLE_THP check To: Yang Shi Cc: Lance Yang , akpm@linux-foundation.org, mhocko@suse.com, david@redhat.com, songmuchun@bytedance.com, peterx@redhat.com, minchan@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 8D7401C0018 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: oj7zd19bty8ygfwz8c5bkb9ddcs7at85 X-HE-Tag: 1706555058-977617 X-HE-Meta: U2FsdGVkX18ms/5WWlnPyo0TkgRrW8fIqpE1f88zQvFjuyD0jeR07ieOEFrE/KzhIYfS2MKdCCJScGrbzk8YYo7TuE4FKTWAFlbekFnv2FsSaznHSri4YZPMTZ6gMCFf56LNwmWSXEQdgFLGhNcxAWwmhiR2R/PqLslpecDfqBaJ3FLw9N6Dutl58C9RV03hhqF3swv7AbEqx/ELgE3IwLcgxtoDFDKo/h1Mwc88bjj6x/64HH//+LLBbYIrgUJzDijfRFMqxaGqEDpASBidJRMShElVCyocKioi79W5cYz/gALcc7lMHqo0OosLvaIDYKm3uG6Cbx0nkyrPeGyIADKhG2DFkvleuMTjF+B7omIL+X9reb+6VyDRiVrf2khJGBibYjC3q/rj5wIx3oZGXOhyjuWuWYY7dSuPCVJXKx/A9j7hd8AgB02/gzKq0vImXQH05uK5F5XYMaMLMLjoPgKgVY0TPTS/KK+7a07QdYAzjobyeRSrU/s8DiJ0NyuiWZmC+FkDZeAJUmSvS+U5+eLr4NfwXVqhkNnRvan117EOWCwyauz8enAz/pVBlek8mVz85bPHLZRbPYWlMP/RFSReYX1w96PuO4nCM35JQHNMqFgiYfJ88z5h2Kh1g19fuX5NA5frUcOFFzdIz2bzt6V+cVxu+XlnCkVdn/31CKIUMajW473kPwHNg04vgq/Ew7Eq3VZI3gdotOhXD21WFvkgrrEMeG9fdfkBwJmNjZVk5qNJcyI4WNLoINEngiycm1Nhk07kQJaombaOzMYBZ9R7H+9K/i/m1QMOAazLkmlEtbYLBn0XfGbpbV6+EERDXNfgxkOjXviM0D+1T8SlYdT0TcR/0W+M00RJji0NskRo4EZXvXw6QK4IaGLsY9qjnbTawbcjtcEll46Kecay01tSd4LTAANQMB57xxxEXb8tt5j5rmVVIruJwKQ4mpZxhnjagAhLcT8Vf4D0OZ1 mjWSFlvR 67LYFw2VQPb3ByTwEFRLnc5CH3r2OfPc+3OOCN+bEtk9Nt3aiHuhWjhFTzUSaXL7Vj2FZhEAdZIty6oUZkDgzXX6Zo33Kl7lm8jJxateP0Sv76c3J+XS7zzwDJqwAsHNjhI2TqZz7cksEJmlvpHbwXtszTZ8nJ3RB5r5T+47YDsWAc8/ZJsYmmj5UBmw3K/q767iISBLGPiTfswR0jepKICS2BJD4VnfPFy3a7T8/PmuLl38SJ+Ik62vx1GuAfLRcY3GDSMjCiycJzIG0riNl1loBMaXEKFJEr5GHbTAl0GPL96A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Jan 29, 2024 at 10:53=E2=80=AFAM Yang Shi wro= te: > > On Sun, Jan 28, 2024 at 9:46=E2=80=AFPM Lance Yang = wrote: > > > > khugepaged scans the entire address space in the > > background for each given mm, looking for > > opportunities to merge sequences of basic pages > > into huge pages. However, when an mm is inserted > > to the mm_slots list, and the MMF_DISABLE_THP flag > > is set later, this scanning process becomes > > unnecessary for that mm and can be skipped to avoid > > redundant operations, especially in scenarios with > > a large address space. > > > > This commit introduces a check before each scanning > > process to test the MMF_DISABLE_THP flag for the > > given mm; if the flag is set, the scanning process > > is bypassed, thereby improving the efficiency of > > khugepaged. > > > > Signed-off-by: Lance Yang > > --- > > mm/khugepaged.c | 18 ++++++++++++------ > > 1 file changed, 12 insertions(+), 6 deletions(-) > > > > diff --git a/mm/khugepaged.c b/mm/khugepaged.c > > index 2b219acb528e..d6a700834edc 100644 > > --- a/mm/khugepaged.c > > +++ b/mm/khugepaged.c > > @@ -410,6 +410,12 @@ static inline int hpage_collapse_test_exit(struct = mm_struct *mm) > > return atomic_read(&mm->mm_users) =3D=3D 0; > > } > > > > +static inline int hpage_collapse_test_exit_or_disable(struct mm_struct= *mm) > > +{ > > + return hpage_collapse_test_exit(mm) || > > + test_bit(MMF_DISABLE_THP, &mm->flags); > > +} > > + > > void __khugepaged_enter(struct mm_struct *mm) > > { > > struct khugepaged_mm_slot *mm_slot; > > @@ -1422,7 +1428,7 @@ static void collect_mm_slot(struct khugepaged_mm_= slot *mm_slot) > > > > lockdep_assert_held(&khugepaged_mm_lock); > > > > - if (hpage_collapse_test_exit(mm)) { > > + if (hpage_collapse_test_exit_or_disable(mm)) { > > /* free mm_slot */ > > hash_del(&slot->hash); > > list_del(&slot->mm_node); > > @@ -2360,7 +2366,7 @@ static unsigned int khugepaged_scan_mm_slot(unsig= ned int pages, int *result, > > goto breakouterloop_mmap_lock; > > > > progress++; > > - if (unlikely(hpage_collapse_test_exit(mm))) > > + if (unlikely(hpage_collapse_test_exit_or_disable(mm))) > > goto breakouterloop; > > > > vma_iter_init(&vmi, mm, khugepaged_scan.address); > > @@ -2368,7 +2374,7 @@ static unsigned int khugepaged_scan_mm_slot(unsig= ned int pages, int *result, > > unsigned long hstart, hend; > > > > cond_resched(); > > - if (unlikely(hpage_collapse_test_exit(mm))) { > > + if (unlikely(hpage_collapse_test_exit_or_disable(mm))) = { > > The later thp_vma_allowable_order() does check whether MMF_DISABLE_THP > is set or not. And the hugepage_vma_revalidate() after re-acquiring > mmap_lock does the same check too. The checking in khugepaged should > be already serialized with prctl, which takes mmap_lock in write. IIUC, there really isn't any correctness race. Claim is just that we can avoid a number of per-vma checks. AFAICT, any task w/ MMF_DISABLE_THP set will always have each and every vma checked (albeit, with a very inexpensive ->vm_mm->flags check) Thanks, Zach > > progress++; > > break; > > } > > @@ -2390,7 +2396,7 @@ static unsigned int khugepaged_scan_mm_slot(unsig= ned int pages, int *result, > > bool mmap_locked =3D true; > > > > cond_resched(); > > - if (unlikely(hpage_collapse_test_exit(mm))) > > + if (unlikely(hpage_collapse_test_exit_or_disabl= e(mm))) > > goto breakouterloop; > > > > VM_BUG_ON(khugepaged_scan.address < hstart || > > @@ -2408,7 +2414,7 @@ static unsigned int khugepaged_scan_mm_slot(unsig= ned int pages, int *result, > > fput(file); > > if (*result =3D=3D SCAN_PTE_MAPPED_HUGE= PAGE) { > > mmap_read_lock(mm); > > - if (hpage_collapse_test_exit(mm= )) > > + if (hpage_collapse_test_exit_or= _disable(mm)) > > goto breakouterloop; > > *result =3D collapse_pte_mapped= _thp(mm, > > khugepaged_scan.address= , false); > > @@ -2450,7 +2456,7 @@ static unsigned int khugepaged_scan_mm_slot(unsig= ned int pages, int *result, > > * Release the current mm_slot if this mm is about to die, or > > * if we scanned all vmas of this mm. > > */ > > - if (hpage_collapse_test_exit(mm) || !vma) { > > + if (hpage_collapse_test_exit_or_disable(mm) || !vma) { > > /* > > * Make sure that if mm_users is reaching zero while > > * khugepaged runs here, khugepaged_exit will find > > -- > > 2.33.1 > >