From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82055C77B61 for ; Fri, 28 Apr 2023 14:02:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9C926B0071; Fri, 28 Apr 2023 10:02:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B25636B0072; Fri, 28 Apr 2023 10:02:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99EB26B0074; Fri, 28 Apr 2023 10:02:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 85ADC6B0071 for ; Fri, 28 Apr 2023 10:02:48 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 303821C67C4 for ; Fri, 28 Apr 2023 14:02:48 +0000 (UTC) X-FDA: 80730965616.21.191601A Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) by imf25.hostedemail.com (Postfix) with ESMTP id 62C7FA0013 for ; Fri, 28 Apr 2023 14:02:43 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=shutemov.name header.s=fm1 header.b="F yQES8z"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=XBTmdq2Q; spf=pass (imf25.hostedemail.com: domain of kirill@shutemov.name designates 64.147.123.21 as permitted sender) smtp.mailfrom=kirill@shutemov.name; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682690563; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=x9gm/YjlJgIBf4QLx8zokV0pDTsVwMfKgmpkz5w/+SA=; b=va+4D9f8HOaJ8zuTv/BdWx6rfpvHsYEr1gVk+gTlM1HibG25nbf2X64j0SrTeq99KJrbkR bcJOgCvgayg74GfDPcHv7pE6PSo/YFgSDaIggey6GtpTr4Q1tAw+Fc6WQdNEaPQALDNEU0 NAZ7v3JgHmRxpy8XpVd/vMBSoXZFp40= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=shutemov.name header.s=fm1 header.b="F yQES8z"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=XBTmdq2Q; spf=pass (imf25.hostedemail.com: domain of kirill@shutemov.name designates 64.147.123.21 as permitted sender) smtp.mailfrom=kirill@shutemov.name; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682690563; a=rsa-sha256; cv=none; b=u/3kc2i1Me0ipP6s7QqSFslGOXbDmQyKvTfq0QB7LfWLFgaqzIOHgDUYA7VGTICKt59iwt E44qyTOQ6T1FhCiLg/PYzjMcFdB+AMrwB30UkPJ1hB7u1NWzJbduw9SykUSJiJaTxx2Q84 K55MJnsHeAVoLAhZ4PY8q77SYNEzx9s= Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 3F1B1320083A; Fri, 28 Apr 2023 10:02:41 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Fri, 28 Apr 2023 10:02:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov.name; h=cc:cc:content-type:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm1; t=1682690560; x= 1682776960; bh=x9gm/YjlJgIBf4QLx8zokV0pDTsVwMfKgmpkz5w/+SA=; b=F yQES8zvSX4C+j1f97ev4Eg9ZxKI3vPyNTPMO6pICuNRv3DdC/B/pFZCokXKYxG18 BnI5ZPysfja/XFr7c3hP2qFn12wE8rPxCTd/oSaTS2l9ldke9PxrpA0+672YsTFY 2k0nSjbDefKCDCIf2ZCQ8FDtzx2OuLXzKYI5H7gi3hEq5M4OFfdHxfisSV7fmWqa xsap/EW7hGN+zFPtrw6rueKsh7cWlPO0H/OjDdSgFRzr/Bt6NFt/f+gOSq6cwOA+ jXoNWEpYBY8i4Q4OY+1VYz0Inb1yBsJAf1rWbPh4fDSbeL5OQL2YrRGJaqunq/Bh Pw+N36BjavwfgXg5/86rw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1682690560; x=1682776960; bh=x9gm/YjlJgIBf 4QLx8zokV0pDTsVwMfKgmpkz5w/+SA=; b=XBTmdq2QsXg6ha0ctUKiEscwoEyKT 6pQxyIIYGdq/3cnuZ2bk8+nwub2rKfK9iElsujOQcKLe/oMaUKp5QheOjcoScIMk q/SZg6fR/6k4cdc5cO8AxBS+WEagYewmvkKgfTC9+seEHnjjiVgKdjfGHLWk2hV+ 79cIssUP576iAZ+Z1v7YAdQF43p/JxlZ59vNTnieFdF/dQPumyEjMudd22+X4kSj NvIgGnqyoDPqOxDkL/eQfDOSTSacKT7qUmPcZqzIcw3PYppuUFTjBqutbjizrXP5 LpTF3AKsaumGQsEJ0lLY2ewys0yh65L4CK4CX9A5y0u4Ak4/A6aspCZPA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfedukedgjedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesthdttddttddtvdenucfhrhhomhepfdfmihhr ihhllhcutedrucfuhhhuthgvmhhovhdfuceokhhirhhilhhlsehshhhuthgvmhhovhdrnh grmhgvqeenucggtffrrghtthgvrhhnpefhieeghfdtfeehtdeftdehgfehuddtvdeuheet tddtheejueekjeegueeivdektdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpehkihhrihhllhesshhhuhhtvghmohhvrdhnrghmvg X-ME-Proxy: Feedback-ID: ie3994620:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 28 Apr 2023 10:02:39 -0400 (EDT) Received: by box.shutemov.name (Postfix, from userid 1000) id 3B9831041AE; Fri, 28 Apr 2023 17:02:36 +0300 (+03) Date: Fri, 28 Apr 2023 17:02:36 +0300 From: "Kirill A. Shutemov" To: "Yin, Fengwei" Cc: linux-mm@kvack.org, akpm@linux-foundation.org, willy@infradead.org, yuzhao@google.com, ryan.roberts@arm.com, ying.huang@intel.com Subject: Re: [PATCH v2 1/2] THP: avoid lock when check whether THP is in deferred list Message-ID: <20230428140236.czx5eii34z373jqq@box.shutemov.name> References: <20230425084627.3573866-1-fengwei.yin@intel.com> <20230425084627.3573866-2-fengwei.yin@intel.com> <20230425123857.k5mp2cdp5c6ldbrk@box.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: 6gygrrfnuetqfzh3tntdd7bcpwcjdca7 X-Rspam-User: X-Rspamd-Queue-Id: 62C7FA0013 X-Rspamd-Server: rspam06 X-HE-Tag: 1682690563-777229 X-HE-Meta: U2FsdGVkX1/AFDXv0pHuekDUcs+U/dBYuJZGRMcrD38XbxYV8rYrZKJ5JG8iI/MOOzidYr2veMlB911OcwS8zQPoLMM52+PcOu7kH56KA5oHfx/k7xdNs1tYInUtY+UtBDtjY5zMQtETrD5xFXfDCjsqQqeRS4JgUuC74qrvTYmYKDOFLSNfCbx98HucQlPDlhZCzgtgHoBLGDPlGxzDsQ8sRmqwfVAPORM/9n1q/Ng5obkp8zbfItzzrZSjq+YU6AaJegyFXjkrZJ/Q7XWfyUOv4Q4PDPaWhtLGecu30L5cGCmOkMg8N07LL4NPiP6kVLfX8lgO9EPUGZlWtBcgev4xSO07NJw/a53lhM5tpFIjz7uTGOBUlEvJhwTt15kxwyN9riMDMmf5LKLSsXQrELSZYD5+t7QvurGqad0tzTKMDA0YD9HkxgS1N7cBzJadblmyIfJRA/GChVZ8mL510igMEMlHMKvOYyKygxPhCtZJnEEEN4T/Of2znUg1FoYJgoZNtZAAwsMuszPGLTP20f8ZW1ng4Y2AZM6j+nq4Lh/+n5mmIOKWKpmLsGIo36F2zVT2o175+uH5jJ5Pg5cpDmPbADECqEfLkJf2y0NVomteyT2hIL8/n2n2kDjVgh9N6/KrdXc2eNS34p3gZ7myewkkQLNWmPBziMmaSsp3yTFbvQurjYTBEs1gAsmrq6mdLJgZCdkVD5Gm4Wnla0wRg5dzu6dSiUILIYvn+EqHHn1jz/z1K9581hU9ySrMwohZSqgJjcFMEqmSuWszreg2MC+q8vs22zL14wehGJ1FVPSdM4brnSf6wkUpmLrB28bzwAxENeemXrEOH+6Ud5Grhz66LWL33KAUjw3MwhL1X+TPX6Lg3NaSbnZVsOsU4Ty973YkFrH2VE7mCM1Yk2I1tsKonPwUB6SWUmHpJ1fJ6VT6InAGsIt4HsBWkS2LLRcBVV5uHix2Q2+5yGe0p4O WcBwj0kw MVifvZetwmBabxganrdh7W6jE+yIzR/TBJQRFCQvE6ZPv4Yb02JfAZmUZx5siYvwlLxsO0BFQVs8Jt0tlh7m+f8zIIfV8Rdt6v0BA7l5Hq3vtTLVvfVkVMrbZv7U5mBEFjc7UGLvYX4l6W1wmdcLeX/8ucyjeEEsPuxAac31DZugQynJ5CpvH6a9SQ4PDDnEZRLZi85kxHA/3AvGi9xjZSaRym7m3mf3gujKhoj9OvMb7AulhLt1RqHPcYn9yMSOmzpkGu90mLeAKArkbSIiMh/4Oeanq8jyi7tuRuv+06gtAbiv/xj8yfxJ64N7fBUPJ4k+1aONKbJNoch1T4ig59mSnIFdVz8iwygDDTmn6atGPfLf1MdxYQQqeyg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Apr 28, 2023 at 02:28:07PM +0800, Yin, Fengwei wrote: > Hi Kirill, > > On 4/25/2023 8:38 PM, Kirill A. Shutemov wrote: > > On Tue, Apr 25, 2023 at 04:46:26PM +0800, Yin Fengwei wrote: > >> free_transhuge_page() acquires split queue lock then check > >> whether the THP was added to deferred list or not. > >> > >> It's safe to check whether the THP is in deferred list or not. > >> When code hit free_transhuge_page(), there is no one tries > >> to update the folio's _deferred_list. > >> > >> If folio is not in deferred_list, it's safe to check without > >> acquiring lock. > >> > >> If folio is in deferred_list, the other node in deferred_list > >> adding/deleteing doesn't impact the return value of > >> list_epmty(@folio->_deferred_list). > > > > Typo. > > > >> > >> Running page_fault1 of will-it-scale + order 2 folio for anonymous > >> mapping with 96 processes on an Ice Lake 48C/96T test box, we could > >> see the 61% split_queue_lock contention: > >> - 71.28% 0.35% page_fault1_pro [kernel.kallsyms] [k] > >> release_pages > >> - 70.93% release_pages > >> - 61.42% free_transhuge_page > >> + 60.77% _raw_spin_lock_irqsave > >> > >> With this patch applied, the split_queue_lock contention is less > >> than 1%. > >> > >> Signed-off-by: Yin Fengwei > >> Tested-by: Ryan Roberts > >> --- > >> mm/huge_memory.c | 19 ++++++++++++++++--- > >> 1 file changed, 16 insertions(+), 3 deletions(-) > >> > >> diff --git a/mm/huge_memory.c b/mm/huge_memory.c > >> index 032fb0ef9cd1..c620f1f12247 100644 > >> --- a/mm/huge_memory.c > >> +++ b/mm/huge_memory.c > >> @@ -2799,12 +2799,25 @@ void free_transhuge_page(struct page *page) > >> struct deferred_split *ds_queue = get_deferred_split_queue(folio); > >> unsigned long flags; > >> > >> - spin_lock_irqsave(&ds_queue->split_queue_lock, flags); > >> - if (!list_empty(&folio->_deferred_list)) { > >> + /* > >> + * At this point, there is no one trying to queue the folio > >> + * to deferred_list. folio->_deferred_list is not possible > >> + * being updated. > >> + * > >> + * If folio is already added to deferred_list, add/delete to/from > >> + * deferred_list will not impact list_empty(&folio->_deferred_list). > >> + * It's safe to check list_empty(&folio->_deferred_list) without > >> + * acquiring the lock. > >> + * > >> + * If folio is not in deferred_list, it's safe to check without > >> + * acquiring the lock. > >> + */ > >> + if (data_race(!list_empty(&folio->_deferred_list))) { > >> + spin_lock_irqsave(&ds_queue->split_queue_lock, flags); > > > > Recheck under lock? > In function deferred_split_scan(), there is following code block: > if (folio_try_get(folio)) { > list_move(&folio->_deferred_list, &list); > } else { > /* We lost race with folio_put() */ > list_del_init(&folio->_deferred_list); > ds_queue->split_queue_len--; > } > > I am wondering what kind of "lost race with folio_put()" can be. > > My understanding is that it's not necessary to handle this case here > because free_transhuge_page() will handle it once folio get zero ref. > But I must miss something here. Thanks. free_transhuge_page() got when refcount is already zero. Both deferred_split_scan() and free_transhuge_page() can see the page with zero refcount. The check makes deferred_split_scan() to leave the page to the free_transhuge_page(). -- Kiryl Shutsemau / Kirill A. Shutemov