From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4A0EC7618E for ; Wed, 26 Apr 2023 08:12:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 35CC86B0087; Wed, 26 Apr 2023 04:12:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 30C536B00A0; Wed, 26 Apr 2023 04:12:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1FCD96B00A1; Wed, 26 Apr 2023 04:12:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1291B6B0087 for ; Wed, 26 Apr 2023 04:12:00 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id DA0E0C020E for ; Wed, 26 Apr 2023 08:11:59 +0000 (UTC) X-FDA: 80722823958.12.40F0929 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf26.hostedemail.com (Postfix) with ESMTP id D5852140003 for ; Wed, 26 Apr 2023 08:11:57 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf26.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682496718; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sBYj9L4DAaGE2wCj0ZCy73msx+01oKLqOwCzdloNjsM=; b=gGofhnfGCvVv/DTRcCL+Xji+LGcdYfpeq/NNBkTAUWQOOO0pt6fpKSh0UMHH4g7+8BYmD2 k5Edfqt5dMfVbhnd4shyPmiX02Y0YHUtIza+/7s1ms89hazZOUfJBT5+THDT1HkPsa8pls sWtPfU+zhSFb87HQvDi5SGBYNqDlJmQ= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf26.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682496718; a=rsa-sha256; cv=none; b=e6Y1id5L/f9oQD0lIPJtWZhjbpquowfKlLUU/xlDP+bnCmSrbt832NtHj8rJBWTMm/lbD9 aYxHVAU1cKHwfhK7oRidBcMCOj6/SPIXN5P6+Oh7ynkJfKkQKMaM9v8aXelW6RYPHqQ8oE rZHyskAvSuFdwzjWekZmbtlPwFytdK4= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AA2F34B3; Wed, 26 Apr 2023 01:12:40 -0700 (PDT) Received: from [10.57.69.78] (unknown [10.57.69.78]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CBCF73F587; Wed, 26 Apr 2023 01:11:55 -0700 (PDT) Message-ID: <9f637a9d-f7d6-b051-74ae-05d4571e8df4@arm.com> Date: Wed, 26 Apr 2023 09:11:54 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: [PATCH v2 1/2] THP: avoid lock when check whether THP is in deferred list To: Yin Fengwei , linux-mm@kvack.org, akpm@linux-foundation.org, willy@infradead.org, yuzhao@google.com, ying.huang@intel.com References: <20230425084627.3573866-1-fengwei.yin@intel.com> <20230425084627.3573866-2-fengwei.yin@intel.com> Content-Language: en-US From: Ryan Roberts In-Reply-To: <20230425084627.3573866-2-fengwei.yin@intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: D5852140003 X-Stat-Signature: 13i9rpfcqk7sx9pc96t41164scy9bwye X-HE-Tag: 1682496717-16313 X-HE-Meta: U2FsdGVkX1/rb+zlSFZVwQOQWEWn2exik7/5ilq+jPAqiiopFbMQxsCpAyT00Wsb63EERQPMN3+AYtkN/Hs0WptFuPZoWZ7bCRx2iWompKlyIcAhR4Zlg5YUMRhgoca0I7zF4+uD682zvrVEerJrTJNOKZkn4zfxbTdMPZL3J7cllT5dHOXMvGfBNSUIKK9E8anum3cLSLMehYGSvnR/i5IvuGEDgJ99fx9NHLYT7cbn3mBHS+JqS/TqxonIOLZ8ejXm1cr+RaQm6iyOiRTdp4enpb8uiF1SRzxkULbwRu2nhBK2n7c89FruQ4bUVJ82/7gfNh/E0Uog9Y7sJ+59kWQNqs514C1GysuZQzkuhT/tHDcZmT0uyyzkzx7vzwIX1412nfzbVTaugZZC4nTBGknLQKdTnL6U41oIhG8Y2hRLgqOJIJq5ceDzmTwlARmJVc/Kjr9pBBICfB3gYxo9sOgn8Pe0JS5Oyc/pEZE/sY9Cz0DHBmJvYY8JtIoKjE7+M10jIZt1buJucQBR15PeiG1jirJWH1no3uQ9pz/AKLr3/+eedemXfXmedkCr/JoYa8qqA4ctljvC3Qwv9XAcqMR4NRQzbse5x9SiE87DZunJlgVqONZuHe2JelK9yXH0cAgq5yGeukfW2aHGhxIXqcempgxfMTx8OZ4N/6eu6R1CSY7s+p4QhVwWEmhzdCrj5tjmivq/VlGxE3zKLlx2Ct4A07d+H/nuPVIxxtAu7JZ5AYXTxAG9T+RRuhU13SHWSxf+3sqnK08gPl1q1D7AqXbnHvF2gaRPFrRqAKSIiV3avIKNYjD9G6ZdKDTDxsKsaZ+Bp3Xo7ABFqnM8UnHVoXsDFZR1JHp+snAlYF4QI6H51AJRgFlGjpvHgew0R/LGefyotvZTgyRytiMq6WZmgByPnAmxcC6yzfYgrFY/OHILaBXqhboVewAkNPwCvBbsNWsNEQtk4fxrRdzZtua AUzh0EGQ Pm35ARELNv/9bkNCzMdd3MH2oYcet6CD1FeXLbFvKqZbew0d8whF8LEnN68kpVM2XCilnpc8SKAr2+z1bM5T57ws5UCmq4ZntG7NG3Uuowz0qWdADDa+4JCuiDAPkYVdfKUtFIthLFgQBw8nhzlqwi7mFFT6cKq1e5LU7BwFNpYScnOH7Auu9LoFT1Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 25/04/2023 09:46, Yin Fengwei wrote: > free_transhuge_page() acquires split queue lock then check > whether the THP was added to deferred list or not. > > It's safe to check whether the THP is in deferred list or not. > When code hit free_transhuge_page(), there is no one tries > to update the folio's _deferred_list. > > If folio is not in deferred_list, it's safe to check without > acquiring lock. > > If folio is in deferred_list, the other node in deferred_list > adding/deleteing doesn't impact the return value of > list_epmty(@folio->_deferred_list). > > Running page_fault1 of will-it-scale + order 2 folio for anonymous > mapping with 96 processes on an Ice Lake 48C/96T test box, we could > see the 61% split_queue_lock contention: > - 71.28% 0.35% page_fault1_pro [kernel.kallsyms] [k] > release_pages > - 70.93% release_pages > - 61.42% free_transhuge_page > + 60.77% _raw_spin_lock_irqsave > > With this patch applied, the split_queue_lock contention is less > than 1%. > > Signed-off-by: Yin Fengwei > Tested-by: Ryan Roberts > --- > mm/huge_memory.c | 19 ++++++++++++++++--- > 1 file changed, 16 insertions(+), 3 deletions(-) > > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > index 032fb0ef9cd1..c620f1f12247 100644 > --- a/mm/huge_memory.c > +++ b/mm/huge_memory.c > @@ -2799,12 +2799,25 @@ void free_transhuge_page(struct page *page) > struct deferred_split *ds_queue = get_deferred_split_queue(folio); > unsigned long flags; > > - spin_lock_irqsave(&ds_queue->split_queue_lock, flags); > - if (!list_empty(&folio->_deferred_list)) { > + /* > + * At this point, there is no one trying to queue the folio > + * to deferred_list. folio->_deferred_list is not possible > + * being updated. > + * > + * If folio is already added to deferred_list, add/delete to/from > + * deferred_list will not impact list_empty(&folio->_deferred_list). > + * It's safe to check list_empty(&folio->_deferred_list) without > + * acquiring the lock. > + * > + * If folio is not in deferred_list, it's safe to check without > + * acquiring the lock. > + */ > + if (data_race(!list_empty(&folio->_deferred_list))) { > + spin_lock_irqsave(&ds_queue->split_queue_lock, flags); > ds_queue->split_queue_len--; > list_del(&folio->_deferred_list); I wonder if there is a race here? Could the folio have been in the deferred list when checking, but then something removed it from the list before the lock is taken? In this case, I guess split_queue_len would be out of sync with the number of folios in the queue? Perhaps recheck list_empty() after taking the lock? Thanks, Ryan > + spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); > } > - spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); > free_compound_page(page); > } >