From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0ED60C77B61 for ; Tue, 25 Apr 2023 12:39:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 626866B0071; Tue, 25 Apr 2023 08:39:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D6B16B0074; Tue, 25 Apr 2023 08:39:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 49E936B0075; Tue, 25 Apr 2023 08:39:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 3B3566B0071 for ; Tue, 25 Apr 2023 08:39:08 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0EC9B14011F for ; Tue, 25 Apr 2023 12:39:08 +0000 (UTC) X-FDA: 80719868376.23.610449B Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) by imf03.hostedemail.com (Postfix) with ESMTP id D22BC2001E for ; Tue, 25 Apr 2023 12:39:04 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=shutemov.name header.s=fm1 header.b="J 3Pw14X"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=Rzx9dE1t; dmarc=none; spf=pass (imf03.hostedemail.com: domain of kirill@shutemov.name designates 64.147.123.20 as permitted sender) smtp.mailfrom=kirill@shutemov.name ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682426345; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SpRnZadvoe4sv7+EWhH/W96lJOWTcZh87++hSC2NpMY=; b=sfNAnMUvpKEVxorKmfEvAqibGXb7R03uOAIUCqH0UwSZSieSDfPIJa8de5oIoeeJX/XJNf uoMx944MeDsxPiBgPFeYFRRAOmTgwq2AmRNb7avUpt/18N4hWDlasBj7vN+hzrZ9qseb2b SxiPxL8+y3w5DdogiZvtD9ataj4Rj/0= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=shutemov.name header.s=fm1 header.b="J 3Pw14X"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=Rzx9dE1t; dmarc=none; spf=pass (imf03.hostedemail.com: domain of kirill@shutemov.name designates 64.147.123.20 as permitted sender) smtp.mailfrom=kirill@shutemov.name ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682426345; a=rsa-sha256; cv=none; b=L8H/olY26U4p/sOJdV4VI8IC4wQjhEgLDFZK0E0f8qxO5iEPljjh9ad+iXdOoE6cKvcIXG ZEK2hCmcUqnsyjQ7oyjpO5Lz/Fu37jMEtwe0i9WR6UGIQ9eO6OX5Aa5is7gK+y5LXSmvwT vWcuV/2pF0K4q38jhhOMa+r3KsaJEQM= Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id A43D63200943; Tue, 25 Apr 2023 08:39:02 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Tue, 25 Apr 2023 08:39:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov.name; h=cc:cc:content-type:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm1; t=1682426342; x= 1682512742; bh=SpRnZadvoe4sv7+EWhH/W96lJOWTcZh87++hSC2NpMY=; b=J 3Pw14XSPIsE00eYilOpi3OBH0PqNycNZeF4SVuqELagmf23hjIP30vR00/d9xlD3 w8JxlCRspZQSKhhCP2dkcNCQ3q3lEcbkEE10iuHWuT8cXK4f6kRFEzNF6IABWoDN SzV6n58DQWpRsQQH4xDo6acHLCdVHAySrnK3FEmRKLgfOlyUlvsaSBW3WHd5Idd/ +RDA04s9BWNE/NaHLl4XOmiqPsZKl7oy/B43jei2svpoK2WxCw9Vv8IxCPkf/xCv n7FHchhVmQW0RMLXGu9cdkeaI/JGsKuoJQASTneIiGzTLzX87JgnfkCTQLxulmYE cFdnJ8FwCBqIu/givUX4Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1682426342; x=1682512742; bh=SpRnZadvoe4sv 7+EWhH/W96lJOWTcZh87++hSC2NpMY=; b=Rzx9dE1t04aAqwE7byKBcQbAJGJlC duJ5BCNH0UA+3mHZmIpssZfXnuIMOLym698f7TTaBb9xWrI78ng67pMaEZEiWDBV i1KMxmUWEyjzo5Ip0RPiIhsd8S2APftUwoU2SFXs7wzm8VncvSG/fH3zYNWMZ54L e4xoxNxwe0wCyUcS0fQbbz8coiGAJAnX2DYr6AZhfrGLlCqoY/OzszxxUldtHCYB 8yOwBPGo38/0hsi/VS4LZLHPD00N8uXi4fHFiFcKjgJG8pxKOiYnbh7JDyvbvt8h NO7eHCacFAjTR8XV+PpYmZhOdCuCv6mi1dm7yNVzUCmMHydMjQyH8fYcQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfeduvddgheehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesthdttddttddtvdenucfhrhhomhepfdfmihhr ihhllhcutedrucfuhhhuthgvmhhovhdfuceokhhirhhilhhlsehshhhuthgvmhhovhdrnh grmhgvqeenucggtffrrghtthgvrhhnpefhieeghfdtfeehtdeftdehgfehuddtvdeuheet tddtheejueekjeegueeivdektdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpehkihhrihhllhesshhhuhhtvghmohhvrdhnrghmvg X-ME-Proxy: Feedback-ID: ie3994620:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 25 Apr 2023 08:39:01 -0400 (EDT) Received: by box.shutemov.name (Postfix, from userid 1000) id DFDE110BAC9; Tue, 25 Apr 2023 15:38:57 +0300 (+03) Date: Tue, 25 Apr 2023 15:38:57 +0300 From: "Kirill A. Shutemov" To: Yin Fengwei Cc: linux-mm@kvack.org, akpm@linux-foundation.org, willy@infradead.org, yuzhao@google.com, ryan.roberts@arm.com, ying.huang@intel.com Subject: Re: [PATCH v2 1/2] THP: avoid lock when check whether THP is in deferred list Message-ID: <20230425123857.k5mp2cdp5c6ldbrk@box.shutemov.name> References: <20230425084627.3573866-1-fengwei.yin@intel.com> <20230425084627.3573866-2-fengwei.yin@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230425084627.3573866-2-fengwei.yin@intel.com> X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: D22BC2001E X-Stat-Signature: r1u4uar36npxzkfkuy7uj3dwk5qmtjbu X-HE-Tag: 1682426344-169673 X-HE-Meta: U2FsdGVkX1+tnC4CgZK73OxnslUc8IVF0bjT63es9cA5GrTFhCkMYkWE4wYlFf75DH5VsbMYlOZOqVeGU8Naqu/rexGKhkRx3pyEvn1uXBMo72VZLlnPcSJsEm19pXOI1mOCq6yrnPiAf/HiqMpzYcNIC4ogFqG2I3iP4byu3MoY8UrJiqlKZTQG5WhTg0Hv8pHtUSVJOhpk9s0sw81Jjk/82KZHbb+PmjmKeWFMjFlbh9RiVhQCroH4CDedI6D1SHCxaxaNdsePeypiIvw1/kwIzXMZhIRpIQFcpqe1+2SgxKfD8zrcbxnVtrU/T7+xTzwSb7t9PTLGNvBsKll1rYmQc3p4h5IBZ4/MJu4WpHumHMolJR6PnkNnsg6bTbRfmyJIExhmKel0WVLZ+9FFFhKq/aLioMDyloGqGqL2YsM/oZ/vh1nFKrkFepPUVMOE+XzTD6Gp+xTI6V/Dt82HvUUsiH/K5VsIR0MD82gxUSztT6E1I2aKPsl7b+wZiR2WBqu4g5DZrtLoOaanDJY4VOO1ByVlNyIIoumhF6M1d0eedwLk1EDPIYoAbWf0PvBJZ+68FURlRfPOHnfj6CSvotAO2NZNFhNGt5YFUt9N9KLf09YRXCXMuKGhMtIb07aeFlYHRTxPf+vn6xK6yY8aE8FoOQjMda74i03JgOHgnzRADqSMNQq+l4TbRChMNzDMLrwvlgM3dVjZl+1wxCiDk67F+mj4Z61wwd6itnS+pGUiFnKN07zamZtFuRizgk4m8j0CaLHuXKyySvgYyzV/zJrwswjQblEA/aY9bV75aVUCrAU/4k0XFlzfDd9huVHxxsb0ceDD1ebyqNf4YZOIExGKNzU22pPN6Jfj+E9ZBJL3h0iy/iRLEaqLGMd7+jWgfjy8Zps6zs0/2dK/S1AAtV3nbjqstJt2GLqjNkxH8SicRM2nlmOXJ+aAYzcjbcumPaLhGubqaUAgV+I0Qy7 qc/6/FxO l2A+9BZrZKAjhQH1AdifK5NAz9aXFOwjW95nnPO/hc4PvWg8fAvuLIeCmd3TrNIzI+mCUL7+sqDtCatwTN/tR+srrPezq98h23GjXNvgljEP7sVRgaS+qXx22+Rvtot+mI8+EOTM32hTa6CZzzXJvh22lTXgV4ZpKC0Ykid77/1XydwroLGgy5IqGhmhJG86lqAMV0Jqnn6myyUxBbzKp2Lzzz+gpZECty9YUU9cs45M0ylyyT1cJaSqdadzjZ4rfXbq2x5kvtusoZpsNQeBU6twqnsoJg8Iagm266lz+wN6AfmMsyfhJCinQrzWqU2PdVG84z/2HsDZK1MY4HQygLwUVTn5EBCT7M7xvoai6WLit/9KJ+vCUnj0ByQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Apr 25, 2023 at 04:46:26PM +0800, Yin Fengwei wrote: > free_transhuge_page() acquires split queue lock then check > whether the THP was added to deferred list or not. > > It's safe to check whether the THP is in deferred list or not. > When code hit free_transhuge_page(), there is no one tries > to update the folio's _deferred_list. > > If folio is not in deferred_list, it's safe to check without > acquiring lock. > > If folio is in deferred_list, the other node in deferred_list > adding/deleteing doesn't impact the return value of > list_epmty(@folio->_deferred_list). Typo. > > Running page_fault1 of will-it-scale + order 2 folio for anonymous > mapping with 96 processes on an Ice Lake 48C/96T test box, we could > see the 61% split_queue_lock contention: > - 71.28% 0.35% page_fault1_pro [kernel.kallsyms] [k] > release_pages > - 70.93% release_pages > - 61.42% free_transhuge_page > + 60.77% _raw_spin_lock_irqsave > > With this patch applied, the split_queue_lock contention is less > than 1%. > > Signed-off-by: Yin Fengwei > Tested-by: Ryan Roberts > --- > mm/huge_memory.c | 19 ++++++++++++++++--- > 1 file changed, 16 insertions(+), 3 deletions(-) > > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > index 032fb0ef9cd1..c620f1f12247 100644 > --- a/mm/huge_memory.c > +++ b/mm/huge_memory.c > @@ -2799,12 +2799,25 @@ void free_transhuge_page(struct page *page) > struct deferred_split *ds_queue = get_deferred_split_queue(folio); > unsigned long flags; > > - spin_lock_irqsave(&ds_queue->split_queue_lock, flags); > - if (!list_empty(&folio->_deferred_list)) { > + /* > + * At this point, there is no one trying to queue the folio > + * to deferred_list. folio->_deferred_list is not possible > + * being updated. > + * > + * If folio is already added to deferred_list, add/delete to/from > + * deferred_list will not impact list_empty(&folio->_deferred_list). > + * It's safe to check list_empty(&folio->_deferred_list) without > + * acquiring the lock. > + * > + * If folio is not in deferred_list, it's safe to check without > + * acquiring the lock. > + */ > + if (data_race(!list_empty(&folio->_deferred_list))) { > + spin_lock_irqsave(&ds_queue->split_queue_lock, flags); Recheck under lock? > ds_queue->split_queue_len--; > list_del(&folio->_deferred_list); > + spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); > } > - spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); > free_compound_page(page); > } > > -- > 2.30.2 > > -- Kiryl Shutsemau / Kirill A. Shutemov