From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D027C7EE21 for ; Fri, 5 May 2023 01:10:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD8BB6B0078; Thu, 4 May 2023 21:10:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D88D4900002; Thu, 4 May 2023 21:10:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C03836B007D; Thu, 4 May 2023 21:10:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by kanga.kvack.org (Postfix) with ESMTP id 5D7426B0078 for ; Thu, 4 May 2023 21:10:07 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1683249007; x=1714785007; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=JIQzyqYXO61GUSI4EofiLvgL7sgEZJhhWwUiTZp0kTA=; b=nvigg/s9rqHJMN/nT9MLnIBflMJXDbA11ZouYnzpuqctEkG0BPrDGJ/W DCS00/qViuEOTPNarufZXQX3EEhGMXe4Uf3/lRXs75FJIZ4ZRiVTbW0uL Z4P0KZfkzLANnujRfcJC9+/UBmXXJo7ABO+wwyG3c8YkjbcPeo0EQdcG/ z5sLwXEB1ffdQKhxE6O1NGLlXrXMgS8l0v0bygh6dSWrREFNBxGJ9uKvm 3jSj72XyEKdPm2/V/JWdJJAo5pCQX48M4x7OhPb4g8/io6O1ZOD2xVtwY VkdAZ37cyETxL1RP28DHdPQDv3fRpam/7MQQvucG7j5mI0IyV5VCQU2jx g==; X-IronPort-AV: E=McAfee;i="6600,9927,10700"; a="414626309" X-IronPort-AV: E=Sophos;i="5.99,250,1677571200"; d="scan'208";a="414626309" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 May 2023 18:10:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10700"; a="674800066" X-IronPort-AV: E=Sophos;i="5.99,250,1677571200"; d="scan'208";a="674800066" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga006.jf.intel.com with ESMTP; 04 May 2023 18:10:06 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Thu, 4 May 2023 18:10:06 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Thu, 4 May 2023 18:10:06 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Thu, 4 May 2023 18:10:06 -0700 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (104.47.74.48) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Thu, 4 May 2023 18:10:05 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ng690viG0hmxT+3RP9vbwomqjTs3qeho2gzufVvHdO+3d+Q4Oi2RZ5U9dXVlFZuzALl6XjX30cLczOxuCUzvOVNTiPywRfgrKgI57u6byD/ifwM+upDPSus1VGxVl8mtcVKHJOhFnHaohrQT3tLy8fbGkQekEZZzfTd80wFPKprenKrqT8mKTsE+IB1O+d/qbt4eIS4VpumYoO3guHf/KVXCzGQJm1HoWPRZmdWmEL631ehJM9Sm5Fd1zEHf0ntorHMRzpp642808EryCbolUB1kSf6kXddkxXzEVMBYMY2bZ0xQv0+skffOYC3WQzr9zAe954hyrXZkcGc84F3bcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=knxrniUewwEXrFjLNt3Q8mWuyAFx4m7oiUFDzcsvDrU=; b=MqxjyhFf1UNQpXjVG+Au/oiO/NN/M8WaKOt+RNIssK4z4fz8LwIMWQkZgUnf5OElvi9qlwr+4+4Ut2QijQ6KcgFyhXbf7EpYOcJq4Pts1n0fiX+50E2+r4/2CbxZOZ1FsChMCGLAkpDWtOl8OxuNww94wZRTLtyYFFn8VHfCy5ZXkEIcXrcgZTscM8tVQ0fJE912/FJwZyBJklv/LRV259Zp3GyWsy0oQFNd4RkxFFFLKpbVYjwCOHSYJqWrzXyuVy2owiaZk6jCdzj7+CmHdbC+HTHodvYUFb+h3ORSfyUcItezE40gFVDxvkqxohf2DT4FNaS6MDp4obtjLqo6Og== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from CO1PR11MB4820.namprd11.prod.outlook.com (2603:10b6:303:6f::8) by PH8PR11MB6928.namprd11.prod.outlook.com (2603:10b6:510:224::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.22; Fri, 5 May 2023 01:10:04 +0000 Received: from CO1PR11MB4820.namprd11.prod.outlook.com ([fe80::d1a0:743b:4659:b72]) by CO1PR11MB4820.namprd11.prod.outlook.com ([fe80::d1a0:743b:4659:b72%7]) with mapi id 15.20.6363.027; Fri, 5 May 2023 01:10:04 +0000 Message-ID: <21df637d-965b-082e-c45c-0992795c35ed@intel.com> Date: Fri, 5 May 2023 09:09:56 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0 Thunderbird/102.10.1 Subject: Re: [PATCH v3 1/2] THP: avoid lock when check whether THP is in deferred list Content-Language: en-US To: "Huang, Ying" CC: , , , , , References: <20230429082759.1600796-1-fengwei.yin@intel.com> <20230429082759.1600796-2-fengwei.yin@intel.com> <877ctnd2ga.fsf@yhuang6-desk2.ccr.corp.intel.com> From: "Yin, Fengwei" In-Reply-To: <877ctnd2ga.fsf@yhuang6-desk2.ccr.corp.intel.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-ClientProxiedBy: SG2PR02CA0042.apcprd02.prod.outlook.com (2603:1096:3:18::30) To CO1PR11MB4820.namprd11.prod.outlook.com (2603:10b6:303:6f::8) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PR11MB4820:EE_|PH8PR11MB6928:EE_ X-MS-Office365-Filtering-Correlation-Id: 2f37adf5-e17e-40bf-c858-08db4d057522 X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0iedBmJPUJHAJaYTD4/KHHHxkZ8mMv6ltZm5SUuBpUNTioNXwq73xlyXlKm0UQO/cwvQqvb+Ok3WfhBwK9xgjiIvdp8wQsy6I3168T8XJQB+tei3b8qK+gI9Z3aSRQQr55fX4nkVYF+uUZL0XgqGLG359CZh2zNZfDZyFYAICk9i7iAfZGgvqbH1Rh/awKL2dvliHi8vvvqFWtvEbkryYN1MxkCjtQ+q5IlHByI8yap1Wb+xbCFow2WCbbQtHL/4AfVdrCjdGDUkYOhA5G2Wc3Jgk5DYIWTroZVFYkvHvFBG0FagL9DXFXfjzcUL923gGhpPV35jccJ+KRuFcJmc2zM1wkFRVHhUgsssCzyBePBvS866ev2QFiec3tAZ7rvGsQFbEa2RCAxdz//T8OWchxposR/mcet084TBwKErGAMOdkwTLGG2ZD58n+H2TgtXcZLDsquHx2T3VRNryKJcFMlaCpfkNMdw4A3xXZ301PuJ1BR7nlJQMACbEnx5hjUa3dyV+Ma0jdNqfXKbJb5aqEQTxE1MsFTR+LhMqovFbmwFZAZlyoQgZiFuFqm3C6V7cL6JvuBusWtNxQDAaP5bue4cfi8KHb20G83dYONI7q7jmDevUeDNkplTyHXRDoJpq1KO0G7hCalgsxFeJFXrvA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR11MB4820.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(39860400002)(376002)(136003)(366004)(346002)(396003)(451199021)(5660300002)(8936002)(6862004)(6486002)(6666004)(36756003)(31696002)(6636002)(86362001)(316002)(66556008)(41300700001)(66476007)(4326008)(66946007)(82960400001)(8676002)(478600001)(38100700002)(31686004)(2906002)(53546011)(37006003)(6512007)(2616005)(6506007)(26005)(83380400001)(186003)(43740500002)(45980500001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aldZKzBHY0JCK1BSdmRadlZINm00Y3FrTW5NaVdPZ0hLL3dsdzdoTkkxQzh5?= =?utf-8?B?NzkzWVNoN0FzdHprL0xSaFFkbkVYWDZ5eVhiaGlJYzkreDZBT0orVDJUOHl6?= =?utf-8?B?Y2FBZ1E1bFY0SmtxU0I3SnVPMktIckNhY0RzWEhXN2dkanEwam5Qb2FoZ0Fl?= =?utf-8?B?YXFub2N4VEVSU2phcEh6dEcrZDJ1Z3BKWHMrZzVWM2xwM0RUYXcyT3F1RXZo?= =?utf-8?B?ajI0RXhuTktIWCtwa2QzUDZxRHZva05oMXU2ZVRrYW5aQThyc3kvZllTVzd2?= =?utf-8?B?RjZlUVRZblVkVlZHRG1EQlhLSm9nWjlQdU1YN1hHUXR0RnFHR0FYajhoVVB1?= =?utf-8?B?ZHFnT1BheXJncTdWQ2lZTDhQWUh1VThqS2R0VEZJZEZsZjFZMVJOTVZrVkhY?= =?utf-8?B?MkRsblFGK05PQm9rVVVsN1NSeHBoZjhxNVYvZ1h1SGpzOFhGUDA5VEszV0J2?= =?utf-8?B?dzBGalNvWnJrcVNWQjVjVHc3WWNzR3RlSzg5ZjdTOFkyTGpWUTFQbVh6K0hx?= =?utf-8?B?R3VENytsUU9rbmZXMWpzMXFLcnRGb1FIQjJ0VnUySlp3YXpwNkp2TzFwUEdW?= =?utf-8?B?cnJIVmZiN2k5eExIOCt3UzI3clFnVS9Yc1RyMDhQWmQ1Zk82SkJHVlhjQkVh?= =?utf-8?B?T2RDaU8zajIwNytPblo5bzhyeFR1cUxlVmpOZnh5OWthdEhEUFVrZmttMlBZ?= =?utf-8?B?Rlp0clVkSndSTEltWS9GZHlRRitVNzdIY2R0ZDRvZ0lOWTlLbEx5YWNabEw5?= =?utf-8?B?amY0U081NUtCeGdyNENXUFJCTzNQcGV1bFJyb01yRXVPcjl3dG5reFk1OXB0?= =?utf-8?B?NGVRb2FhemxhZ1ZIdkZYZWNFRnZBRTF5NE9qUWMrak1JcFhtRS92MFFLSGln?= =?utf-8?B?QXFUamNlQm52WkdVQm9Sa0tIM2NBdlJEQXZDOHZuMlZUY3ZjdGhrMFBjZXox?= =?utf-8?B?di8xbzZxYlRuRjRSN2ltQ2tldHF4MGFtV3ZjTmtlMGVkMXp1cFpid3RqQjFF?= =?utf-8?B?cmIzQ0V5aDRlSVZ5VXovK0dRZlZ1cWc5SktNa21Ub3AyamVVdjQxZWNGYzY3?= =?utf-8?B?eUtvL01XdTdxaUVmTk51UlVPbWhCRW9MK29SQlExRHduZ3JiQW1DUFB0UDlU?= =?utf-8?B?ZUczNjUzNnZ4YitTK01yUFQ5bjhGRUtmM3pLQjdYSFRjMnlFcGZBeVdMRWZV?= =?utf-8?B?TklYM1UvdVkrRi9yZy90WUJJTkdBVWhzTjc0OUpkRWYwZHlOSE92US9sVUZr?= =?utf-8?B?VVg0aWdNbVpCSk9YTUJpeFQwV1RjTkhyeEwzdkc5Wlhqcm8vRFpQTTFvRTJV?= =?utf-8?B?RllETzAzK2tsclF1UWs3dDRvMWJCMWpQenBZSkFLRUdHbi9yMkpranV4Z1Fh?= =?utf-8?B?OWhDd3JiSGlWbmRmS3hVS3l1bWJHQzhObkZvdTJqRmhvR1JlS3BWT3J2NXB3?= =?utf-8?B?MytQN2RUTHNjazRoTTZVNm9PbVJlcEM0b3U3bXR6eHgyNS9aVTB3cTdFR1NQ?= =?utf-8?B?NjJaQUhjamFjSlc5aTFwbS81N01XNDB1WHcxWklLa2FocTZNd3QwZ3ljWnBy?= =?utf-8?B?R3RlZUgvV1VpemRpeHk4b2Y2VHB2SUZxQXNvNCs4amppajBxclIzMS81ejkz?= =?utf-8?B?UlFVRjN0MWJmN2h1SFVnZU1zK2dQTVJqU0hiVEVJazRkZUNNazRuTU50Tm56?= =?utf-8?B?L2tDT1l5WktNNUVFenljelJGd2kxbHdST1B2MURyL2lPR3lhS05lNG9PdDlL?= =?utf-8?B?bXNUcmRBSk11TXV2M04wN25ac2pyZlN3UCt2SFljUEt5NDY0MllyR0c4dkVj?= =?utf-8?B?Rkl3d0tYc1ovWS9lOGJadW9zSld2TlBuMlFwbnVZM3NoNTQyY29LSXN1VlpL?= =?utf-8?B?RDR4UTR4Nk02K3NBS1MyS2MwbDFMNThKVk15VkxESitNU0ZmU2NtNHNrTmpm?= =?utf-8?B?NE8zaFdoM09tY0N0Q0loZVJpaWppaDROQ2hzSk15WkU5azJZb2M4N3plQnl5?= =?utf-8?B?cVQySytDSEFpaGNjOHNIWlFoTTQwWjZpUzE5TENZUWpKNDMwdWZOU2dOVlNY?= =?utf-8?B?aTRyclVZRXNLdjhkb3ZMYWE5ZEdkNkUrbksyZiticllFQ014UGl6cGV2NHlY?= =?utf-8?B?M0NIMUNveHBIRXpsUVRqem8yS1hlZk1uVUJyREdEYWhSWGFBSjViREs3QkZK?= =?utf-8?B?b0E9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: 2f37adf5-e17e-40bf-c858-08db4d057522 X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB4820.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 May 2023 01:10:03.9715 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BdQB1+MmedtkpEs33UqJ4qfiBqLoXes3ZGztCmQauFAzl7hjk4ojiEW9GO6zUV7gb0AcA0Q4DMBt4+3bgVE2Eg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR11MB6928 X-OriginatorOrg: intel.com X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Ying, On 5/5/2023 8:52 AM, Huang, Ying wrote: > Yin Fengwei writes: > >> free_transhuge_page() acquires split queue lock then check >> whether the THP was added to deferred list or not. It brings >> high deferred queue lock contention. >> >> It's safe to check whether the THP is in deferred list or not >> without holding the deferred queue lock in free_transhuge_page() >> because when code hit free_transhuge_page(), there is no one >> tries to add the folio to _deferred_list. >> >> Running page_fault1 of will-it-scale + order 2 folio for anonymous >> mapping with 96 processes on an Ice Lake 48C/96T test box, we could >> see the 61% split_queue_lock contention: >> - 63.02% 0.01% page_fault1_pro [kernel.kallsyms] [k] free_transhuge_page >> - 63.01% free_transhuge_page >> + 62.91% _raw_spin_lock_irqsave >> >> With this patch applied, the split_queue_lock contention is less >> than 1%. >> >> Signed-off-by: Yin Fengwei > > Thanks! > > Reviewed-by: "Huang, Ying" Thanks a lot for reviewing. Regards Yin, Fengwei > >> --- >> mm/huge_memory.c | 17 ++++++++++++----- >> 1 file changed, 12 insertions(+), 5 deletions(-) >> >> diff --git a/mm/huge_memory.c b/mm/huge_memory.c >> index 032fb0ef9cd1..2a1df2c24c8e 100644 >> --- a/mm/huge_memory.c >> +++ b/mm/huge_memory.c >> @@ -2799,12 +2799,19 @@ void free_transhuge_page(struct page *page) >> struct deferred_split *ds_queue = get_deferred_split_queue(folio); >> unsigned long flags; >> >> - spin_lock_irqsave(&ds_queue->split_queue_lock, flags); >> - if (!list_empty(&folio->_deferred_list)) { >> - ds_queue->split_queue_len--; >> - list_del(&folio->_deferred_list); >> + /* >> + * At this point, there is no one trying to add the folio to >> + * deferred_list. If folio is not in deferred_list, it's safe >> + * to check without acquiring the split_queue_lock. >> + */ >> + if (data_race(!list_empty(&folio->_deferred_list))) { >> + spin_lock_irqsave(&ds_queue->split_queue_lock, flags); >> + if (!list_empty(&folio->_deferred_list)) { >> + ds_queue->split_queue_len--; >> + list_del(&folio->_deferred_list); >> + } >> + spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); >> } >> - spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); >> free_compound_page(page); >> }