From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85B5FC64EC7 for ; Wed, 1 Mar 2023 11:03:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97CB86B0071; Wed, 1 Mar 2023 06:03:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 92D066B0072; Wed, 1 Mar 2023 06:03:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F4A56B0073; Wed, 1 Mar 2023 06:03:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 717106B0071 for ; Wed, 1 Mar 2023 06:03:22 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 413CC1C6E98 for ; Wed, 1 Mar 2023 11:03:22 +0000 (UTC) X-FDA: 80520043044.22.7591FB5 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by imf14.hostedemail.com (Postfix) with ESMTP id 1393E100019 for ; Wed, 1 Mar 2023 11:03:18 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=alibaba.com; spf=pass (imf14.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677668600; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=325nPCwmbW3i4u+T+dDi/2bYw/5pVsyWh6JeGsZyvm0=; b=TBFsel5+TzUItT3Kl0zkzfJ7y67HIXyp2PFZC9bV2ac8wRucyjoIr4T2XaVI5AITxhC1Xu weXIKXh7r0xg9D+HG2ZsPiCHT2yXQ/aTi9+azhjLhuEZCoJ8xjCYdxgnezgH6nfBvyW2Ta hdoBGJ+RsN655A0wJ2hI3iF1UP4PR3Q= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=alibaba.com; spf=pass (imf14.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677668600; a=rsa-sha256; cv=none; b=F+LisrwURYX+5IFdDGtrAJMX1GY7he/3sifwNqXlpnIU57nAGj+rFuLVBREuQNyHAhEhLP K+Z3/nvqFSSGqrt8aViNBUzDs9Uu6P3wZ3ZwF5BLI++47L6rq5xp5Nv8R6Gj3xs+G+8VET s3WnmN2+G+1VlNBPpv8bmfLsUHEHfME= X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R191e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045170;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=14;SR=0;TI=SMTPD_---0Vct2H0z_1677668593; Received: from 30.97.48.59(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0Vct2H0z_1677668593) by smtp.aliyun-inc.com; Wed, 01 Mar 2023 19:03:14 +0800 Message-ID: Date: Wed, 1 Mar 2023 19:03:20 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.7.2 Subject: Re: [PATCH 3/3] migrate_pages: try migrate in batch asynchronously firstly To: "Huang, Ying" Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Hugh Dickins , "Xu, Pengfei" , Christoph Hellwig , Stefan Roesch , Tejun Heo , Xin Hao , Zi Yan , Yang Shi , Matthew Wilcox , Mike Kravetz References: <20230224141145.96814-1-ying.huang@intel.com> <20230224141145.96814-4-ying.huang@intel.com> <87zg8x9epg.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Baolin Wang In-Reply-To: <87zg8x9epg.fsf@yhuang6-desk2.ccr.corp.intel.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 1393E100019 X-Stat-Signature: e6sfq91o3nuwjyufpbphhexi98sbcanm X-Rspam-User: X-HE-Tag: 1677668598-961477 X-HE-Meta: U2FsdGVkX1+DU0Dov97kwOc7TTnelj+/JiOUZzgxkYoWIz/2dU5nDZydfamKRssbPlG3eDoSK8Txq7cQrKPDvhlJlsjRjHCOzYUU3tZSLkmnKUXMNZiWCi0L8mseT1TsY0C1b7Amfb+tWahJFp47zJCiFbF4WoHv/QwsQqvtPcpoBHwb/eSj6ph8uj6yyPRImZmb5QyXDgJ0onbCALIRVAfnfUOYXgQtN8jMG/I7iq0+JpMsEIVLzwalWCFV0UB2ICSwgUSSOcarhkFDgTFfG9OhR8w7FAR+Dm8a1/QoQIp81CuUDdr7wN65gUcgiK/9TZPpjRXlStj7no9QUCcrhfnzEdtGWtkh1ZjKt7NC/sN+GF9e7yODEg0W5QAcFKkTjOp0X/UAzC2hgDxixshj3ZTfAm+iWyXBQHGsSXy0RYWChOyV6iELNuAswDXfn82ZajeVs2VVa/tCU0JV4+OUl0voE+5TssoipVrsUdp/D0/wXpvuDeV9qiA7UprZ6riBXLaxq0g0nJ5g1foR9RtaDw/C22d6rNQgSPL6UZf2rVKyVyDuMAdAF+HloBZYYeqApj+SzlbqgIKtU42uEq1E+oFelD1fE7X7Qn+0r8Am9dFiEwl4mXsyMTTDPjCdXxgd6DVm0ttlSGeEcNjiiMrs1BQcNxo3Jg8lbHRfXt7RH2PWwiMafGK8lVHLOeui2EiQk+YHaJ39QadPbPYPCWKbY7LrTclTvcWJUKO6KGYCl9RRwtSC17sp5khDghLB5o/oxJMwgws1eIEiDyKW0ihPdfv+/qUNM5TvZAY4ic34PdmDKhnWEYOL+UyzV9xeBtg1hE4NJMG2jjHNQ+ikGjADAmkjGaxXMmYBNUDqWYZUEkwfVzPO5zRe07M0uWgo1kVq1CHbObODsC5fy3c7XuGALEYLuRMy0s32BbclLxvZFikTBdpjngV4+aj4CxK62Pyh7PdQ73IIcTRFeTBi+QX I4MgCYtO u8jXJWazw0YfRzYlQgcUzQl1O9ahpMSQF4lMPhwMujjdIOcO0xZkMrWwH8G/ylPEuJRbCwa1BLVT8DEvBIWdoe/tTWdMK4I+OV5OEdWoi+MKqL+jPySYBrjf+0khGmG5ZSv37xpmwDbdf3I0hb80dfEaLK7W3mVYtGc3BIb5Gv8SQa18YWiXLxO546YkMZV4Xl8HzfplB6Mzrd+zgKW0JWOKrOBxbDQ8HT3dsts+fEgCNW2rCbuMiVB52To5vjIoi9bm62QPv9RatKfTEbyCXbUUkQ0W4v2X3SH3Y4iibxuAUM/JZBS+aW/im4x6vVlSR39jTRZV90KOGfKi1p8SChUsNhxudEkgAKGTtiqBOPiQxAKI1MVCALcSV2lVmRvYT7PVvEh6MoOoqbPwCjSU8Ax0DC4B07m+WCdxf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 3/1/2023 2:18 PM, Huang, Ying wrote: > Baolin Wang writes: > >> On 2/24/2023 10:11 PM, Huang Ying wrote: >>> When we have locked more than one folios, we cannot wait the lock or >>> bit (e.g., page lock, buffer head lock, writeback bit) synchronously. >>> Otherwise deadlock may be triggered. This make it hard to batch the >>> synchronous migration directly. >>> This patch re-enables batching synchronous migration via trying to >>> migrate in batch asynchronously firstly. And any folios that are >>> failed to be migrated asynchronously will be migrated synchronously >>> one by one. >>> Test shows that this can restore the TLB flushing batching >>> performance >>> for synchronous migration effectively. >>> Signed-off-by: "Huang, Ying" >>> Cc: Hugh Dickins >>> Cc: "Xu, Pengfei" >>> Cc: Christoph Hellwig >>> Cc: Stefan Roesch >>> Cc: Tejun Heo >>> Cc: Xin Hao >>> Cc: Zi Yan >>> Cc: Yang Shi >>> Cc: Baolin Wang >>> Cc: Matthew Wilcox >>> Cc: Mike Kravetz >>> --- >>> mm/migrate.c | 65 ++++++++++++++++++++++++++++++++++++++++++++-------- >>> 1 file changed, 55 insertions(+), 10 deletions(-) >>> diff --git a/mm/migrate.c b/mm/migrate.c >>> index 91198b487e49..c17ce5ee8d92 100644 >>> --- a/mm/migrate.c >>> +++ b/mm/migrate.c >>> @@ -1843,6 +1843,51 @@ static int migrate_pages_batch(struct list_head *from, new_page_t get_new_page, >>> return rc; >>> } >>> +static int migrate_pages_sync(struct list_head *from, new_page_t >>> get_new_page, >>> + free_page_t put_new_page, unsigned long private, >>> + enum migrate_mode mode, int reason, struct list_head *ret_folios, >>> + struct list_head *split_folios, struct migrate_pages_stats *stats) >>> +{ >>> + int rc, nr_failed = 0; >>> + LIST_HEAD(folios); >>> + struct migrate_pages_stats astats; >>> + >>> + memset(&astats, 0, sizeof(astats)); >>> + /* Try to migrate in batch with MIGRATE_ASYNC mode firstly */ >>> + rc = migrate_pages_batch(from, get_new_page, put_new_page, private, MIGRATE_ASYNC, >>> + reason, &folios, split_folios, &astats, >>> + NR_MAX_MIGRATE_PAGES_RETRY); >>> + stats->nr_succeeded += astats.nr_succeeded; >>> + stats->nr_thp_succeeded += astats.nr_thp_succeeded; >>> + stats->nr_thp_split += astats.nr_thp_split; >>> + if (rc < 0) { >>> + stats->nr_failed_pages += astats.nr_failed_pages; >>> + stats->nr_thp_failed += astats.nr_thp_failed; >>> + list_splice_tail(&folios, ret_folios); >>> + return rc; >>> + } >>> + stats->nr_thp_failed += astats.nr_thp_split; >>> + nr_failed += astats.nr_thp_split; >>> + /* >>> + * Fall back to migrate all failed folios one by one synchronously. All >>> + * failed folios except split THPs will be retried, so their failure >>> + * isn't counted >>> + */ >>> + list_splice_tail_init(&folios, from); >>> + while (!list_empty(from)) { >>> + list_move(from->next, &folios); >>> + rc = migrate_pages_batch(&folios, get_new_page, put_new_page, >>> + private, mode, reason, ret_folios, >>> + split_folios, stats, NR_MAX_MIGRATE_PAGES_RETRY); >>> + list_splice_tail_init(&folios, ret_folios); >>> + if (rc < 0) >>> + return rc; >>> + nr_failed += rc; >>> + } >>> + >>> + return nr_failed; >>> +} >>> + >>> /* >>> * migrate_pages - migrate the folios specified in a list, to the free folios >>> * supplied as the target for the page migration >>> @@ -1874,7 +1919,7 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page, >>> enum migrate_mode mode, int reason, unsigned int *ret_succeeded) >>> { >>> int rc, rc_gather; >>> - int nr_pages, batch; >>> + int nr_pages; >>> struct folio *folio, *folio2; >>> LIST_HEAD(folios); >>> LIST_HEAD(ret_folios); >>> @@ -1890,10 +1935,6 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page, >>> if (rc_gather < 0) >>> goto out; >>> - if (mode == MIGRATE_ASYNC) >>> - batch = NR_MAX_BATCHED_MIGRATION; >>> - else >>> - batch = 1; >>> again: >>> nr_pages = 0; >>> list_for_each_entry_safe(folio, folio2, from, lru) { >>> @@ -1904,16 +1945,20 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page, >>> } >>> nr_pages += folio_nr_pages(folio); >>> - if (nr_pages >= batch) >>> + if (nr_pages >= NR_MAX_BATCHED_MIGRATION) >>> break; >>> } >>> - if (nr_pages >= batch) >>> + if (nr_pages >= NR_MAX_BATCHED_MIGRATION) >>> list_cut_before(&folios, from, &folio2->lru); >>> else >>> list_splice_init(from, &folios); >>> - rc = migrate_pages_batch(&folios, get_new_page, put_new_page, private, >>> - mode, reason, &ret_folios, &split_folios, &stats, >>> - NR_MAX_MIGRATE_PAGES_RETRY); >>> + if (mode == MIGRATE_ASYNC) >>> + rc = migrate_pages_batch(&folios, get_new_page, put_new_page, private, >>> + mode, reason, &ret_folios, &split_folios, &stats, >>> + NR_MAX_MIGRATE_PAGES_RETRY); >>> + else >>> + rc = migrate_pages_sync(&folios, get_new_page, put_new_page, private, >>> + mode, reason, &ret_folios, &split_folios, &stats); >> >> For split folios, it seems also reasonable to use migrate_pages_sync() >> instead of always using fixed MIGRATE_ASYNC mode? > > For split folios, we only try to migrate them with minimal effort. > Previously, we decrease the retry number from 10 to 1. Now, I think > that it's reasonable to change the migration mode to MIGRATE_ASYNC to > reduce latency. They have been counted as failure anyway. Sounds reasonable. Thanks for explanation. Please feel free to add: Reviewed-by: Baolin Wang