From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6002C64EC7 for ; Wed, 1 Mar 2023 06:10:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B2656B0071; Wed, 1 Mar 2023 01:10:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 262C46B0072; Wed, 1 Mar 2023 01:10:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 129976B0073; Wed, 1 Mar 2023 01:10:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id F2E1C6B0071 for ; Wed, 1 Mar 2023 01:10:09 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C4EA21611A1 for ; Wed, 1 Mar 2023 06:10:09 +0000 (UTC) X-FDA: 80519304138.12.D03462A Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by imf30.hostedemail.com (Postfix) with ESMTP id BB78780013 for ; Wed, 1 Mar 2023 06:10:06 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=K3m8E8g7; spf=pass (imf30.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.31 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677651008; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=N7cUy9TKHzJ+epyI6fERiaaAKyByLQ8z5egufTYEeog=; b=f89LxgsT8vaDIBgo+s8LNU7g9bja74dKrdcZ2HtQcAhJU+Vio1Lrdu2OBxDH6Q02XQXzSC qT7wZi5LN15V++QfWeFGS2RCOJUrkgh4ylvJQXHirizjOcbTsZJN6qSYnQrrmM6wjQ6P4E Jpsat+pBSI6rI5fmKBV0hsZrQHkuPgM= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=K3m8E8g7; spf=pass (imf30.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.31 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677651008; a=rsa-sha256; cv=none; b=7dHNrNitaojR3TVm/5a4S6/kuj3OU0OSKI51Apq1grEMr3R7rnzx5lru3U01NO5zypR07+ EkqCqRZ4ALX6peGFES5qfUD7jC6f16Ksm6LLwo14Dw4GNXWNa7iQOxXyjxYZ3mN7L51oWW E1k4DIciZjcoWTLgb0NYyZg2Fj6SLFg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677651006; x=1709187006; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=WWshl4dK3+R9OwEb435b+D4HXgqFVC207zqrd+7KQQc=; b=K3m8E8g7VTg1wa6LQiQFXcFymD9e05Z0eo7oQD89g8U6n2hcBV1VDIO6 NeGLmG2uOm8/RMLHzMT5G6lPGHMLGcUW+ELOPDeUub1CKfX63qHeDeZC9 8H3pncaS3JQtGxJqykWS6c8ojd7LUq7AWXyCtPflkRk4R59IxAAvtLlPh P1TT5P5hV2HzhE2IzLaeTYnmDo906m3XxJJXZLB0ZqHQYocF2hhlzWORh gkD0AFTMUmPQZqUvvusK9PZhRvJaawPY0KDVuWbEBQlNohHMBbeLUt4I8 GgtpufAVxPmpXJwVBKIPLuGBTnFLKfiLuQN2j0PMMFfqyDHxMytln4sH2 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10635"; a="396897801" X-IronPort-AV: E=Sophos;i="5.98,224,1673942400"; d="scan'208";a="396897801" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Feb 2023 22:10:05 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10635"; a="651878705" X-IronPort-AV: E=Sophos;i="5.98,224,1673942400"; d="scan'208";a="651878705" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Feb 2023 22:10:01 -0800 From: "Huang, Ying" To: Hugh Dickins Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Xu, Pengfei" , Christoph Hellwig , Stefan Roesch , Tejun Heo , Xin Hao , Zi Yan , Yang Shi , Baolin Wang , Matthew Wilcox , Mike Kravetz Subject: Re: [PATCH 3/3] migrate_pages: try migrate in batch asynchronously firstly References: <20230224141145.96814-1-ying.huang@intel.com> <20230224141145.96814-4-ying.huang@intel.com> <87cz5ub5dr.fsf@yhuang6-desk2.ccr.corp.intel.com> <070f71-9af-c29a-30b9-758b5cdf6766@google.com> Date: Wed, 01 Mar 2023 14:08:56 +0800 In-Reply-To: <070f71-9af-c29a-30b9-758b5cdf6766@google.com> (Hugh Dickins's message of "Tue, 28 Feb 2023 13:22:59 -0800 (PST)") Message-ID: <874jr5atqf.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: BB78780013 X-Stat-Signature: u8rj6cotyh5tafj4wz7nr1ctzetjtsr1 X-HE-Tag: 1677651006-802084 X-HE-Meta: U2FsdGVkX1/ofmtYxJrKIReTCLj4mpFvSr/RJWJUrAjf+bQ/3anvdJbtYKNiu6ZmkxI2UmChdkzD/bZ9Fzhc9II3Zmon8qUmguSe0d1VU4FJvFMqlYE6dcJnF7TTiLIsS+nQ1dsxqYNUAIsaA6P69I3kgADFld/c0NVmQzFsE8XNg+Aya2KdyMeCst7jzvMOiLRv3WurvHYDoo4LFbaPh/tivntDtJS1WAh8XHHA6h5KUILaFdHEbvsmejoF/75lCghG4cVDqZNYhDtQJJXEm9omB5Sxi4PCVVqMb5vIxbDsHkm9zFhRCtB8BWe/4z5yGaMqZ7pwRRtfH3+LoX9g+P/aJbzTCgaFtwyGE5ORVG0Ct9QtTEdNWjz8t7ECIjwIQoDfVr7mFxJBwx9mSbmYHnb3kQ6JTHrJYArFZqPP1MKfcTLQuMv8raTiLTcpJLOsLeOzBOtg7xzCVvQYIqhP8roRDPu/boX54DvKwe/ZfU/MCJZ3w5B+KT2jEfUEjGzcoOsEo/+GnQegsd9ATyjHA1GXzmAVnkSfY/mf5SCI/rO1K3wfE8Up3P1sg91SQwoUcDVJ0fzpyxVq572tIy4DbVDEOztlojKXrzgRmkBdSZeVVhE5OscE7U27N6hVYt/EudBvJiXEbd3CeUteBBkmgY8fvJn5RoWQ6+0c3gpuXNUBFghQ9q2If0tsDhQvTTUDU0Eh/i1n1F+2TDTpSpJLxY1uUHKlftjgVBTx+EedZL0YSylYrqfUSGBJfRrGiwhz/S8VeFBXRPCyrQaMBz46h/PxWnUOGdW7r4FH7Bbtk64ly4RDU9HXFhOd5aFBVBwSQ3HCsy2bUAztm5GJekktF8WqQpCdKBtmNHXwM9E3FJCMydL6VIEB1XlFBPrF5DInhHhg81e2s0E4xwDyg0fc2m933FHsPZPJ+p4csbDTCgclTo+GBn/rOt5UGdcK76FOXTpOrgQ3GwODyvOa+8j wnwmJfDA JkhGQMH8ImijEpkN3hrpkkgQaoKLOJFDHgSQt+mROSoeciMsEKp0GmX1dkXQTNKyZOhaCctnISPfBC44CMaBIJAlr76A1pmspg8vobCLNtEtEJJ3TMwmWhmcrUucpHHr/d7oa7eEkhkauWWWQ1sRec2aGgrjWZLFTrmJs8ZGj8cbLmkBI7YlZP3TBxuPY9aMotUt8wUrU8IYUE3w= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hugh Dickins writes: > On Tue, 28 Feb 2023, Huang, Ying wrote: >> Hugh Dickins writes: >> > On Fri, 24 Feb 2023, Huang Ying wrote: >> >> >> >> diff --git a/mm/migrate.c b/mm/migrate.c >> >> index 91198b487e49..c17ce5ee8d92 100644 >> >> --- a/mm/migrate.c >> >> +++ b/mm/migrate.c >> >> @@ -1843,6 +1843,51 @@ static int migrate_pages_batch(struct list_head *from, new_page_t get_new_page, >> >> return rc; >> >> } >> >> >> >> +static int migrate_pages_sync(struct list_head *from, new_page_t get_new_page, >> >> + free_page_t put_new_page, unsigned long private, >> >> + enum migrate_mode mode, int reason, struct list_head *ret_folios, >> >> + struct list_head *split_folios, struct migrate_pages_stats *stats) >> >> +{ >> >> + int rc, nr_failed = 0; >> >> + LIST_HEAD(folios); >> >> + struct migrate_pages_stats astats; >> >> + >> >> + memset(&astats, 0, sizeof(astats)); >> >> + /* Try to migrate in batch with MIGRATE_ASYNC mode firstly */ >> >> + rc = migrate_pages_batch(from, get_new_page, put_new_page, private, MIGRATE_ASYNC, >> >> + reason, &folios, split_folios, &astats, >> >> + NR_MAX_MIGRATE_PAGES_RETRY); >> > >> > I wonder if that and below would better be NR_MAX_MIGRATE_PAGES_RETRY / 2. >> > >> > Though I've never got down to adjusting that number (and it's not a job >> > to be done in this set of patches), those 10 retries sometimes terrify >> > me, from a latency point of view. They can have such different weights: >> > in the unmapped case, 10 retries is okay; but when a pinned page is mapped >> > into 1000 processes, the thought of all that unmapping and TLB flushing >> > and remapping is terrifying. >> > >> > Since you're retrying below, halve both numbers of retries for now? >> >> Yes. These are reasonable concerns. >> >> And in the original implementation, we only wait to lock page and wait >> the writeback to complete if pass > 2. This is kind of trying to >> migrate asynchronously for 3 times before the real synchronous >> migration. So, should we delete the "force" logic (in >> migrate_folio_unmap()), and try to migrate asynchronously for 3 times in >> batch before migrating synchronously for 7 times one by one? > > Oh, that's a good idea (but please don't imagine I've thought it through): > I hadn't realized the way in which your migrate_pages_sync() addition is > kind of duplicating the way that the "force" argument conditions behaviour, > It would be very appealing to delete the "force" argument now if you can. Sure. Will do that in the next version. > But aside from that, you've also made me wonder (again, please remember I > don't have a good picture of the new migrate_pages() sequence in my head) > whether you have already made a *great* strike against my 10 retries > terror. Am I reading it right, that the unmapping is now done on the > first try, and the remove_migration_ptes after the last try (all the > pages involved having remained locked throughout)? Yes. You are right. Now, unmapping and moving are two separate steps, and they are retried separately. After a folio has been unmapped successfully, we will not remap/unmap it 10 times if the folio is pinned so that failed to move (migrate_folio_move()). So the latency caused by retrying is much better now. But I still tend to keep the total retry number as before. Do you agree? Best Regards, Huang, Ying