From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0832C46467 for ; Mon, 2 Jan 2023 23:54:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E3248E0002; Mon, 2 Jan 2023 18:54:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 493458E0001; Mon, 2 Jan 2023 18:54:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3825E8E0002; Mon, 2 Jan 2023 18:54:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 29E5D8E0001 for ; Mon, 2 Jan 2023 18:54:33 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id EC0CB1A0789 for ; Mon, 2 Jan 2023 23:54:32 +0000 (UTC) X-FDA: 80311515984.10.C31F822 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by imf25.hostedemail.com (Postfix) with ESMTP id 95D58A0006 for ; Mon, 2 Jan 2023 23:54:30 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=c7sZCJ0J; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf25.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.31 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672703671; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=95CQbU+zcOknYECwMv5ASW6ZetRKOqEhOAEFfojbLro=; b=mJWOVlABFUf06kRk2vA6gSReY+w833lZ62qR3NrbicyxLTPfWepcfU1a8Ti/rVXgM02lK1 6QWGNRdOcY4NeWtYiLJTL89K/+I2B1KLEMRtDeBdQJdu1Gd9u81J8uora5emHeJM/C6tFC DM3JG548iOf8aiIwMvoUVy4jIRNZJQI= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=c7sZCJ0J; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf25.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.31 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672703671; a=rsa-sha256; cv=none; b=OuDMrbtlhBO7ql9v0D6uPXEF1kesB7hwed92q+xYgqVlOA2TnOVt/qB3RXeo7tfwxmiPAT nMtFY4K5+rDYqHmOvb/UB4Pg65+4jMrqmE8hs/ewuQTH9bB6TTRUSYRZwjA2jvLKDG4h17 1iRrSMk6Ej3dwW5AMwm81lEPmCXpTGw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1672703670; x=1704239670; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=aR3K3ex1SxuJ6C4KEDB9xBXcz30I39NL7soRBCm7lKo=; b=c7sZCJ0JyB28h4/+dHKK1vC1LsCy7Wy3y4hMZLoFANKzPYhUvvnw4GJJ Uh+Mw/tZBP63Cy5rsHQUfH0uQ8hirds48jLoad+xc1f4ca5b9gtCH4gif B7G+uEKzsZzCF+3hQ/dAzh5W4ygcIXjeoJmKfgUBb0HUXNds9Wo5yhMun ydV137SPKz2hHMlbAcR6brXyGrSyYzZ0CyhW13rJlO3zOawq/LZGPFTor uBf7+/FjourKOdShOPe5j8s/jwSlmc8++rG8sdlB1mcqC9ypkME5us1fJ JwI5VwZ9ob+fJSkNAAnORCSWKa01+aLeMC3iu4s9vjPG7yVKpAUq/8A5Q A==; X-IronPort-AV: E=McAfee;i="6500,9779,10578"; a="383850912" X-IronPort-AV: E=Sophos;i="5.96,295,1665471600"; d="scan'208";a="383850912" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2023 15:54:28 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10578"; a="685197615" X-IronPort-AV: E=Sophos;i="5.96,295,1665471600"; d="scan'208";a="685197615" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2023 15:54:25 -0800 From: "Huang, Ying" To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Zi Yan , Yang Shi , Baolin Wang , Oscar Salvador , Matthew Wilcox , Bharata B Rao , Alistair Popple , haoxin Subject: Re: [PATCH 2/8] migrate_pages: separate hugetlb folios migration References: <20221227002859.27740-1-ying.huang@intel.com> <20221227002859.27740-3-ying.huang@intel.com> <20221228151735.e855bde30c1782bb45b97930@linux-foundation.org> Date: Tue, 03 Jan 2023 07:53:31 +0800 In-Reply-To: <20221228151735.e855bde30c1782bb45b97930@linux-foundation.org> (Andrew Morton's message of "Wed, 28 Dec 2022 15:17:35 -0800") Message-ID: <87a630xzlw.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Queue-Id: 95D58A0006 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: oyh4jqzz5m9gbfy4tb7z8rjwf7m5upc6 X-HE-Tag: 1672703670-637509 X-HE-Meta: U2FsdGVkX18EaRgHBNB/joOf1KYQasXaCOcrndnmIf7jb9psNnk+oh7IIRE0y7nb/33uAChmWOeK/Ci/301HS7uSPfjmKdlfEv5+/6aHBokZDLztv0QfKGVWL9pFYvPP7AuMW8zNo+uPk8IE3NycaEfo3FhdGfBKjIR0U5qGuhswejsxQDqwnSio/+jOuiR77Q25pN8nlGHHQ0pmoY5eLZ2VkJTmBEzJLB2SwvF7xGu5vtz1zo3uJOlcGMYv1uaHV4TuBFNvf95guRe1iFsCCHm+b2f9rBI0YkpD/eRRYcYUrl/T9y99urWPg/KTifVjnOv3ZBcNWNnUqZHL9K+v3TgOpFIwggyazCoolq8CniIDdty9pOWJfiK1iRuT7p4C622SMWT28YMFOLXO5SDx8w54DZLCTgwiFvKzcWwnYC4DDirtgSnJP7fAN+YA+qMfgHpwFMrY7gy1rdLkzpK5n6ObZKmzJHWD8wdwWpEfh2WtD67V8iNOjOE+2MUhSacDKeKbsnM1fHc6FcUlOjpMyEdCCNz2AyfVH8VyqMRug9QdKxz8qVzcRK5/g2AqXx7DecyRku70GVXsCmFtjCABycx9lLRWonuwlydIBgcf1v2XksEardKFb0vjIqH3G+THhggs3Ex11BIT7QmbxV2g1qTB8k4RybN7mT08LyJIanWySJ0+GuzBqFsXyEqOmnITJ4w2qJyG95ULzwjk41DTQTZxZo2X7+OLF1w6fPd59QW9/eihE6+hEi3nAz5kCEZLgx3Tj0rQO7FfbaIDa96RfwMIEqQUc9K6XSdACx3rrgjjQWdHCn6Tq43xay8smryqvFFs1TaYODlVrIl8uaaxO7aTohEXAxd2aDwXSyNUpqrkYg6StqGs2g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Andrew Morton writes: > On Tue, 27 Dec 2022 08:28:53 +0800 Huang Ying wrote: > >> This is a preparation patch to batch the folio unmapping and moving >> for the non-hugetlb folios. Based on that we can batch the TLB >> shootdown during the folio migration and make it possible to use some >> hardware accelerator for the folio copying. >> >> In this patch the hugetlb folios and non-hugetlb folios migration is >> separated in migrate_pages() to make it easy to change the non-hugetlb >> folios migration implementation. >> >> ... >> >> --- a/mm/migrate.c >> +++ b/mm/migrate.c >> @@ -1404,6 +1404,87 @@ struct migrate_pages_stats { >> int nr_thp_split; >> }; >> >> +static int migrate_hugetlbs(struct list_head *from, new_page_t get_new_page, >> + free_page_t put_new_page, unsigned long private, >> + enum migrate_mode mode, int reason, >> + struct migrate_pages_stats *stats, >> + struct list_head *ret_folios) >> +{ >> + int retry = 1; >> + int nr_failed = 0; >> + int nr_retry_pages = 0; >> + int pass = 0; >> + struct folio *folio, *folio2; >> + int rc = 0, nr_pages; >> + >> + for (pass = 0; pass < 10 && retry; pass++) { > > Why 10? This is inherited from the original max pass number from migrate_pages(). Which is introduced in commit 49d2e9cc4544 ("[PATCH] Swap Migration V5: migrate_pages() function"). From the code and commit message, I don't find out why. I guess that we need some magic number anyway. Now, because the magic number is used in 2 places (migrate_pages() and migrate_hugetlbs()), it's better to define it as a constant macro? >> + retry = 0; >> + nr_retry_pages = 0; >> + >> + list_for_each_entry_safe(folio, folio2, from, lru) { >> + if (!folio_test_hugetlb(folio)) >> + continue; >> + >> + nr_pages = folio_nr_pages(folio); >> + >> + cond_resched(); >> + >> + rc = unmap_and_move_huge_page(get_new_page, >> + put_new_page, private, >> + &folio->page, pass > 2, mode, >> + reason, ret_folios); >> + /* >> + * The rules are: >> + * Success: hugetlb folio will be put back >> + * -EAGAIN: stay on the from list >> + * -ENOMEM: stay on the from list >> + * -ENOSYS: stay on the from list >> + * Other errno: put on ret_folios list >> + */ >> + switch(rc) { >> + case -ENOSYS: >> + /* Hugetlb migration is unsupported */ >> + nr_failed++; >> + stats->nr_failed_pages += nr_pages; >> + list_move_tail(&folio->lru, ret_folios); >> + break; >> + case -ENOMEM: >> + /* >> + * When memory is low, don't bother to try to migrate >> + * other folios, just exit. >> + */ >> + nr_failed++; >> + stats->nr_failed_pages += nr_pages; >> + goto out; >> + case -EAGAIN: >> + retry++; >> + nr_retry_pages += nr_pages; >> + break; >> + case MIGRATEPAGE_SUCCESS: >> + stats->nr_succeeded += nr_pages; >> + break; >> + default: >> + /* >> + * Permanent failure (-EBUSY, etc.): >> + * unlike -EAGAIN case, the failed folio is >> + * removed from migration folio list and not >> + * retried in the next outer loop. >> + */ >> + nr_failed++; >> + stats->nr_failed_pages += nr_pages; >> + break; >> + } >> + } >> + } >> +out: >> + nr_failed += retry; >> + stats->nr_failed_pages += nr_retry_pages; >> + if (rc != -ENOMEM) >> + rc = nr_failed; >> + >> + return rc; >> +} > > The interpretation of the return value of this function is somewhat > unobvious. > > I suggest that this function be fully commented. > > Why does a retry contribute to nr_failed. What is the interpretation > of nr_failed. etcetera. Sure. Will do that in the next version. Best Regards, Huang, Ying