From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0306C433FE for ; Tue, 8 Nov 2022 08:25:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 011D48E0001; Tue, 8 Nov 2022 03:25:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F04776B0073; Tue, 8 Nov 2022 03:25:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DCC608E0001; Tue, 8 Nov 2022 03:25:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C9AED6B0071 for ; Tue, 8 Nov 2022 03:25:05 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 79C901410CE for ; Tue, 8 Nov 2022 08:25:05 +0000 (UTC) X-FDA: 80109589770.30.DE3FAC7 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf10.hostedemail.com (Postfix) with ESMTP id 8AFEDC0004 for ; Tue, 8 Nov 2022 08:25:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667895903; x=1699431903; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=XZ203FU2gWgveJJQiR5r1L+9v3x85zvQR5KRdSLBM7U=; b=dgkHOCBsq1lfaLQ5t2hbe+07p7Ic57Uencj93ri55bUgK3BOH1r2B0kY yzjDBuKNKyDA07EAoJzdI8INfL/HZF2Z+niXUETEuwjIJT5kXkIvHgyNW XE5d+oxh0qxJ6LoupcnoB9v86wZA80WYBQIjiKXe3ZctnZXjBLoQzOmtR rYKHeLpYw3dbkfW+GL4zr/MxNDpeEroF53tWxeHQUx3zKJkdycEe21LcA XF3Xf8M1B6Egw34WjaI1ayfZY7FQ5lw//pmOm3v2fqEq5o35F6bl0ui2D 3TgwAcrpKrFPJm+XyxV3bhP7sI8FEP/19Y8rlixMYIAGcZJ2BOdjFSOwd w==; X-IronPort-AV: E=McAfee;i="6500,9779,10524"; a="396935044" X-IronPort-AV: E=Sophos;i="5.96,147,1665471600"; d="scan'208";a="396935044" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2022 00:24:59 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10524"; a="630798092" X-IronPort-AV: E=Sophos;i="5.96,147,1665471600"; d="scan'208";a="630798092" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2022 00:24:57 -0800 From: "Huang, Ying" To: Baolin Wang Cc: , , Andrew Morton , Zi Yan , Yang Shi , Oscar Salvador , Matthew Wilcox Subject: Re: [PATCH 2/2] migrate: convert migrate_pages() to use folios References: <20221104083020.155835-1-ying.huang@intel.com> <20221104083020.155835-3-ying.huang@intel.com> Date: Tue, 08 Nov 2022 16:24:18 +0800 In-Reply-To: (Baolin Wang's message of "Mon, 7 Nov 2022 16:13:54 +0800") Message-ID: <87leol4zwt.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667895904; a=rsa-sha256; cv=none; b=TQE/OllsbXesM9VfgmBbnogII9sSWA4qXR+QQc5tupmSW9He+g4DdOmDvbsXWZxRRUxcde oT2viux8WbO/9zexRvgKTdaJjStMDjyIU0ZPg7bRc8mNZ03BiWg+WRP/dOK6euSDgJ9ajY 30s7UQJ9Xb1uuPBRYy7yQd2h9e0lPeI= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=dgkHOCBs; spf=pass (imf10.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667895904; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1U83dq8B2cEq0pJ5KAdzNDJCk82pPSyFl5YSJGmnYtg=; b=mpo8gl2QZmYqEd485KScYVW8NjLK/DGhM+ILRHItad+C4nmqHO4vYA9Vc5e9W83xobN6fS UTc37ze8HfubWWFo3+HfGpk8N1BuhM01mXWvOzD9BSKoSel51lcVAZN/8ujGAlPaUykfN0 BehFeiCp93A7ajjXOXY6zQw6UE7RLxs= X-Stat-Signature: ri84kmetmx3fhh91mc15cfyb458g4rzw X-Rspamd-Queue-Id: 8AFEDC0004 Authentication-Results: imf10.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=dgkHOCBs; spf=pass (imf10.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1667895903-280891 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Baolin Wang writes: > On 11/4/2022 4:30 PM, Huang Ying wrote: >> Quite straightforward, the page functions are converted to >> corresponding folio functions. Same for comments. >> THP specific code are converted to be large folio. >> Signed-off-by: "Huang, Ying" >> Cc: Andrew Morton >> Cc: Zi Yan >> Cc: Yang Shi >> Cc: Baolin Wang >> Cc: Oscar Salvador >> Cc: Matthew Wilcox >> --- >> mm/migrate.c | 201 +++++++++++++++++++++++++++------------------------ >> 1 file changed, 107 insertions(+), 94 deletions(-) >> diff --git a/mm/migrate.c b/mm/migrate.c >> index f6dd749dd2f8..b41289ef3b65 100644 >> --- a/mm/migrate.c >> +++ b/mm/migrate.c >> @@ -1373,218 +1373,231 @@ static int unmap_and_move_huge_page(new_page_t get_new_page, >> return rc; >> } >> -static inline int try_split_thp(struct page *page, struct >> list_head *split_pages) >> +static inline int try_split_folio(struct folio *folio, struct list_head *split_folios) >> { >> int rc; >> - lock_page(page); >> - rc = split_huge_page_to_list(page, split_pages); >> - unlock_page(page); >> + folio_lock(folio); >> + rc = split_folio_to_list(folio, split_folios); >> + folio_unlock(folio); >> if (!rc) >> - list_move_tail(&page->lru, split_pages); >> + list_move_tail(&folio->lru, split_folios); >> return rc; >> } >> /* >> - * migrate_pages - migrate the pages specified in a list, to the free pages >> + * migrate_pages - migrate the folios specified in a list, to the free folios >> * supplied as the target for the page migration >> * >> - * @from: The list of pages to be migrated. >> - * @get_new_page: The function used to allocate free pages to be used >> - * as the target of the page migration. >> - * @put_new_page: The function used to free target pages if migration >> + * @from: The list of folios to be migrated. >> + * @get_new_page: The function used to allocate free folios to be used >> + * as the target of the folio migration. >> + * @put_new_page: The function used to free target folios if migration >> * fails, or NULL if no special handling is necessary. >> * @private: Private data to be passed on to get_new_page() >> * @mode: The migration mode that specifies the constraints for >> - * page migration, if any. >> - * @reason: The reason for page migration. >> - * @ret_succeeded: Set to the number of normal pages migrated successfully if >> + * folio migration, if any. >> + * @reason: The reason for folio migration. >> + * @ret_succeeded: Set to the number of folios migrated successfully if >> * the caller passes a non-NULL pointer. >> * >> - * The function returns after 10 attempts or if no pages are movable any more >> - * because the list has become empty or no retryable pages exist any more. >> - * It is caller's responsibility to call putback_movable_pages() to return pages >> + * The function returns after 10 attempts or if no folios are movable any more >> + * because the list has become empty or no retryable folios exist any more. >> + * It is caller's responsibility to call putback_movable_pages() to return folios >> * to the LRU or free list only if ret != 0. >> * >> - * Returns the number of {normal page, THP, hugetlb} that were not migrated, or >> - * an error code. The number of THP splits will be considered as the number of >> - * non-migrated THP, no matter how many subpages of the THP are migrated successfully. >> + * Returns the number of {normal folio, large folio, hugetlb} that were not >> + * migrated, or an error code. The number of large folio splits will be >> + * considered as the number of non-migrated large folio, no matter how many >> + * split folios of the large folio are migrated successfully. >> */ >> int migrate_pages(struct list_head *from, new_page_t get_new_page, >> free_page_t put_new_page, unsigned long private, >> enum migrate_mode mode, int reason, unsigned int *ret_succeeded) >> { >> int retry = 1; >> + int large_retry = 1; >> int thp_retry = 1; >> int nr_failed = 0; >> int nr_failed_pages = 0; >> int nr_retry_pages = 0; >> int nr_succeeded = 0; >> int nr_thp_succeeded = 0; >> + int nr_large_failed = 0; >> int nr_thp_failed = 0; >> int nr_thp_split = 0; >> int pass = 0; >> + bool is_large = false; >> bool is_thp = false; >> - struct page *page; >> - struct page *page2; >> - int rc, nr_subpages; >> - LIST_HEAD(ret_pages); >> - LIST_HEAD(thp_split_pages); >> + struct folio *folio, *folio2; >> + int rc, nr_pages; >> + LIST_HEAD(ret_folios); >> + LIST_HEAD(split_folios); >> bool nosplit = (reason == MR_NUMA_MISPLACED); >> - bool no_subpage_counting = false; >> + bool no_split_folio_counting = false; >> trace_mm_migrate_pages_start(mode, reason); >> -thp_subpage_migration: >> - for (pass = 0; pass < 10 && (retry || thp_retry); pass++) { >> +split_folio_migration: >> + for (pass = 0; pass < 10 && (retry || large_retry); pass++) { >> retry = 0; >> + large_retry = 0; >> thp_retry = 0; >> nr_retry_pages = 0; >> - list_for_each_entry_safe(page, page2, from, lru) { >> + list_for_each_entry_safe(folio, folio2, from, lru) { >> /* >> - * THP statistics is based on the source huge page. >> - * Capture required information that might get lost >> - * during migration. >> + * large folio statistics is based on the source large > > Nit: s/large/Large Thanks. Will change this. >> + * folio. Capture required information that might get >> + * lost during migration. >> */ >> - is_thp = PageTransHuge(page) && !PageHuge(page); >> - nr_subpages = compound_nr(page); >> + is_large = folio_test_large(folio) && !folio_test_hugetlb(folio); >> + is_thp = is_large && folio_test_pmd_mappable(folio); >> + nr_pages = folio_nr_pages(folio); >> cond_resched(); >> - if (PageHuge(page)) >> + if (folio_test_hugetlb(folio)) >> rc = unmap_and_move_huge_page(get_new_page, >> - put_new_page, private, page, >> - pass > 2, mode, reason, >> - &ret_pages); >> + put_new_page, private, >> + &folio->page, pass > 2, mode, >> + reason, >> + &ret_folios); >> else >> rc = unmap_and_move(get_new_page, put_new_page, >> - private, page_folio(page), pass > 2, mode, >> - reason, &ret_pages); >> + private, folio, pass > 2, mode, >> + reason, &ret_folios); >> /* >> * The rules are: >> - * Success: non hugetlb page will be freed, hugetlb >> - * page will be put back >> + * Success: non hugetlb folio will be freed, hugetlb >> + * folio will be put back >> * -EAGAIN: stay on the from list >> * -ENOMEM: stay on the from list >> * -ENOSYS: stay on the from list >> - * Other errno: put on ret_pages list then splice to >> + * Other errno: put on ret_folios list then splice to >> * from list >> */ >> switch(rc) { >> /* >> - * THP migration might be unsupported or the >> - * allocation could've failed so we should >> - * retry on the same page with the THP split >> - * to base pages. >> + * Large folio migration might be unsupported or >> + * the allocation could've failed so we should retry >> + * on the same folio with the large folio split >> + * to normal folios. >> * >> - * Sub-pages are put in thp_split_pages, and >> + * Split folios are put in split_folios, and >> * we will migrate them after the rest of the >> * list is processed. >> */ >> case -ENOSYS: >> - /* THP migration is unsupported */ >> - if (is_thp) { >> - nr_thp_failed++; >> - if (!try_split_thp(page, &thp_split_pages)) { >> - nr_thp_split++; >> + /* Large folio migration is unsupported */ >> + if (is_large) { >> + nr_large_failed++; >> + nr_thp_failed += is_thp; >> + if (!try_split_folio(folio, &split_folios)) { >> + nr_thp_split += is_thp; >> break; >> } >> /* Hugetlb migration is unsupported */ >> - } else if (!no_subpage_counting) { >> + } else if (!no_split_folio_counting) { >> nr_failed++; >> } >> - nr_failed_pages += nr_subpages; >> - list_move_tail(&page->lru, &ret_pages); >> + nr_failed_pages += nr_pages; >> + list_move_tail(&folio->lru, &ret_folios); >> break; >> case -ENOMEM: >> /* >> * When memory is low, don't bother to try to migrate >> - * other pages, just exit. >> + * other folios, just exit. >> */ >> - if (is_thp) { >> - nr_thp_failed++; >> - /* THP NUMA faulting doesn't split THP to retry. */ >> - if (!nosplit && !try_split_thp(page, &thp_split_pages)) { >> - nr_thp_split++; >> + if (is_large) { >> + nr_large_failed++; >> + nr_thp_failed += is_thp; >> + /* Large folio NUMA faulting doesn't split to retry. */ >> + if (!nosplit && !try_split_folio(folio, &split_folios)) { > > I am not sure if need to add a is_thp validation before calling > try_split_folio()? IIUC try_split_folio() can deal with large folio with arbitrary order now. > BTW, you should rebase on the mm-unstable branch, since I've added a > retry for THP split. Yes. Andrew remind me too. Best Regards, Huang, Ying >> + nr_thp_split += is_thp; >> break; >> } >> - } else if (!no_subpage_counting) { >> + } else if (!no_split_folio_counting) { >> nr_failed++; >> } >> - nr_failed_pages += nr_subpages + >> nr_retry_pages; >> + nr_failed_pages += nr_pages + nr_retry_pages; >> /* >> - * There might be some subpages of fail-to-migrate THPs >> - * left in thp_split_pages list. Move them back to migration >> + * There might be some split folios of fail-to-migrate large >> + * folios left in split_folios list. Move them back to migration >> * list so that they could be put back to the right list by >> - * the caller otherwise the page refcnt will be leaked. >> + * the caller otherwise the folio refcnt will be leaked. >> */ >> - list_splice_init(&thp_split_pages, from); >> + list_splice_init(&split_folios, from); >> /* nr_failed isn't updated for not used */ >> + nr_large_failed += large_retry; >> nr_thp_failed += thp_retry; >> goto out; >> case -EAGAIN: >> - if (is_thp) >> - thp_retry++; >> - else if (!no_subpage_counting) >> + if (is_large) { >> + large_retry++; >> + thp_retry += is_thp; >> + } else if (!no_split_folio_counting) { >> retry++; >> - nr_retry_pages += nr_subpages; >> + } >> + nr_retry_pages += nr_pages; >> break; >> case MIGRATEPAGE_SUCCESS: >> - nr_succeeded += nr_subpages; >> - if (is_thp) >> - nr_thp_succeeded++; >> + nr_succeeded += nr_pages; >> + nr_thp_succeeded += is_thp; >> break; >> default: >> /* >> * Permanent failure (-EBUSY, etc.): >> - * unlike -EAGAIN case, the failed page is >> - * removed from migration page list and not >> + * unlike -EAGAIN case, the failed folio is >> + * removed from migration folio list and not >> * retried in the next outer loop. >> */ >> - if (is_thp) >> - nr_thp_failed++; >> - else if (!no_subpage_counting) >> + if (is_large) { >> + nr_large_failed++; >> + nr_thp_failed += is_thp; >> + } else if (!no_split_folio_counting) { >> nr_failed++; >> + } >> - nr_failed_pages += nr_subpages; >> + nr_failed_pages += nr_pages; >> break; >> } >> } >> } >> nr_failed += retry; >> + nr_large_failed += large_retry; >> nr_thp_failed += thp_retry; >> nr_failed_pages += nr_retry_pages; >> /* >> - * Try to migrate subpages of fail-to-migrate THPs, no nr_failed >> - * counting in this round, since all subpages of a THP is counted >> - * as 1 failure in the first round. >> + * Try to migrate split folios of fail-to-migrate large folios, no >> + * nr_failed counting in this round, since all split folios of a >> + * large folio is counted as 1 failure in the first round. >> */ >> - if (!list_empty(&thp_split_pages)) { >> + if (!list_empty(&split_folios)) { >> /* >> - * Move non-migrated pages (after 10 retries) to ret_pages >> + * Move non-migrated folios (after 10 retries) to ret_folios >> * to avoid migrating them again. >> */ >> - list_splice_init(from, &ret_pages); >> - list_splice_init(&thp_split_pages, from); >> - no_subpage_counting = true; >> + list_splice_init(from, &ret_folios); >> + list_splice_init(&split_folios, from); >> + no_split_folio_counting = true; >> retry = 1; >> - goto thp_subpage_migration; >> + goto split_folio_migration; >> } >> - rc = nr_failed + nr_thp_failed; >> + rc = nr_failed + nr_large_failed; >> out: >> /* >> - * Put the permanent failure page back to migration list, they >> + * Put the permanent failure folio back to migration list, they >> * will be put back to the right list by the caller. >> */ >> - list_splice(&ret_pages, from); >> + list_splice(&ret_folios, from); >> /* >> - * Return 0 in case all subpages of fail-to-migrate THPs are >> - * migrated successfully. >> + * Return 0 in case all split folios of fail-to-migrate large folios >> + * are migrated successfully. >> */ >> if (list_empty(from)) >> rc = 0;