From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81A64C4332F for ; Wed, 28 Dec 2022 23:17:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E08C68E0002; Wed, 28 Dec 2022 18:17:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DB9028E0001; Wed, 28 Dec 2022 18:17:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C816A8E0002; Wed, 28 Dec 2022 18:17:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id BB8B58E0001 for ; Wed, 28 Dec 2022 18:17:40 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 9AFDE1A0D85 for ; Wed, 28 Dec 2022 23:17:40 +0000 (UTC) X-FDA: 80293279080.16.6688740 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf24.hostedemail.com (Postfix) with ESMTP id EB579180009 for ; Wed, 28 Dec 2022 23:17:38 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Eq9n3BwV; spf=pass (imf24.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672269459; a=rsa-sha256; cv=none; b=JIE8CiP9eVkxoQAyHqKojksGR7qem2Jrgy9JllxS5PAMt1Jq6p6k9MPgQtBiu/aKnWhJT8 IKA6NiyOJsskypKYV3Da+GHFmaRmqo5W8KnNWwTUtgJj++vdiEhzJbJOOGBtxmKypAKRH/ kWRQGK2Ex1mV+uvnmXP8Ve4VCdR/jUg= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Eq9n3BwV; spf=pass (imf24.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672269459; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=raUhdUuM59ezc3dod3/lMT8e1Tv5xKqAnrr3SIxD6Ic=; b=G+AG+ObVV+mmFqBayAqRqPs7ANLbqk5/rbQKM+Xfoh2+vHtD4ZIWGTjOQljzvFpZrNwmoL v1iOqc2xho7G5cNazi00AmF3Z54doJYA+qycDSlaZsliNwMF46LRJemUFnHusG07gQ5hwF wB83UIYnV3tlTE/ZjDoSDGHUWVEIM3w= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E634C61644; Wed, 28 Dec 2022 23:17:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CE2FFC433EF; Wed, 28 Dec 2022 23:17:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1672269457; bh=w3Rr9ivFjnV3esspD85qFy2NhTPDUghMYW5Ck9kdk10=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Eq9n3BwVPz9gSgeeWeqlP+VVTCilTbLEiOWa7G9/tNTYlilXjazrIGwcV5D0NwaWF KVLQcC4Eq5BNimUIMLMOi9rdEPTAUMvytFEBUaJR+pZOLA9IiLukXjRcFDb0nyqBFD 19oTrX8kCxgCZciOFh3YUtD70hZF0q8v6Wp4MMH0= Date: Wed, 28 Dec 2022 15:17:35 -0800 From: Andrew Morton To: Huang Ying Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Zi Yan , Yang Shi , Baolin Wang , Oscar Salvador , Matthew Wilcox , Bharata B Rao , Alistair Popple , haoxin Subject: Re: [PATCH 2/8] migrate_pages: separate hugetlb folios migration Message-Id: <20221228151735.e855bde30c1782bb45b97930@linux-foundation.org> In-Reply-To: <20221227002859.27740-3-ying.huang@intel.com> References: <20221227002859.27740-1-ying.huang@intel.com> <20221227002859.27740-3-ying.huang@intel.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: EB579180009 X-Stat-Signature: btmyqh8kxqoadij1feuhxi6pn78yx41q X-HE-Tag: 1672269458-668795 X-HE-Meta: U2FsdGVkX1/xR+rkRjRI2N8JYcTQ7Jgk0MwwzWmlWKArVHZXZz77ml8Z2XXJGAUkTtTDe2pVsQ6mUCgC0dBzLRvwokN4ZrYbk3P2V+3CE7lQJL+xFvcb7vUPotNTHUn7yd4ybwU15xWaMpS0TVp1vGqBC39ZK9xF4WKqlFvPku+lNE8k8Iv8fAGe4Wl6SVYoHllNqRX6Sy7VAIQesRFDY1Zr57HrQMNQjKWwxpaX43Ps+5pjqowMK/QT56Eyl/IqrYbVWWnwuCmGTI16PbEPdLXrmWu6WF38ivjpESjhEyyv+jFqAg5thoB5iWPN4XgKKC0+43uO4Sm0ugnJPyRawSJuO3GEwFXOsXRcqqIo1zCD6MyBqyjSj1P6mIXoyvZpT7jwmvu2lAJ/IrlHZIWzkjNnjvKVyeyoYposMm/Ts4oz1Jz9IHGWYz1C/S5fhkCnhybE+9tddqZy+BnecvvM04DEVv1zIpwPmGPgyLq4BuBiK29scPeM2+GN/m6GdxOeUBUEQuhv176ERYf0vx+JOl2XSAW/ZhdiqgVZnDnRT8ga6ijET56q4SnBITS2+QLpZsf1aFc8W93lcUcGLDWeIvxGZfb6XkrKk5D9dCmNb75xZvmxXhi0Y1mGS3iAtZY0BvEb0sdZPqPfpXjfcCd8ysLmPyouCs5tCuo897zMwv6ci4+Gs2Ihs7BO6dcQXBOcA4becGnAzQfzYJXU9yiY4g7L6+mY5EPppaS/QsCH0ANEODIX+iqz/196G1kjD2Rb4bDxvqE44YrFSOWaf0jgSKme421q/t6klcpXDJUFv9yWy53nJ9b1O+//y2JVpvjbruscqFKybMMFjVVLgn9muoAOfjjxTkLQqdAfsOAxvuiRTaPhKzKDZM2qRagpUlA00vQ+Tq8IoEMmlftRhYIXFG0W/QU9HIGkPYjHc3+jT2HPyupdna5mxEHfY+t0K8cdDoa80BA28uppXLLdZlj JCwUf22R ktrkl/xNpWiNI/g//lMdfkqDfK6y8eD/BIzjus0nCgKYGSNEkIJVvekZs3yxv4L9gKtFkgES/dN7GUwg8iviwn4Xcq9l2elnuGQpwDJHt4QRboqmj/Oq8BwQK7A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, 27 Dec 2022 08:28:53 +0800 Huang Ying wrote: > This is a preparation patch to batch the folio unmapping and moving > for the non-hugetlb folios. Based on that we can batch the TLB > shootdown during the folio migration and make it possible to use some > hardware accelerator for the folio copying. > > In this patch the hugetlb folios and non-hugetlb folios migration is > separated in migrate_pages() to make it easy to change the non-hugetlb > folios migration implementation. > > ... > > --- a/mm/migrate.c > +++ b/mm/migrate.c > @@ -1404,6 +1404,87 @@ struct migrate_pages_stats { > int nr_thp_split; > }; > > +static int migrate_hugetlbs(struct list_head *from, new_page_t get_new_page, > + free_page_t put_new_page, unsigned long private, > + enum migrate_mode mode, int reason, > + struct migrate_pages_stats *stats, > + struct list_head *ret_folios) > +{ > + int retry = 1; > + int nr_failed = 0; > + int nr_retry_pages = 0; > + int pass = 0; > + struct folio *folio, *folio2; > + int rc = 0, nr_pages; > + > + for (pass = 0; pass < 10 && retry; pass++) { Why 10? > + retry = 0; > + nr_retry_pages = 0; > + > + list_for_each_entry_safe(folio, folio2, from, lru) { > + if (!folio_test_hugetlb(folio)) > + continue; > + > + nr_pages = folio_nr_pages(folio); > + > + cond_resched(); > + > + rc = unmap_and_move_huge_page(get_new_page, > + put_new_page, private, > + &folio->page, pass > 2, mode, > + reason, ret_folios); > + /* > + * The rules are: > + * Success: hugetlb folio will be put back > + * -EAGAIN: stay on the from list > + * -ENOMEM: stay on the from list > + * -ENOSYS: stay on the from list > + * Other errno: put on ret_folios list > + */ > + switch(rc) { > + case -ENOSYS: > + /* Hugetlb migration is unsupported */ > + nr_failed++; > + stats->nr_failed_pages += nr_pages; > + list_move_tail(&folio->lru, ret_folios); > + break; > + case -ENOMEM: > + /* > + * When memory is low, don't bother to try to migrate > + * other folios, just exit. > + */ > + nr_failed++; > + stats->nr_failed_pages += nr_pages; > + goto out; > + case -EAGAIN: > + retry++; > + nr_retry_pages += nr_pages; > + break; > + case MIGRATEPAGE_SUCCESS: > + stats->nr_succeeded += nr_pages; > + break; > + default: > + /* > + * Permanent failure (-EBUSY, etc.): > + * unlike -EAGAIN case, the failed folio is > + * removed from migration folio list and not > + * retried in the next outer loop. > + */ > + nr_failed++; > + stats->nr_failed_pages += nr_pages; > + break; > + } > + } > + } > +out: > + nr_failed += retry; > + stats->nr_failed_pages += nr_retry_pages; > + if (rc != -ENOMEM) > + rc = nr_failed; > + > + return rc; > +} The interpretation of the return value of this function is somewhat unobvious. I suggest that this function be fully commented. Why does a retry contribute to nr_failed. What is the interpretation of nr_failed. etcetera.