From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 370E0C3DA6F for ; Thu, 24 Aug 2023 03:13:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 917B0280055; Wed, 23 Aug 2023 23:13:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C7EB8E0011; Wed, 23 Aug 2023 23:13:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B658280055; Wed, 23 Aug 2023 23:13:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 67E208E0011 for ; Wed, 23 Aug 2023 23:13:44 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2C9571C904D for ; Thu, 24 Aug 2023 03:13:44 +0000 (UTC) X-FDA: 81157528368.26.AE941B2 Received: from out30-101.freemail.mail.aliyun.com (out30-101.freemail.mail.aliyun.com [115.124.30.101]) by imf04.hostedemail.com (Postfix) with ESMTP id 3A30440005 for ; Thu, 24 Aug 2023 03:13:40 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.101 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692846822; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3V+m3y+o7WPMccFaGRMGWAIdW/mcTOrjXycEcwtLQKw=; b=rL7oXvzmJ3AiN4jYe9iT7FuggnszsgvbbOFQbAcocjBOksBTJNBjAV2dv9aG2v27TmCdb/ 5ujeux3Nnl0qKgPH5nxy0QXdNw1wn/HCT6SwU+LLj3cHhXmDMNM7FGRsU8Hi1L3ILY+Nxm BGWX04ndN2yCjdgQFq6+s6ugl01PO7M= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692846822; a=rsa-sha256; cv=none; b=1ayNNn6l2r18sPdbP3kCajK5io/QkXxXWCLI13a2M25vop+rMrJBfgV+v8xwc9/5WOtUD6 vINCDtJucV7bSpfPUsc1/4JzbZ9gfuD7uYgzT8UJzww25hY3dgq6Yx7fjIxlMAjX2fq6Fx KH45iSKaQvfWYz27INM0yVP65ukKDPQ= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.101 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=alibaba.com X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=7;SR=0;TI=SMTPD_---0VqSIdrF_1692846810; Received: from 30.97.48.68(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VqSIdrF_1692846810) by smtp.aliyun-inc.com; Thu, 24 Aug 2023 11:13:31 +0800 Message-ID: Date: Thu, 24 Aug 2023 11:13:32 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 Subject: Re: [PATCH v2 0/4] Extend migrate_misplaced_page() to support batch migration To: "Huang, Ying" Cc: akpm@linux-foundation.org, mgorman@techsingularity.net, shy828301@gmail.com, david@redhat.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <87pm3fbxos.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Baolin Wang In-Reply-To: <87pm3fbxos.fsf@yhuang6-desk2.ccr.corp.intel.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 3A30440005 X-Rspam-User: X-Stat-Signature: rkksg4w3cmqcyngijodw4it145f7wz9s X-Rspamd-Server: rspam03 X-HE-Tag: 1692846820-186318 X-HE-Meta: U2FsdGVkX1/wSGa1BAJrvu2cHGFOll0aCytlQhApeXgMjIs4E260E1MNhdsYOep0ZonD8d9c2Xy6GnrAE/3qlrfyhY5HvJ8N6LhZOYj1Mo0VpitNRfsMwU0RSYJnaZzKcmMwoTOPFEySgPn/j9/I/0uRqdzbe2Eqw46vwrkANzqWvsOacOGon3jC17LCtGnVbu8YA+Mpzxw0Ht/9Ec56Guq1QULalJrZ4xssUnh8s8MlIJXSGtxQxl7CIZvFsbaI6GtxOE2C3sJzv25fgzH/GaFuUS9Z/Bd1fNk/CY+57zih2BfWQBtxyCqDCsBQrlBD0RStUD7Y6A8SDSlYpw9UnUDt/0CbCfftpzHnAPkpzsKN/cIBUcBzT3VGA2PmQiuySG6pUvgRHqE8TdG1GmPaoYTA5KhBFyHUCR0YZhwJZquBoelWLM4NwHEZrmPYfuhUlWXr+qBOslFBFscIW/JQIlrP2f0rqpDPfDgzGKS1kjcd6sbSL89UyDDlWVYFAtGm1/OPh7s70pbnw52hatcHmTHZgSMdLQ9x6BMzS+Cag9KSS/JfpMlc350v+DthwtXXVjA0yrYrnupgfiJGJkgfq9OKLwVoka2RebLBwvuA3w0X57LNApCwDTajVc8yJCuhoPQ6iUgzNNY7gKhDGqnQgYc81qs/l67Ke00+Lq/RZ5A7wwvSS/H4ijXHFtDWkn9kAtDYsxjzgAHaZUCJehNlf1opTPc+/GTHCXjTlczXnFwYFXqrZecWiTMi72x9Xf9dW40A38qwGr0MnrCtygGnBSiKDPO79QDpRQyyjodKPHUCgKFFLigoxOMLOP2u2l7GGjuC27KOVM4jkldOfyIr6k/0IinS8QRDNkMBTTYL1yD23KruwgfUQyipP32dnYBQpba81YVoOfB5T6ruyYCWskzbRmtWE7xVCyJxkiBX4yQKx6qd2lFmHsuf2JMc4KQob3jlOv5iSKvYruX+cJz TOIa99oZ YD+hTzvwRTT7WQ2dOpcWp6yMWM2ZWqV5tGijw8cuz0qK1sFED+4WZb3GfN+1ct7qgFXDvDRfNNb90P7CsPAaAs3C2sE9zwZf2Xr+vu2GiVj6X7F0x1hCyUS0sEoOvDsJp3OKhHOG5oTWBSaaw9UPHc27eu6ZYJVLamdOJUlILnmSR3siBQkMyze6Ryzi596clJ8lxB/zhc4sPtGWgU3YK4DByTwwS/kRFT93Vn3vLkGSKkPYvrgioT7jm+TgOyymsDKBE5Tc8YlQx+oKKZ9ufR3MOkoEtKVK8L7an X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 8/22/2023 10:47 AM, Huang, Ying wrote: > Baolin Wang writes: > >> Hi, >> >> Currently, on our ARM servers with NUMA enabled, we found the cross-die latency >> is a little larger that will significantly impact the workload's performance. >> So on ARM servers we will rely on the NUMA balancing to avoid the cross-die >> accessing. And I posted a patchset[1] to support speculative numa fault to >> improve the NUMA balancing's performance according to the principle of data >> locality. Moreover, thanks to Huang Ying's patchset[2], which introduced batch >> migration as a way to reduce the cost of TLB flush, and it will also benefit >> the migration of multiple pages all at once during NUMA balancing. >> >> So we plan to continue to support batch migration in do_numa_page() to improve >> the NUMA balancing's performance, but before adding complicated batch migration >> algorithm for NUMA balancing, some cleanup and preparation work need to do firstly, >> which are done in this patch set. In short, this patchset extends the >> migrate_misplaced_page() interface to support batch migration, and no functional >> changes intended. >> >> In addition, these cleanup can also benefit the compound page's NUMA balancing, >> which was discussed in previous thread[3]. IIUC, for the compound page's NUMA >> balancing, it is possible that partial pages were successfully migrated, so it is >> necessary to return the number of pages that were successfully migrated from >> migrate_misplaced_page(). > > But I don't find the return number is used except as bool now. As I said above, this is a preparation for batch migration and compound page NUMA balancing in future. In addition, after looking into the THP' NUMA migration, I found this change is necessary for THP migration. Since it is possible that partial subpages were successfully migrated if the THP is split, so below THP numa fault statistics is not always correct: if (page_nid != NUMA_NO_NODE) task_numa_fault(last_cpupid, page_nid, HPAGE_PMD_NR, flags); I will try to fix this in next version. > Per my understanding, I still don't find much value of the changes > except as preparation for batch migration in NUMA balancing. So I still IMO, only patch 3 is just a preparation for batch migration, but other patches are some cleanups for migrate_misplaced_page(). I can drop the preparation patches in this series and revise the commit message. > think it's better to wait for the whole series. Where we can check why > these changes are necessary for batch migration. And I think that you > will provide some number to justify the batch migration, including pros > and cons. > > -- > Best Regards, > Huang, Ying > >> This series is based on the latest mm-unstable(d226b59b30cc). >> >> [1] https://lore.kernel.org/lkml/cover.1639306956.git.baolin.wang@linux.alibaba.com/t/#mc45929849b5d0e29b5fdd9d50425f8e95b8f2563 >> [2] https://lore.kernel.org/all/20230213123444.155149-1-ying.huang@intel.com/T/#u >> [3] https://lore.kernel.org/all/f8d47176-03a8-99bf-a813-b5942830fd73@arm.com/ >> >> Changes from v1: >> - Move page validation into a new function suggested by Huang Ying. >> - Change numamigrate_isolate_page() to boolean type. >> - Update some commit message. >> >> Baolin Wang (4): >> mm: migrate: factor out migration validation into >> numa_page_can_migrate() >> mm: migrate: move the numamigrate_isolate_page() into do_numa_page() >> mm: migrate: change migrate_misplaced_page() to support multiple pages >> migration >> mm: migrate: change to return the number of pages migrated >> successfully >> >> include/linux/migrate.h | 15 +++++++--- >> mm/huge_memory.c | 23 +++++++++++++-- >> mm/internal.h | 1 + >> mm/memory.c | 43 ++++++++++++++++++++++++++- >> mm/migrate.c | 64 +++++++++-------------------------------- >> 5 files changed, 88 insertions(+), 58 deletions(-)