From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52929C2D0CD for ; Wed, 21 May 2025 18:46:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D55716B0096; Wed, 21 May 2025 14:45:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D06046B0098; Wed, 21 May 2025 14:45:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF7B06B0099; Wed, 21 May 2025 14:45:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9BF166B0096 for ; Wed, 21 May 2025 14:45:59 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 443195EB8C for ; Wed, 21 May 2025 18:45:59 +0000 (UTC) X-FDA: 83467794438.07.60C81DC Received: from tor.source.kernel.org (tor.source.kernel.org [172.105.4.254]) by imf17.hostedemail.com (Postfix) with ESMTP id A2D3640005 for ; Wed, 21 May 2025 18:45:57 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=oQJwvDpI; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747853157; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GKb5KmM9lmVzU1Z2PRSmi3zwfjg8yhK696kRN9kTpP4=; b=aaPfOxIIrjgVnBg7fzTTUZCppX7SZrJWktAqfi4Nvt2ux0Fzek0J3nO+OeWAxUGxDbDt3A r/Y+6d1qdrHKAXTBQ4d7Xb2JBM/1RAvPs1TOTTsBsUxV7CI/UDn1SaCfclDmmi+vjv5sBQ Fl/ke0J3kSzbCL8iFfrDWMtNSUhQ6wU= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=oQJwvDpI; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747853157; a=rsa-sha256; cv=none; b=ADM6nrfKc5RA/oZmreHUpltDaxWMqKQWjhP8SkL7v4HVjvFVLWNyCQ3VjCkDJWjievqYyH 6LSDtaahDPVR3dA18rZKGoWzMCByV7/hcC+vTOwSttAi6j7F1BQ6hj8szfwec3CP7XKxw3 c0p6HkhIcOFhUwITGAV4y5lx7bzXqKk= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by tor.source.kernel.org (Postfix) with ESMTP id A165C62A00; Wed, 21 May 2025 18:45:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2EE22C4CEE4; Wed, 21 May 2025 18:45:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1747853156; bh=LYjLJmThU61NbIP8XRAsnYD4C0EoiLBTx7NC9szONOA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oQJwvDpICuPfG+tPW0YPx6MuMk/qNCEGqyoMPXpDoIeetZN/SbhMnu4DEJexUtUJn glggBEFb6xjQ543Go87CfI7NAWfpbhhYQgHYHxOpyLqpnQZUev5Jo7GYst1q8yA2y6 cwJZn1H5bBt2plXq6tunwLBggmglUHNWu8EGQbwZDtkXjjuHxatOM/zFNwCQd684E/ 5dRMgeeN5tlPAdcpB5Su9hh12/RkVPZ8BTazI6x3GOcdtW5YJUviag2Hr/4VRLv5en c+G5lho/YqAKd3gwYqcVcHTd1RUzr+zCuavmntAaN1SkRXKupL8YLZeOpvw64OqZj5 zvwG6RJYiqG2Q== From: SeongJae Park To: Bharata B Rao Cc: SeongJae Park , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Jonathan.Cameron@huawei.com, dave.hansen@intel.com, gourry@gourry.net, hannes@cmpxchg.org, mgorman@techsingularity.net, mingo@redhat.com, peterz@infradead.org, raghavendra.kt@amd.com, riel@surriel.com, rientjes@google.com, weixugc@google.com, willy@infradead.org, ying.huang@linux.alibaba.com, ziy@nvidia.com, dave@stgolabs.net, nifan.cxl@gmail.com, joshua.hahnjy@gmail.com, xuezhengchu@huawei.com, yiannis@zptcorp.com, akpm@linux-foundation.org, david@redhat.com Subject: Re: [RFC PATCH v0 0/2] Batch migration for NUMA balancing Date: Wed, 21 May 2025 11:45:52 -0700 Message-Id: <20250521184552.46414-1-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250521080238.209678-1-bharata@amd.com> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Stat-Signature: 5nzqwwonzbc8cg3xthqzej91p6euwjqo X-Rspamd-Queue-Id: A2D3640005 X-Rspamd-Server: rspam11 X-HE-Tag: 1747853157-688814 X-HE-Meta: U2FsdGVkX1+GvNthueHCWw3ujjrSM5kyYD6tGx3mEMvobxYxcL5pSYSh6f7IfjYcJqbDtxKo8v5cRkaLF0RyvV24sT7dDN/viElsEVNwqo8L8/MOwla+xfvZ/lg7IRtZZpj5ZZ5f+Pnrda75NeIv4tDYrWSvvZi3DXwCvY4rDtD88OE1yT7fbQhzcDmmThBjCYuIqLErkvJ9pqu2cswbpE0aiqs8Yw82o/O4UZzb6vHjvkc4ihiSJncMHprpzcCDKjlM6uXKT82qvUQRWv4YH9p3+ZdXH2wpmfP8p7k+dWpkgAbWNJDajT/xvmiG6CcFr6SqYR/S2OewJzEfHIyMwDo6mZnCsuAxyzEI/x9cG78jHNy1tIRmGGvdSucDPzhAtMI6Wgn3fyyag3WC3sb0Y068vv8jTSua1qqjpeugYM8jLRelnmbKQEKO4bJ3W0AOb18Wai0nX6isSm/r+tX+6kGAT3cWvdeofZPgpG1+Vecc51gYSGYi0h2MaP+iVmiezKeK7w0B4giLBNB+C/bWPoxsXWuUE2my444iQfUJRwfSDym5O5Jpxbmbu+159qteYYG3YQIIog4y3AUdTKMLeyCJLKF9o28srLQ2qO4t73s3twZpCFBF4pr05DSsARzg08+7tHoctM16CVTBkmw1ZD7RL3LwRtSXLopucX4MGGVfytxi1xJedwMpeyqYYr9Vj15y1D3Y/1ONM59db+HrL92DHrEQQq1NOwzcOJ91BaEhAKOeG1HZkGAUyjMsXwIkO1/Dn80ZQ6tl/UZmikvxL1oOFbe5pIy6hcJTmWR21u7PPKuEncduT6i6awiVIEy4krp2OwTxtrHoIeShzxuJPafGRUXGq/JMjNGYwhxhzoCL8UPJgI5yp7aAcj7LEguBmQBF1hYdo6QsdjthsaBfKFakruMDTLTeH5VXoDebQBGsy1Zc5D3Ok5DFI7DzCQiDZeyUJlQkpU6Sl4C5HDV Qofoni/H 7Z1nVz9ILAbiHjibDQDNi/go67P2XVYDoHkNzI5bplDe0AP3ddfzLVT4aa2E7M9jEtSQ+3922R1OEx5lW1ndtpMWcBpSz104bwQAVf33VYvIJULHIEeEeapZoZWjpLHYBXSSja+6WvJPfBTyziQsybXXRGBeBRiM++B559qCvpnU2WP58542aOIFao9bKHo/LFR5NBQpAw3J3v/iatEp0fEQ6Kri/oIHWsEEChrfEfT8KgWsQrGAbukLGQXG4xCQspDXYmUQgL6/nIaDTK2IgihhId8hfE79AYX8YEaJwg0bWtgQ6kpKXt+bQ+FhjqslnH8y+ayPdNvn+KpEi5kGITdLwLAYjfeXjt0pcGxreHJCzf9Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Bharata, On Wed, 21 May 2025 13:32:36 +0530 Bharata B Rao wrote: > Hi, > > This is an attempt to convert the NUMA balancing to do batched > migration instead of migrating one folio at a time. The basic > idea is to collect (from hint fault handler) the folios to be > migrated in a list and batch-migrate them from task_work context. > More details about the specifics are present in patch 2/2. > > During LSFMM[1] and subsequent discussions in MM alignment calls[2], > it was suggested that separate migration threads to handle migration > or promotion request may be desirable. Existing NUMA balancing, hot > page promotion and other future promotion techniques could off-load > migration part to these threads. Or if we manage to have a single > source of hotness truth like kpromoted[3], then that too can hand > over migration requests to the migration threads. I am envisaging > that different hotness sources like kmmscand[4], MGLRU[5], IBS[6] > and CXL HMU would push hot page info to kpromoted, which would > then isolate and push the folios to be promoted to the migrator > thread. I think (or, hope) it would also be not very worthless or rude to mention other existing and ongoing works that have potentials to serve for similar purpose or collaborate in future, here. DAMON is designed for a sort of multi-source access information handling. In LSFMM, I proposed[1] damon_report_access() interface for making it easier to be extended for more types of access information. Currenlty damon_report_access() is under early development. I think this has a potential to serve something similar to your single source goal. Also in LSFMM, I proposed damos_add_folio() for a case that callers want to utilize DAMON worker thread (kdamond) as an asynchronous memory management operations execution thread while using its other features such as [auto-tuned] quotas. I think this has a potential to serve something similar to your migration threads. I haven't started damos_add_folio() development yet, though. I remember we discussed about DAMON on mailing list and in LSFMM a bit, on your session. IIRC, you were also looking for a time to see if there is a chance to use DAMON in some way. Due to the technical issue, we were unable to discuss on the two new proposals on my LSFMM session, and it has been a bit while since our last discussion. So if you don't mind, I'd like to ask if you have some opinions or comments about these. [1] https://lwn.net/Articles/1016525/ Thanks, SJ [...]