From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 543A0C77B7C for ; Wed, 2 Jul 2025 20:15:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E49FA6B009A; Wed, 2 Jul 2025 16:15:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DD2A06B00AD; Wed, 2 Jul 2025 16:15:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C9A716B00AE; Wed, 2 Jul 2025 16:15:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B2D126B009A for ; Wed, 2 Jul 2025 16:15:01 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8D283160473 for ; Wed, 2 Jul 2025 20:15:01 +0000 (UTC) X-FDA: 83620428402.19.1A546EB Received: from mail-yw1-f173.google.com (mail-yw1-f173.google.com [209.85.128.173]) by imf05.hostedemail.com (Postfix) with ESMTP id 9CE9610000A for ; Wed, 2 Jul 2025 20:14:59 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ac6uJzdT; spf=pass (imf05.hostedemail.com: domain of bijan311@gmail.com designates 209.85.128.173 as permitted sender) smtp.mailfrom=bijan311@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1751487299; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bVqvDTgUdbQeaLQbOWegVEyiOQ5l1FCYExImfHBr7Tc=; b=4ImZQIoAaoVpSSyDQkdkGgpSgKlp5j6xX+Mw/E3l6KtI/11yfvv/54MzV6k/LZUWJUaKXZ rkrESDp/pSnnxBvDogHQ9MVOrfpgk1+HcRN+UWEBCOXgAI995DsRY8WSyjE3GryPeXaaAH ya4QqFF4Svq8Mz//V3PgD1vmXUqgtaw= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ac6uJzdT; spf=pass (imf05.hostedemail.com: domain of bijan311@gmail.com designates 209.85.128.173 as permitted sender) smtp.mailfrom=bijan311@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1751487299; a=rsa-sha256; cv=none; b=gJL6IUi7dNO12CSEIP5uUpK98avA+krweZvs0G1S4ZBu4LAzBVjAXP3TxuxzqN+kzA55jV 7s5H6OPLVnagyeElON0O/zbcaJQ6VTDfIPCCjdd6YWmyfx1TmYtG/b0WmJf1GaXUPKU/k8 zylyrTki1uEvLf2vJkiJCOPae6wrAZM= Received: by mail-yw1-f173.google.com with SMTP id 00721157ae682-70e3e0415a7so3652327b3.0 for ; Wed, 02 Jul 2025 13:14:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1751487299; x=1752092099; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bVqvDTgUdbQeaLQbOWegVEyiOQ5l1FCYExImfHBr7Tc=; b=ac6uJzdTmHc42M7qV1AkywTCVA8pd7l5MXb6oppr2p9NzkUwRThNIwZCxiZWn++BXc E++RmeP5Rfz5lpNLSq/nNtapEXZQ5c95DoAW1ctVwouqBxls5xpqtN+xl5HnEMpEWhKI Vzpp3MHisvCvEdaRJbYlTskf7nW+KB8bBLmXFk73+tsJT2bPpGXhEvZdy6ApWhNTWb3Q Qz3hzgcLDTLP8DHF5IERuwl5mGPKFZ24YhprAIgbZidofiDZBSa4EazPfoh/72GccfSH bO3QtjFNanBJDXUJv3SybgcgRxNoUt4YsZ7712bbXfC+nqGua+bAO5x7opsVpWG6Q3GA UXQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751487299; x=1752092099; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bVqvDTgUdbQeaLQbOWegVEyiOQ5l1FCYExImfHBr7Tc=; b=keJEWIedS9FLuFF6oaxcM5eLZ2kOoHEuzjlWXF9wxn5qLSVwMu36KlgZjnXVY5m8WY R43t5ckENS8ZYFtiB5Dzd1b7nZbD4ykog0Vtr5qLTuRBM2CVUSnj5W1xgwTBfDxYMK2I D3hEM0IJynqNaf1FzMJK3KMvHOAq8rEJbej18b390l3Mupw66A4+QM/F9YYxclsHgTtt 2xWGdEfkhHg5zxIyfTweOD5+k9gHJ7NbrLGLW5FLbgW+TYSR6KLvfVXLSTfMbgZ9rKLi L+Fpk/DL7OVhIvqmHa7SUsQDcXOtK63A4Zs6ljF9h8CVHcBcI1ArCC6mnC4ocblNUag0 3uiQ== X-Forwarded-Encrypted: i=1; AJvYcCXbNuYbxDtQzooaT9+tBfnb0fyWR4YRh7llSdUUAdBT5Rl9RHJHcpBhrd7fiVPPB5Xxv9kz1fAcfw==@kvack.org X-Gm-Message-State: AOJu0YwNHpoj5lp5w1Tyulysax+v/NbwS8f/RbQvYzBzbHiwVAiufwyr 6ofbB+glrx3zifoVFx353ZP0KP+7r8M4/MffITWji0NhLHICG4WXVao8 X-Gm-Gg: ASbGncvJqqxibWq3yasMQTGBntk8u8D051dbUInUwxJJ+f4aBQQRUnCjCLI1ckRhgOc WZkR+LNM1aMh6HjZior2gFKDN+dO/W6xb3xvAaufAZenrvrcfeCPI8lTrEO0f5lQv2sNMWH4P3+ z5TwHHz4CAdZZfMzAX9GFYYkhbB7gTBTUVDMDEEDDmL5aAP/S0R3RxxJY7P94hU8MZQMHtqyD2W Lt73D1f9jeaFaHdzfCl1EUEFIwST78huPFgQJIkvErhwz47QwF9Am5RH02AITQKNGNTNMlQOXy4 LxsjguQ+f1vj/Bw5etySQXOZfyJS5X3hrTpQXEi2Vv4wxzQ2/hYRsQcm1Oa/XS7WfFqpcoIn1FF fvPo9DadeJQjjQ8lEBw== X-Google-Smtp-Source: AGHT+IHCCc83L/Ed/tXXORICBO5ubDFIQHrqXVUSwSXIp5c0xPoL3pJjjGIyHzRoVNsFfggSP3dVNw== X-Received: by 2002:a05:690c:b17:b0:70e:7158:6fda with SMTP id 00721157ae682-716590c8c4amr15208317b3.9.1751487298340; Wed, 02 Jul 2025 13:14:58 -0700 (PDT) Received: from bijan-laptop.attlocal.net ([2600:1700:680e:c000:873e:8f35:7cd8:3fe3]) by smtp.gmail.com with ESMTPSA id 00721157ae682-71515cb4347sm26124157b3.83.2025.07.02.13.14.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jul 2025 13:14:57 -0700 (PDT) From: Bijan Tabatabai To: damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org Cc: sj@kernel.org, akpm@linux-foundation.org, corbet@lwn.net, joshua.hahnjy@gmail.com, bijantabatab@micron.com, venkataravis@micron.com, emirakhur@micron.com, ajayjoshi@micron.com, vtavarespetr@micron.com, Ravi Shankar Jonnalagadda Subject: [RFC PATCH v3 11/13] mm/damon/vaddr: Use damos->migrate_dests in migrate_{hot,cold} Date: Wed, 2 Jul 2025 15:13:34 -0500 Message-ID: <20250702201337.5780-12-bijan311@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250702201337.5780-1-bijan311@gmail.com> References: <20250702201337.5780-1-bijan311@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 9CE9610000A X-Stat-Signature: ksfqxhb56ybksmu981794psjg1wpxwjk X-HE-Tag: 1751487299-346285 X-HE-Meta: U2FsdGVkX1/HD+ho1julNNTz7o4tuT9i3FSjRQMrRAXO2RNqKZ5bBoGF6H2FjicSHtAY/LUVpPAv12iRBxX0VWULEYNzraSHuV4gxKidnGzVPsdziGzIxAws/kcaA/0JA8UBY58Cqy1u3BVx0UJfP0n9DT7S8JuYbGqMuBOpKtuACMdknCvBiUG3V1d4e27Uhp1Esbx5YJS6nIaKIi5h/7UWStR7ToaFHWMLunpG7WU/1XKX7qi+zAOPrSyPk4WSNQxFDMxmQgxPmSZKtuSsBTNXcSNcAigwRZoS2LEkBg8X07a6eBK6+cz/tNeSpJ9pGgWxmFAXWG+IWAEqcE/4UPvpTNtFHfrNrPGkcdjFeaYwmSZ+ZJ7mb0qqnA4lWG6pPweb1688xGIxa9wvaE9BYIvqeRJ21BmXFA6Fshb+ZLH8NzcfO/+71XmQDR1CJMdXx8X9KWxQKj74I1fKKZBY7jt4NcULedImXOYT0dPdgkodoz6ojliPbSdRK7/ccBHVjNd/0As9kC9scbpi6J45oqg4JrK65rF9OEAFngrtAGjLl+0OskDSG7+qCLoAMyp7j1SBt/wSeVtoaOxQOWqZzpG+7a3S1OzNfZuw7HOA4ZFL+2jn/1K/a42IGjYs0GomkqsGZ4Gy9mhpB8N6iyXAOyJWPw8EGdbY+pAC4UMs8w41bQB+j15GMMlOulFn+lR7n+UPc7yS0ymiUPMAS5Xnrz4jnQyLp2XCw+VHcb+079AEdjTR59GGkgwOTNh1IhUhIYywMezHr7GRJnlG+gGuZ/g67xjjao+z+y2aqKh2J7QdS+WIBF+t2TdVcxFpxj3r3F2y6AH0STJP52fB7ss4TZDx3sGcy/HNIj8v5qfpfLpSgm9QS0ensE2QmF7NqBl08irw7AEkS4r2aTJIeb+K/7rJmFctBhbsnuafv0WeQkjZw/UDBDenk5Pu++7a9sUTedcCpBlUilwZKqUqMP3 JsbgF9sY SQsffqJJ0ZRFxtVlvkz/PuUm2w3ez/6dC9EBnJJh3LaNpS/+UhfHGAcOblAuDB/VBcmLifXUxJ3qxGktmX4kLRXYvXIh45qp5f9ZdA9fFDGZP8n5Su29b6NYwaRUbqDKWk7rpSCiD29BeJW26i0K4NFthP7+d+62zrEvQGG6ld4Dapm7Vv0eJyQLhh6nFUaYR9l/PxwWNVOfst+FwQKaoUWcLuv9Vss6vrw3vg9vkbUyDf5xfobebQMKUDYCSLwYrTmwlz/2VHTW7FBTMzcSTs/UOsKJgEMnK4Tv0JE9RNW11DX1MRk9uxOPyx81CRnQsPAQ0cCprewU/Wo33Wck4cehtHgBQDEWQ0q1/fPMfnrmdXCVF8IzzZZr7QbLU+gnELXqmFJVizQM1GJw9VF1OdE84XIzqK15IOx3tRYaHO322MROyF5Ej282YUKw5rJvSsacZmrOxJkyE57DKz2JJcjA9Jvh/uh9RDvfwOZGkeRN3YKcjkWoSRU+TUHzHFNWSUL9E0UmCqxJG+PYKcgIVB2jdwuKpD9fYictCqEZOhOoQq8G5TsGY5m86lOom5lErWccTqwekmku8Hkn1hTCdReAQeqejAUd86Nf8jMR+i9MpTtSSeF5Ata2V5fGJ0c+fGT2iJ9UI1jp+k+E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Bijan Tabatabai damos->migrate_dests provides a list of nodes the migrate_{hot,cold} actions should migrate to, as well as the weights which specify the ratio pages should be migrated to each destination node. This patch interleaves pages in the migrate_{hot,cold} actions according to the information provided in damos->migrate_dests if it is used. The interleaving algorithm used is similar to the one used in weighted_interleave_nid(). If damos->migration_dests is not provided, the actions migrate pages to the node specified in damos->target_nid as before. Co-developed-by: Ravi Shankar Jonnalagadda Signed-off-by: Ravi Shankar Jonnalagadda Signed-off-by: Bijan Tabatabai --- mm/damon/vaddr.c | 114 ++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 97 insertions(+), 17 deletions(-) diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 5cdfdc47c5ff..5f230a427fdc 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -611,11 +611,76 @@ static unsigned int damon_va_check_accesses(struct damon_ctx *ctx) return max_nr_accesses; } +struct damos_va_migrate_private { + struct list_head *migration_lists; + struct damos *scheme; +}; + +/* + * Place the given folio in the migration_list corresponding to where the folio + * should be migrated. + * + * The algorithm used here is similar to weighted_interleave_nid() + */ +static void damos_va_migrate_folio(struct folio *folio, + struct vm_area_struct *vma, unsigned long addr, + struct damos_migrate_dests *dests, + struct list_head *migration_lists) +{ + pgoff_t ilx; + int order; + unsigned int target; + unsigned int weight_total = 0; + int i; + + /* + * If dests is empty, there is only one migration list corresponding + * to s->target_nid. + */ + if (!dests->nr_dests) { + i = 0; + goto isolate; + } + + order = folio_order(folio); + ilx = vma->vm_pgoff >> order; + ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order); + + for (i = 0; i < dests->nr_dests; i++) + weight_total += dests->weight_arr[i]; + + /* If the total weights are somehow 0, don't migrate at all */ + if (!weight_total) + return; + + target = ilx % weight_total; + for (i = 0; i < dests->nr_dests; i++) { + if (target < dests->weight_arr[i]) + break; + target -= dests->weight_arr[i]; + } + + /* No index being chosen indicates a mistake in the algorithm */ + if (i == dests->nr_dests) { + WARN_ONCE(1, "Error determining target interleave node"); + return; + } + +isolate: + if (!folio_isolate_lru(folio)) + return; + + list_add(&folio->lru, &migration_lists[i]); +} + #ifdef CONFIG_TRANSPARENT_HUGEPAGE static int damos_va_migrate_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long next, struct mm_walk *walk) { - struct list_head *migration_list = walk->private; + struct damos_va_migrate_private *priv = walk->private; + struct damos *s = priv->scheme; + struct list_head *migration_lists = priv->migration_lists; + struct damos_migrate_dests *dests = &s->migrate_dests; struct folio *folio; spinlock_t *ptl; pmd_t pmde; @@ -630,12 +695,8 @@ static int damos_va_migrate_pmd_entry(pmd_t *pmd, unsigned long addr, if (!folio) goto unlock; - if (!folio_isolate_lru(folio)) - goto put_folio; - - list_add(&folio->lru, migration_list); + damos_va_migrate_folio(folio, walk->vma, addr, dests, migration_lists); -put_folio: folio_put(folio); unlock: spin_unlock(ptl); @@ -648,7 +709,10 @@ static int damos_va_migrate_pmd_entry(pmd_t *pmd, unsigned long addr, static int damos_va_migrate_pte_entry(pte_t *pte, unsigned long addr, unsigned long enxt, struct mm_walk *walk) { - struct list_head *migration_list = walk->private; + struct damos_va_migrate_private *priv = walk->private; + struct damos *s = priv->scheme; + struct list_head *migration_lists = priv->migration_lists; + struct damos_migrate_dests *dests = &s->migrate_dests; struct folio *folio; pte_t ptent; @@ -660,12 +724,8 @@ static int damos_va_migrate_pte_entry(pte_t *pte, unsigned long addr, if (!folio) return 0; - if (!folio_isolate_lru(folio)) - goto out; - - list_add(&folio->lru, migration_list); + damos_va_migrate_folio(folio, walk->vma, addr, dests, migration_lists); -out: folio_put(folio); return 0; } @@ -717,34 +777,54 @@ static unsigned long damos_va_migrate(struct damon_target *target, struct damon_region *r, struct damos *s, unsigned long *sz_filter_passed) { - LIST_HEAD(folio_list); + struct damos_va_migrate_private priv; struct task_struct *task; struct mm_struct *mm; + int nr_dests; + int nid; + bool use_target_nid; unsigned long applied = 0; + struct damos_migrate_dests *dests = &s->migrate_dests; struct mm_walk_ops walk_ops = { .pmd_entry = damos_va_migrate_pmd_entry, .pte_entry = damos_va_migrate_pte_entry, .walk_lock = PGWALK_RDLOCK, }; + use_target_nid = dests->nr_dests == 0; + nr_dests = use_target_nid ? 1 : dests->nr_dests; + priv.scheme = s; + priv.migration_lists = kmalloc_array(nr_dests, + sizeof(struct list_head), GFP_KERNEL); + if (!priv.migration_lists) + return 0; + + for (int i = 0; i < nr_dests; i++) + INIT_LIST_HEAD(&priv.migration_lists[i]); + task = damon_get_task_struct(target); if (!task) - return 0; + goto free_lists; mm = damon_get_mm(target); if (!mm) goto put_task; mmap_read_lock(mm); - walk_page_range(mm, r->ar.start, r->ar.end, &walk_ops, &folio_list); + walk_page_range(mm, r->ar.start, r->ar.end, &walk_ops, &priv); mmap_read_unlock(mm); mmput(mm); - applied = damon_migrate_pages(&folio_list, s->target_nid); - cond_resched(); + for (int i = 0; i < nr_dests; i++) { + nid = use_target_nid ? s->target_nid : dests->node_id_arr[i]; + applied += damon_migrate_pages(&priv.migration_lists[i], nid); + cond_resched(); + } put_task: put_task_struct(task); +free_lists: + kfree(priv.migration_lists); return applied * PAGE_SIZE; } -- 2.43.5