From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C1E1C83F09 for ; Wed, 9 Jul 2025 01:01:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E07168D000B; Tue, 8 Jul 2025 21:01:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D90698D0001; Tue, 8 Jul 2025 21:01:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C09F48D000B; Tue, 8 Jul 2025 21:01:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AF0768D0001 for ; Tue, 8 Jul 2025 21:01:26 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 66023802D8 for ; Wed, 9 Jul 2025 01:01:26 +0000 (UTC) X-FDA: 83642922972.30.0066E30 Received: from mail-yb1-f182.google.com (mail-yb1-f182.google.com [209.85.219.182]) by imf02.hostedemail.com (Postfix) with ESMTP id 84EBB80008 for ; Wed, 9 Jul 2025 01:01:24 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RxbLr8xY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of bijan311@gmail.com designates 209.85.219.182 as permitted sender) smtp.mailfrom=bijan311@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752022884; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IZC5q8LZ5pc6UFxmCdVHkQ+xcSY0x1BRxLFRLW7GuXk=; b=k8KeFUs2ZNsbMoTi0dXo1/184oPd29IwPNhydXjG4ECfC2T2ss694K4cBRdG0AXUadiNrz Q9tIm9M+bd2r4ieUnSQgpk4nJ0o/dGXdT9Xlk7xG8eBl4DUjBTCj/XQ8vi/vjZE7gIF38R MGFgrrQlfH5PMn9kwcUvcD2GNu4QoNQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752022884; a=rsa-sha256; cv=none; b=qPHsPVUHgqs2QAd1y0wgDW1JzCzW7//GbRbQ/L1wK0r6oeXOP63Q4yl4+lu9Q7Onfi2NSP eCbtrq9w6EGNUrK/PlJXqtCCtHAKVaPn98B2hFLhi3uAxD0BPAyPnshG2Vk2f9Hg4xhcLM LZOW2uFov+DwkjrYvF9UgWCnpvEk2Es= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RxbLr8xY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of bijan311@gmail.com designates 209.85.219.182 as permitted sender) smtp.mailfrom=bijan311@gmail.com Received: by mail-yb1-f182.google.com with SMTP id 3f1490d57ef6-e75668006b9so5041509276.3 for ; Tue, 08 Jul 2025 18:01:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1752022883; x=1752627683; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IZC5q8LZ5pc6UFxmCdVHkQ+xcSY0x1BRxLFRLW7GuXk=; b=RxbLr8xYD7efYsLAH6BPY3fEjs4MyxWZDsZW51tc7WxlPrkDGnb6gtAK6NOdHLDGer GwH5d8qMCR3KmUYbYCFv/lP+Vdbhbi+gjaOwUPR2j3/IZ+G55gj4KV7vSs563MKHDnjt G8LGFhsS5IVpISGDumXVh+KNsb0CVlVBZQSD1bwwdMBA1hDnsSNHd9x7lMqSVsaVkEpC KbMp6IGV9WjVUy59/3NCNW3pkYeb1kKNDGte5PZJ0OYfjK6zYo9cIMerZpHJ9bKbaGZF JEZwoLhZJhkSCHQsHb2WzAtYbrB4azUWLxO9p8NpYYEB0JFmzkWsS0DSTn6nQ7DUESY/ VWYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752022883; x=1752627683; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IZC5q8LZ5pc6UFxmCdVHkQ+xcSY0x1BRxLFRLW7GuXk=; b=VZQjlA6SllNb3EStEXDX3XzAC88+qes2WU3wcpu1Pb2+83zBykGvlAAfYQyMLAfZiD 0mSOoQB8RNO6ODHh83SvQSBeRmhuckJkNbn52/xfkpvElnDgvJrm23eCNoK6T2TgL4Bp fc76bPQ5lo5hCHoJbfRjUrRRNVLXK5b8GL8FJFkn0zvPy8FG7PAcnxrdJqmCPcGmRFQN exgiXIVozyl7oMAdo3NTnSjuJFi6uhR7RdZI3En7AY3bPxLPZYuQb5ueWFMl6CMAV+nJ wLJp7JnoWnR4ErDYJFFbfTh1ITLQoORhvRqUmJDFipFsc7XGTTlVl0rfOJaRlSONQk9G uYyQ== X-Forwarded-Encrypted: i=1; AJvYcCX7SSzQLo+V9BYtf6CQd88bquUdPZ+nklLZG7ODvLz2mAdDn6X4/HngXZPvcm0NrCNdSAsyXKMvSQ==@kvack.org X-Gm-Message-State: AOJu0YzkyhVf1GUWxu72on6dknz81mdUCnthDQUdUCxsyrpAAZ7v5bBD Eeh2E95oXqBQczdualGKaio0uqvuLCaurfiILYH+nlqtxmLsibbLB1JN X-Gm-Gg: ASbGncsakdOw4lNSkzC9iSsv3GNDnkcMD/LilknVjkfrIbJ7t7H0ENTQtgtSwP7OKED Coe/wXWvfYFq5Nk3khYmemNXuYWcfVySULTTKaUHxbCH153T6nfacSx/QeCsnbn85gPhnNIeCid u5hMyuCS++imkX3ZTeWwZNEMMJEjM3hfs+wErcSznD+GHdBzAgkot7MSCpzC088zt4LsepL/J0y v53Qxz/c7h9plpp+w4cD3XwlG5HefeXGkIa+I71sp/LHgsrbtMs8HFl1mfYopExB9UldRDd/YDr OQCPTqf4zwteEK+crVLpYKrUNx10ZHsJcNbOX1G9YGTilglSEESygiFoOZ08lA8eVKmNILcuSMQ z1aAR7HU= X-Google-Smtp-Source: AGHT+IGBVL08fCEmXie0BqOThtC53srTZke6UZuD3rNGM0GVPqcqyBQP9Gr6apPaTrB4/7lyEi3sSA== X-Received: by 2002:a05:6902:2609:b0:e8b:444a:9298 with SMTP id 3f1490d57ef6-e8b6e0de728mr1064674276.5.1752022883567; Tue, 08 Jul 2025 18:01:23 -0700 (PDT) Received: from bijan-laptop.attlocal.net ([2600:1700:680e:c000:235f:99bb:f36e:a060]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e899c43ed8asm3656043276.26.2025.07.08.18.01.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jul 2025 18:01:23 -0700 (PDT) From: Bijan Tabatabai To: damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org Cc: sj@kernel.org, akpm@linux-foundation.org, corbet@lwn.net, bijantabatab@micron.com, venkataravis@micron.com, emirakhur@micron.com, vtavarespetr@micron.com, ajayjoshi@micron.com, Ravi Shankar Jonnalagadda Subject: [PATCH v4 13/13] mm/damon/vaddr: Apply filters in migrate_{hot/cold} Date: Tue, 8 Jul 2025 19:59:43 -0500 Message-ID: <20250709005952.17776-14-bijan311@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250709005952.17776-1-bijan311@gmail.com> References: <20250709005952.17776-1-bijan311@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 84EBB80008 X-Stat-Signature: k413i5edh7h57ii6656c7ofgmc33tosi X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1752022884-609730 X-HE-Meta: U2FsdGVkX18k8yWWV2EunXR/U4RFJj1IFMJ3CTkgsFZJhH48HlN20hCE6pS8v6JVjdnqbPIBjpUHxlJ4Udn/qGzNyZ5HCcg7soA60VSRrE/ThHjpVVQ8Z1hIwavmUfIxtoLgPqOJn0SUqS9fflrjvHENpDLVixr5Wc0c56THOidxfnfBJ5MJ09K0TeGhclGlBQifm63Zwm6xDZH0DtV7SZB1u3Ik1hLEePAzR0CZpIkMSgVOdeKLJIJ7ZtG6IOoSsEW/U86ny2z9h2XR1IKG/IiOsGiMVO+oqi4f+V8vsocVkW/+lLG3Vgf9LZfGYa6UtvV0lo7U62FPOvKh6rCHfIU3UHBZmbcHxwLzeBnZA07fZXDtZ/ZE+i+WxW6RgF4eNY+yokPJcVd/f6vJ97fsuPgzQnMgKAIaPasooLlmDDOgqfk4AleK5/JyrvXjIjWSQo1lTEs1ab5DeghQWXWS2F08B/GBe3MwtXmxdlUr9vd8i+t7Y/9TJu7CI12gbC1SJjvPra+KLBNf6U5rnP1KVw6a1x9wD5OGxZ9nngvAwhqexYMNx4NUnv8IOkLlnBsLTK1DeWkk0RFqtHmGrM1f566gPw8OdHujd77fEHV7SckVYgRaTv+zXHpymt2T5Woa3zTnxCu4m6BUY1nFSzopl0fBU8Rw3CJvyK1lLSxSktxkHiGssvBm9fDxRatpcKP1E3Vz8VDmcEGLLiNdGXpN2MAjhcPtX1HnVq26OQlqTMqIQl0zBwlK8arnWizSaOZ24qeLqukKF8qC91XPN5vC2z38015aHczNfOxUFjCks37MLTf4JCLn6DJbO6egivuE9t1moMpf6kiCuD2O9drRW7O2DoDBzS15g2F5KLnyZh+ouar7uema+uWKvTfOciSgdqf2NtVMk4nzzumr4Ma9dFjwbk1NP2vXqIJuPCSGNMi2N2X6G8JxU3hsfbvxxE8LW68uOn5ikzodOI+VgMT J6cDp9f4 XZKbYk55pbNqW3e52v1GgQL4BjL1iuTXuntQ+omIj/FdhyjT7ZKystI7R0/QzsmyjmICl7xpqIjFe/9Pc4enBdFOwgdBf0W8kqxQzFfQwcZrRX/iLLdwKWjnUfO3qn0Gn3weFY4nm6b/0uCas1dPyFUwCaOMlI1CN+DDhq5JFwAzqxUx6P4bIGC71nv1TrYRUXl/Kb/TcwugzruImiWgchG/FC10VZ/JKvVzzT/RlgnGjHBzQufzKIcrWfw+eBaXAaxuR61SL94p5eqqlm0mlQoFNTkgPoWZOVEwY3hGJNs7VGesBtp4cIvxkVx1+DBTgeLHqwoZlMEXenDlRy4RFOOfTVKft7dunZpqhnbcL/6s/06f2pWGstOGMvTeMlI2isbKSEQb0tnlCg3KEnvseghI5039s0MGqqpqAPNL6apqRrEHQtzGJyQLwSjv6P1uUszmDYCydTps/pFFdTwBL+QOfVO27Gc03ckuFNiUVCNOmeyoXAT8jp5ic/GDs9VYezgGSBFer+6ADmGX4yrbrSYEbp1lwMPGoXHSDMKcLDmfvxFo2ebnS7Ly1wG2X3CucVw1a X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Bijan Tabatabai The paddr versions of migrate_{hot/cold} filter out folios from migration based on the scheme's filters. This patch does the same for the vaddr versions of those schemes. The filtering code is mostly the same for the paddr and vaddr versions. The exception is the young filter. paddr determines if a page is young by doing a folio rmap walk to find the page table entries corresponding to the folio. However, vaddr schemes have easier access to the page tables, so we add some logic to avoid the extra work. Co-developed-by: Ravi Shankar Jonnalagadda Signed-off-by: Ravi Shankar Jonnalagadda Signed-off-by: Bijan Tabatabai --- mm/damon/vaddr.c | 69 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 65 insertions(+), 4 deletions(-) diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 47d5f33f89c8..7f5dc9c221a0 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -611,9 +611,60 @@ static unsigned int damon_va_check_accesses(struct damon_ctx *ctx) return max_nr_accesses; } +static bool damos_va_filter_young_match(struct damos_filter *filter, + struct folio *folio, struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, pmd_t *pmdp) +{ + bool young = false; + + if (ptep) + young = pte_young(ptep_get(ptep)); + else if (pmdp) + young = pmd_young(pmdp_get(pmdp)); + + young = young || !folio_test_idle(folio) || + mmu_notifier_test_young(vma->vm_mm, addr); + + if (young && ptep) + damon_ptep_mkold(ptep, vma, addr); + else if (young && pmdp) + damon_pmdp_mkold(pmdp, vma, addr); + + return young == filter->matching; +} + +static bool damos_va_filter_out(struct damos *scheme, struct folio *folio, + struct vm_area_struct *vma, unsigned long addr, + pte_t *ptep, pmd_t *pmdp) +{ + struct damos_filter *filter; + bool matched; + + if (scheme->core_filters_allowed) + return false; + + damos_for_each_ops_filter(filter, scheme) { + /* + * damos_folio_filter_match checks the young filter by doing an + * rmap on the folio to find its page table. However, being the + * vaddr scheme, we have direct access to the page tables, so + * use that instead. + */ + if (filter->type == DAMOS_FILTER_TYPE_YOUNG) + matched = damos_va_filter_young_match(filter, folio, + vma, addr, ptep, pmdp); + else + matched = damos_folio_filter_match(filter, folio); + + if (matched) + return !filter->allow; + } + return scheme->ops_filters_default_reject; +} + struct damos_va_migrate_private { struct list_head *migration_lists; - struct damos_migrate_dests *dests; + struct damos *scheme; }; /* @@ -673,7 +724,8 @@ static int damos_va_migrate_pmd_entry(pmd_t *pmd, unsigned long addr, { struct damos_va_migrate_private *priv = walk->private; struct list_head *migration_lists = priv->migration_lists; - struct damos_migrate_dests *dests = priv->dests; + struct damos *s = priv->scheme; + struct damos_migrate_dests *dests = &s->migrate_dests; struct folio *folio; spinlock_t *ptl; pmd_t pmde; @@ -691,9 +743,13 @@ static int damos_va_migrate_pmd_entry(pmd_t *pmd, unsigned long addr, if (!folio) goto unlock; + if (damos_va_filter_out(s, folio, walk->vma, addr, NULL, pmd)) + goto put_folio; + damos_va_migrate_dests_add(folio, walk->vma, addr, dests, migration_lists); +put_folio: folio_put(folio); unlock: spin_unlock(ptl); @@ -708,7 +764,8 @@ static int damos_va_migrate_pte_entry(pte_t *pte, unsigned long addr, { struct damos_va_migrate_private *priv = walk->private; struct list_head *migration_lists = priv->migration_lists; - struct damos_migrate_dests *dests = priv->dests; + struct damos *s = priv->scheme; + struct damos_migrate_dests *dests = &s->migrate_dests; struct folio *folio; pte_t ptent; @@ -720,9 +777,13 @@ static int damos_va_migrate_pte_entry(pte_t *pte, unsigned long addr, if (!folio) return 0; + if (damos_va_filter_out(s, folio, walk->vma, addr, pte, NULL)) + goto put_folio; + damos_va_migrate_dests_add(folio, walk->vma, addr, dests, migration_lists); +put_folio: folio_put(folio); return 0; } @@ -790,7 +851,7 @@ static unsigned long damos_va_migrate(struct damon_target *target, use_target_nid = dests->nr_dests == 0; nr_dests = use_target_nid ? 1 : dests->nr_dests; - priv.dests = dests; + priv.scheme = s; priv.migration_lists = kmalloc_array(nr_dests, sizeof(*priv.migration_lists), GFP_KERNEL); if (!priv.migration_lists) -- 2.43.0