From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DA70EEB57F for ; Sat, 9 Sep 2023 03:37:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C5FFE6B0103; Fri, 8 Sep 2023 23:37:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C0F1E6B0104; Fri, 8 Sep 2023 23:37:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92C186B0107; Fri, 8 Sep 2023 23:37:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7E1576B0103 for ; Fri, 8 Sep 2023 23:37:31 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5A1471A11B6 for ; Sat, 9 Sep 2023 03:37:31 +0000 (UTC) X-FDA: 81215649102.22.EC3606B Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf17.hostedemail.com (Postfix) with ESMTP id 857884000D for ; Sat, 9 Sep 2023 03:37:29 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=tVutAxv4; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694230649; a=rsa-sha256; cv=none; b=6HBPMeMq7VWqLSm6vzxihyVIaVsOp6IVUOZIEm+G5OSjIu9SvVDmbrnDsTuC39xijJIOz4 2azHINSa1LyerGM6aimPQ71znIt7QaMPWSkvHIF5sbWcISIOplrOJhBRKgbu2OEsNpm4fW L59PcsKRNhY535TLrY7fTYDWaJBhl0o= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=tVutAxv4; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694230649; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=k3COegnpRC19JcbUcYxuzktOO63xpTUqnEuucER9Yc8=; b=Pe1anRh5KISqCzKOdj3sNTbyalhXUxqjYc+JEkvi5xmhtNMRYKghxKE2IG4JKoQP8eKkBF DRZ/pGAT8UVzoEonO/xsTEVobiayhxDjtR/QYMKRDPk5obNKcj+ezf5Rb+IdCN4tfDjr9H hhGwNjm0oapXzBbRvyzenxdHfn+NxOs= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 34B27B82336; Sat, 9 Sep 2023 03:37:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 805D7C433C9; Sat, 9 Sep 2023 03:37:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694230647; bh=ugkHqkw7yl130d3cjNnQjhDRwZWu/ujQKtPoxpJY3zc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tVutAxv4NGTAsH9JU8KYYDMBIibZhveJyySt4YiREVpzBp4Kgnvnr6Lhff2Uvnkvv WnOpiFOHRBmjWqJ4pAAWXWiiNP1cm6M76idSJnmIcEPp1RID5/jgf0Ce6JMdbsr4et hxtMYclLYMlcSZRhLUtDVINrsCb3MS/i70CjffezmLzikjkNeehVjsoXQMR1ekndIb 00/hgnsL+Fbe6GfYHF8wncDZN582NEtDhPZyikUTOvaNaSmtnJ4GvUjJzjk+sFb6GF 7E2oJ8kxURwt9U1Mgw9kCCqLsnN6FtMRfRnzaO4May4NrAgppDWAETNS/jDDJG28+5 BSPmzIkC83nWw== From: SeongJae Park To: Cc: SeongJae Park , Andrew Morton , damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC 6/8] mm/damon/core: use pseudo-moving sum for nr_accesses_bp Date: Sat, 9 Sep 2023 03:37:09 +0000 Message-Id: <20230909033711.55794-7-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230909033711.55794-1-sj@kernel.org> References: <20230909033711.55794-1-sj@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 857884000D X-Stat-Signature: zkxng9du6icqs58pzn7xzh8s1wud8ngw X-Rspam-User: X-HE-Tag: 1694230649-216476 X-HE-Meta: U2FsdGVkX18xdTIRmmCuBZMmTd3l3WGJeZtOn0A1P6k8j4NxSvQFi2zaQ9AX9HqBbDkFwQ/oZ1CG0fytCx+3E+tFOLNYbYMqnkeQ44A5bqCl4a1pWyvScX1XRO/tzGKw6it2me8QuWdyx5fcse9Nh/hHxcin/vl2OsNyV8kAJWte6VrXrrASk6eTKJYV16sPwaDffUe1tjlz3IMc0lFQaz6AFmkXBESMFSJUgTt6bWtsYdDgh3epq7+RqPF5WmVaIt+FuWGQdzF1Sqv13vR8sOcUgxcfIOq/gvXda8P4S1M1iUrsSCYqvFSjEWc+cWXLAFU86O4iNKL4rwXA4cG20LiZtErAfvXGVLfnyoAg9+nopmn0bl73cKaTNkqF5HwRkT8uz+Q+q2ArD8Sws1IqUKgie1QS1fuSh3emKhTcHUp0KBIo2Imb00o8DfjSAffBpuVQiE2kog/Wf8UHcAawLHYFl0wyu8v/HW3oLWrqSqJNDBbEwpo8SJEWc0JgSfaXUgxAuXhaJQCOjR01Cti2effBGgJlx+0jvoM33gPWqtG6EKPtoikAWLmuNsiY/+2h7e9QP7Q0Mapk0UUaulnBL2mEJyZuiGYCRJgXe9fmSN4W6oszlw0BU30nITE6RHD/siOHqsHe2QA8iZg+etF0DBOOjXxm9cq3/pecqxVRJm8JvhNxpq0dMyoLIOiU59ZNDzRnJh+nzdaGD6ad2G/+0QqTJfPBM+Qrj0Twp+UdcDs9uRML46ntz/DK/tYD4vONLv8ZnkEeUyPV6uE5RZS//crAPY7QzZ7+Y8wyRXmFKXo9Yl02Mgt6oqOTYLdtJAtWkGzVnJN+y88Fi/m+ucAk9b5JelSnpnZsvkcBwigpKSgmA/skY62g966xRNhXPvaxgzBFmOXXCRG1H4TPS7rt/1pcuNGURw/bVMsYOwHqZiXmVgNYQ2esFaDAPNGGLdpJWiO4zp6JO2+fldP/wGr xhEBInB8 eoCuhkvOnZwcX64MJiePxF2RHfGS61ZGPTuc4EJ85nVT7WJO5WrDrzwGQuifvckt56lCUZ2+3+SdX84OdPibJWIAR4ocPolxuu8P+hBBJtSHqauuIAb1lMLNfleHJmOwwL8iArI0ivGd3lG56OJoVk1+RMN3YkwNLJXw1lQUBr62ArZikmA1J2UHs6sR9DUd27JGg3J31K3Xyp47v6FHTiHpoMORUX7YM57jdO+SGmyDuUJD+WmXGI+qLKdUd8w8C9jTjVWE9aBv/MYHrD+exnRlp5uR58iMoEEtMBT5Y3yWHTwpcy0oOvbp/jA/lC1cblZ/lZo1wL+h3BUPh260dlO1QFchecZroB9Cvs9UeZr98z7OPpWWSDRd6e2JiwkImL0U2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let nr_accesses_bp be calculated as a pseudo-moving sum that updated for every sampling interval, using damon_moving_sum(). This is assumed to be useful for cases that the aggregation interval is set quite huge, but the monivoting results need to be collected earlier than next aggregation interval is passed. Signed-off-by: SeongJae Park --- include/linux/damon.h | 12 +++++++++--- mm/damon/core.c | 16 +++++++++++++++- mm/damon/paddr.c | 9 +++++---- mm/damon/vaddr.c | 12 +++++++----- 4 files changed, 36 insertions(+), 13 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 15f24b23c9a0..0fe13482df63 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -40,7 +40,8 @@ struct damon_addr_range { * @ar: The address range of the region. * @sampling_addr: Address of the sample for the next access check. * @nr_accesses: Access frequency of this region. - * @nr_accesses_bp: @nr_accesses in basis point (0.01%). + * @nr_accesses_bp: @nr_accesses in basis point (0.01%) that updated for + * each sampling interval. * @list: List head for siblings. * @age: Age of this region. * @@ -51,7 +52,11 @@ struct damon_addr_range { * damon_update_region_access_rate(). * * @nr_accesses_bp is another representation of @nr_accesses in basis point - * (1 in 10,000) that updated every aggregation interval. + * (1 in 10,000) that updated for every &damon_attrs->sample_interval in a + * manner similar to moving sum. By the algorithm, this value becomes + * @nr_accesses * 10000 for every &struct damon_attrs->aggr_interval. This can + * be used when the aggregation interval is too huge and therefore cannot wait + * for it before getting the access monitoring results. * * @age is initially zero, increased for each aggregation interval, and reset * to zero again if the access frequency is significantly changed. If two @@ -629,7 +634,8 @@ int damon_set_regions(struct damon_target *t, struct damon_addr_range *ranges, unsigned int nr_ranges); unsigned int damon_moving_sum(unsigned int mvsum, unsigned int nomvsum, unsigned int len_window, unsigned int new_value); -void damon_update_region_access_rate(struct damon_region *r, bool accessed); +void damon_update_region_access_rate(struct damon_region *r, bool accessed, + struct damon_attrs *attrs); struct damos_filter *damos_new_filter(enum damos_filter_type type, bool matching); diff --git a/mm/damon/core.c b/mm/damon/core.c index a97026489301..c813407c09b4 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -1639,14 +1639,28 @@ unsigned int damon_moving_sum(unsigned int mvsum, unsigned int nomvsum, * damon_update_region_access_rate() - Update the access rate of a region. * @r: The DAMON region to update for its access check result. * @accessed: Whether the region has accessed during last sampling interval. + * @attrs: The damon_attrs of the DAMON context. * * Update the access rate of a region with the region's last sampling interval * access check result. * * Usually this will be called by &damon_operations->check_accesses callback. */ -void damon_update_region_access_rate(struct damon_region *r, bool accessed) +void damon_update_region_access_rate(struct damon_region *r, bool accessed, + struct damon_attrs *attrs) { + unsigned int len_window = 1; + + /* + * sample_interval can be zero, but cannot be larger than + * aggr_interval, owing to validation of damon_set_attrs(). + */ + if (attrs->sample_interval) + len_window = attrs->aggr_interval / attrs->sample_interval; + r->nr_accesses_bp = damon_moving_sum(r->nr_accesses_bp, + r->last_nr_accesses * 10000, len_window, + accessed ? 10000 : 0); + if (accessed) r->nr_accesses++; } diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 44f21860b555..081e2a325778 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -148,7 +148,8 @@ static bool damon_pa_young(unsigned long paddr, unsigned long *folio_sz) return accessed; } -static void __damon_pa_check_access(struct damon_region *r) +static void __damon_pa_check_access(struct damon_region *r, + struct damon_attrs *attrs) { static unsigned long last_addr; static unsigned long last_folio_sz = PAGE_SIZE; @@ -157,12 +158,12 @@ static void __damon_pa_check_access(struct damon_region *r) /* If the region is in the last checked page, reuse the result */ if (ALIGN_DOWN(last_addr, last_folio_sz) == ALIGN_DOWN(r->sampling_addr, last_folio_sz)) { - damon_update_region_access_rate(r, last_accessed); + damon_update_region_access_rate(r, last_accessed, attrs); return; } last_accessed = damon_pa_young(r->sampling_addr, &last_folio_sz); - damon_update_region_access_rate(r, last_accessed); + damon_update_region_access_rate(r, last_accessed, attrs); last_addr = r->sampling_addr; } @@ -175,7 +176,7 @@ static unsigned int damon_pa_check_accesses(struct damon_ctx *ctx) damon_for_each_target(t, ctx) { damon_for_each_region(r, t) { - __damon_pa_check_access(r); + __damon_pa_check_access(r, &ctx->attrs); max_nr_accesses = max(r->nr_accesses, max_nr_accesses); } } diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index e36303271f9d..af2cb82e1fad 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -557,26 +557,27 @@ static bool damon_va_young(struct mm_struct *mm, unsigned long addr, * r the region to be checked */ static void __damon_va_check_access(struct mm_struct *mm, - struct damon_region *r, bool same_target) + struct damon_region *r, bool same_target, + struct damon_attrs *attrs) { static unsigned long last_addr; static unsigned long last_folio_sz = PAGE_SIZE; static bool last_accessed; if (!mm) { - damon_update_region_access_rate(r, false); + damon_update_region_access_rate(r, false, attrs); return; } /* If the region is in the last checked page, reuse the result */ if (same_target && (ALIGN_DOWN(last_addr, last_folio_sz) == ALIGN_DOWN(r->sampling_addr, last_folio_sz))) { - damon_update_region_access_rate(r, last_accessed); + damon_update_region_access_rate(r, last_accessed, attrs); return; } last_accessed = damon_va_young(mm, r->sampling_addr, &last_folio_sz); - damon_update_region_access_rate(r, last_accessed); + damon_update_region_access_rate(r, last_accessed, attrs); last_addr = r->sampling_addr; } @@ -593,7 +594,8 @@ static unsigned int damon_va_check_accesses(struct damon_ctx *ctx) mm = damon_get_mm(t); same_target = false; damon_for_each_region(r, t) { - __damon_va_check_access(mm, r, same_target); + __damon_va_check_access(mm, r, same_target, + &ctx->attrs); max_nr_accesses = max(r->nr_accesses, max_nr_accesses); same_target = true; } -- 2.25.1