From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14064C2D0A8 for ; Mon, 28 Sep 2020 10:36:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 70BC1206F7 for ; Mon, 28 Sep 2020 10:36:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="tbJlE3Cc" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70BC1206F7 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E8FD86B0068; Mon, 28 Sep 2020 06:36:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DFB5D6B006C; Mon, 28 Sep 2020 06:36:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D05B96B006E; Mon, 28 Sep 2020 06:36:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id BAB0C6B0068 for ; Mon, 28 Sep 2020 06:36:56 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 7E553181AE863 for ; Mon, 28 Sep 2020 10:36:56 +0000 (UTC) X-FDA: 77312117232.07.camp35_32155bf27180 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id 655741803F9B8 for ; Mon, 28 Sep 2020 10:36:56 +0000 (UTC) X-HE-Tag: camp35_32155bf27180 X-Filterd-Recvd-Size: 12188 Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Mon, 28 Sep 2020 10:36:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1601289416; x=1632825416; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=c/MK2pCSgOMGVOP2p7S4cmlEP6k8s8TkAwQl4lTQDDY=; b=tbJlE3CcVuNa1n7weeepTnRud12OB7gTNHEsVLdebpXLFwvH4VnPr9Ah SEZ3PKs7+s6LkNSRVUvIclyV/bnXtwV8G2ZIJ1V5IK5jf5GUT0mbeMGQF 9rjAV603HQj9ScqTvrydns+NxB1eXORDQCXGS0ox+5l2LluXteVU/sMnj U=; X-IronPort-AV: E=Sophos;i="5.77,313,1596499200"; d="scan'208";a="56638137" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-1a-715bee71.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP; 28 Sep 2020 10:36:56 +0000 Received: from EX13D31EUA001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan3.iad.amazon.com [10.40.163.38]) by email-inbound-relay-1a-715bee71.us-east-1.amazon.com (Postfix) with ESMTPS id AE83DA1DB6; Mon, 28 Sep 2020 10:36:43 +0000 (UTC) Received: from u3f2cd687b01c55.ant.amazon.com (10.43.161.237) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 28 Sep 2020 10:36:25 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 2/5] mm/damon: Separate DAMON schemes application to primitives Date: Mon, 28 Sep 2020 12:35:25 +0200 Message-ID: <20200928103528.4256-3-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200928103528.4256-1-sjpark@amazon.com> References: <20200928103528.4256-1-sjpark@amazon.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.43.161.237] X-ClientProxiedBy: EX13D10UWA003.ant.amazon.com (10.43.160.248) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park DAMON-based operation schemes feature is implemented inside DAMON 'core'. Though the access pattern based schemes target region tracking part makes sense to reside in the 'core', applying the scheme action would better to be reside in the 'primitives', as the work highly depends on the type of the target region. For the reason, this commit moves the part to 'primitives' by adding one more context callback, 'apply_scheme' and implementing it in the reference primitives implementation for the virtual address spaces. Note that this doesn't add the implementation for the physical address space, as it didn't exist before. Nonetheless, the extension for physical space would be easily done in this way in future. Signed-off-by: SeongJae Park --- include/linux/damon.h | 8 +++++ mm/damon/core.c | 65 ++------------------------------------ mm/damon/damon.h | 28 ----------------- mm/damon/primitives.c | 73 ++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 82 insertions(+), 92 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 264958a62c02..505e6261cefa 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -170,6 +170,7 @@ struct damos { * @check_accesses: Checks the access of target regions. * @target_valid: Determine if the target is valid. * @cleanup: Cleans up the context. + * @apply_scheme: Apply a DAMON-based operation scheme. * @sample_cb: Called for each sampling interval. * @aggregate_cb: Called for each aggregation interval. * @@ -193,6 +194,9 @@ struct damos { * monitoring. * @cleanup is called from @kdamond just before its termination. After this * call, only @kdamond_lock and @kdamond will be touched. + * @apply_scheme is called from @kdamond when a region for user provided + * DAMON-based operation scheme is found. It should apply the scheme's action + * to the region. * * @sample_cb and @aggregate_cb are called from @kdamond for each of the * sampling intervals and aggregation intervals, respectively. Therefore, @@ -229,6 +233,8 @@ struct damon_ctx { unsigned int (*check_accesses)(struct damon_ctx *context); bool (*target_valid)(struct damon_target *target); void (*cleanup)(struct damon_ctx *context); + int (*apply_scheme)(struct damon_ctx *context, struct damon_target *t, + struct damon_region *r, struct damos *scheme); void (*sample_cb)(struct damon_ctx *context); void (*aggregate_cb)(struct damon_ctx *context); }; @@ -312,6 +318,8 @@ void kdamond_prepare_vm_access_checks(struct damon_ctx *ctx); unsigned int kdamond_check_vm_accesses(struct damon_ctx *ctx); bool kdamond_vm_target_valid(struct damon_target *t); void kdamond_vm_cleanup(struct damon_ctx *ctx); +int kdamond_vm_apply_scheme(struct damon_ctx *context, struct damon_target *t, + struct damon_region *r, struct damos *scheme); void damon_set_vaddr_primitives(struct damon_ctx *ctx); /* Reference callback implementations for physical memory */ diff --git a/mm/damon/core.c b/mm/damon/core.c index d85ade7b5e23..ba52421a2673 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -701,68 +701,6 @@ static void kdamond_reset_aggregated(struct damon_ctx *c) } } -#ifndef CONFIG_ADVISE_SYSCALLS -static int damos_madvise(struct damon_target *target, struct damon_region *r, - int behavior) -{ - return -EINVAL; -} -#else -static int damos_madvise(struct damon_target *target, struct damon_region *r, - int behavior) -{ - struct task_struct *t; - struct mm_struct *mm; - int ret = -ENOMEM; - - t = damon_get_task_struct(target); - if (!t) - goto out; - mm = damon_get_mm(target); - if (!mm) - goto put_task_out; - - ret = do_madvise(t, mm, PAGE_ALIGN(r->ar.start), - PAGE_ALIGN(r->ar.end - r->ar.start), behavior); - mmput(mm); -put_task_out: - put_task_struct(t); -out: - return ret; -} -#endif /* CONFIG_ADVISE_SYSCALLS */ - -static int damos_do_action(struct damon_target *target, struct damon_region *r, - enum damos_action action) -{ - int madv_action; - - switch (action) { - case DAMOS_WILLNEED: - madv_action = MADV_WILLNEED; - break; - case DAMOS_COLD: - madv_action = MADV_COLD; - break; - case DAMOS_PAGEOUT: - madv_action = MADV_PAGEOUT; - break; - case DAMOS_HUGEPAGE: - madv_action = MADV_HUGEPAGE; - break; - case DAMOS_NOHUGEPAGE: - madv_action = MADV_NOHUGEPAGE; - break; - case DAMOS_STAT: - return 0; - default: - pr_warn("Wrong action %d\n", action); - return -EINVAL; - } - - return damos_madvise(target, r, madv_action); -} - static void damon_do_apply_schemes(struct damon_ctx *c, struct damon_target *t, struct damon_region *r) @@ -781,7 +719,8 @@ static void damon_do_apply_schemes(struct damon_ctx *c, continue; s->stat_count++; s->stat_sz += sz; - damos_do_action(t, r, s->action); + if (c->apply_scheme) + c->apply_scheme(c, t, r, s); if (s->action != DAMOS_STAT) r->age = 0; } diff --git a/mm/damon/damon.h b/mm/damon/damon.h index fc565fff4953..4315dadcca8a 100644 --- a/mm/damon/damon.h +++ b/mm/damon/damon.h @@ -5,31 +5,3 @@ /* Get a random number in [l, r) */ #define damon_rand(l, r) (l + prandom_u32() % (r - l)) - -/* - * 't->id' should be the pointer to the relevant 'struct pid' having reference - * count. Caller must put the returned task, unless it is NULL. - */ -#define damon_get_task_struct(t) \ - (get_pid_task((struct pid *)t->id, PIDTYPE_PID)) - -/* - * Get the mm_struct of the given target - * - * Caller _must_ put the mm_struct after use, unless it is NULL. - * - * Returns the mm_struct of the target on success, NULL on failure - */ -static inline struct mm_struct *damon_get_mm(struct damon_target *t) -{ - struct task_struct *task; - struct mm_struct *mm; - - task = damon_get_task_struct(t); - if (!task) - return NULL; - - mm = get_task_mm(task); - put_task_struct(task); - return mm; -} diff --git a/mm/damon/primitives.c b/mm/damon/primitives.c index d7796cbffbd8..e762dc8a5f2e 100644 --- a/mm/damon/primitives.c +++ b/mm/damon/primitives.c @@ -38,8 +38,11 @@ #endif /* - * Functions for the initial monitoring target regions construction + * 't->id' should be the pointer to the relevant 'struct pid' having reference + * count. Caller must put the returned task, unless it is NULL. */ +#define damon_get_task_struct(t) \ + (get_pid_task((struct pid *)t->id, PIDTYPE_PID)) /* * Get the mm_struct of the given target @@ -62,6 +65,10 @@ struct mm_struct *damon_get_mm(struct damon_target *t) return mm; } +/* + * Functions for the initial monitoring target regions construction + */ + /* * Size-evenly split a region into 'nr_pieces' small regions * @@ -788,6 +795,68 @@ void kdamond_vm_cleanup(struct damon_ctx *ctx) } } +#ifndef CONFIG_ADVISE_SYSCALLS +static int damos_madvise(struct damon_target *target, struct damon_region *r, + int behavior) +{ + return -EINVAL; +} +#else +static int damos_madvise(struct damon_target *target, struct damon_region *r, + int behavior) +{ + struct task_struct *t; + struct mm_struct *mm; + int ret = -ENOMEM; + + t = damon_get_task_struct(target); + if (!t) + goto out; + mm = damon_get_mm(target); + if (!mm) + goto put_task_out; + + ret = do_madvise(t, mm, PAGE_ALIGN(r->ar.start), + PAGE_ALIGN(r->ar.end - r->ar.start), behavior); + mmput(mm); +put_task_out: + put_task_struct(t); +out: + return ret; +} +#endif /* CONFIG_ADVISE_SYSCALLS */ + +int kdamond_vm_apply_scheme(struct damon_ctx *ctx, struct damon_target *t, + struct damon_region *r, struct damos *scheme) +{ + int madv_action; + + switch (scheme->action) { + case DAMOS_WILLNEED: + madv_action = MADV_WILLNEED; + break; + case DAMOS_COLD: + madv_action = MADV_COLD; + break; + case DAMOS_PAGEOUT: + madv_action = MADV_PAGEOUT; + break; + case DAMOS_HUGEPAGE: + madv_action = MADV_HUGEPAGE; + break; + case DAMOS_NOHUGEPAGE: + madv_action = MADV_NOHUGEPAGE; + break; + case DAMOS_STAT: + return 0; + default: + pr_warn("Wrong action %d\n", scheme->action); + return -EINVAL; + } + + return damos_madvise(t, r, madv_action); +} + void damon_set_vaddr_primitives(struct damon_ctx *ctx) { ctx->init_target_regions = kdamond_init_vm_regions; @@ -796,6 +865,7 @@ void damon_set_vaddr_primitives(struct damon_ctx *ctx) ctx->check_accesses = kdamond_check_vm_accesses; ctx->target_valid = kdamond_vm_target_valid; ctx->cleanup = kdamond_vm_cleanup; + ctx->apply_scheme = kdamond_vm_apply_scheme; } void damon_set_paddr_primitives(struct damon_ctx *ctx) @@ -806,6 +876,7 @@ void damon_set_paddr_primitives(struct damon_ctx *ctx) ctx->check_accesses = kdamond_check_phys_accesses; ctx->target_valid = NULL; ctx->cleanup = NULL; + ctx->apply_scheme = NULL; } #include "primitives-test.h" -- 2.17.1