From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BE6CC02181 for ; Fri, 24 Jan 2025 17:21:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B3D4A280080; Fri, 24 Jan 2025 12:21:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AEE95280079; Fri, 24 Jan 2025 12:21:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 98D90280080; Fri, 24 Jan 2025 12:21:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7ACA5280079 for ; Fri, 24 Jan 2025 12:21:37 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id D423F1A15E1 for ; Fri, 24 Jan 2025 17:21:36 +0000 (UTC) X-FDA: 83043012192.29.8732B41 Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by imf10.hostedemail.com (Postfix) with ESMTP id CEB56C0018 for ; Fri, 24 Jan 2025 17:21:34 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=IiVg4RLG; spf=pass (imf10.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.181 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737739295; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JpBMlffQq/AmVURDmJqZIAknLMu8Pltvl96VGpNVI6w=; b=4xzqY5i6uN53zHDsLB5Zu04QWjy+9irsLzbwfLDUWTfkc48fyCGTfpMkJHCazl/oDs/iUI Wj5voAKRCSUnemZcY3oFvyvjAxvfCkszaN/hlOETno15mjP3Ghna3uW9Z1I5CtntLA8ozi Bcf3A078bF2eaYTub9qyGhiBv36xaJM= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=IiVg4RLG; spf=pass (imf10.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.181 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737739295; a=rsa-sha256; cv=none; b=ikeHqAjtb12blpInfvJQ/PHgOEhUXJkHfI+MGbtoLra0shLa3zfrVlkIi2kcUZypEgz2eM W2xbl01Kgl7afRupTIPW35c3cZNf+Yowrh42sIetV7ckI57xOdD2M9w9LLegsOSPu2dfZF aTQMy7FgHJy9j0f/7c6Napq4X0i8Mz0= Received: by mail-qk1-f181.google.com with SMTP id af79cd13be357-7b6ed9ed5b9so330198785a.2 for ; Fri, 24 Jan 2025 09:21:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1737739294; x=1738344094; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=JpBMlffQq/AmVURDmJqZIAknLMu8Pltvl96VGpNVI6w=; b=IiVg4RLGzFaCLfu3opVb/Xelr8+BChV/1ZlQtIJfPXXqHOJ9otF0N2YbTkjzv8XAOn FnXooyQ8Xjrhc/E0hCeAk75KaWNmCrS0XNh1d76yxUv0HHQdOGt3xXLktJZUrOjJoMub R3ODUY1v8Wt9syYXF/dtbGV1Squ7nJXvEAgXeemuhpnVhog/LixLNnt1/OQz0b86AG3n fBr8m6QK5pJdMp7ZNcrz05rSAJWKv1NWl4qnV9Ho/GrG5jr5WiEkw+QQe+1fezPkuQ0P LyxLHgRu2ZTHrEWFFU+yZYz4lzS1whH9SeIpXZ9VjduglShIH8uzEt8VBknk2IhOHlju /vjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737739294; x=1738344094; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=JpBMlffQq/AmVURDmJqZIAknLMu8Pltvl96VGpNVI6w=; b=juJnXuniTccmxCnZ/sUqurITQ1x2LES/DgpUrF8Zp7q+bo4S4iETA2DOMjn8synGaP 11XaQ4XD7qtxEVhNnE4i0Wi1/iIzy7eq9ZK3vt+Cip8hlgVifDNNtEms8sANfoLPOlXi 54ZrVs5oHP+jf4UFf+bn65WTK7pbyfGeVN0xKyOmIxiuhvu36+qLOEzn+k4hxp5rlc8i nqbrGzYFItG5bnoGV+NzEefqukqbxlDF3j+w3N1AqnN0erfNijvh2Yca2ZwjG5fR+S/G 9O7UtO9sbbGAUP+H9m2kpdT4wdg77K9SDbominHlXmFd1QiBC2Tc1Q8x1cGsm9a4TMt2 ODRg== X-Forwarded-Encrypted: i=1; AJvYcCXTeqlof8WliSeG5MvcvbM+1Z/B4hg4jdeHJZ06LjhxvlpexAzdKkdL6t2cZ0zBdnJnXW/PBKXkjA==@kvack.org X-Gm-Message-State: AOJu0YzZ5rUD4+xWdboedq4vlVYisclaJFoOfvr+DPKEDsuaDys08dFE PaIaaGOpdBMnHzupi1myr9RuFbmEl2hrUuQu7Y/AJxfg2T36aLur3KZbqKfs2lY= X-Gm-Gg: ASbGnctrACo1HtWmBl3cDIVwOQ/iLR3cfTPyluGOqmsf6lcGiZkPLQq0X0/MI+iEYZo CUYoVEwv+Jna9O+M/WBwnCctSPB6OtasbcK6ENabk4w2FlOLfoQu6UJzj2o8ZCxwsdFv+EVyPZj 94S24+q8lCqzX82CB1uGlDDFJOug0F2fIW/cJbUorFQdWtdSbiCnwxjrhWzwa4K5jYsBJusHjlk 6ZhNlA0e89BLY4mHzxZsEdNsLDv/IrCIHp/g6WPyVL8OI4qNSa7TCNzOBGHRg3+TZQBB9okQdSF 651QGMtj1KwEXje26zuvIMRhChTmL++F0LUt7LbY8bZNcRyk/R5QDioJyDSAo8w= X-Google-Smtp-Source: AGHT+IH3fhXb3bOkGIW4xZiUVxwtmeBpzr0Mn/GhdnkWq3CbqtmVe+oX1U58/Sxdrz8CjUkPr8Greg== X-Received: by 2002:a05:620a:24c8:b0:7b6:eab3:cdd1 with SMTP id af79cd13be357-7be6325dc77mr5522744185a.49.1737739293908; Fri, 24 Jan 2025 09:21:33 -0800 (PST) Received: from gourry-fedora-PF4VCD3F (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7be9af0d1efsm112515585a.101.2025.01.24.09.21.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jan 2025 09:21:33 -0800 (PST) Date: Fri, 24 Jan 2025 12:21:31 -0500 From: Gregory Price To: SeongJae Park Cc: lsf-pc@lists.linux-foundation.org, damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Raghavendra K T , Yuanchu Xie , Jonathan Cameron , Kaiyang Zhao , Jiaming Yan , Honggyu Kim Subject: Re: [LSF/MM/BPF TOPIC] DAMON Requirements for Access-aware MM of Future Message-ID: References: <20250124021153.103622-1-sj@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250124021153.103622-1-sj@kernel.org> X-Rspamd-Queue-Id: CEB56C0018 X-Stat-Signature: n45gjmidsxmj69f6r3o9prg5t6tcgmrr X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1737739294-244065 X-HE-Meta: U2FsdGVkX1/K+X3YhJ+mE63S70WVjH6U8i1dvFonPGV+lonTREofNz9DVOtAwtlXIkCwGTzGYJGB+zmErglQALGGzgaQ97HeCcB8CurPewXldroz+vE5h7nWIe2QTiCUyU3fucdbsJKmHWqiNaf9Jxl6tlTNeY2uKcnrHwQAj0XA0LsglI+TEDjt1YoF3VBi3JdzKGsK1zMwJVKbxe0jvFBoou6ournmleBGB7AHiqlHgskJ+xld2/3WXecgY9iZGpsPSPRrSB73JAfu0EwGSCxhJ6o3Ca3DyYIDdneQwxUlQfXK7e5g153xYJQ9+NbXGTt7fex0xeyCDlO1ep/RA4WgqI1iumk7gNJF7JJj8NzxJqXbAlg4Jx6OtuWKnwkMx4TEmdmD7nqps4oLvAMTsvk0hyWcZiIFiO7T2Uk0KSkWkpjkMMFFjgriHlopAWBBQPLaj4dSxY5gPlH6LUjtzcTCcoN+8iOqQmSYfCLgDOYpGpGN5r3Ri73EdYaFgHVc52Y68R0mfiAaWNGLYN9jZ3CfMFuUJ6uxCP1yyYhCcRfzmSjqV+slJx/GZbM98s2IBoHDF+Ssift+nWCq1Y09l9N+3UT7p+Glfx6WhL31aYwW6m5Q48eb+5pR6iAnBI3n/TINY0+9JG8P6CxhRlnwDqrUbB9rOFuL48anJ9FPR9TmoyQCjx4pVcZeO+N3qetL2vPlKhSRPrAMNu/lmG5l39ydkghMmff6VFidpvgFnmE7SEeSSzj96QEkWxoN5sgr4Pi1LLnspJdhBsQAtljK8NA3XBgusEb4AC5fM07dNXiycoLPv2/15+AKUux3Hy3gkDK2RyL9+2puJ96ZSUocQfPZ9LB4UpzpLDS8usA+RUIG5s15RL6dUWSOh3YL2mNy/VDmD23H9ZrG8xgGsNv8PG0HSjZMrm38+lxfx7s7dsyWRPX3mVr9lwEOLdX8N8b5kuIbB8tgtBebBq5+lmw Rc9xr4lx PUh37uwDnExaFVfua/RtCLcLCKRJbp8aBXtebSad9PlhTqR/pxRK9Odk7PIFmtFrRM22sktFirfrOraqCmbruZeVg5XYcJRu/dcY1fX3710uvzNOumKx8iAPj6xOiTsyonJD5SqmkCrO98AubtgKnfWn4kUWg+yGSR33PpVUJoDK1zLdaf16wRseGwRikqaybbywxv3zTRo7D6a4TyxKJGbd7k06qFzHvhaDCUGJ2lDS0vIL+WjEpQ8Svz7jZcHgU0dxeIcUsvjjfV2HJmWPRxjNpbi6ssm0klvEe4RfcXqsppPshAXR2eSrrLKfiLmeWp8z0MUh5RzEhKzE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jan 23, 2025 at 06:11:53PM -0800, SeongJae Park wrote: > Hello Gregory, > > > 1) The cost of checking promotion candidacy can be problematic > > > > In my microbenchmark in the last RFC version, I showed that while > > the performance upside (~22-25%) is substantial, there was a > > non-trivial cost associated with injecting even a single global > > boolean check in the file_read() path. This was unexpected. > > > > I can probably optimize the disabled case with a likely() clause, > > but I did not expect such sensitivity. This tells me injecting > > an unconditional call into DAMON may be too much overhead. > > I cannot agree more with you about the point that the mechanism for finding the > promotion/demotion (and any access-aware system operation) candidates should > induce only modest or at least controllable overhead. Actually it was the one > of biggest motivations of DAMON design, and I haven't imagined adding > unconditional calls to DAMON here. > > Nonetheless, injecting an unconditional call here should be avoided for not > only DAMON calls but any expensive calls? I'm also not pretty sure what DAMON > call you are thinking about. > Just any call, DAMON or otherwise. The explicit check injecting ~2-3% overhead on my microbench was a simple + } else if (!folio_test_isolated(folio) && + (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && If this is causing additional overhead, call me skeptical that trying anything more complicated will turn out better. > > > > I would need to explore this further - including whether it is > > feasible to inject such a large dependency into swap.c > > I understand DAMON is not small in terms of the code size, and has many > limitations that makes it unusable in many use cases. But, again, I'm not > pretty sure what kind of DAMON usage in swap.c you're thinking about, and > therefore not easy to understyand what part of DAMON is considered as a large > dependency that concerns you. It would be great if we can make more concrete > example as a result of this topic session at LSFMMBPF. > It's not a matter of code size - it's a matter of tightly coupling core components of the kernel to extraneous ones. Adding additional dependencies between components increases overall system complexity and makes it hard to reason about the behavior of the system. For example, in the prior snippet: + } else if (!folio_test_isolated(folio) && + (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && + numa_pagecache_promotion_enabled) { + promotion_candidate(folio); This amounts to if (some condition && feature enabled) mark a folio as a candidate for promotion The promotion_candidate() function is contained within migrate.c and uses (mostly) migrate.c mechanisms (from a task_work). All you need to understand the behavior is between swap.c and migrate.c. If instead you aggregate this to DAMON, understanding the behavior of swap can require you to understand what DAMON is actually doing with this information. Now you need to understand swap.c, migrate.c, AND DAMON. It makes it more difficult to reason about the system when something goes wrong. This increases the maintenance burden for maintainers (and onboarding complexity for anyone new to the kernel, for that matter). That doesn't mean we shouldn't consider doing this - it just means that benefit needs to outweight the complexity/maintenance cost. > FYI, I also not having specific idea for helping unmapped pages promotion for > now. That's my assignment that I will do by LSFMMBPF. But, a few things that > I naively thinking DAMON might be able to help unmapped promotions are, > > 1. Using DAMON for profiling how much hot and cold unmapped pages are in which > tier, and use the information for unmapped pages promotion optimization. > 2. Using DAMOS to target-promote hot unmapped pages while using page > faults-based promotion for mapped pages. > 3. Using DAMOS to promote both mapped and unmapped hot pages. > This missing the scenario where DAMOS/DAMON is not suitible for deployment in someone's environment. The kernel should still do *something*. And that is kind of the point - we can expose more complexity to the users with DAMON, but the kernel should be able to do some reasonable promotion action without this additional system. > > but I > > also see value is making tasks handle promotion as a form of fairness. > > I agree that could be good in terms of fairness. I want to learn more about > the significance of it, though. > Fairness in this scenario is simple. If one task is causing an outsizes number of promotions to occur, and it causes some ASYNC system to handle those promotions, it is effectively acquiring more CPU time via that ASYNC system than other residents. Trying to charge this time back to the noisey task is harder than just having the task incur the cost of migration. But doing it inline can cause the task to slow down. So it's difficult to predict how it's going to pan out. Need evidence. > I agree. DAMON allows combining multiple different mechanisms with its core > logic, so I beleive it migt be a place that can aggregate the different > identification mechanisms. > > DAMON's access monitoring results based system operations feature, namely > DAMOS, also has its own aggressiveness control logic, and resides in the core > layer, so could be used consistently with different promotion candidates > identification mechanisms. > Without data this is a nice thought, but we have existing mechanisms that work and can be improved - lets not disrupt that. Finding an aggregated promotion solution helps everyone move forward without disrupting development in these areas (and makes the different indentification mechanisms play nice with each other). Trying to also create a voltron "one indentification system to rule them all" is a nice thought, but it's heavy-weight compared to adding a folio flag check and a call to mpol_migrate_misplaced(). We need to respect that reality and not regress the existing mechanisms by trying to over-engineer a generalized solution. ~Gregory