From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37E93C02180 for ; Tue, 14 Jan 2025 03:06:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF8766B0085; Mon, 13 Jan 2025 22:06:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A8BA76B0089; Mon, 13 Jan 2025 22:06:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 921F26B008A; Mon, 13 Jan 2025 22:06:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7366C6B0085 for ; Mon, 13 Jan 2025 22:06:15 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E1EB11404EE for ; Tue, 14 Jan 2025 03:06:14 +0000 (UTC) X-FDA: 83004568668.08.B8DD553 Received: from mail-qv1-f42.google.com (mail-qv1-f42.google.com [209.85.219.42]) by imf21.hostedemail.com (Postfix) with ESMTP id F3FAA1C0004 for ; Tue, 14 Jan 2025 03:06:12 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b="cAVj5Z/F"; spf=pass (imf21.hostedemail.com: domain of gourry@gourry.net designates 209.85.219.42 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736823973; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HtUZDx9l37Yn313XZC7GTvlIbUPEDAdSGSYveecq8vs=; b=IWxZwAuoeCTwFyaTgL+XIfIYy81UCp2ks56LwUqpBR7qqwJ6icQ04WjDqhFr6fQP7wbovs 62GIhu+m+3AbsBFg/WDQEOxpg949+jQEqL2jhFQybJuosQle2HzAyWTcsIls11gxPgB8N4 OA7OHiOgdlVtWbvm/ZpE3YoRNImEPmU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736823973; a=rsa-sha256; cv=none; b=Fu4lpgR9riUQ/gR6IuhkZs0kg8UWCXE8eaCw45AYTPTF+33vWYAdJgSR+7Ojh9az6VTs8p FAr7pzqdrg/58WVfedIYBHfqBIuPhu+0HCpIg6A+/Q0MS6pLL7tT0r1l5RaTSZVVA4PVVJ Hx/m/fyaB5LNyeS/3vfztjctKtugSc0= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b="cAVj5Z/F"; spf=pass (imf21.hostedemail.com: domain of gourry@gourry.net designates 209.85.219.42 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none Received: by mail-qv1-f42.google.com with SMTP id 6a1803df08f44-6d8edad9932so34877946d6.0 for ; Mon, 13 Jan 2025 19:06:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1736823972; x=1737428772; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=HtUZDx9l37Yn313XZC7GTvlIbUPEDAdSGSYveecq8vs=; b=cAVj5Z/Fj4qKwlgnP2YBIMdW277dzZFCvKCAU39vhmUTtybXDK8ibKLYP4FlHPKccQ f+Yz/sa3/T+eO2wbY0bfpTm9dh2i785kq9RBje7SFdD9ftTOLXezyfj4cCskdf0pNu7h QwllZ0nKC4IPe7/6vnZET3Yl16vWYjqp+fjFdq3NROHcg8dhKftczXXdKqduI4q7FW6v oCuZvtw/I8kB2FleG9P3AAzB6In+mmUAx8Rn24Sz0twwjZIeAEChRWbQzFABuN7nm0Ek SSck0Xir1Bo33FuUFUir/3jYDmQJB+YQjYljk9ZLGg8G8/yhds9pwoZzBKnrif5Nm4+/ qxBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736823972; x=1737428772; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=HtUZDx9l37Yn313XZC7GTvlIbUPEDAdSGSYveecq8vs=; b=LD9eQYaYfNZNDqf6JDbIa4pNPtW/7cxV5mvO01LOvjV16OKX2kfMXun+MNvkAtBWVO CI/x3YZjfue7Gw6eGLMJ0xPuwaf902vQvxjnS53p668cw/p3rh7BsE+y9cmRHUPBrM/v lLSyjTIV1JE+W74remPXt58d0z/d4jz86TYbFAeha6xwE18ZY23ZN0A8HM3L8/7fnx6t 4WIl1wynB/bHvcPuSN0nQ0xC6itiSYuk+RR+3Ifdm1Jo4X5eqrGBYjoxir58D8IP0Dw4 vTtj/KfcFCX8JU6HN9B1wIrVJSQaRlDHzqPmVdHraRixe5hnKmowXM39fu+K0cDKnvUE g/KQ== X-Forwarded-Encrypted: i=1; AJvYcCVRlpuZYqXWj1laDVPrigvPJP2Xx6I/IYO2GPlyrYJpE7vEDXk1JG8+UzM/fnXS8E3M3JfXbTHwew==@kvack.org X-Gm-Message-State: AOJu0YxpoFKJVA9c/uisDmmwttk0/xxmLY2HuuQYf5APbYrZOzzFzN5i iumwhYWmXtzYR1MPWm//33b5vnexsTdq/2wfqsD6NjldocG4Y+SP2TS0CQHdjCQ= X-Gm-Gg: ASbGncsvc6WkO9UVbaZYb/PXzJXDFLpD4FJwN1QAJvvcApT47YcxrEBecQE1ZRkTmLf gIZ2SyqzmoNXUG+amfJjqzrqtG9UemxnbvE/x65h09eUirmi4AkT0enfIrvMGBaqeoOF5V/fvr7 CKLBL+2yfMqmuUISVv90arK4p8mih7RUkajgVQiWN8iQ/RihKJJgLbYrP3XZcSVWIhHvLUWG2dx ZrPXes2GJA4R3YqROzwHXYICKInXWPtucG5BHBramBxEUcSgGgIeZ7tTc7lOCMrpllELoNhoAvP iejiUphM0n2SWdOE1U3sJp0YJtHq2e3d4/csGRQ= X-Google-Smtp-Source: AGHT+IEWuwaWcprp4Gpl+J4gz4gbL/TitiB33tFONeErcmRD00ZkT3QMNRoDGqF+c3sisNy1TFSHKQ== X-Received: by 2002:a05:6214:2f91:b0:6d8:a027:9077 with SMTP id 6a1803df08f44-6df9b1b8fe6mr375286116d6.5.1736823972082; Mon, 13 Jan 2025 19:06:12 -0800 (PST) Received: from gourry-fedora-PF4VCD3F (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6dfade7327csm48211256d6.69.2025.01.13.19.06.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jan 2025 19:06:11 -0800 (PST) Date: Mon, 13 Jan 2025 22:06:09 -0500 From: Gregory Price To: SeongJae Park Cc: lsf-pc@lists.linux-foundation.org, damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Raghavendra K T , Yuanchu Xie , Jonathan Cameron , Kaiyang Zhao , Jiaming Yan , Honggyu Kim Subject: Re: [LSF/MM/BPF TOPIC] DAMON Requirements for Access-aware MM of Future Message-ID: References: <20250101222039.74565-1-sj@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250101222039.74565-1-sj@kernel.org> X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: F3FAA1C0004 X-Stat-Signature: s46usritp4m3r5rfr4g5jp3a3hicai46 X-Rspam-User: X-HE-Tag: 1736823972-260873 X-HE-Meta: U2FsdGVkX19Y1ympgWdy8eD6jHvGe5PVZlit2+GTdEu33nPyuogkLu2+5uBAw9zg3UfGV8OxiOEdYFUSgjzDMyopyFObx0Artt4Gvp/VPMC8yWxPPskgDRMJS1KHXX1ksIRCKrk4h50/Nm6LQCCO/JYso637YRo4Kl2ltwG651parlQhOPbYQTWxCYxATiDOnC9yS4dYB4LtgMJ7lGabvJlCEp1LSUnJzJ0LI5BCtjI18eFnrQSgOtgI4L/4be7qC1Iv5UyYOSws8Pvw4jEPwaAoa7YXWk7rdFW00LaY1ImMeDif1E35rotOYEkbgh9wI/Dl0KaH23iljFkFENszrg9E/fxrHYDqGBnPv3TClyejswSLHI2IuWIIZ5S4ge4L7ekU/gvU4CFK01ox4j7zj+7xB3q8t9TurCI0B3Z6VWb9yZyev0QD0TLeFqaMR9+GJNHqN0PF8ujMHuEDqpY1cBD3ZQLkcMlvj4INHadHSSrHO4P4LLrHc/rd92xxqHywPIVDuYswyqIdRMhJkfXRKeZyTn5xcFirl0lqVWUzTHkHAlu0//GqhH+tB64ZFl2LGZDmgwz03JX/J3iPK3zSPhvTxQ4Jdabsqe7IFpk1/I1WTzo5LVZWT40sBq6FHkosXkYujt7LmLpsrXFIkZLLT6bJ+RExEJCKgat//uyUO1IMIYdR18eOdH3lgmgx0tPBFi0B/wttyYv3YACk/hKNaSyr+o8RtwTBel5rySpCWBJaShGhKytbvIDTTNv/HKyrS4kI3eQnZdFMXG7Ekzhk0uBh6tJNWRPXh/Kla07VPFAc7mlN/H4XOTVSezhv8frCeT7yRkC6GFDPx8rzlzjtXqE5nCEUwFObIxGvXfvDDSvWrCQBQGtagFEMsfph0amOAdE3XlEOlsyO72IwQb9VmW3oUAB6h0V99/Q4zWFLLIdIf5eBgNMZNge15YGuGVQCxrS03xeu1gLgFRmJd8U rNEyFWkY EOPv76p5+rwdr3/6GozN5b3nQXzOfO94oc7Jis/4OTbo8WV1Kz1Lhc+uAontUV3VQFwfANetee6+AFssnEhbGArixOeun7+ujA/BnYdKVOEG+CqNbqwQW+Lt8PPp0du7NsA/SCIqTsrhNU183SBV05aBBdQsz2VABHnLRJUAz2dS1dZw9ANl6gJUx/QmUlW+xAfn7vx9BTc/olEpg9AeXFtauI99cdBdplL2O95ObQUr8JT45bDLHcibZvo+ulGUkKffYG8zWYgrznvPabS5M+3qoE4yOUeyIxx99AkxvjE6dsq8J+aQDf349BmrYCMyk/mfwCrb2/4/WTEPp4PwUAxl3mzEr+RXaQO32gDAl0Jnn4ueaDgwVNf+snDqtgA4defPuj8sUunzQiQNbaxeNRQ2KLhECG8JA2k1N X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jan 01, 2025 at 02:20:39PM -0800, SeongJae Park wrote: > Hi all, > > > I find a few interesting and promising projects that aim to do efficient access > pattern-aware memory management of near future, including below (alphabetically > sorted). > > - Promotion of unmapped page cache folios > (https://lore.kernel.org/20241210213744.2968-1-gourry@gourry.net) I'll break down a few observations I made while hacking on unmapped page cache promotion - and my concerns for a leveraging DAMON here. Additionally some other concerns I've seen raised about duplicating promotion logic across various kernel components. Latest RFC: https://lore.kernel.org/linux-mm/20250107000346.1338481-1-gourry@gourry.net/ Basic Premise: Use folio_mark_accessed() as a measure of hotness for promotion. Defer promotion to task_work due to locking complexities. My major concerns / lessons learned from this exercise include: 1) The cost of checking promotion candidacy can be problematic In my microbenchmark in the last RFC version, I showed that while the performance upside (~22-25%) is substantial, there was a non-trivial cost associated with injecting even a single global boolean check in the file_read() path. This was unexpected. I can probably optimize the disabled case with a likely() clause, but I did not expect such sensitivity. This tells me injecting an unconditional call into DAMON may be too much overhead. I would need to explore this further - including whether it is feasible to inject such a large dependency into swap.c This may not affect all cases, but it does affect at least this one. 2) The complexity of "when it is safe" to promote a folio is subtle at best, and "actively hostile" at worst. I learned in v1 of the RFC that promotion inline with fma() is not feasible due to a few contexts (task dying in particular) in which migration is not safe. I deferred to task work because I noticed prior attempts (in development notes) had seen similar issues. Adding a folio reference and/or page flag to defer that migration to another context (i.g. async kthread) solves this at the expensive of implementation complexity. (leaked folios if done wrong) I'd have to look at whether it's worth the increased complexity to aggregate this (particular) identification mechanism - but I think there is clear value to aggregating promotion. I could see some value in pumping tracking bits into DAMON - but I also see value is making tasks handle promotion as a form of fairness. 3) There were expressed opinions on runtime fairness WRT to promotion. There's two competing thoughts: A) Making accessing tasks eat inline promotion cost captures that cost in their runtime slice, promoting fairness in scheduling. B) Aggregating promotion to an external thread can reduce inline faults and tail latencies, but may hides per-task cost. This is a concern if one task drives all the promotions, effectingly stealing an entire core by nature of the async design. I don't have a good answer to this, just an observation that charging promotion time to the identifying task was a concern that was raised. 4) TPP and Unmapped Page Promotion may affect each other. There is a rate-limiting mechanism in the migration path that was intended to prevent over-pressuring bandwidth with aggressive migrations - prevent major memory stalls. By adding more pressure on this limit from an additional source, we're obviously increasing the time it takes to converge. This is probably the greatest argument for creating a new, aggregated promotion mechanism to serve all of these identification mechanism. This would make it easier for us to determine whether/what identification mechanisms can be aggregated while enabling forward progress on each of them separately. 5) Scarce resources We need to be careful not to consume excessive amounts of resources in an attempt to track all these identifying mechanisms. Even 1 byte per folio is 256MB on a 1TB machine. This gets out of hand quick. With task-work, I was able to add no additional resource consumption, but deferring to a fully async scenario and needing to track things like last-accessing CPU, timestamps, and etc. We'll need to examine this closely if we decide to aggregate either of these mechanisms. ~Gregory