From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 983CCC7115B for ; Fri, 20 Jun 2025 21:47:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D80326B0088; Fri, 20 Jun 2025 17:47:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D30D06B0089; Fri, 20 Jun 2025 17:47:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C1F5E6B008A; Fri, 20 Jun 2025 17:47:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id AF1B36B0088 for ; Fri, 20 Jun 2025 17:47:42 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 990F781016 for ; Fri, 20 Jun 2025 21:47:41 +0000 (UTC) X-FDA: 83577116322.17.8B3F995 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) by imf25.hostedemail.com (Postfix) with ESMTP id D1926A0005 for ; Fri, 20 Jun 2025 21:47:39 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=cN2rgX+v; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf25.hostedemail.com: domain of bijan311@gmail.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=bijan311@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750456059; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4/b8SGEJQFlDd4fd5rMq8B6RnUjovzraV/pGDKTi6mA=; b=X7Mez0vb/CqkrMjsACDlI0e3xtJADb+FdDia1UT2nMLrIyCgRUczgdzAexqM6q2k/CitKm xaJoGkVJQIXb/lxJs8QoW+FGueNv4T4A++ygJ6J8qpojU9zT+kojZSxAo+dWF3C3KT7qiG 7ams6QwtzrH2cJbvVj9klx3xAlg0GSI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750456060; a=rsa-sha256; cv=none; b=A6MA60SudVzlZqt5fEC51+n6lAHOPlll6mpJ12UTzi6mhestBa20VyR+FG2iujT9DDV8dw 7R2EP8RJP1e0KT4kPI7LXMsSt5ASIRvKtFeFiweYBHR5ueOV55gj+3qyD+AXeWqKukGlRm J5ApNgWGmhp09eKjKscISLXZeT9auSc= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=cN2rgX+v; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf25.hostedemail.com: domain of bijan311@gmail.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=bijan311@gmail.com Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-ad1b94382b8so431303566b.0 for ; Fri, 20 Jun 2025 14:47:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1750456058; x=1751060858; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=4/b8SGEJQFlDd4fd5rMq8B6RnUjovzraV/pGDKTi6mA=; b=cN2rgX+vDZbN2TylmdjbTVAUcLtzdk8zpy5nysbBjcTndOdMQdcr1IaV1WkcfwwVxG A5GfhNWxz4iyrAJh62mcVwwdS4Rlxnd6Wv0OynRnsewz9lVrjtaGi79xuTn7vesz0XN+ ENXrmvUE0t36sn3Z9HEO4jxVqZzG1x31As7xnV41cU+1a5e35Y7jrSDTr7RSfisZyD8v GyuC94kNKySsahHW+L4BcCpa1vLuluJj3JVwqtTS2fs46CfhqYSdVnSnyvBrRmYnlvI6 xFYPU6A1a5BXheWzWdRWyWMd6qm7MMdOyhAkZMqmJ2ooOkJsFZ6O7mDrh//MrcEJWRwk l7lA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750456058; x=1751060858; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4/b8SGEJQFlDd4fd5rMq8B6RnUjovzraV/pGDKTi6mA=; b=SRAvR20sLF91I7xuXwIXipqo4SGcT+6v7k1MhYgCvtV1Chx8KdotuxF95OS14anOd8 MGgUbiKA0cgDhi3KHQg8Z9dS2+MNU8vlHh3WgJK78lQnphkbtA4zQmjWEDIFj8zSiH6b QaIZxinwc2uPJNB1RrXendCC9fjcwaYyzOLeK7jRquNnhCHVu3G41IZYkWaKijqhuH+n F6V1wgWtfi82dIRkNBTqzUZi0LlYkXuYdjQ5xGJPX3Ya2PbvgT/+ZHouuTEc9maZmq+E KI9vOCCnfjiz0a/tCxQ7qLMECzdwe4Wg9ZRPFG+6gLWzNccXt0ljG3SVAgEK0jCyiJJx dANQ== X-Forwarded-Encrypted: i=1; AJvYcCXPLALZx9jbjYw6Nqi6A3wNRkBRMxZV7EdjEfphr1WzGu+AS/uWVrzSKRWib3R1y7YT4NGdcJ20BQ==@kvack.org X-Gm-Message-State: AOJu0YzJP/zee8USUyaWB/LBW8B/4SN4/bgmE07ssWvJ/74eIZ/2JMvv RuzEeE3a+k5NH0ykoBrW4ZjNavuMOTmMdW89gqRWXOx/zJsb+NC20fRJB4Q1r3e7XNpcAS8OZE6 bWmjKFo/cfAwt7Vpm8xNYcsQ033q6NXQ= X-Gm-Gg: ASbGncssz4vufS4FGBAJyxBRMSjs25UHrnLj0QQuu6U/o6ipdwuS7rbBe2pOkEH16g4 aXtQoV2CUnlReP64fecu7eB5Gfy4sLKQKWrRhzT0HAjATNOxhdjpQLUf5w2NRT3YBjk6WewEnhC dVOmN/o7BIwDI9T04qlqZflzj2rccbI5ink6mw2CVJzxgsswZ9hu6ZxHtWhbfqxGkrCpii5a4Zl 0l02FeGNqEUTcQ5 X-Google-Smtp-Source: AGHT+IFBMnvlkmoY7H6omzg7UDAqSYXQfxSICjnkdbKgo7nV5h7TFBh+q5W8gVbHtCvl4GGytB5w6tjyIEcHnyR3jsA= X-Received: by 2002:a17:907:7289:b0:add:f191:d851 with SMTP id a640c23a62f3a-ae057b60511mr397434566b.32.1750456057834; Fri, 20 Jun 2025 14:47:37 -0700 (PDT) MIME-Version: 1.0 References: <20250620180458.5041-1-bijan311@gmail.com> <20250620202155.98021-1-sj@kernel.org> In-Reply-To: <20250620202155.98021-1-sj@kernel.org> From: Bijan Tabatabai Date: Fri, 20 Jun 2025 16:47:26 -0500 X-Gm-Features: AX0GCFswQvDPvHTlBd2hX3fSa1Acsc4GALQizBGHTv_3IJ0fB0gZkfA3pdqdjFg Message-ID: Subject: Re: [RFC PATCH v2 0/2] mm/damon/paddr: Allow interleaving in migrate_{hot,cold} actions To: SeongJae Park Cc: damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, david@redhat.com, ziy@nvidia.com, matthew.brost@intel.com, joshua.hahnjy@gmail.com, rakie.kim@sk.com, byungchul@sk.com, gourry@gourry.net, ying.huang@linux.alibaba.com, apopple@nvidia.com, bijantabatab@micron.com, venkataravis@micron.com, emirakhur@micron.com, ajayjoshi@micron.com, vtavarespetr@micron.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: D1926A0005 X-Stat-Signature: zhciu5rmqrtk8cdmn5td95o5heesb8rk X-HE-Tag: 1750456059-392473 X-HE-Meta: U2FsdGVkX19hL2fxTvjLTdX0uN/v3+DK4iPXPc0gGoBuG5Vnz6kibwLS8p4DkA2hPIj0qc3YGtNh+jA2jSgzXl1ZF8QQLeuGyzhI7jQMt0rIOp4b3bjCIbgLjeo3tQzHdlB7PAoe+CSBlEXAh3p4satM3nvXbkRm5j8gXZis3DbFIDYi+YzBMYT3HD3eW/48fsxmcbgY8xkoZsDeUXX7Z8QPnUTjTa//zB62A/5302gkpXA17SKjPiiZ63I+B5LTTooWuiQ30oMMQpHe1teUmgBoswJfRl23OZb7I5F3MXLYtlr4IqessCyiX7f1hCc0U8D5p1B9GJZKMbTtnv6r9F8ourmIW9FYe61H39qfoFUIQfbZiUsZwI4ayGRXTI7UY103JNWbkQ9Mkwn6DjcD3SsS0+j8xU6iG3MUpTmbq3pfSoCZhGHntQiatY8kE9pxSr1vl0KbHEJREbnQwCfntK96DpDtMiZ5ivaoYlStp0icK/5/JCquVCYHv7SWKLhLFJku0KteYoD/VjZpte+DPSEWX/RjJD0WJi8625bD28UcEHInhqxP/cv9z+qNSvj5D9Ng6iNPsg4vjIdhXeVjCi4UhtDAG9Mu69CW3oHj1v4tZeNW9cPJlOk3nbjcoR0/9FnjePxGoN2KLZbkhSbOHEasxv3DVv0QHxQX3eyKrR2gROygXJtLd78R8LduPOXwL4QnmrxLtqPBD9fkiLjlOVhlFuAFpdmB17WpqpoLo5dT/RVcHw4gb2jY8wMbfmI7JMAmMstUgJRmldjjKNwiVXr/EBXLYfIPQfANAOc69rE6cBU5XZKUZaBuFzKeI8qghrZDMFZCxr2nZZ62+HZfEVMio7x678yrCG7nHI9UsNJy9hIprc5rxboA6TED4Td5tHNr0nWIS9+sFGWR51PojNS82BrLOSTzBq7CAdJqsxLHZVc/GbvED8oBmGshSPnEH6OIuwwfGEAHM9BpJuZ DDKza3ZS TH0pumzspSny/MmKIdpAziQlL2LevvGJdWetddvoBBvRFYc+VJ3yMvdXT+BQYsm/Vkgq1HLKfYyR/n9X0ja+e0Q7lEhs52KcllUOH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi SeongJae, On Fri, Jun 20, 2025 at 3:21=E2=80=AFPM SeongJae Park wrote= : > > Hi Bijan, > > On Fri, 20 Jun 2025 13:04:56 -0500 Bijan Tabatabai w= rote: > > [...] > > As a toy example, imagine some application that uses 75% of the local > > bandwidth. Assuming sufficient capacity, when running alone, we want to > > keep that application's data in local memory. However, if a second > > instance of that application begins, using the same amount of bandwidth= , > > it would be best to interleave the data of both processes to alleviate = the > > bandwidth pressure from the local node. Likewise, when one of the proce= sses > > ends, the data should be moves back to local memory. > > > > We imagine there would be a userspace application that would monitor sy= stem > > performance characteristics, such as bandwidth utilization or memory ac= cess > > latency, and uses that information to tune the interleave weights. Othe= rs > > seem to have come to a similar conclusion in previous discussions [3]. > > We are currently working on a userspace program that does this, but it = is > > not quite ready to be published yet. > > So, at least in this toy example, we have user-space control. Then, I th= ink we > could decouple DAMON and weighted interleaving, and ask the usr-space too= l to > be the connection between those. That is, extend DAMOS_MIGRATE_{HOT,COLD= } to > let users specify migration target nodes and their weights. And ask the > user-space tool to periodically read weighted interleaving parameters tha= t > could be auto-tuned, and update DAMOS_MIGRATE_{HOT,COLD} parameters > accordingly. Actually the user-space tool on this example is making the > weights by itself, so this should be easy work to do? > > Also, even for general use case, I think such user-space intervention is = not > too much request. Please let me know if I'm wrong. You are correct. The userspace tool would be coming up with the weights, so it would not be hard for it to write those weights to two places. I coupled the weights used in DAMON and weighted interleaving for this revision and the previous because I could not think of a use case where you would want to use different weights for allocation time and migration, so it felt silly to have two different places with the same data. However, I don't feel too strongly about this, so I'm willing to defer to your judgement. Also, our userspace tool updates these weights somewhat frequently, several times per minute, when it detects a change in the bandwidth utilization of the system to calibrate the interleave ratio. I am concerned about how frequent changes to the scheme via the sysfs interface will affect the effectiveness of DAMON's page sampling. From what I understand, updates to the sysfs aren't saved until the user writes to some sysfs file to commit them, then the damon context is recreated from scratch. Would this throw away all the previous sampling work done and work splitting and merging regions? I am not super familiar with how the sysfs interface interacts with the rest of the system, so this concern might be entirely unfounded, but I would appreciate some clarification here. [...] > > > > Questions for Reviewers > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > 1. Are you happy with the changes to the DAMON sysfs interface? > > I'm happy with it for RFC level implementation. And in my opinion, you n= ow > proved this is a good idea. For next steps toward mainline landing, I'd = like > to suggest below interface change. > > Let's allow users specify DAMOS_MIGRATE_{HOT,COLD} target nodes and weigh= ts > using only DAMON interface. And let the user-space tool do the synchroni= zation > with weighted interleaving or other required works. > > This may require writing not small amount of code, especially for DAMON s= ysfs > interface. I think it is doable, though. If you don't mind, I'd like to > quickly make a prototype and share with you. > > What do you think? That sounds good to me! Having a prototype from you for the sysfs interface would certainly be helpful, but if you're busy, I can take a pass at it as well. > > 2. Setting an interleave weight to 0 is currently not allowed. This mak= es > > sense when the weights are only used for allocation. Does it make se= nse > > to allow 0 weights now? > > I have no opinion, and would like to let mempolicy folks make voices. Bu= t if > we go on the decoupling approach as I suggested above, we can do this > discussion in a separate thread :) > > [...] > > Revision History > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > Changes from v1 > > (https://lore.kernel.org/linux-mm/20250612181330.31236-1-bijan311@gmail= .com/) > > - Reuse migrate_{hot,cold} actions instead of creating a new action > > - Remove vaddr implementation > > - Remove most of the use of mempolicy, instead duplicate the interleave > > logic and access interleave weights directly > > - Write more about the use case in the cover letter > > - Write about why DAMON was used for this in the cover letter > > - Add correctness test to the cover letter > > - Add performance test > > Again, thank you for revisioning. Please bear in mind with me at next st= eps. > I believe this work is very promising. Thank you for your help and feedback! Bijan [...]