From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D48B6C25B74 for ; Fri, 24 May 2024 05:24:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 313366B007B; Fri, 24 May 2024 01:24:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C37A6B0083; Fri, 24 May 2024 01:24:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 18D246B0085; Fri, 24 May 2024 01:24:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id EF5A36B007B for ; Fri, 24 May 2024 01:24:37 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 691D04082A for ; Fri, 24 May 2024 05:24:37 +0000 (UTC) X-FDA: 82152149394.17.E187E48 Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) by imf13.hostedemail.com (Postfix) with ESMTP id C68B620003 for ; Fri, 24 May 2024 05:24:34 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=ajou.ac.kr header.s=google header.b=qp8TGMWt; dmarc=pass (policy=reject) header.from=ajou.ac.kr; spf=pass (imf13.hostedemail.com: domain of tome01@ajou.ac.kr designates 209.85.215.179 as permitted sender) smtp.mailfrom=tome01@ajou.ac.kr ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716528275; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dIV5RE5g4iad/Y9kEiDw/i3UIuirpCFKtvdMSetkisA=; b=l0UWpM8qPSO4hC++ufFJo3rWCYoT4az6oEInMh0Mi91f2uDrA+Y/KO70Yi10eUznbSarDT 6wXvmm4h+4ahsp33Bkicn+gWpOyPlf8N0XCdH1Q7CtASB2PyRSv0Cn0XMX0HVR5Vq+y/Yo hkAozxLSXsQWKxS34oIvosV5tpoJBXI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716528275; a=rsa-sha256; cv=none; b=k4rTc71qpyT6ThsYpaAIOZOJLaLQ1pydWYKQ1u6EjGt5PkjWzNvnn/DAiEZ4QfCY/N/EQJ +8Klv8gvbicM66xvBVk+jYa7UthIvwnoQ6f2D+TDDWfiS4zfrMsSYdGUm3KZjSpxJkG19P 5yzg1yDHb4FBDkglbCD4B2f7epmL+2Q= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=ajou.ac.kr header.s=google header.b=qp8TGMWt; dmarc=pass (policy=reject) header.from=ajou.ac.kr; spf=pass (imf13.hostedemail.com: domain of tome01@ajou.ac.kr designates 209.85.215.179 as permitted sender) smtp.mailfrom=tome01@ajou.ac.kr Received: by mail-pg1-f179.google.com with SMTP id 41be03b00d2f7-60585faa69fso1836396a12.1 for ; Thu, 23 May 2024 22:24:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ajou.ac.kr; s=google; t=1716528273; x=1717133073; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=dIV5RE5g4iad/Y9kEiDw/i3UIuirpCFKtvdMSetkisA=; b=qp8TGMWtkqtEYJ0/OF5JgTYE3muDxZ9NGKdfKe+Pw2GL/1ELA/105oeSMUzhIG9y1U AKGG57BluXLJXs167KahFl/YnnTXy2+rZFnfzbCbmUyP+qwsLzCCnDLNC2+8cxgoSJf8 CYLRUD0siCfDnX3gmx+ZLJhRGQlOJ68xGtOj8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716528273; x=1717133073; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dIV5RE5g4iad/Y9kEiDw/i3UIuirpCFKtvdMSetkisA=; b=sVwQUlD+8NgyxKOAvX/n7HubdBea+6a6zf7W5Vyb3SxFQdws+HNx4a3TMZ/gPDGYRs PuGkgyFaDv0um47VJu2uztp06TgKCdFbkZKhUTCLaHML6+O6CI7BAsuct67HxabGYQzZ xLyn6OIb+MMIaTn6VTk9aIok6+CxxSt6ka5fKA7N6snNvqRtgzuUTNR5ZCYLa713nV3a kx+TpftMtqWOJgpTgx1eNqvdH4Y9hbki73PW5OV6Rk+TCNQEHlzxzLeMUUkE3OSb64H0 YUz2MLFek0nlE9vApAkDdQbuDcXhne5HgUQGYUASEhms2asfxYnDyPF1/6e3BwFntuI2 tDKw== X-Forwarded-Encrypted: i=1; AJvYcCUbU1Uqu/9XG5kgEn9ONphiCxSGR45AUgZlvkRMhCKxMHCwfXeuBAT2YxwMagGXwyWwH5McBZbCl9rigxL/LVcYZx4= X-Gm-Message-State: AOJu0YxtsTLZOlmxzq354tAqdUqzPAMmRRJL9TYvtbM4HN+OoN+M001y qFTZQpKT6dEciQ0bF47TRumAbIC58886u9hyqyde8Byxg3nmyC9CGHrRTpn/QaM= X-Google-Smtp-Source: AGHT+IGC0OeejHOahOcls4+0SSq+j0qs0QDYYujqimaD7K7PmQNUXYf5rNmSGHvF6o4treKv5WkUvQ== X-Received: by 2002:a17:902:d4c4:b0:1f4:5072:e094 with SMTP id d9443c01a7336-1f45072e339mr7589545ad.9.1716528272832; Thu, 23 May 2024 22:24:32 -0700 (PDT) Received: from swarm08 ([210.107.197.32]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f44c9b87b4sm5121745ad.264.2024.05.23.22.24.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 May 2024 22:24:32 -0700 (PDT) Date: Fri, 24 May 2024 14:24:28 +0900 From: Jonghyeon Kim To: SeongJae Park Cc: damon@lists.linux.dev, linux-mm@kvack.org, Andrew Morton , linux-kernel@vger.kernel.org Subject: Re: [RFC PATCH v2 0/3] Add NUMA-aware DAMOS watermarks Message-ID: <20240524052428.GA368050@swarm08> References: <20240520143038.189061-1-tome01@ajou.ac.kr> <20240522010034.79165-1-sj@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20240522010034.79165-1-sj@kernel.org> X-Stat-Signature: 8zxu7h34z95e9btsxhbckoqb3z6u78r6 X-Rspamd-Queue-Id: C68B620003 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1716528274-622032 X-HE-Meta: U2FsdGVkX1+cXS9zmt/1h7yHWhSmGz4PZmYnH3Zbg08sNNk90RzFrgghS9k728DAtArnYK3rxq6ROhQNQGElP2IRwpSBzCj6ECl7zvkidGPVgxTF3XOWeZFAZHn3JSYMpCbTxLbuR81zRAQRJDBp+ypZg6C1ZqQIFYdq3X8PeyAsHMH61xhVO/OqP8WmFnh1QhkUxHWGaWM4hqClpFs1bk+rXGA9jIavzxZ4wGdRY5pLUANif8gB9Tgd+NwfbDOHtskQ7vzOS9Xmb/RXdFswDSVgnEWEyzleOE9c2cXGdNE688QyglJnlDUu+/t92jveHUGc4gNSz013JH8YtCdJojvFd+QjqUREX0TUqP2Lfd1ReB4FXHn55QHcucrrw3dD3cErL6h65YWb2EMl/1OkF7jRmZ1akNMau9EDr21I3WSQJ4WIFwprikSyxsXuDFHTuXGLCU8ZAF7+0WPsyH7obIL/0th40jsPf/tzSA+CDA+eYSdc3YQEDnWoq9bh6q7vUm3LMVCo/2Qvrb3c3kyP9xv+nzbocoi5MpJIE9KfiHJSPhoW8g1OvXS3u/XesHuyLCPjkoimpxuOONzFJ1SvibyzCVGsRpLK7LcO1WHelysNB+/3JRe25v899d6mTAIt5ANv2dkYUfW/DezRGXl829c0N5Wjxr9HIZlYT4BIHO0pmFHCwalBCZvPWBLLrrSVPAf/+XSmdq9frzgKhHX7KusiblIXXDBkHLUjqfKjvZ8Y8LjoKl7xSdiYbGiJOYplBr61UdGLkemNwI3A/sq86BpX14mus0vhGKTSo1jA4TAXBv5kFxAkVbrxL6ZEajCDzAL4MfEm8FkVOymsBWEad3hkFhr0OMyOiCF5qcwHKS8R4OQ9IKQtKrQuMbDuAuhNs0eV+eav5dX2Yna+SU2eDU5zNhXQEHyVmF3yKpvdqboQEC+p2cZd16IWv8NA8fS0pSraAHdpbBQPPtXpdaT R5j81MYO xBhoVnyDgtouNNY3Ru0J69WR2v5Yw9xt/Ajwzaxy2lGWRqRpElaORKSlf7/+efPHpZERkyxn6RzZnc94= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, May 21, 2024 at 06:00:34PM -0700, SeongJae Park wrote: > Hi Jonghyeon, > Hi, SeongJae > On Mon, 20 May 2024 14:30:35 +0000 Jonghyeon Kim wrote: > > > Current DAMOS schemes are not considered with multiple NUMA memory nodes. > > For example, If we want to proactively reclaim memory of a one NUMA node, > > DAMON_RECLAIM has to wake up kdamond before kswapd does reclaim memory. > > However, since the DAMON watermarks are based on not a one NUMA memory > > node but total system free memory, kdamond is not waked up before invoking > > memory reclamation from kswapd of the target node. > > > > These patches allow for DAMON to select monitoring target either total > > memory or a specific NUMA memory node. > > I feel such usage could exist, but my humble brain is not clearly imagining > such realistic usage. If you could further clarify the exampected usage, it > would be very helpful for me to better understand the intention and pros/cons > of this patchset. Especially, I'm wondering why they would want to use the > watermark feature, rather than manually checking the metric and turning DAMON > on/off, or feeding the metric as a quota tuning goal. > The goal of this patchset is to manage each NUMA memory node individually through DAMON. Also, the main target scheme is memory reclaim (or demotion in tiered memory). By allowing DAMON to be managed by each NUMA node, I expect that users can easily set up memory reclaim for each node. Additionally, I think that a watermark for each node is an appropriate metric for activating DAMON_RECLAIM, because the kswapd reclaim logic also follows a watermark of free memory for each node. There are two use cases. Let's assume two NUMA nodes are constructed of 32GB (node0) and 16GB (node1), respectively. The first case is when using DAMON module. If users do not specify a monitoring region, DAMON's module finds the biggest size of the two NUMA memory nodes and designates it as the monitoring region (node0, 32GB). Even if we want to enable DAMON_RECLAIM to node0, it does not work proactively because the watermark works based on the total system memory (48GB). Similarly, if the users want to enable DAMON_RECLAIM to node1, users have to manually designate the monitoring region as the address of node1. Nonetheless, since DAMON still follows the default watermark (total memory, 48GB), proactive reclaim will not work properly. Below is an example. # echo Y > /sys/module/damon_reclaim/parameters/enabled # cat /sys/module/damon_reclaim/parameters/monitor_region_start 4294967296 # 0x100000000 # cat /sys/module/damon_reclaim/parameters/monitor_region_end 36507222015 # 0x87fffffff # dmesg | grep node ... [0.012812] Early memory node ranges [0.012813] node 0: [mem 0x0000000000001000-0x000000000009ffff] [0.012815] node 0: [mem 0x0000000000100000-0x000000005e22dfff] [0.012817] node 0: [mem 0x000000005e62c000-0x0000000069835fff] [0.012818] node 0: [mem 0x000000006f2d3000-0x000000006f7fffff] [0.012819] node 0: [mem 0x0000000100000000-0x000000087fffffff] < target [0.012825] node 1: [mem 0x0000002800000000-0x0000002bffffffff] ... When we use DAMON_RECLAIM by default, DAMON_RECLAIM targets node0 memory (32GB). However, DAMON runs differently from the initial goal because the watermark works based on the combined node0 and node1(48GB). DAMON_LRU_SORT also faces the same situation. The second case is when we apply DAMON to a process. If a process allocates memory that exceeds a single NUMA node(node0), some users could want to reclaim the cold memory of the process in that node. In my humble opinion, the reclaim scheme(DAMOS_PAGEOUT) is effective in this case. Unlike the DAMON module, since DAMON monitors process memory using a virtual address, it is hard to decide whether to enable a DAMOS_PAGEOUT due to a lack of node memory stats. Even though we use watermarks for DAMOS_PAGEOUT, it works the same with the above module case (thresholds based on total memory, 48GB). To overcome this problem, I think the dedicated watermark (for node0) can be an answer. > > > > --- > > Changes from RFC PATCH v1 > > (https://lore.kernel.org/all/20220218102611.31895-1-tome01@ajou.ac.kr) > > - Add new metric type for NUMA node, DAMOS_WMARK_NODE_FREE_MEM_RATE > > - Drop commit about damon_start() > > - Support DAMON_LRU_SORT > > > > Jonghyeon Kim (3): > > mm/damon: Add new metric type and target node for watermark > > mm/damon: add module parameters for NUMA system > > mm/damon: add NUMA-awareness to DAMON modules > > Following up to the above question, why they would want to use DAMON modules > rather than manually controlling DAMON via DAMON sysfs interface? IMHO, some users want to use DAMON feature without mannualy configurating via DAMON sysfs due to complexity. Since this patchset can be adopted to sysfs interface, I will update supporting NUMA-aware watermarks for sysfs interface in the next version. Best Regards, Jonghyeon > > > Thanks, > SJ > > > > > include/linux/damon.h | 11 +++++++++-- > > mm/damon/core.c | 11 ++++++++--- > > mm/damon/lru_sort.c | 14 ++++++++++++++ > > mm/damon/modules-common.h | 4 +++- > > mm/damon/reclaim.c | 14 ++++++++++++++ > > mm/damon/sysfs-schemes.c | 35 +++++++++++++++++++++++++++++++++-- > > 6 files changed, 81 insertions(+), 8 deletions(-) > > > > -- > > 2.34.1