From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63199C54E58 for ; Wed, 20 Mar 2024 07:09:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97D6F6B0092; Wed, 20 Mar 2024 03:09:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 906956B0093; Wed, 20 Mar 2024 03:09:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 780556B0095; Wed, 20 Mar 2024 03:09:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 616D36B0092 for ; Wed, 20 Mar 2024 03:09:18 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 1A787A0E0C for ; Wed, 20 Mar 2024 07:09:18 +0000 (UTC) X-FDA: 81916541196.27.9FDD364 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf01.hostedemail.com (Postfix) with ESMTP id 858EF4000C for ; Wed, 20 Mar 2024 07:09:13 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf01.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710918556; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aDuPWc0weZjb6GlasLdB6yuVXRFvwEXbF25+jDWImS0=; b=VSkWoWI7xrZbncdQyl2VQg1N/d9u+X1K+XKfK8mUiIWKoIHzyU4hpjTLz2NEaLjKB4UWaZ eJHELkyYspOROrqX99aSpBxrJQQiGgEl/IbIWCu4dBinE6pZp5WgbSVIub4Ce09MTl45UB Tsob0IFYQNazgMnFEXwwiqopGrda+n0= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf01.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710918556; a=rsa-sha256; cv=none; b=S8v2ddnB738D+hvixmcirBr7cuBgiwGIxLi7/aU2TAJZBndXdha9uqSHBau9HP4O3PQrVR h1TznG7zZCAK5b9ufUpO2lsciQ/8y3/252rU3/mp/3PP+yTn73MvmtpFN6ntMcZRz6eVcc r/TuAFHInYuF2bdKNj2zTDhXa9CChAA= X-AuditID: a67dfc5b-d85ff70000001748-32-65fa8b94d389 From: Honggyu Kim To: SeongJae Park Cc: damon@lists.linux.dev, linux-mm@kvack.org, akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com, art.jeongseob@gmail.com Subject: Re: [RFC PATCH v2 0/7] DAMON based 2-tier memory management for CXL memory Date: Wed, 20 Mar 2024 16:07:48 +0900 Message-ID: <20240320070754.2165-1-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240318190721.99659-1-sj@kernel.org> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrEIsWRmVeSWpSXmKPExsXC9ZZnoe7U7l+pBtdOcVlM7DGwmLN+DZvF rhshFvcfvGa3+L/3GKPFk/+/WS1O3Gxks+j8vpTF4vKuOWwW99b8Z7U4sv4si8Xms2eYLRYv V7PY1/GAyeLw1zdMFpMvLWCzeDHlDKPFyVmTWSxmH73H7iDssfT0GzaPDU1AYuesu+weLftu sXss2FTq0XLkLavH4j0vmTw2repk89j0aRK7x4kZv1k8dj609HixeSajR2/zOzaPz5vkAvii uGxSUnMyy1KL9O0SuDL2PMkpeJRQcXRJRQNjj1cXIyeHhICJxNHD+xlh7PMnFrOD2GwCahJX Xk5iArFFBBQlzj2+yNrFyMXBLDCDReLwwi6wBmGBYIm5Z/+D2SwCqhLnG3YDFXFw8AqYSdzd LQYxU1Pi8fafYDM5BYwlfm09BGYLCfBIvNoAsZdXQFDi5MwnLCA2s4C8RPPW2cwguyQE9rFL LD3fygQxSFLi4IobLBMY+Wch6ZmFpGcBI9MqRqHMvLLcxMwcE72MyrzMCr3k/NxNjMCYW1b7 J3oH46cLwYcYBTgYlXh4DQp+pgqxJpYVV+YeYpTgYFYS4WXnBgrxpiRWVqUW5ccXleakFh9i lOZgURLnNfpWniIkkJ5YkpqdmlqQWgSTZeLglGpgjJjwUOdywst/blacv7/WX77XMy+YMSKm 2lBC5uedY7f9n3Efeyr7s9Oev/7e8ttuNmUdyzYUNv/2CY/9sifzyWspjx/MDMfO/FK75dp3 4vm5vbzzkzp5dyyQ2jw5/3uIn/d9NenFXDcMdj24Ok0kTfD5UwafdVuzeZSnXH7FePvGF87P x3cU2CuxFGckGmoxFxUnAgCBe6AwtQIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprOIsWRmVeSWpSXmKPExsXCNUNLT3dK969Ug45uS4uJPQYWc9avYbPY dSPE4v6D1+wW//ceY7R48v83q8WJm41sFp1PvjNaHJ57ktWi8/tSFovLu+awWdxb85/V4sj6 sywWm8+eYbZYvFzN4tC156wW+zoeMFkc/vqGyWLypQVsFi+mnGG0ODlrMovF7KP32B3EPJae fsPmsaEJSOycdZfdo2XfLXaPBZtKPVqOvGX1WLznJZPHplWdbB6bPk1i9zgx4zeLx86Hlh4v Ns9k9Ohtfsfm8e22h8fiFx+YPD5vkgsQiOKySUnNySxLLdK3S+DK2PMkp+BRQsXRJRUNjD1e XYycHBICJhLnTyxmB7HZBNQkrrycxARiiwgoSpx7fJG1i5GLg1lgBovE4YVdjCAJYYFgibln /4PZLAKqEucbdgMVcXDwCphJ3N0tBjFTU+Lx9p9gMzkFjCV+bT0EZgsJ8Ei82rAfrJVXQFDi 5MwnLCA2s4C8RPPW2cwTGHlmIUnNQpJawMi0ilEkM68sNzEzx1SvODujMi+zQi85P3cTIzC6 ltX+mbiD8ctl90OMAhyMSjy8BgU/U4VYE8uKK3MPMUpwMCuJ8LJzA4V4UxIrq1KL8uOLSnNS iw8xSnOwKInzeoWnJggJpCeWpGanphakFsFkmTg4pRoYdU/tfbS62Hk9I1/P9OJFllcsJ39k /evDaltdXSZ7QoNtpmfZ5Fn5ujO2nVl6bIq4Cu/9jrMhPg1KK2xWxF5uft3ruDjt/5MyZQmb 1n+cTaVFuXOCee13MkpyrvgXfbemzu+ZorvDHasJX1w/iErs8sw+HHgniu/4fesVrG9FSzwC 0lqfxu5TYinOSDTUYi4qTgQACV31OaoCAAA= X-CFilter-Loop: Reflected X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 858EF4000C X-Stat-Signature: dmqhcedtow6cq3wxfyfdnytca8ixcn8g X-Rspam-User: X-HE-Tag: 1710918553-89962 X-HE-Meta: U2FsdGVkX19IX4ys4SuXEOAOlvjfWCRXLRoBihn3P/E3KVV2Z390Uoof2WwVq4EJHzK/1SsDgNiZtrFJWPNyG9zQVonNo1xtljgVSjun2CE/lX0VxQo5VfLS2shDV1tiqW1RTwBq36YKrHgKm7q2Rx+IZ3bno127ugMPMiw10oXwRXves35/v8UR+VjrwBq8MVVKE7wk6/myUPEAIr26qwZrJ+SxlVv7IIX/NREXPp7n0ACc4oMd0JcVzrwC2GPp44Pn2241Brztj+lgoQVVVvUnAMbLx0zskgBjc5vw4kkxcHCd2r3ShuYPyyf8ZJjyD52PHX5DrsIKPQnEKVAAL4k/d34BHLsTnA7gjiSz3M3hzNnhz7Dd/h2gss08hMiLO94HNM27PBc3CQr7SnuUMvy1veeOyWfWUe6PdAjZD4ylKaNXJVoCJG13dgG0RFCHALAPDsKimt0O73X/WvbetxYoTOc/WVLM5aqxQTSu8f9CEULeVyfNR3kqlj7g0ElymfWW4JqeCINHtIL0hpgp+Q21GXZktkkwe3gfhl9U+Zd7jPu/T9PPPSOI21ittau2nXhJMeRBkKPednOqPqxugieRM9jvZlevBUeBUl+fpDACpqiGHNqE6LRmWHbHDmaZ+/MjME08YCV8G7F1ts1OMvJjcdiUbA7yHWnVC4pURzlT4GGS5fpXhZ1ExV86L/CAgne/UvN7OwT55ANgQYcrl0M4Ex/lU4FVU/ULOTtxLXwxKV4VA+GEIoF4buLpQAT3PdNR/H7zIouGK9KsOVPXZXY/fPmAV3jMdqzbWQ39XR1jyIcZe6QiIWTYu38NS3xASyssCXyei9vL5YQ4W1CaUvnM2YVLn/gBvODC1odDgPsBXTY9qPsCrf+8G4pciJOTf4af5JHqVN7VMJBX/uNyvNsrV/9yqDyoCV3wXLSOlmRorjwkOu+VP8L5k/GYmOAHXai74hrYTld6aa7RW2Y /H3L8Gng ySfJq0XXwJjyrSh33OFRwfUMPF46xlIc0RH4OQFBRxII4RI6uRCiCr3NNtlx40VBtJBIs3VpDWDbgR+WObugS5dy3lqjMhXlo2c9XrM/XwDU96g7Tlz2kUax5pEbtk+GsviZJ8W2b2xzpyJNm0Go4cL3cpkohvKYGi2lLhrCwsiv6XgAxF8hIzzilreRb6YqzZNbbpl1+ZSssikLMIPEXhoLW3k6VirjnnHEybfHxiyOvci9igXTIPgY6PtJ0OSTa/i/85v+fA+puZgo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi SeongJae, On Mon, 18 Mar 2024 12:07:21 -0700 SeongJae Park wrote: > On Mon, 18 Mar 2024 22:27:45 +0900 Honggyu Kim wrote: > > > Hi SeongJae, > > > > On Sun, 17 Mar 2024 08:31:44 -0700 SeongJae Park wrote: > > > Hi Honggyu, > > > > > > On Sun, 17 Mar 2024 17:36:29 +0900 Honggyu Kim wrote: > > > > > > > Hi SeongJae, > > > > > > > > Thanks for the confirmation. I have a few comments on young filter so > > > > please read the inline comments again. > > > > > > > > On Wed, 12 Mar 2024 08:53:00 -0800 SeongJae Park wrote: > > > > > Hi Honggyu, > > > > > > > > > > > > -----Original Message----- > > > > > > > From: SeongJae Park > > > > > > > Sent: Tuesday, March 12, 2024 3:33 AM > > > > > > > To: Honggyu Kim > > > > > > > Cc: SeongJae Park ; kernel_team > > > > > > > Subject: RE: Re: [RFC PATCH v2 0/7] DAMON based 2-tier memory management for CXL memory > > > > > > > > > > > > > > Hi Honggyu, > > > > > > > > > > > > > > On Mon, 11 Mar 2024 12:51:12 +0000 "honggyu.kim@sk.com" wrote: > > > > > > > > > > > > > > > Hi SeongJae, > > > > > > > > > > > > > > > > I've tested it again and found that "young" filter has to be set > > > > > > > > differently as follows. > > > > > > > > - demote action: set "young" filter with "matching" true > > > > > > > > - promote action: set "young" filter with "matching" false > > > > Thinking it again, I feel like "matching" true or false looks quite > > vague to me as a general user. > > > > Instead, I would like to have more meaningful names for "matching" as > > follows. > > > > - matching "true" can be either (filter) "out" or "skip". > > - matching "false" can be either (filter) "in" or "apply". > > I agree the naming could be done much better. And thank you for the nice > suggestions. I have a few concerns, though. I don't think my suggestion is best. I just would like to have more discussion about it. > Firstly, increasing the number of behavioral concepts. DAMOS filter feature > has only single behavior: excluding some types of memory from DAMOS action > target. The "matching" is to provide a flexible way for further specifying the > target to exclude in a bit detail. Without it, we would need non-variant for > each filter type. Comapred to the current terms, the new terms feel like > implying there are two types of behaviors. I think one behavior is easier to > understand than two behaviors, and better match what internal code is doing. > > Secondly, ambiguity in "in" and "apply". To me, the terms sound like _adding_ > something more than _excluding_. I understood that young filter "matching" "false" means apply action only to young pages. Do I misunderstood something here? If not, "apply" means _adding_ or _including_ only the matched pages for action. It looks like you thought about _excluding_ non matched pages here. > I think that might confuse people in some > cases. Actually, I have used the term "filter-out" and "filter-in" on > this and several threads. When saying about "filter-in" scenario, I had to > emphasize the fact that it is not adding something but excluding others. Excluding others also means including matched pages. I think we better focus on what to do only for the matched pages. > I now think that was not a good approach. > > Finally, "apply" sounds a bit deterministic. I think it could be a bit > confusing in some cases such as when using multiple filters in a combined way. > For example, if we have two filters for 1) "apply" a memcg and 2) skip anon > pages, the given DAMOS action will not be applied to anon pages of the memcg. > I think this might be a bit confusing. No objection on this. If so, I think "in" sounds better than "apply". > > > > Internally, the type of "matching" can be boolean, but it'd be better > > for general users have another ways to set it such as "out"/"in" or > > "skip"/"apply" via sysfs interface. I prefer "skip" and "apply" looks > > more intuitive, but I don't have strong objection on "out" and "in" as > > well. > > Unfortunately, DAMON sysfs interface is an ABI that we want to keep stable. Of > course we could make some changes on it if really required. But I'm unsure if > the problem of current naming and benefit of the sugegsted change are big > enough to outweighs the stability risk and additional efforts. I don't ask to change the interface, but just provide another way for the setting. For example, the current "matching" accepts either 1, true, or Y but internally keeps as "true" as a boolean type. $ cd /sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/schemes/0/filters/0 $ echo 1 | tee matching && cat matching 1 Y $ echo true | tee matching && cat matching true Y $ echo Y | tee matching && cat matching Y Y I'm asking if it's okay making "matching" receive "out" or "skip" as follows. $ echo out | tee matching && cat matching out Y $ echo skip | tee matching && cat matching skip Y > Also, DAMON sysfs interface is arguably not for _very_ general users. DAMON > user-space tool is the one for _more_ general users. To quote DAMON usage > document, > > - *DAMON user space tool.* > `This `_ is for privileged people such as > system administrators who want a just-working human-friendly interface. > [...] > - *sysfs interface.* > :ref:`This ` is for privileged user space programmers who > want more optimized use of DAMON. [...] > > If the concept is that confused, I think we could improve the documentation, or > the user space tool. But for DAMON sysfs interface, I think we need more > discussions for getting clear pros/cons that justifies the risk and the effort. If my suggestion is not what you want in sysfs interface, then "damo" can receive these more meaningful names and translate to "true" or "false" when writing to sysfs. > > > > I also feel the filter name "young" is more for developers not for > > general users. I think this can be changed to "accessed" filter > > instead. > > In my humble opinion, "accessed" might be confusing with the term that being > used by DAMON, specifically, the concept of "nr_accesses". I also thought > about using more specific term such as "pg-accessed" or something else, but I > felt it is still unclear or making it too verbose. > > I agree "young" sounds more for developers. But, again, DAMON sysfs is for not > _very_ general users. I worried the developer term is also going to be used for "damo" user space tool as "young" filter. But if you think it's good enough, then I will follow the decision as I also think "accessed" is not the best term for this. > And the term is used commonly on relcaimation part and > LRU, so I think this is not too bad as long as we provide nice documentation or > abstraction from user-space tool. > > > > > The demote and promote filters can be set as follows using these. > > > > - demote action: set "accessed" filter with "matching" to "skip" > > - promote action: set "accessed" filter with "matching" to "apply" > > > > I also think that you can feel this is more complicated so I would like > > to hear how you think about this. > > To my humble brain, this looks intuitive for this use case. But as I also > mentioned above, I worry if this could keep simple and intuitive in complicated > filter usages. > > > > > > > > > > > > > > > > > DAMOS filter is basically for filtering "out" memory regions that matches to > > > > > > > the condition. > > > > Right. In other tools, I see filters are more used as filtering "in" > > rather than filtering "out". I feel this makes me more confused. > > I understand that the word, "filtering", can be used for both, and therefore > can be confused. I was also spending no small times at naming since I was > thinking about both coffee filters and color filters (of photoshop or glasses). > But it turned out that I'm more familiar with coffee filters, and might be same > for DAMON community, since the community is having beer/coffee/tea chat series > ;) (https://lore.kernel.org/damon/20220810225102.124459-1-sj@kernel.org/) Yeah, I thought about filter for including pages for given config as follows. \ / \ / only matched items pass this filter. || But the current DAMOS filter is about excluding pages for given config as follows just like a strainer. ___ /###\ |#####| matched items are excluded via this filter. \###/ --- I think I won't get confused after keeping this difference in mind. > That said, I think we may be able to make this better documented, or add a > layer of abstraction on DAMON user-space tool. > > [...] > > > > > > Yes, I understand "promote non-non-young pages" means "promote young pages". > > > > > > This might be understood as "young pages are NOT filtered out" for promotion > > > > > > but it doesn't mean that "old pages are filtered out" instead. > > > > > > And we just rely hot detection only on DAMOS logics such as access frequency > > > > > > and age. Am I correct? > > > > > > > > > > You're correct. > > > > > > > > Ack. But if it doesn't mean that "old pages are filtered out" instead, > > > > > > It does mean that. Here, filtering is exclusive. Hence, "filter-in a type of > > > pages" means "filter-out pages of other types". At least that's the intention. > > > To quote the documentation > > > (https://docs.kernel.org/mm/damon/design.html#filters), > > > > > > Each filter specifies the type of target memory, and whether it should > > > exclude the memory of the type (filter-out), or all except the memory of > > > the type (filter-in). > > > > Thanks for the correction. > > > > > > then do we really need this filter for promotion? If not, maybe should > > > > we create another "old" filter for promotion? As of now, the promotion > > > > is mostly done inaccurately, but the accurate migration is done at > > > > demotion level. > > > > > > Is this based on your theory? Or, a real behavior that you're seeing from your > > > setup? If this is a real behavior, I think that should be a bug that need to > > > be fixed. > > > > I have observed this in the hot_cold example, but I also found that the > > promotion is done very quickly because the age for promote action is set > > to 0 to max in my json setup. It makes most pages of the region are > > young because there is not enough time for those pages being old. That > > means I was wrong. > [...] > > > > I understand the function name of internal implementation is > > > > "damos_pa_filter_out" so the basic action is filtering out, but the > > > > cgroup filter works in the opposite way for now. > > > > > > Does memcg filter works in the opposite way? I don't think so because > > > __damos_pa_filter_out() sets 'matches' as 'true' only if the the given folio is > > > contained in the given memcg. 'young' filter also simply sets 'matches' as > > > 'true' only if the given folio is young. > > > > > > If it works in the opposite way, it's a bug that need to be fixed. Please let > > > me know if I'm missing something. > > > > No, it was also my misunderstanding. I used to set the matching false > > using my script. > > Thank you for confirming. I understand we found no bug at the moment. > > > To summarize my opinion again, > > 1. I agree the concept and names of DAMOS filters are confusing and not very > intuitive. > 2. However, it's unclear if the problem and the benefit from the suggested new > names are huge enough to take the risk and effort on changing ABI. > 3. We could improve documentation and/or user-space tool. I think improving "damo" can be a good solution. > Thank you again for the suggestion and confirmations to my questions. Likewise, thank you for the explanation in details. Honggyu > > > Thanks, > SJ > > [...]