From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6931FC27C4F for ; Fri, 14 Jun 2024 03:11:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EAC006B0093; Thu, 13 Jun 2024 23:06:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E34296B0098; Thu, 13 Jun 2024 23:06:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C87BA6B0099; Thu, 13 Jun 2024 23:06:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A73676B0093 for ; Thu, 13 Jun 2024 23:06:04 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 5365512040B for ; Fri, 14 Jun 2024 03:06:04 +0000 (UTC) X-FDA: 82228005048.17.EDAB3CA Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf01.hostedemail.com (Postfix) with ESMTP id 52DB64000A for ; Fri, 14 Jun 2024 03:06:00 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718334361; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O9RCl36Sr/T3L9aY/kSc8muE2yIiWIe9X6hTnOc3ADw=; b=CAjf3zXa0Pyghkg/iKYE+r7+I5FXVo8EMoQ4pL5tHr7ABaEDAhXS5vrc6ncGBkkpIE04Ou 7QDNRADmtFv12r4n4awRjT9EQGRjXMNLVHVOyYlKNBfM/Y8VpI60cZIQ/lXLzhanj4ohPW +vDFFdwALVRFP/HkNfvhIW77Ka/kyR4= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718334361; a=rsa-sha256; cv=none; b=UFaSz5JuD8TISV/L/vYaYSjMJ6c5mP34/xuKQdooAuxPfGNp7j+3CtwnaIFvfvloOnOnSc PacxXigqO98at6xqDU/sW6Zk5yGfxguCCbq5NJu2P5iF3UaMRfs7FUcfdgs2LiD4mVaRFe ukSuGNtlAWrQfe7A90p/LsYO5jlmwBI= X-AuditID: a67dfc5b-d6dff70000001748-61-666bb3973391 From: Honggyu Kim To: SeongJae Park Cc: damon@lists.linux.dev, Andrew Morton , Masami Hiramatsu , Mathieu Desnoyers , Steven Rostedt , Gregory Price , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, 42.hyeyoo@gmail.com, art.jeongseob@gmail.com, kernel_team@skhynix.com, Hyeongtak Ji , Rakie Kim , Yunjeong Mun , Honggyu Kim Subject: Re: [PATCH v5 0/8] DAMON based tiered memory management for CXL memory Date: Fri, 14 Jun 2024 12:05:51 +0900 Message-ID: <20240614030554.784-1-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240613174604.63629-1-sj@kernel.org> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrDLMWRmVeSWpSXmKPExsXC9ZZnoe70zdlpBvc6LSwm9hhYzFm/hs3i /oPX7BZP/v9mtWhoesRicXnXHDaLe2v+s1ocWX+WxWLz2TPMFouXq1ns63jAZHH46xsmBx6P paffsHnsnHWX3aNl3y12j02rOtk8Nn2axO5xYsZvFo8Xm2cyemz8+J/d4/MmuQDOKC6blNSc zLLUIn27BK6Mow0/GQtuK1csPHGNvYHxumQXIyeHhICJxJSmh2ww9v45u5lBbDYBNYkrLycx gdgiAooS5x5fZO1i5OJgFtjKInGj8wQjSEJYwF/i6ZU9YEUsAqoSD+bOYAexeQVMJfZMbGOB GKop8Xj7T7A4p4CxxPs3J8DqhQR4JF5t2M8IUS8ocXLmE7B6ZgF5ieats5lBlkkI/GeT+LFp GyPEIEmJgytusExg5J+FpGcWkp4FjEyrGIUy88pyEzNzTPQyKvMyK/SS83M3MQLjYFntn+gd jJ8uBB9iFOBgVOLh9XiWlSbEmlhWXJl7iFGCg1lJhHfWQqAQb0piZVVqUX58UWlOavEhRmkO FiVxXqNv5SlCAumJJanZqakFqUUwWSYOTqkGRsdb/Mua+I/qPX0jfX6f3vE9/r8Tvku5bs9O kDqlydRoa+wTUNjvzCpzZ5rF6zJbPv05wcuqBGYzzlJmjRaLPnHzXtW+p4liv6ZOyxI5viX/ jeLDG6GzTVfeYnqVGDTv6NTadsv59//+MF91gkf33JTcY4zB27YneCd1x5SfmKRdp+KzQJXl lxJLcUaioRZzUXEiAO8NfpR/AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNLMWRmVeSWpSXmKPExsXCNUNLT3f65uw0g4aVKhYTewws5qxfw2Zx /8Frdosn/3+zWjQ0PWKx+PzsNbNF55PvjBaH555ktbi8aw6bxb01/1ktjqw/y2Kx+ewZZovF y9UsDl17zmqxr+MBk8Xhr2+YLH5vW8HmIOix9PQbNo+ds+6ye7Tsu8XusWlVJ5vHpk+T2D1O zPjN4vFi80xGj40f/7N7fLvt4bH4xQcmj8+b5AK4o7hsUlJzMstSi/TtErgyjjb8ZCy4rVyx 8MQ19gbG65JdjJwcEgImEvvn7GYGsdkE1CSuvJzEBGKLCChKnHt8kbWLkYuDWWAri8SNzhOM IAlhAX+Jp1f2gBWxCKhKPJg7gx3E5hUwldgzsY0FYqimxOPtP8HinALGEu/fnACrFxLgkXi1 YT8jRL2gxMmZT8DqmQXkJZq3zmaewMgzC0lqFpLUAkamVYwimXlluYmZOaZ6xdkZlXmZFXrJ +bmbGIFRsKz2z8QdjF8uux9iFOBgVOLh9XiWlSbEmlhWXJl7iFGCg1lJhHfWQqAQb0piZVVq UX58UWlOavEhRmkOFiVxXq/w1AQhgfTEktTs1NSC1CKYLBMHp1QDo7Hsyr3RO4/+b14k+nCV 11N2own3TbXuGfjN2J9WuWpdckwqf35YUc1s91tWck2Na76Eb/u0aYsJ2+XKFP6z70ymGudk 3TDiSZr+Z+LWCY9DXgc7LNsTJbF69ZSd5isufdJyV/+y6XWWeeoDkxcHevbvYc+Y71u3vsM8 WeKIX3vmHdu4vc1zJimxFGckGmoxFxUnAgCvQf2+fgIAAA== X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 52DB64000A X-Stat-Signature: qzwuwdhy4wyg6xm83dtckcqd3t1if41q X-HE-Tag: 1718334360-857354 X-HE-Meta: U2FsdGVkX18yHUmRhQQPmrGhkcm44QWfz4quEK/ZbNloSD9HjXFOuc66R3Uy3G9n1pofc2H5X+Wrqr21GeydjvYvaPx+RVvCyJk3L+71rDWt8cZ18X3V0xP8Cvc1uBSy8+7Khr8wSlG7beDXyV4G1Ky+s5KII0yt/54ZQKO/fLwZotk0bLSxQ8S8+zlHh3BgSeQF9FRu2uf/IocmRzY+HcGdE3E7UjVr5TLNNqpZBuBdbD2ZsQs8dDcd8mEGMf2IeoU8Jtwtv8or6y4Oz3F+ZYyG/mvjtVg0xBKLlS6tGtPVSbff8bVHrtKGZNJY94QocAomFGBKav6g5x9M1DlfySSAhhAalvWKXZ9Ky6ndp6gYQJRX6ls8vX+cVyvWNSLpS8qz0SrGqprd2w/1O4zfzRX2xcpE7pGz12KpWDuSTU6zQOZ39vfbFYsZja3LU8GIdIaZaaJn2JmKIWgBZ+EOYbxogXTEsFOfpqEWmhfvAcgflaqemg61e9qlQZ/o1vD2CyEkifBj9HQf80+0iW6nPZXpvLtr+6TduO8kVh0OgeOvcwT6n5itzV3FQ1d2npRedruUCi65fjG2dPJIrO4oKiz53pOCiHOeQmOYg8736SbvEhw/IqW1B6YzAFHO17g3b/9XZ2hLZbV1AfRYPdiwz8Rgg60cuLPrLOBwwmBrUueYQX+AsuAunmNanuspPOALrjYRMbtjdKg9IjfvQqTIUjfpuynqcy0r68N/O8oeoFHUmejfCo/vi6Uz76JPiPKa5rQVTWLgEZJcC3peCKBBDqEKr9F+maYpvwGVJnVHG2YzIvr8gPTT3lQINyf0A6c33GufOvJ93SV3Agu+bQGsk2GNIFc84ONu/P+UvjQjGqotUte8iGi6BcYmTh4+yGrFUfQUb7p9i629kYjmvp6VsvAV4+O/RwUZ/jSLFfKRARZrykYdd3omZnlNmfnKDe6yr1pMtXb8Mj5RiLzqL0m 38gaJ5eA 5jWb/CeqejZKVgNHZP1TMRBDvAiSurV1TK/h+SJ8uEmIKt3RkIgJw6pzZZbUJ2I7mdnSkRX+J17Vf9GSeXtmKx8s03Pm0DflCGqTc0diaIG///67nGpReFrMyy75rRqGKFes61KeqM1OWqgVmQfLCRDWPNB75k6wG2xVp X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi SeongJae, On Thu, 13 Jun 2024 10:46:04 -0700 SeongJae Park wrote: > Hi Honggyu, > > On Thu, 13 Jun 2024 22:20:47 +0900 Honggyu Kim wrote: > > > There was an RFC IDEA "DAMOS-based Tiered-Memory Management" previously > > posted at [1]. > > > > It says there is no implementation of the demote/promote DAMOS action > > are made. This patch series is about its implementation for physical > > address space so that this scheme can be applied in system wide level. > > > > Changes from RFC v4: > > https://lore.kernel.org/20240512175447.75943-1-sj@kernel.org > > 1. Add usage and design documents > > 2. Rename alloc_demote_folio to alloc_migrate_folio > > 3. Add evaluation results with "demotion_enabled" true > > 4. Rebase based on v6.10-rc3 > > I left comments on the new patches for the documentation. > > [...] > > > > Evaluation Results > > ================== > > > > All the result values are normalized to DRAM-only execution time because > > the workload cannot be faster than DRAM-only unless the workload hits > > the peak bandwidth but our redis test doesn't go beyond the bandwidth > > limit. > > > > So the DRAM-only execution time is the ideal result without affected by > > the gap between DRAM and CXL performance difference. The NUMA node > > environment is as follows. > > > > node0 - local DRAM, 512GB with a CPU socket (fast tier) > > node1 - disabled > > node2 - CXL DRAM, 96GB, no CPU attached (slow tier) > > > > The following is the result of generating zipfian distribution to > > redis-server and the numbers are averaged by 50 times of execution. > > > > 1. YCSB zipfian distribution read only workload > > memory pressure with cold memory on node0 with 512GB of local DRAM. > > ====================+================================================+========= > > | cold memory occupied by mmap and memset | > > | 0G 440G 450G 460G 470G 480G 490G 500G | > > ====================+================================================+========= > > Execution time normalized to DRAM-only values | GEOMEAN > > --------------------+------------------------------------------------+--------- > > DRAM-only | 1.00 - - - - - - - | 1.00 > > CXL-only | 1.19 - - - - - - - | 1.19 > > default | - 1.00 1.05 1.08 1.12 1.14 1.18 1.18 | 1.11 > > DAMON tiered | - 1.03 1.03 1.03 1.03 1.03 1.07 *1.05 | 1.04 > > DAMON lazy | - 1.04 1.03 1.04 1.05 1.06 1.06 *1.06 | 1.05 > > ====================+================================================+========= > > CXL usage of redis-server in GB | AVERAGE > > --------------------+------------------------------------------------+--------- > > DRAM-only | 0.0 - - - - - - - | 0.0 > > CXL-only | 51.4 - - - - - - - | 51.4 > > default | - 0.6 10.6 20.5 30.5 40.5 47.6 50.4 | 28.7 > > DAMON tiered | - 0.6 0.5 0.4 0.7 0.8 7.1 5.6 | 2.2 > > DAMON lazy | - 0.5 3.0 4.5 5.4 6.4 9.4 9.1 | 5.5 > > ====================+================================================+========= > > > > Each test result is based on the exeuction environment as follows. > > Nit. s/exeuction/execution/ Thanks. Fixed it. > [...] > > In summary, the evaluation results show that DAMON memory management > > with DAMOS_MIGRATE_{HOT,COLD} actions reduces the performance slowdown > > compared to the "default" memory policy from 11% to 3~5% when the system > > runs with high memory pressure on its fast tier DRAM nodes. > > > > Having these DAMOS_MIGRATE_HOT and DAMOS_MIGRATE_COLD actions can make > > tiered memory systems run more efficiently under high memory pressures. > > Thank you very much for continuing this great work. > > Other than trivial comments on documentation patches and the above typo, I have > no particular concern on this patchset. I'm looking forward to the next > version. I have addressed all your comments and resent v6 again. Please have a look again. https://lore.kernel.org/20240614030010.751-1-honggyu.kim@sk.com Thanks very much for your review! Thanks, Honggyu > > Thanks, > SJ > [...]