From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F65DC27C4F for ; Thu, 13 Jun 2024 17:46:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 902F46B00D5; Thu, 13 Jun 2024 13:46:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B2B26B00D6; Thu, 13 Jun 2024 13:46:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 753E86B00D7; Thu, 13 Jun 2024 13:46:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 530A36B00D5 for ; Thu, 13 Jun 2024 13:46:13 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 02D6C401F4 for ; Thu, 13 Jun 2024 17:46:12 +0000 (UTC) X-FDA: 82226594226.18.35B32BE Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf04.hostedemail.com (Postfix) with ESMTP id 53B5840019 for ; Thu, 13 Jun 2024 17:46:10 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=qmmo3kTk; spf=pass (imf04.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718300770; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MS2q0pepRz2o7PYFdGiBo8oOasR+l53w0tK1U1sWclQ=; b=qg0iM4NRxCshEKuYyrl6xjPT2CUQkFHW0OrUNL2bwboq3geKUvtr0OaJ4/hH4MlU7DF0/x kY6md59Z+WXhmby7HNdZ6VLc/vQBLlCUyE+H5eKw5C+he04kbKnpqtmGnf/Q68XUsN+Cuk oUIOleqXxUWSe50id0F6lLN+5o7DTu8= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=qmmo3kTk; spf=pass (imf04.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718300770; a=rsa-sha256; cv=none; b=V0o7GTsr6eVc5f66ucnXmCMM955MoHfoFd7lYo4dWt4bpzamUEhiE2p5Csn3xTJeEEjS59 ceX4/yufGC++9Cs40hWAq9oarbuukpQahufAwQSz4dLHyrVxSGd/6u4iPkfreUVtlFJvjq Zq/wGQ9Fb7jWmhPDSMNTZ4ASNpoIJj4= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6381861BC4; Thu, 13 Jun 2024 17:46:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D03FAC2BBFC; Thu, 13 Jun 2024 17:46:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1718300769; bh=2xnDcVWPpc4uuyAJqazQ/b3kOV5JH1ETZDCMyQNsypo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qmmo3kTk6tF9BXrVmVWjY1p5n6Pd446uMdNoBChrb4ymvL901wCvMoCoKvNddFlnR fDflClD/H2Fm5xPdLL7GIPyPtY2b4TWPo/lhP3rm06Zv1D1TAejbA9KMdaXWALKPoL VreIDaSBz1gBMPumkK5lJIHgAE4RYLctYX+eAUzW0/zjbFJoml7HsuprIHRDLbK0lZ JzUICyUy3c1n16Ncz+eYTobvyIhiQrcdn61YJGUINrN4PS3QtxRSiLLk/4NtCI4zvu x9qszP68I701jdUHGB8q6u+pjR7I82JLJcMJU4ulb31MviqOEzGUSosikPqBDZY7Dm +XYO0kr9/4U+Q== From: SeongJae Park To: Honggyu Kim Cc: SeongJae Park , damon@lists.linux.dev, Andrew Morton , Masami Hiramatsu , Mathieu Desnoyers , Steven Rostedt , Gregory Price , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, 42.hyeyoo@gmail.com, art.jeongseob@gmail.com, kernel_team@skhynix.com, Hyeongtak Ji , Rakie Kim , Yunjeong Mun Subject: Re: [PATCH v5 0/8] DAMON based tiered memory management for CXL memory Date: Thu, 13 Jun 2024 10:46:04 -0700 Message-Id: <20240613174604.63629-1-sj@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240613132056.608-1-honggyu.kim@sk.com> References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 53B5840019 X-Stat-Signature: xnku7cs4s6aaj7te1g3z3x5r6okr4nbk X-HE-Tag: 1718300770-528928 X-HE-Meta: U2FsdGVkX1+wSUT6ir2C8XSNKSH047kL1HxG/62oiqGdRct3aWjlDwRW70VY4fcYGlH8axSHQijcrv0cwWmmG2QAW9FVKe22uvUgv2V1Kqx5nGdQioMT1DJpqHTLTcBEx/x9qjiYMmMsVzMKLKR5rKT3TqJMSg4SO6zRfVFytZnWm9bXSpIUbcEs/Kzn60SZiBhS18BbPcKvjByhInKym5yMGsdijWbstRQ8U5oaRhVm9ICnXWnBDO2+Je5RXnIIkcviQZAlS/xCF5B1daoTSiC2ApOhwtkZFYG9Oa0WJr+Cw8bMvcrITNBpsSUQ868lnMaJreVqMOiigoky9Bqa4DdU1rzZN7lhFttOts3aqFMSt9Ldpr5DPOjWYfkPEH4lrGKkJT0s5nEexZuuBr/sLGSF1j829eL/qlqCfKzYo1Hr49q2u7QZ/aubTWlPAGaskPLiUW9J4hLi04SeDQ23E01qIkdbLDYNiXEaTnvs0zB8BRFQVUvgxA0WSETzjp12f9M4ODxZciD6JPraNWLr58OJfJAi5nuxPaNFog4Jxzdzl61GV+iG7wiMdx2RrjzD4Kq66adS65NjqnzGQ0ObvmmsCe/Zo9e9fgNaiaNW2+U9IfIV2ml1pnMCR+vLB/aF6qhQyOzuDN/Qy2+Hf0bUB5FbLSY+pTPNvxmXf+B6A7jl9NQW2NiS9sErLrpXHBiXm7JLfc3kQU0gnGMf/xTElZVVbpgtS3rbucgr/kQBWRK29kjQ0DQ9jlXu6MOYf1lVpme33B8Jmg+TfacwLHk9q5EkTOtteriXRsp4BN3q2W4CAja5aYvx74Sg/g0mVh9AuDjT0Q1Dvh0aUOXDcZTmsV8LlrxRvIGsHYzZvsT3gmp2hzDaLKGIkPNM6P8W2MNcIivWs1KY99OZ0yd5bLbnQr8IP204miFUk5THHd9XWsQWmoKlb2ZDMh1+ND89oItp02XMbTZ2pxYXZotmAaR 2J/sj4c6 KJlEp+g7WEGyWxtTEpF4MUW3HaKKPM4cTq4S81kS5wtSTatzFBhwOiMnLPAmQ2fPGwv49kODQAfINznNujq/QxlbULV/A/ugqcqI0cAxJOzoqtTjGCYQJNIQ1Y906te7B7lz1tdHCv3njs57kn4oQ0wRB6i6O07VWWm1OhUstq511TuaisfZdqhLabsulxQ1tN4aGyf6tuSrk/wTY0ba52N6SgKMKMltS0doW+mGyS1WgPLC4c1RLM121KoXvXW2du/V0mZd3GG6lGXA2GEMnZbGj8BoagyypQ3yM/TbBt6JgzCkotLfUnKW3Jex0NLXr42gdZu4uy9+E2HytIRWRUOfqhknnBbmdy9tPaKu1ZpmGvfW5wL6MjmczDQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Honggyu, On Thu, 13 Jun 2024 22:20:47 +0900 Honggyu Kim wrote: > There was an RFC IDEA "DAMOS-based Tiered-Memory Management" previously > posted at [1]. > > It says there is no implementation of the demote/promote DAMOS action > are made. This patch series is about its implementation for physical > address space so that this scheme can be applied in system wide level. > > Changes from RFC v4: > https://lore.kernel.org/20240512175447.75943-1-sj@kernel.org > 1. Add usage and design documents > 2. Rename alloc_demote_folio to alloc_migrate_folio > 3. Add evaluation results with "demotion_enabled" true > 4. Rebase based on v6.10-rc3 I left comments on the new patches for the documentation. [...] > > Evaluation Results > ================== > > All the result values are normalized to DRAM-only execution time because > the workload cannot be faster than DRAM-only unless the workload hits > the peak bandwidth but our redis test doesn't go beyond the bandwidth > limit. > > So the DRAM-only execution time is the ideal result without affected by > the gap between DRAM and CXL performance difference. The NUMA node > environment is as follows. > > node0 - local DRAM, 512GB with a CPU socket (fast tier) > node1 - disabled > node2 - CXL DRAM, 96GB, no CPU attached (slow tier) > > The following is the result of generating zipfian distribution to > redis-server and the numbers are averaged by 50 times of execution. > > 1. YCSB zipfian distribution read only workload > memory pressure with cold memory on node0 with 512GB of local DRAM. > ====================+================================================+========= > | cold memory occupied by mmap and memset | > | 0G 440G 450G 460G 470G 480G 490G 500G | > ====================+================================================+========= > Execution time normalized to DRAM-only values | GEOMEAN > --------------------+------------------------------------------------+--------- > DRAM-only | 1.00 - - - - - - - | 1.00 > CXL-only | 1.19 - - - - - - - | 1.19 > default | - 1.00 1.05 1.08 1.12 1.14 1.18 1.18 | 1.11 > DAMON tiered | - 1.03 1.03 1.03 1.03 1.03 1.07 *1.05 | 1.04 > DAMON lazy | - 1.04 1.03 1.04 1.05 1.06 1.06 *1.06 | 1.05 > ====================+================================================+========= > CXL usage of redis-server in GB | AVERAGE > --------------------+------------------------------------------------+--------- > DRAM-only | 0.0 - - - - - - - | 0.0 > CXL-only | 51.4 - - - - - - - | 51.4 > default | - 0.6 10.6 20.5 30.5 40.5 47.6 50.4 | 28.7 > DAMON tiered | - 0.6 0.5 0.4 0.7 0.8 7.1 5.6 | 2.2 > DAMON lazy | - 0.5 3.0 4.5 5.4 6.4 9.4 9.1 | 5.5 > ====================+================================================+========= > > Each test result is based on the exeuction environment as follows. Nit. s/exeuction/execution/ [...] > In summary, the evaluation results show that DAMON memory management > with DAMOS_MIGRATE_{HOT,COLD} actions reduces the performance slowdown > compared to the "default" memory policy from 11% to 3~5% when the system > runs with high memory pressure on its fast tier DRAM nodes. > > Having these DAMOS_MIGRATE_HOT and DAMOS_MIGRATE_COLD actions can make > tiered memory systems run more efficiently under high memory pressures. Thank you very much for continuing this great work. Other than trivial comments on documentation patches and the above typo, I have no particular concern on this patchset. I'm looking forward to the next version. Thanks, SJ [...]