From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40169CEACFA for ; Tue, 1 Oct 2024 19:14:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D72C2800DE; Tue, 1 Oct 2024 15:14:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 95F69280068; Tue, 1 Oct 2024 15:14:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 803092800DE; Tue, 1 Oct 2024 15:14:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 5AA18280068 for ; Tue, 1 Oct 2024 15:14:34 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 02639A064D for ; Tue, 1 Oct 2024 19:14:33 +0000 (UTC) X-FDA: 82625984868.13.C6C68DD Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf12.hostedemail.com (Postfix) with ESMTP id 6336440006 for ; Tue, 1 Oct 2024 19:14:32 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=t4dXciEH; spf=pass (imf12.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727809945; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=l4BnxHV0RK8HNZC1JJ7AZOR1T2ZyRtjtylRcTCR6e2U=; b=bQ9yuDL0mYZ3hQVWcmrIOmpmL/BeBtSany8RHaQ4DmXO8QF895sGAukQrwWfyhv2drZoDh L7qKgb7CQTJrMamooBx24dQAL4Xqlrya8yDR6TatDgzgOvb7nwFsetXx3wI9maMcEsCUuC k2JP6MrSdAL6KMjmyGsWxZRZ0INJv7I= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727809945; a=rsa-sha256; cv=none; b=59GBAtXZMh2r/Olf8eBQMFmgA9u/vqxkxQsoR6Pz+HuvT6h2MjicSEaFZFAQOXWzJCXnJG 1UC4DwGQRwulZGUiowFKGBwhMtp8fwjX3Mk4nqAHNaNAYRqvJsoqGbPfKwuJdG8tKWg3r6 scnFxwBKPQUjVj9qOrptEk29wojvNlg= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=t4dXciEH; spf=pass (imf12.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 573235C53F7 for ; Tue, 1 Oct 2024 19:14:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EF9C5C4CEC6; Tue, 1 Oct 2024 19:14:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727810071; bh=1o3xkKDxJdtkE8fgqL+WN2XSbXjNaOd+iHfQfqSiNLY=; h=From:To:Cc:Subject:Date:From; b=t4dXciEHyaKIkUR03pyVn5lnVZEGNg7JvOVwcOx9OOH2n6PuxRZDwXQz5cYV2fACi yTqQqTjUkPN8uhrrustRtTTQWKS3ZNOEemLNam/SBbkoMpQsf4BkMgLUqaOQmsJsoO 2tQISWwvsktjKXQv/CUkZArnmElzGMQEeSmmxZaGZNn6LxTzhFApuzb2kGQem6a6e2 /VVifbZMoOCpqMCgepS+y8qlBaEaBLBJmc8XfQ+21hV9RyqQBFMkWrW+D/dBJp94kK jfWLiKWlkOLsCamFJ+TfLjJ0YY+BuBt+EKmFm2MJf4CQTov3ylipp9KXe8ZUJC/3By uEZ7OG2NZnn4A== From: SeongJae Park To: Cc: SeongJae Park , damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: DAMON Quaterly News Letter (2024 Q3) Date: Tue, 1 Oct 2024 12:14:24 -0700 Message-ID: <20241001191425.588219-1-sj@kernel.org> X-Mailer: git-send-email 2.46.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 6336440006 X-Stat-Signature: 5cxuptb8npnhdp69931bqz7buq3o76mp X-HE-Tag: 1727810072-742722 X-HE-Meta: U2FsdGVkX1+VuibaC+DztK2N4Lrln85B1PE3jqSd+LGBO1QKpbI4+kCyjF8zvMhUiO9LviH74U1eJIKClQWD501NQs/C+z59Dqgd3qOMpfPoPxWSdmFT2c4u25U47fGAQcGvyt6F+JGZJFioPj4ATPzkQoeGahN0bBpduBxnU6vgL9b8T/r6r0OSM1KhJgJc817ELGk0r4okqoQG++8KBjO/z3fPGuPeP8DxVw6VRBMvaYN8i3vJqeBtpil8iPV84XuWhDn2fgC1TUNPKmRaegrCeb6Njvatwgm4E+SOHuHbG16m9g588YGFy9EXS2MvL4QpT2EL+Khc+YIHy9jMrvykWbwHwb+VbM6aUOrguKPg2K2kkt2m0A7huxC5+f5BRhXQI2+U4u0ppA1bU0tW0bnpPCMU6bMYuJCp9CItEb+FyoWJrBHJysO5kqpojbLAPd+/O8TQd6cX44/GSRIEevE87tIdMReCDIL5CoH1d2jMREn121XBli6DszM8ZZ9hj416NKO2+GdbkRIKOiIi5JMQ93PAOb3oT2mdpHAfXJAb+2nzH3Hof/sOnQRb4thXW7hIrRyj3vjUp5rZjLZoWP5vfvbO/K/+SuSplasdom3wtg2jPQL6KX6UEUhsG3nHsAd/GlRaaflO6+uUTMz75rbBeEUEcBFyGN1SIyNISHNHholppSiX9lgshCN6Bce4o9IlDjc8wSPuT8A9FE8hFAYybUzBpRLOMna8RlFsMf+R2FlNc0ieY1Ut8rrh0xc+iKmvSOKvCkiCakRAf7WLB2QtNmfU4JMn6mf2eat6zyx0utLt6PJAmvAsVS1OOYyk0WCz2/tBvHguxvbZg8eyzT4mU3k0e7dYb5UpOA9CHkiTDHrlWgTjTxjbpQ6CF3gNVnXJXSJPASjNr196aC/Yb9oLwMKIGPHUqV9ZLncuGv3T9bFBJOySETUtxzHxy4uAJXCwHWU9+wrNf3oAVpa RTNVlIQV FoSsTkrgMbrObctGylhkuMbg+mTE3lDgAAkCWe/8rjHeWG1eOkA15ye324s0RJyzGol9OK0e8DFNcIno9tZhcAGx9D5ZB6jMTAmvZSLC0SgfaAK+grRlSA7/NQiZER2YlifxZpZYgNsknr3IFeSxswfLaTMZc2C9OMkVFDqt4F6MVE2bKnnAZfOfrTB6YqSRsBZ9y2/9NCp5E+R50Lo0jHylhSwdcx1Yj6BKl7KRQvhxPQgSduzi2enmIvP0Zc8XgTa03VCXSM/FPMZcKzrhAunrIAjOhJ3a2yyy7NLxNOge1xxJpGH607b78IY2v3vYZiexVP4uAhaeIboS+fZqgm/9Jwab6Ae/E407gIQ7yv8vxMXnAx1I3FVZd8D9TiKBdV2e7aliS2P+FTx6pey7ge/4XQYzuAagxHLBPIc0keOmGuiMgvVonSpN0qQVyxJdW1SlCCSJ36J4C6dVUwKJW/tRXBdMBhzhqDHjUWVFjs55Qzo0rZC+zEvpkHdwDpWShiNUPTjLBBJLmfS2R84yX69RUo8ZLgmZ/M6DGYxREiGXdVyhm30Dfl/O2mnTB8EtLbESXxkDD1LWXkkkr1OLGFx3yDuJKGA0ldijp0VTLIHrQ/JMKCLFNqnJ2PzFo1UBeY2HZWHzic6RivBq3AH/bcWy26T78K7fix28rxpI1A+dibQQH1oTdE19OIFTmiNjixnjtIi5pGEhzgrDUFgxBmS8/4RgK9tKgYRdgcb+D5vwJAx2Dxx78SVXCmg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hello DAMON community, Three more months have passed since the last DAMON quaterly news letter[0]. Let's look back what happened in the last three months. Highlights of Highlights ======================== - New features. DAMON-based CXL memory tiering patchset has merged[1] into the mainline. - More users. DAMON usages from AWS Aurora Serverless v2 has unveiled[6]. CONFIG_DAMON has enabled[9] on Debian kernels. Six more academic papers that exploring and/or utilizing DAMON has published or archived on OSDI[2,3], ATC[4,5], VLDB[6], and ArXiv[7]. Monthly PyPI downloads of DAMON user-space tool (DAMO) has significantly increase. - Re:org. DAMON github repos have moved[8] from awslabs org to damonitor org. - Conferences: Detailed DAMON usages from products and labs have presented on OSS EU[10] and the VLDB paper[6]. Long-term DAMON plans for kernel that extensible and just works have shared and discussed[11] on LPC. More detailed list of news per each month follows. July 2024 ========= A couple of OSDI'24 papers[2,3] for memory tiering that references and exploring DAMON as a part of them are published. ATC'24 also published two DAMON-citing papers at the same time. The first one[4] proposes a way to improve monitoring accuracy of DAMON, while the second one[5] mentions DAMON can be useful for extensible memory management. VLDB paper about Aurora Serverless V2, which reveals their usage of DAMON on the product, is uploaded[6]. Memory Management subsystem pull request for Linux v6.11-rc1 with DAMON changes for CXL memory tiering has merged[1]. August 2024 =========== GitHub repos for non-kernel parts of DAMON project including 'damo', 'damon-tests' and 'damoos' have announced[8] to be moved from awslabs org to damonitor org. As of this writing, the reorganization is completed. September 2024 ============== CONFIG_DAMON is enabled[9] on Debian kernels. An academic paper preprint that optimizes THP using DAMON and BPF has uploaded[7] to ArXiv. DAMON usages from labs and real-world products for profiling-guided optimizations, proactive reclamation (Auroa Serverless v2), and CXL memory tiering (SK hynix HMSDK) have shared[10] on Linux Conference at OSSummit EU'24. DAMON long-term plans for making kernel extensible and just works have shared and discussed on Kernel Memory Management Micro-Conference[11] at LPC'24. Monthly PyPI downloads of DAMON user-space tool (DAMO) surpassed 4,000. It took about 7 months since the record of 2,000 (2024-02), which took about 18 months from the beginning. The speed is apparently growing. Previous quaterly news letters ============================== - 2024 Q2: https://lore.kernel.org/20240701212244.52288-1-sj@kernel.org - 2024 Q1: https://lore.kernel.org/20240402191224.92305-1-sj@kernel.org More Past DAMON News ==================== If you're interested in a humblie list of more DAMON events that happened in past and curated by DAMON maintainer, please reach out to the project site's news page [12]. References ========== [0] "DAMON Quaterly News Letter (2024 Q3)", https://lore.kernel.org/20240701212244.52288-1-sj@kernel.org [1] "MM updates for 6.11-rc1", https://lore.kernel.org/20240721145415.fbeb01a853962ef91334f3d1@linux-foundation.org [2] "Nomad: Non-Exclusive Memory Tiering via Transactional Page Migration", https://www.usenix.org/conference/osdi24/presentation/xiang [3] "Managing Memory Tiers with CXL in Virtualized Environments", https://www.usenix.org/conference/osdi24/presentation/zhong-yuhong [4] "Telescope: Telemetry for Gargantuan Memory Footprint Applications", https://www.usenix.org/conference/atc24/presentation/nair [5] "FBMM: Making Memory Management Extensible With Filesystems", https://www.usenix.org/conference/atc24/presentation/tabatabai [6] "Resource management in Aurora Serverless", https://www.amazon.science/publications/resource-management-in-aurora-serverless [7] "eBPF-mm: Userspace-guided memory management in Linux with eBPF", https://arxiv.org/pdf/2409.11220 [8] https://lore.kernel.org/all/20240813232158.83903-1-sj@kernel.org/ [9] https://salsa.debian.org/kernel-team/linux/-/merge_requests/1172 [10] "DAMON Recipes: Ways to Save Memory Using a Linux Kernel Subsystem in the Real World", https://sched.co/1ej2S [11] "DAMON: Long-term Plans for Kernel That {Just Works,Extensible}", https://lpc.events/event/18/contributions/1768/ [12] https://sjp38.github.io/post/damon_news/ Wrapup ====== So, those were yet another grateful three months. Looking forward to get more news! If you have any question regarding DAMON, please feel free to reach out to the community and/or me. Thanks, SJ