From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 248DFC4708C for ; Fri, 28 May 2021 12:44:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 970B4610A0 for ; Fri, 28 May 2021 12:44:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 970B4610A0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 259706B0036; Fri, 28 May 2021 08:44:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 22FD26B006E; Fri, 28 May 2021 08:44:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0F7C36B0070; Fri, 28 May 2021 08:44:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0205.hostedemail.com [216.40.44.205]) by kanga.kvack.org (Postfix) with ESMTP id D0DA26B0036 for ; Fri, 28 May 2021 08:43:59 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 637798249980 for ; Fri, 28 May 2021 12:43:59 +0000 (UTC) X-FDA: 78190606998.10.C91F341 Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) by imf04.hostedemail.com (Postfix) with ESMTP id 70C5F3956 for ; Fri, 28 May 2021 12:43:54 +0000 (UTC) Received: by mail-qk1-f173.google.com with SMTP id q10so3905502qkc.5 for ; Fri, 28 May 2021 05:43:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=aSGlQM/XH/4dqoGaUVEoegdMj3CQ0k2Q1uZIVms6JyE=; b=VmQ2B6FVflLQlxCG3jjQAirYKIoA6nnByBl5Llwnh+8UmYK7Syi/Ihl3BdwX/iPdRg QhFRDq/ZirOXC0DZSMux2f0d9wsGTp/V8jxPe8+tPQCHPXTy45odyZdZwiD6weUU4rL2 Q31sbAmyRICsqPcsTFSaqvGEvtczyTQ7Uw+RNdD0nesUatPbRsTUupVOnyFT4wDimRQz 1MabxnK2XemZm+BIyf0iMCKMKqzDZ74wBU/5SzhhIILVb0IN6iuvBynAyoQm9QSV8CDR fMY5iH9Acqe7eBCuthWDEV+RZ0jZaNCPVZLiMUNIFz6bEwraC+yHaoLHrmRSvrGnQVkY FEIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=aSGlQM/XH/4dqoGaUVEoegdMj3CQ0k2Q1uZIVms6JyE=; b=mc9oLRVZXkBsIyAq/YO1mTSECCeGTon0iYcWfakvBCAZRcBtmxbxIIpW5UcrJ4bLv5 0G+euctnd4lNk2+04GmakdATry88jJj+hcRUX+0QvR1NMfYp//YT4FS5jQsH7MBidl17 mXn1ue/bfTn90KZPB6jo83basjfJdEa95zdJkU9EM+AmCjmCbSsDb1uqKqume8P+CKt1 ehRAI23ykZF8aJ8enshpLT84sMX72u6lcWN1p77dQEPy6WLNeXaDMcekEeyOEU+D8AcS h68tR9U6jw7r0f3nO9pzLX3ankvEM2EbYkvUl/k0Lk9vuuumFng/rw3wKbgVTN0hweOn USAg== X-Gm-Message-State: AOAM531G+mS72RYwi6V0hK+aT6YqkeTwbMRYWAR3QrCwbuf6SwSLGLn0 zOStFVBT3WW0us4cdBkV1kw= X-Google-Smtp-Source: ABdhPJwD/P+zQDuakBQd4ilWxP4M27/YGz6pBQUtgSIzZLwsd5vhcahrXFyb/KrSDjAhnRIp+6xoWg== X-Received: by 2002:a05:620a:42:: with SMTP id t2mr3710915qkt.423.1622205838412; Fri, 28 May 2021 05:43:58 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id z1sm3351959qki.47.2021.05.28.05.43.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 May 2021 05:43:57 -0700 (PDT) From: SeongJae Park X-Google-Original-From: SeongJae Park To: lsf-pc@lists.linux-foundation.org Cc: shakeelb@google.com, gthelen@google.com, rientjes@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [LSF/MM TOPIC] Fine-grained Data Access Pattern and Memory Management Optimizations Date: Fri, 28 May 2021 12:43:50 +0000 Message-Id: <20210528124350.9140-1-sjpark@amazon.de> X-Mailer: git-send-email 2.17.1 X-Rspamd-Queue-Id: 70C5F3956 Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=VmQ2B6FV; spf=pass (imf04.hostedemail.com: domain of sj38park@gmail.com designates 209.85.222.173 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam04 X-Stat-Signature: 7ccbxx8niegfb5t1s43snbwirh4zjyia X-HE-Tag: 1622205834-494870 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park DAMON and DAMOS --------------- DAMON[1] is a framework for general data access monitoring of kernel subsystems. It provides best-effort high quality monitoring results while incurring only minimal and upper-bounded overhead, due to its practical overhead-accuracy tradeoff mechanism. On a production machine utilizing 70 GB memory, it can repeatedly scan accesses to the whole memory for every 5ms, while consuming only 1% single CPU time. On top of it, a data access pattern-oriented memory management engine called DAMON-based Operation Schemes (DAMOS) is implemented. It allows clients to implement their access pattern oriented memory management logic with very simple scheme descriptions. We implemented fine-grained access-aware THP and proactive reclamation using this engine in three lines of scheme and achieved remarkable improvements[2]. As of this writing (2021-05-28), the code is not in the mainline but available at its development tree[3], and regularly posted to LKML as patchsets[4,5,6]. Nevertheless, the code has already merged in the public Amazon Linux kernel trees[7,8], and all Amazon Linux users can use DAMON/DAMOS off the box. We are also supporting the two latest upstream LTS stable kernels[9,10]. Agenda ------ Now we can improve Linux kernel's memory management using DAMON's fine-grained access information. We could do that by modifying existing subsystems to use DAMON internally, or implementing new ones on top of DAMOS and let users select. The two ways would have their own pros and cons. I am currently working for implementation of another practical and lightweight proactive reclamation in the latter way, because this is a new subsystem. I'd like to further extend this work to other parts of the kernel including THP, NUMA and tiered-memory balancing, LRU list arrangement, and KVM, though I don't have concrete idea as of now but only shallow thought. I'd like to briefly introduce DAMON/DAMOS, my progress and plans, and discuss with people for - thoughts on the programming interface of DAMON/DAMOS, - pros and cons of the two approaches for this kind of work, - how DAMON/DAMOS could collaborate with other similar works, - concerns regarding my planned future works, and - what additional DAMON/DAMOS-based optimizations would be possible. [1] https://damonitor.github.io (https://damonitor.github.io/) [2] https://damonitor.github.io/doc/html/latest/vm/damon/eval.html [3] https://https://github.com/sjp38/linux/tree/damon/master (https://https//github.com/sjp38/linux/tree/damon/master) [4] https://lore.kernel.org/linux-mm/20210520075629.4332-1-sj38.park@gmail.com/ [5] https://lore.kernel.org/linux-mm/20201216084404.23183-1-sjpark@amazon.com/ [6] https://lore.kernel.org/linux-mm/20201216094221.11898-1-sjpark@amazon.com/ [7] https://github.com/amazonlinux/linux/tree/amazon-5.4.y/master/mm/damon [8] https://github.com/amazonlinux/linux/tree/amazon-5.10.y/master/mm/damon [9] https://github.com/sjp38/linux/tree/damon/for-v5.4.y [10] https://github.com/sjp38/linux/tree/damon/for-v5.10.y