From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CB92C4167B for ; Thu, 30 Nov 2023 09:14:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CBD946B0453; Thu, 30 Nov 2023 04:14:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C450B6B0454; Thu, 30 Nov 2023 04:14:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ABF436B0455; Thu, 30 Nov 2023 04:14:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 96F0E6B0453 for ; Thu, 30 Nov 2023 04:14:35 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7AA041A013C for ; Thu, 30 Nov 2023 09:14:35 +0000 (UTC) X-FDA: 81514060110.15.BB07776 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf29.hostedemail.com (Postfix) with ESMTP id A3BF712001E for ; Thu, 30 Nov 2023 09:14:33 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=G+0lQdeo; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf29.hostedemail.com: domain of cuiyangpei@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=cuiyangpei@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701335673; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kkZm04lfRlD3mzOUGiHtFxV8qZ1vw4L1VTf3erF9dac=; b=HTeGWWdpNRzd3nJjKO1ySINx+M22tDxJRjUfSo8Rs9afiAWvvbvjzVt2SmW5iwCnFC2fce dmAjk1+CzTvqzOHXHMY5ohjW6aFhz0FROWD2pf7mDd5kbg/Y5gwRRVQNwTeQpgC7dvOGBC xSLiJ88tLkLKgksZv6Y/PfTxRQxKLfQ= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=G+0lQdeo; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf29.hostedemail.com: domain of cuiyangpei@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=cuiyangpei@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701335673; a=rsa-sha256; cv=none; b=SzokPnsjnt51+C0Wcm7SCrJK6ACLpyZMYmcvf1bBDOT7PUy99MTDP8Vg38YKlzkvfnCFeM l70iUe1GW77DMGDbsuaMWfNRnQsFSk9QDb3Yx+ZyEniTXNecgfCGduGSO9fqaK0C1+81eM Zho1NfjCKzzAXc0EpKhJgV2zbVpGAbI= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1d03bcf27e9so1403025ad.0 for ; Thu, 30 Nov 2023 01:14:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701335672; x=1701940472; darn=kvack.org; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=kkZm04lfRlD3mzOUGiHtFxV8qZ1vw4L1VTf3erF9dac=; b=G+0lQdeo7wMwHUd3B8DK6WdbNS99/vMNRt9uvGdzJISxQ4qQN0OrFN66+T0ICrAyFx rBulu3MiEE7bAct7xjGC2BbyCt8/iDz2lSfYXXwMd2imdAhNLdOPA/8VXChmgoJF4o0+ +nCvz5faH9+f33F4+beUzQqsw8011gaRUnA2syB5HCvqbG9OtvGGYykilVK157ugZs1N urGx8clL55Hn+TluJv/7qHOOrc6z/Uyou6nnEVn2iUY7AjEftOz5qHe2E2MdcQ8px2Lq lThGaeipk0HxuKOmd643xn8DtT6DK9ycxFyL119orj5DGv/UHRuQFwV+y5iqfOsiks5m 6eQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701335672; x=1701940472; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kkZm04lfRlD3mzOUGiHtFxV8qZ1vw4L1VTf3erF9dac=; b=qKncTasSwkpdataTCJSgfwP9OevSzaIOXsDNtTu/tend+fFA15LWyLxjdMZ1xM4k0s jDOoGxOUYVpj4Y6VR4zXH1WHeAHbOfUS2/sqtmf7e9vUfzjNvLQHM2hXfEvt7K+WdM5J tsOTt8oju/Xi1iH97vbqM8Vt+jChZ3+wnxmyX/V/rp5qjbH5xl5nQSBs5ewFBYxSHxJW Cw/d/o3vAwfGEFmNh6PTTnEsdfYprnfgVm2WHVd37zjgaGpSV8P9ylyIh2FFA9UeqNHE aIzZLlDp1A+wQnLxJNQ/I3EPqK09/7JOxVVfTNiuQgo6L52qPA3K6W1AUHRyDh1xwwGC MHPw== X-Gm-Message-State: AOJu0Yy8CAB8qra7M2f4a82LgYGEeQKLLSmn63WQZ/WK9nQBB2yHoEGa MvVvXBnF63tmAKlK74LJ/w8= X-Google-Smtp-Source: AGHT+IFcncNN/ZTUn02k8CmbBs5KH3TQJCqee4HEtCviqsR8oz+yW8pmspfox950SyTatF7XsUr+2g== X-Received: by 2002:a17:902:db09:b0:1cf:cc0d:b269 with SMTP id m9-20020a170902db0900b001cfcc0db269mr15176117plx.40.1701335672265; Thu, 30 Nov 2023 01:14:32 -0800 (PST) Received: from cuiyangpei ([43.224.245.227]) by smtp.gmail.com with ESMTPSA id y4-20020a170902700400b001cfc46baa40sm835498plk.158.2023.11.30.01.14.29 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 30 Nov 2023 01:14:31 -0800 (PST) Date: Thu, 30 Nov 2023 17:14:26 +0800 From: cuiyangpei To: SeongJae Park , akpm@linux-foundation.org, damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: xiongping1@xiaomi.com Subject: Re: [PATCH 1/2] mm/damon/sysfs: Implement recording feature Message-ID: <20231130091426.GA13946@cuiyangpei> References: <20231129131315.GB12957@cuiyangpei> <20231129171058.50270-1-sj@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231129171058.50270-1-sj@kernel.org> User-Agent: Mutt/1.9.4 (2018-02-28) X-Rspamd-Queue-Id: A3BF712001E X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 6uubkgaa3podyb8u6djkqdbtbptzb74y X-HE-Tag: 1701335673-270869 X-HE-Meta: U2FsdGVkX18uyOpzr5qYfj+0/+g0lrAaBdYeEYuX6sWg5aRsYJZOI1RRJCj/MonNc6UAixBdTxPrMqqCmlTg0BbXCuhoUS+EhiL8CKE2RnMkNZvAtN8fAXOPh21oXAw3jZM9kL+GWBoByzQ1EfryMfB3e2xBdjnE5B0bwmeAUAcw03u2oBH4QfqH65kQsxcqCKB17yopYAeogsseU57JKRgFRAHEcN2+bPgNl91y5vj+nseMn00tNkA+Q50nEdONsBIyk0KSZG2QZC57kMUVKtmtEm+Riy+vF2DJYa/LIVsERn321OLtquppnA0dnMJ80goW2TvIGm6anJ45iSsvtRh5AaC3D6V4bHzY6bbh/HCQXZlKc2fg5ByzYHJTGiacqfPqEFX8uwX2UTfjTocRr8MoykeOctMVfhUjD1soHYTDvrfOtTCaKrm9XfDPsmDh2sodyVSR11UF+LjeIBIjPUPy73/4Jkm144NN/RqeyTOnyk6PW4MuDemeUjtnGeAam2dI+/YW2TlihzuwJQ2w9hKgsbc0b6oY6Mvi1Sq1YGIsGdE5ZGAxHRH1htrOyXvYXu61xJrofr6gsCUQ3XJtTohE6V5wAe/YdSNl651djnrrdCfIJQxHxz99z/7kQO9isLbn2tuS5JIfC2J9bPUr+BebLEDBpPd2ablpTPQO+pwcX2JQdg5ou7SYyRwvaZ2U+8DqeoCiXD+ERSfyPtOoT5JdhpsSYH7yjA7N1clN6GkIpkl1Nu8JkqeOAcItzZ55xn7I/PoXUgXK0CUNGRyfhk+920dUcEkRLFl1uZuGQjR4D0dHyfyf5h6N8kXfuRKZu2z/s3RuHUI9vjVffZScaGrxE1ofuNdicTh+tA1u8EHS7Q8KS85+LA8h7mnQXpPTdvm9XOvlqOGVfjXbKIeybfecS4svMkXtm3iWoBNSqEUhK4Vsz080eyq6G6z3hK11cqd+/6aVRjMqcuORdT1 qgCWNWNF 6TZHJu3HGhfvyBK0fahW2Evbfc1QBHOlx0tVWGPzd1HEgzoHtX4SLkk3QFA8/NvbRmbYApK69OsT9fkMJJYp+jhdGL7uisKpHUynmeJwFhTHQJQzFbveWheM0sPHwJwu6H3I+q+5f70qydsk1a9A+qzugFUWqzONyw1mNqKm44oJ/qIL6eivmKQOAIpymhk+pwno4D1sibETrtfCj/QDano6IUO8WYD6c6J8+WWeZ5zgX32hGnZeqtlIii5zQMhSl29g3/eLVXWYTvFU6RgtKCWBpXe9UFMOaBJPbtVjWDBifg/jhjRxA4zQBcwMBwHlLE/nlMP1T40CUrOjk0UqmBU3PKZHXT5Kfe9t4aevWij5DqpbxoDExFRbvw5QdOYK9SF/D+i6Ad+kvbTCEjbyswJafmNRqSjpQMlA2lmMtWbRFjJygTqLKgXntNYn2Ra5SVDRWyUUO2b2oqnr4sfH0dMQQlzfr0nmhDhmnoCsqqFDOVobM3EK7KTfEbVDsIBh1jY6/8cnmh32dS44y/UtXRfW9TU+I26OKbrex X-Bogosity: Ham, tests=bogofilter, spamicity=0.335763, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi SeongJae, We also investigated the operation schemes you mentioned, but we don't think it can fit our needs. On android, user will open many apps and switch between these apps as needs. We hope to monitor apps' memory access only when they are on foreground and record the memory access pattern when they are switched to the background. When avaliable memory reaches a threshold, we will use these access patterns with some strategies to recognize those memory that will have little impact on user experience and to reclaim them proactively. I'm not sure I have clarified it clearly, if you still have questions on this, please let us know. Thanks. On Wed, Nov 29, 2023 at 05:10:58PM +0000, SeongJae Park wrote: > Hi Cuiyangpei, > > On Wed, 29 Nov 2023 21:13:15 +0800 cuiyangpei wrote: > > > Hi SeongJae, > > > > We are using damon on the Android operating system. It starts monitoring > > when app comes to the foreground, stops monitoring and save the > > monitoring results when app goes to the background. > > Thank you so much for sharing this detailed use case. This will be very > helpful for us at understanding real usage of DAMON and making it better for > that together. > > > > > The two methods that you mentioned, > > > > 1.tracepoint events > > This method requires opening the tracepoint event and using the > > 'perf-record' tool to generate the perf.data file. Then parsing the > > perf.data file. However, the user's phone is not enabled tracepoint > > events. Additionally, the generated file is quite complex, and we only > > need memory addresses and access frequency informations. > > That's fair points, thank you for kindly explaining this. > > > > > 2. damos > > There is no direct Python runtime environment on android phones. > > > > Both of these methods provide results that are not very intuitive and > > require complex parsing. We save the results in the format of starting > > address, region size, and access frequency. When the available memory > > reaches a threshold, the user space reclaim memory with low access > > frequency by calling 'process_madvise' function. > > Again, very fair points. So, if I understood correctly, you want to reclaim > cold pages proactively when the available memory reaches a threshold, right? > DAMON could do that directly instead of you[1]. Using that, you don't need to > save the access pattern and parse but just ask DAMON to find memory regions of > specific access frequency range and reclaim. Have you also considered using > that but found some problems? > > I understand the feature may not perfectly fit for your use case, and I want to > learn from you how it could be better. > > [1] https://docs.kernel.org/mm/damon/design.html#operation-schemes > > > Thanks, > SJ > > > > > Thanks. > > > [...]