From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5284EC46CA0 for ; Wed, 29 Nov 2023 13:13:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B88066B03D8; Wed, 29 Nov 2023 08:13:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B387D6B03D9; Wed, 29 Nov 2023 08:13:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A2ACD6B03DA; Wed, 29 Nov 2023 08:13:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 916126B03D8 for ; Wed, 29 Nov 2023 08:13:24 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 68E62C0373 for ; Wed, 29 Nov 2023 13:13:24 +0000 (UTC) X-FDA: 81511033128.18.A3EDABB Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf15.hostedemail.com (Postfix) with ESMTP id 363EEA002A for ; Wed, 29 Nov 2023 13:13:21 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=SXW8B5ej; spf=pass (imf15.hostedemail.com: domain of cuiyangpei@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=cuiyangpei@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701263602; a=rsa-sha256; cv=none; b=oYESrirAZYl+L6UossiknfcFOjib6izehivmf8mkpVSW+FRt9QHu0UVf9v99bPqgQk1zvY Lk50kYBv5CoU3n8hm0HvnTjYTMd/Rp+Bpwz3bZxtBEGSVUP8m5MPYSJeO+jPQjPHb7p703 Ke4vrS0Nz2JtzX+tXbzYs5y7h4nPyIw= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=SXW8B5ej; spf=pass (imf15.hostedemail.com: domain of cuiyangpei@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=cuiyangpei@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701263602; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=y1YT23JDUOStR2+efKckGHd7K0OmUB2BTaDMNtRq3C4=; b=boJ9H6zNmjk1VKG/crtLppz1ggU8Y2D+LVDCexjjRx+MCui3VsiZ05ZVJwcivaNNbCinPz YFAJzI34WDtzUFSTe+t06gjkv3DIakoMa8YhFMpIoENYlTQFMrxnSsmle/U8oVb4pfNHVr aGY4uiqWZKkQjJYU18IJC+0UrA/8XKw= Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-1cfc3f50504so30809325ad.3 for ; Wed, 29 Nov 2023 05:13:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701263601; x=1701868401; darn=kvack.org; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=y1YT23JDUOStR2+efKckGHd7K0OmUB2BTaDMNtRq3C4=; b=SXW8B5ejdtIx9ZmmbqITH2ujI5Jh8uiFR9TZRkamyfhb2ZKGXIkLuZhyaHzpuI3oy8 wTGPi0Odzf2/mvSOS7PvMEhenzpNu4TZkjjZmglZp3P4upt6Ms1Nj88tnMI8SZEsb8Z2 JPav2qtbk9jiqabknfWk25jGtGUbg0oZUXqJEEhXVtGmhzGuNrEls1qd420e5mraB0EC oXcsFMc863ochJvXK+8cA0WCwf4x3q5PDfaVfDLpQtuCovW9cIBp4eRWwGP4yc9+nsB7 /xfay2zze/NgKEMfrxukGz6gGLJo5lzDdkPIUENrVfgsngDiFzVtEiE45t2qwk1Y33xK pjtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701263601; x=1701868401; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y1YT23JDUOStR2+efKckGHd7K0OmUB2BTaDMNtRq3C4=; b=L3NcWEaWRsxUdbzkvszjue/iiUZKVGTU7WQ64FCOINrl4RN49LRxU8xFJTtHtKAgMf RSsfkOJ1gNphHQzDIycerI9CTcRMu3tCuitLeQLu/2MkPkNJNGYsKldWaz86z0fWhNTo DNA/Tm87o8tqDkgFI/1ZtRifDE4N6pAAGK4G8dEu+/gubCNWBFHfbqsqGFJxHyfzoEJP GipNu33azed3qzRkn/H8vFjfwrCb1rOwwAEgSghY9rjbS23guLW48f0D7gt8M6uJxFYS AXqy9oWE7jfqj57P4XTbZkq+zXTxbRDeDux1otvbQTho+1/inXYBAu2VkN8xpeESMMsn A6Gg== X-Gm-Message-State: AOJu0YxoDrMhZjsMf2FUcsT5FAy/AXVh1rhQXbVqPBkXix8nIjF1cDZJ eJuJqaTcvN4KVU6YlWxRbNA= X-Google-Smtp-Source: AGHT+IFXMxElkHGt+LImGYb5PKuu4f0cgQnpCZoBKnnVa9EEfjK0Xu8vKDp4M5bqcmmcons7LMjGXw== X-Received: by 2002:a17:902:b194:b0:1cf:f359:ce37 with SMTP id s20-20020a170902b19400b001cff359ce37mr5474542plr.2.1701263600844; Wed, 29 Nov 2023 05:13:20 -0800 (PST) Received: from cuiyangpei ([43.224.245.227]) by smtp.gmail.com with ESMTPSA id 13-20020a170902e9cd00b001cfca7b8ec4sm6314215plk.101.2023.11.29.05.13.18 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 29 Nov 2023 05:13:20 -0800 (PST) Date: Wed, 29 Nov 2023 21:13:15 +0800 From: cuiyangpei To: SeongJae Park , akpm@linux-foundation.org, damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: xiongping1@xiaomi.com Subject: Re: [PATCH 1/2] mm/damon/sysfs: Implement recording feature Message-ID: <20231129131315.GB12957@cuiyangpei> References: <20231128073440.11894-1-cuiyangpei@xiaomi.com> <20231128185739.47916-1-sj@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231128185739.47916-1-sj@kernel.org> User-Agent: Mutt/1.9.4 (2018-02-28) X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 363EEA002A X-Stat-Signature: oyyu9udxatk13zeaga9eihstuqhrd7po X-Rspam-User: X-HE-Tag: 1701263601-113335 X-HE-Meta: U2FsdGVkX1/76Fr+phxrmtRqj/Cz0qBbWb8hWBCGgNT/9izrZjC3blTJb+WGrDddNHw9Y9fJmvLkzP+ybHPaByuV5Vtgnz21dyJ7c+5bCHOjasIYNTpuB7JfzuyJtuiiZdUTQuQ9tpIzNa30iltlT0sy1wVngRvWRYIuA9S/wAc6v7RZOQ4FwmdailzHHU1hG4lbIC8M3ms1nYr+dFpSDmhZ4qL3UXSLoSa6FZnFetjPru9zbnKaraXItavgmQK6t64+so+y0GFk1M5HFq7kpBuGG+9Nq/0MVR6Y8Y9aQu2bmCajTvUVv73eC8ib2FONIToZjNqcUyiDWudju1oXjL13xw88xIBuxgUSfK11GojlaayP6tuvoU7ugldTx0iJVk8I5ewjIbHXE8EkjOoFPexRxvGh2IjZ45WjSlxqHz8C9gik8B+tNtHZfGLKDka9dvua1tiqY70quTEhcVW/W9RwOYNzQ7e+aqtOk8K15vsWXd6oxs9CIYbNNhkWL1Yt7hy6dyItcFl4Ot7fihnu/waXjyUILuqdPtUdyufLwN8M6SJkTJgbTpc406uTSkDcA45uXQthCvPiGKKv3hZQNN4roK9GDcnX4O/dvAEEsWcX3uJRAaGq149+gBxRiALuMW+7ePaFS3guQeCCXjSX+CieeWBZTqhktfzGYJjMIcALOKcpZQMNXM5wGBhNf9lOlvUi3cmmEw65zR3tZ8gCuJuabV1dr6nH6ixRqQ3po0hCaZjV0njFBHHtp4CPBZXpiZJIRfkB7vrwGZk/TcJacbFtj2VcASbJyc6Pw/2G0cSD9D+09V8a929qrIfhH3ppPF9Gj5F4Tjyjiq7w1w8OpgJjrxv/MIH8r4G9Rp6JAzj8istr+66bd72fuF3v3i3R8DDF/t0GLePMF3KAzQztkN/iVJvZFnWu6EiHNlxUXxVBiGT2Yw/L0om/+htJ60AgumMYWpHPJZIAcbGZ0LE 4H5OybkQ bGMYDeC33dhljSG5CNeqRoWjH6n3FbjolUnE/t/pLEnhhujjovdZN+2KvqJwQQsWgB94PHxhGmuwrwTW/H4NGJMGLShru58aXYXYHFl3S6hOjXk93NHbAGWnWAKWHvnKifPa/whKhd5EB35YcFt4wGVeNTFOe0KqI3XqCrEtC7kxhwe8l4lCZtuVxJ+faNqy72abuRrNmtvjfbhHGvA3QmGNuBJrNu6XpsJH/eAj4luJEK0ZCdO5awgXvcm9zoEWxbE1mXnTqgRwXDY2gbRJvXzm+YokWDpHUdjgdJF2yt0I8DqY1zbesOu1YW2rYFPJ6pK9ifQk8Lz1ZV3niiE0zm9kxSV2kiibinAWNOTbYG1Hl+iTFjDUyRtPD/1If8sERTvg+OlbWA1rgkbDeXeku1gfQQQ/GlQN1b/KqgLRt7UpcLcLlUXvN4u/gkrYP1nubx9O7vK3V+BzkOOKbaKR1gA6oByrvR+zesW9FLhdvs3NToX4zQthAQUvv6f6frLFhFmJmsb09yiTgyrg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.060158, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi SeongJae, We are using damon on the Android operating system. It starts monitoring when app comes to the foreground, stops monitoring and save the monitoring results when app goes to the background. The two methods that you mentioned, 1.tracepoint events This method requires opening the tracepoint event and using the 'perf-record' tool to generate the perf.data file. Then parsing the perf.data file. However, the user's phone is not enabled tracepoint events. Additionally, the generated file is quite complex, and we only need memory addresses and access frequency informations. 2. damos There is no direct Python runtime environment on android phones. Both of these methods provide results that are not very intuitive and require complex parsing. We save the results in the format of starting address, region size, and access frequency. When the available memory reaches a threshold, the user space reclaim memory with low access frequency by calling 'process_madvise' function. Thanks. On Tue, Nov 28, 2023 at 06:57:39PM +0000, SeongJae Park wrote: > Hi Cuiyanpei, > > > Thank you for this nice patchset. > > On Tue, 28 Nov 2023 15:34:39 +0800 cuiyangpei wrote: > > > The user space users can control DAMON and get the monitoring results > > via implements 'recording' feature in 'damon-sysfs'. The feature > > can be used via 'record' and 'state' file in the '/kernel/mm/ > > damon/admin/kdamonds/N/' directory. > > > > The file allows users to record monitored access patterns in a text > > file. Firstly, users set the size of the buffer and the path of the > > result file by writing to the ``record`` file. Then the recorded > > results are first written in an in-memory buffer and flushed the > > recorded results to a file in batch by writing 'record' to the > > ``state`` file. > > > > For example, below commands set the buffer to be 4 KiB and the result > > to be saved in ``/damon.txt``. :: > > > > # cd /kernel/mm/damon/admin/kdamonds/N > > # echo "4096 /damon.txt" > record > > # echo "record" > state > > This reminds me the record feature of DAMON debugfs interface[1], which still > not merged in the mainline. I deprioritized the patchset to have a better > answer to Andrew's questions on the discussion (nice definition of the binary > format and quatization of the benefit), and later I realized I don't have real > use case that this makes real benefit, so I'm no more aiming to make this > merged into the mainline. > > More specifically, I'm now thinking the feature is not really needed since > trace event based recording works, and we found no problem so far. The DAMON > user-space tool (damo)[2] also dropped support of the in-kernel record feature, > but we received no problem report. > > Also, I believe DAMOS tried regions like feature could provide some level of > information, since it provides snapshot of the monitoring result, which > contains a time data, namely 'age'. > > Could you please further elaborate your aimed use case of this feature and the > advantage compared to other alternatives (tracepoint-based recording or DAMOS > tried regions based snapshot collecting) I mentioned above? > > [1] https://lore.kernel.org/linux-mm/20211011093057.30790-1-sj@kernel.org/ > [2] https://github.com/awslabs/damo > > > Thanks, > SJ > > > > > Signed-off-by: cuiyangpei