From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73B27C4345F for ; Sun, 5 May 2024 05:26:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E58876B0082; Sun, 5 May 2024 01:26:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E08E36B0087; Sun, 5 May 2024 01:26:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D001D6B0088; Sun, 5 May 2024 01:26:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B34446B0082 for ; Sun, 5 May 2024 01:26:31 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5BAC7C04A2 for ; Sun, 5 May 2024 05:26:31 +0000 (UTC) X-FDA: 82083206982.12.BC3A98D Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf03.hostedemail.com (Postfix) with ESMTP id 9868920016 for ; Sun, 5 May 2024 05:26:28 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="h+/CzvMB"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf03.hostedemail.com: domain of irogers@google.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=irogers@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714886788; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eb/EPhjcbAC4KAB4gdgqUAeq8oCgWvlhJ9vOj/P63R8=; b=O6GNDob4M23gZyn90ceICtU1DPyTRAtrV8ygPjYpc41cX8Qng5huZI6+X79OlhmjDIgQCw 8m4Grxoj60uYGbxsGjSkEiDFAgAd2ndqp+LE6SHaFaxIqcFoAiDUow+ZtZSAW6vznJ9QhZ xM8+SKQhQeV4uhACiE9QPbhd6hjvp60= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="h+/CzvMB"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf03.hostedemail.com: domain of irogers@google.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=irogers@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714886788; a=rsa-sha256; cv=none; b=ZRA47oPWz/5127KBHX1b39Rsky9GmBbkE4ymUjSPOUHirEdkWqv/nh+zY1bTEt2WwCJlV+ dFOJVy+Uh/5W+nFKtw3iEVgAcYqK43As/CmC9+Fh9YhjoKxS5Ykk/+fJ25n/jWeEG0MxUc UTcqm1xigNCgyPKtzYlLXShIQG4keKA= Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-43d361a2124so65361cf.0 for ; Sat, 04 May 2024 22:26:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1714886788; x=1715491588; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=eb/EPhjcbAC4KAB4gdgqUAeq8oCgWvlhJ9vOj/P63R8=; b=h+/CzvMBLgkhFWoQpNQPfmHbN1x85E+1baxUjwkdkWruot8JQhMoWkuqQQwDzZG//i a7Cujj7oCIOBFafwB5JTAH8T/O2qAlJuIQlfZhYunAa0O0vFbDaiJmS9PZGnwq9mNYC8 AU+5VFAvLcoYKnone4IDo8rRxY/iIO4pIM4lIbiKR2xkcGp7vJ1/JSXTgPjp+l5DkjE5 u/wq93pJE9JI6ni3MPuHN5iVBNGM+jmMx+4ked72eslRHQwLd9SOrhWzkxOcsGGfA8N1 K7EtFx/VmTLW9BKgefQg1wKiL7Sjd3zJxDJDEpIPRqQzdbPLAyeNj8BdpqUsAPpcwQkP ewqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714886788; x=1715491588; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eb/EPhjcbAC4KAB4gdgqUAeq8oCgWvlhJ9vOj/P63R8=; b=qVJ8kci/juNjCdzV30uK6iOhetKxcjdgfEvDH+awdi+zLqDAsx92uMQYdRg/AV0yva +oJIJbfMWGrwyCMiOrLHfabzsrv2l08FepOXLwgyMkvAuK1T5mGndpi2TzzHQJCzgg8D 3FZGUmiM8+mExBJv9+GtlSGBEE1oc88XQ/9WNMMQu73G11MeYVOdlyUB3eSUX795jo4L VgJx6zqe/bNlCfAU7zskxwvbOeUC9+mw90WDIB4CcIp12hd/reBqtXHsR+R0hmKKeIDS 6Rvf0G4FV8d+r9kaK/wTuR+IWt1j74sUsENeq0POEj5S+dCUBFkuuAzD45cG8+ne/4+a qUgQ== X-Forwarded-Encrypted: i=1; AJvYcCUtOEjdffiMJkQbugXR9QJexAKQKBPjDqeqzGDu0lgvfRaESDXY3c934Tg9NOrv+LUKuFOpS5U6EODRBTIOwEY477c= X-Gm-Message-State: AOJu0Yx6C4rAlJO6rKDCpVZUA3sJXIn1qRC7KRboYHGPv4T8Rp1xmRuA vptsolb0k5AFRFtCOoe4NCtXFlA2u7XQLPZzfpu/WHcD8HPQmboMJWLE0r8AcqxSny9PT5Y55aH 0ZjHLYOamMtpmt/U2VLX/++j1qUBOy+52zWVe X-Google-Smtp-Source: AGHT+IG/zlrczb1H5X6lo6ctQdhFV8VmRyTeunZU4W3zP3ofbT54pBnHB9dhcXJHuB/0kmY7mhPIE4g+wjaH12P2Sn4= X-Received: by 2002:a05:622a:1305:b0:43a:2e2b:eec with SMTP id d75a77b69052e-43d030b28f1mr2462581cf.2.1714886787525; Sat, 04 May 2024 22:26:27 -0700 (PDT) MIME-Version: 1.0 References: <20240504003006.3303334-1-andrii@kernel.org> In-Reply-To: <20240504003006.3303334-1-andrii@kernel.org> From: Ian Rogers Date: Sat, 4 May 2024 22:26:16 -0700 Message-ID: Subject: Re: [PATCH 0/5] ioctl()-based API to query VMAs from /proc//maps To: Andrii Nakryiko Cc: linux-fsdevel@vger.kernel.org, brauner@kernel.org, viro@zeniv.linux.org.uk, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, gregkh@linuxfoundation.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 9868920016 X-Stat-Signature: ikqbg3n3deayzucn14yrmw36xaenye7u X-HE-Tag: 1714886788-197568 X-HE-Meta: U2FsdGVkX19OB0dEcQUj6MKW90sDKh78ftksT2n0PI66TGFwABn7WrEd6+cprGeB21qV9wq2L975KsFtHdEnzNqVNSYDfZgTNsHvTemgpJ7Q6FW2Pm5nltIfbj4E8dtOoEKi0W0PqSzWFKcwdryVeo8OHlmoNxL02VOjRpmCTF6bMDVlIoVH/kUcdBBEKDLzTHJvXwpqKvdyEsMyu/N7xQ+QJDMRJATfgnc7KAcYcTDxgcnF5BbPPWEtw9WCgbkHvjt0ulUKzX0zaHDuf4gzCkfg9uo8Us4yqKH2VcZ4QNy0CCFPKd8w/0F3QGrX8cKntatpMbMoVNzAq0Q+5mEbd1LLwboGYoEreWy6ETpgEQLHO787tQ94+co4CVsPyRTeo9Iifyi3Nzg1DDV3NTX7MSsQZy9Ek8zKF9gljC/nNbh7HnTtBpoU9mO6ivkgZRYgecitOfOMJPh7tyS5l64gXH61K487KH8Lc4FziZGEGNoUPttGR2PiVrutZUPKFBnKIEdz1oqFpH8MIyytyYLNK1V5OqKy3phwwH7EdxoEumexFzCN9hLXAKdBeuBLpz6YNsrHLbhohLAgLtTwegNoooyZT222f7i+2kO6DtyDMYKwS0zLnoZff7bHW7kZOfjnppzmMKaQ6LzLlvp+SiCB8DI1WDKqSyvakdafRcrftqkwN9m0JuPJx5WIxkCLCO/C5YbgJwxZYpX3uiRZSf/hbEplkeMVkXn8Rzj1zMWeCWbItkdtr0DvlqdXtd9rq+vLxzLJEwyhdP74bOv3fDHh56bzIecEkUQyaKKUxmEUtcaPC/46JPtAiH3h1YXw6jA4+7GtGzLE8AtVeX3x6vSeu9dWLm7B+I8/jPCDlj4P0C3KXo0OnPDinwLJhQQ2LvW3WAW2TOma6YGE/zV8w6rBBkrHdr5Evm0vHYRRRTwff4tdMQGWX96Qw9rwkM3Z6ROhM2DpEqko1fwwILiVZ1d /IGKkjIu AkMvn64gXR+FHCKbug8PpKxnjFaAEj1hKbXRmftVDILn17MfMGsbXCi1nazSR/dRHUBo1tMDjPTSeLJgz5OO+gxmPNFw5RYC8Ssxk5HaggQ3jZ/bZDUwfcWdaf90L07h68ykpootrm00GCiXLiI7nHQWntQhUsZArupUJW8JnGMks+hcxNO6cvRJrGH4uEWSr2fwCPGKumvNTa3Pk23vqPG+LIiz1izTSqEc4kig6btahg3We3mTBz+fGVmWiNV+FBChUnOVE1191wMhef1wkaD0sYwcVqYwAcuLK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, May 3, 2024 at 5:30=E2=80=AFPM Andrii Nakryiko = wrote: > > Implement binary ioctl()-based interface to /proc//maps file to allo= w > applications to query VMA information more efficiently than through textu= al > processing of /proc//maps contents. See patch #2 for the context, > justification, and nuances of the API design. > > Patch #1 is a refactoring to keep VMA name logic determination in one pla= ce. > Patch #2 is the meat of kernel-side API. > Patch #3 just syncs UAPI header (linux/fs.h) into tools/include. > Patch #4 adjusts BPF selftests logic that currently parses /proc//ma= ps to > optionally use this new ioctl()-based API, if supported. > Patch #5 implements a simple C tool to demonstrate intended efficient use= (for > both textual and binary interfaces) and allows benchmarking them. Patch i= tself > also has performance numbers of a test based on one of the medium-sized > internal applications taken from production. > > This patch set was based on top of next-20240503 tag in linux-next tree. > Not sure what should be the target tree for this, I'd appreciate any guid= ance, > thank you! > > Andrii Nakryiko (5): > fs/procfs: extract logic for getting VMA name constituents > fs/procfs: implement efficient VMA querying API for /proc//maps > tools: sync uapi/linux/fs.h header into tools subdir > selftests/bpf: make use of PROCFS_PROCMAP_QUERY ioctl, if available > selftests/bpf: a simple benchmark tool for /proc//maps APIs I'd love to see improvements like this for the Linux perf command. Some thoughts: - Could we do something scalability wise better than a file descriptor per pid? If a profiler is running in a container the cost of many file descriptors can be significant, and something that increases as machines get larger. Could we have a /proc/maps for all processes? - Something that is broken in perf currently is that we can race between reading /proc and opening events on the pids it contains. For example, perf top supports a uid option that first scans to find all processes owned by a user then tries to open an event on each process. This fails if the process terminates between the scan and the open leading to a frequent: ``` $ sudo perf top -u `id -u` The sys_perf_event_open() syscall returned with 3 (No such process) for event (cycles:P). ``` It would be nice for the API to consider cgroups, uids and the like as ways to get a subset of things to scan. - Some what related, the mmap perf events give data after the mmap call has happened. As VMAs get merged this can lead to mmap perf events looking like the memory overlaps (for jits using anonymous memory) and we lack munmap/mremap events. Jiri Olsa has looked at improvements in this area in the past. Thanks, Ian > fs/proc/task_mmu.c | 290 +++++++++++--- > include/uapi/linux/fs.h | 32 ++ > .../perf/trace/beauty/include/uapi/linux/fs.h | 32 ++ > tools/testing/selftests/bpf/.gitignore | 1 + > tools/testing/selftests/bpf/Makefile | 2 +- > tools/testing/selftests/bpf/procfs_query.c | 366 ++++++++++++++++++ > tools/testing/selftests/bpf/test_progs.c | 3 + > tools/testing/selftests/bpf/test_progs.h | 2 + > tools/testing/selftests/bpf/trace_helpers.c | 105 ++++- > 9 files changed, 763 insertions(+), 70 deletions(-) > create mode 100644 tools/testing/selftests/bpf/procfs_query.c > > -- > 2.43.0 > >