From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 449D6CFA45C for ; Wed, 23 Oct 2024 17:54:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C8A716B0088; Wed, 23 Oct 2024 13:54:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C39F06B008A; Wed, 23 Oct 2024 13:54:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B00C46B008C; Wed, 23 Oct 2024 13:54:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 926B86B0088 for ; Wed, 23 Oct 2024 13:54:50 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2AAE7A0D5C for ; Wed, 23 Oct 2024 17:54:18 +0000 (UTC) X-FDA: 82705617372.28.CF2D5F4 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf25.hostedemail.com (Postfix) with ESMTP id 05B71A0027 for ; Wed, 23 Oct 2024 17:54:35 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=aSoY5+CF; spf=pass (imf25.hostedemail.com: domain of andrii.nakryiko@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=andrii.nakryiko@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729705935; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uzC0zojgdAaM2rfn3YXXNhX5GBn8pCVofhKBdnpRhHA=; b=26zxgxjyy49JG0LUTwI2JfHjiv467P9ohbv6picIyZ3XdlJjhvo2kLZO3ouAIlr/iTjB4G dI6j3WwiCWnIBKtw8/dIDSX9005EDUXN4sur6CiyUp3hjHBBBV79CEwTJqppjB0NQzmDjU 42gNEGLqTbC79EnXo+3PpcQ96nfYhpQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729705935; a=rsa-sha256; cv=none; b=ZnPkivgbkArEa9RvHvPd0FCLWz8SsrVfjHSVDb1jVFByym/AcRLhsEhHtGtUyIEiuzun0k 436dQh/0NhWi9ukOQ1YXHKUy4znHZ94F1vhd/bvNvx+C9yCPpGUbWll2OK7lAyF1lkazIX BTjrUdiBMc9B05GMdWHkpVL5XdSj2C0= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=aSoY5+CF; spf=pass (imf25.hostedemail.com: domain of andrii.nakryiko@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=andrii.nakryiko@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-71e67d12d04so73508b3a.1 for ; Wed, 23 Oct 2024 10:54:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729706087; x=1730310887; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=uzC0zojgdAaM2rfn3YXXNhX5GBn8pCVofhKBdnpRhHA=; b=aSoY5+CF3Gkvunwj28EEMupIWzbVD6vUZxjN6xa226sQJLCjFz15hzkgOm/GzdQxLk 6wxmaiBRq4RUL5aixPGp/Cysl/soZxxOKpKY8lRErWJKBTC75NDBylAVv1NSQ5jpOKKL FAxNSp+UV0mj46r/26+oXkw9/GcKFHKohDdmtjmESbkZqxK1tX4UuyYu2Ia8IkVXKQg4 kqgxesTTOrZZaIg5uBI6QP7lmdrTmsaSkiig+v9XIDnLewdd6JvtQptqzmvqhfLj7lHb 570uFF1q5xZWH5lKJ7/T4nHtJ63wVkkyNMJtnuaBB5Qkia5M4qoppMYrwQVtQLLgHcBA iV0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729706087; x=1730310887; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uzC0zojgdAaM2rfn3YXXNhX5GBn8pCVofhKBdnpRhHA=; b=ENfubfu9FpkjQfFaOBX3cq6JLhfADcqccX7V0OsyPyXxUhzStm6zNyRFJOzGPQnG3A HxW2rpSqbo1MrGeZ9SsByjfGwR5RTevi+pimz4P9e3Bw4FwG4XXmbqEjCHHxN9tTNbwc P02cDt87EM6Kui4SiP9iE/pBrSp0oJfLriIy+btBJ+P97foT45BjlTum5NnA6s57/8j1 DyoX0RWmno9tMEiR6Me1/oHUD4Roq+pLkYK8EpMC4eHMtLBYJeqwSyvD2gdI7kiIjgMB 13sAOdQ0cB9eF3xaQv/LqweKXzzQUkSgQ8TwzIMf6Cq5w3B8Ud/0yVCZZ+RG7FfuKogH 4JVA== X-Forwarded-Encrypted: i=1; AJvYcCWDZTr7wTuQVviOjTCsw6Cot4LHdt3y7oR0IbYtgylhBOtR+cJxSZYm+QvMAbVk2vVRxO0Ovexj/Q==@kvack.org X-Gm-Message-State: AOJu0YxV7IZDiTTfBeylR+yaLHg4KlOfBvmJ5UUPYbP4bEAU0NSyER02 O6yzoncwefR/Z9ECYVORyRnmFhfIBOQauIc3TxC0p4GSWwj1w9RxRgkP0mKaICx4km2vsj8xjEz wXGPx10VLB4opYFD/1f91B5o/OMY= X-Google-Smtp-Source: AGHT+IH8r6pSZYJrhlvyswY9keD0vhNapeUVfc4WgwLysEX15Sns9APZ9b0HudC/bFg3ZXdudugL5NyKX/kM9UGRi0Y= X-Received: by 2002:a05:6a00:811:b0:71e:680d:5e94 with SMTP id d2e1a72fcca58-72030b61979mr5015362b3a.19.1729706087069; Wed, 23 Oct 2024 10:54:47 -0700 (PDT) MIME-Version: 1.0 References: <20241010205644.3831427-1-andrii@kernel.org> In-Reply-To: <20241010205644.3831427-1-andrii@kernel.org> From: Andrii Nakryiko Date: Wed, 23 Oct 2024 10:54:34 -0700 Message-ID: Subject: Re: [PATCH v3 tip/perf/core 0/4] uprobes,mm: speculative lockless VMA-to-uprobe lookup To: Andrii Nakryiko Cc: linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org, peterz@infradead.org, oleg@redhat.com, rostedt@goodmis.org, mhiramat@kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, jolsa@kernel.org, paulmck@kernel.org, willy@infradead.org, surenb@google.com, akpm@linux-foundation.org, mjguzik@gmail.com, brauner@kernel.org, jannh@google.com, mhocko@kernel.org, vbabka@suse.cz, shakeel.butt@linux.dev, hannes@cmpxchg.org, Liam.Howlett@oracle.com, lorenzo.stoakes@oracle.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 05B71A0027 X-Stat-Signature: 1qm64ka1g574zx794ba4mgkkdi3ybkmf X-HE-Tag: 1729706075-426791 X-HE-Meta: U2FsdGVkX1/94G16m/nxHI2AK8IVG4CfIDmb3NIvkF7N4MiSNXPS2QNXpB7ocnJ5wmn170Ggv83Ydv1LoQ+rd5nZpRTKF8NdfKmKGhpVaHIed363qqh0tkTiSOGWyPWWIg65TyM0silA6gv+9bUcbAI3RoHqQm8AUuKBEwguHA9NQhOkY0HL0gohzS3CvhzA5+IMZ7A3sELHrK9Uonfsst8EYjA3XXUfIHTvB2usjQQ9DD9kEaQ3vBQJW+Brw+VXW8WwxtwOHrP65S4zSxrpr4pWl2qarNejxGVIELpxgtk7iY2Kq2mBRbtBg75aaXpBw3ewGYIxgs8B5oJq2tB4AsKijiLeBlyAOkp3bm/cHjx9sTcV7QZVhCi05aRhEhpoHybqc+7UtdI9r/AUnJwKYOMs6RgJ+VxKkykeb+GZMceD2Q3S1V2+1wVTTSuvR5mPATPsPOhjA/+HBmFIZxwQQehSNSS4NpBKnuofi9DZTeMhelSjRfPXn9gytea/I8+obO0qmaQI57/xvm9UNipQ88BA+tJsGR09GQ8kbMvkG+qEtULozvSrREZHBh+OFIUDeMziwSlMim8z/agCc+cSHF8eX5iHVU7pS+omDow05cNrvsWDTAaF8jIrIxWpjmz/ZntP3mgimzYdtd7REQesZ1SIfLoIvfju2+HNmntj+/2t0CiIdNsiXA8tCi++yzD0hkhnXDcJNxCtVrWyK+qJLMfkuyngpIPDH5/CUNJypkpMQsYFVKnNMt6qNxGYWkdaxMbEW+yOo2Atd2lm/Y52jKAfUpJXss/m9iA9AiRW1wH/luHG4C0QSCtYeZWBukPl9lYwJjCA3Nd8vL2SlIhrXnH3/RNW5ttjY4vPo6OKVRb0aBEg5lSA4aTSPyHyfllIEfrbZ2bZ7UJfVAFQwibzn+IlRw9QFAH/Qxj5HAOpnDXYJyY9grhKlA/pHsnx7mR1k8JmAlXFWwrcTc6htA1 uJijtONv 6edofLZWkOATY3JvXoOgHJNDhEAnzt0jL+NnN/dB5f+YrAa36JECA2oeCohElUsVsiXHSlB4985Wh+wS1KkEvXfp9VPVl4L9xMdTEGSnAjElQsasi0n8Q/jqaNF+Kxacfm+SnPszjRgihh/Eure2C0qfFvaeNODI5DAPxV2nG0Vok4F0QzOhO2T4u48A6SRNV/JM14B1Q7Ldvo2IB8y+pr3AR7KJcFOFduJ9H1YBO+NBBKvyYvIbBazSmgGWX3pSFE6Qm8pzXGPvCRfi4TYj+uIos+VJU1V0TSREJmWdrvQZNP+r/4kb2KMH8OmlYPo57g6NtZ+6iQSzjDOm6k3IwQ6rA0g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Oct 10, 2024 at 1:56=E2=80=AFPM Andrii Nakryiko = wrote: > > Implement speculative (lockless) resolution of VMA to inode to uprobe, > bypassing the need to take mmap_lock for reads, if possible. Patch #1 by = Suren > adds mm_struct helpers that help detect whether mm_struct was changed, wh= ich > is used by uprobe logic to validate that speculative results can be trust= ed > after all the lookup logic results in a valid uprobe instance. Patch #2 > follows to make mm_lock_seq into 64-bit counter (on 64-bit architectures)= , as > requested by Jann Horn. > > Patch #3 is a simplification to uprobe VMA flag checking, suggested by Ol= eg. > > And, finally, patch #4 is the speculative VMA-to-uprobe resolution logic > itself, and is the focal point of this patch set. It makes entry uprobes = in > common case scale very well with number of CPUs, as we avoid any locking = or > cache line bouncing between CPUs. See corresponding patch for details and > benchmarking results. > > Note, this patch set assumes that FMODE_BACKING files were switched to ha= ve > SLAB_TYPE_SAFE_BY_RCU semantics, which was recently done by Christian Bra= uner > in [0]. This change can be pulled into perf/core through stable > tags/vfs-6.13.for-bpf.file tag from [1]. > > [0] https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git/commit/= ?h=3Dvfs-6.13.for-bpf.file&id=3D8b1bc2590af61129b82a189e9dc7c2804c34400e > [1] git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git > > v2->v3: > - dropped kfree_rcu() patch (Christian); > - added data_race() annotations for fields of vma and vma->vm_file which = could > be modified during speculative lookup (Oleg); > - fixed int->long problem in stubs for mmap_lock_speculation_{start,end}(= ), > caught by Kernel test robot; > v1->v2: > - adjusted vma_end_write_all() comment to point out it should never be ca= lled > manually now, but I wasn't sure how ACQUIRE/RELEASE comments should be > reworded (previously requested by Jann), so I'd appreciate some help th= ere > (Jann); > - int -> long change for mm_lock_seq, as agreed at LPC2024 (Jann, Suren, = Liam); > - kfree_rcu_mightsleep() for FMODE_BACKING (Suren, Christian); > - vm_flags simplification in find_active_uprobe_rcu() and > find_active_uprobe_speculative() (Oleg); > - guard(rcu)() simplified find_active_uprobe_speculative() implementation= . > > Andrii Nakryiko (3): > mm: switch to 64-bit mm_lock_seq/vm_lock_seq on 64-bit architectures > uprobes: simplify find_active_uprobe_rcu() VMA checks > uprobes: add speculative lockless VMA-to-inode-to-uprobe resolution > > Suren Baghdasaryan (1): > mm: introduce mmap_lock_speculation_{start|end} > > include/linux/mm.h | 6 ++-- > include/linux/mm_types.h | 7 ++-- > include/linux/mmap_lock.h | 72 ++++++++++++++++++++++++++++++++------- > kernel/events/uprobes.c | 52 +++++++++++++++++++++++++++- > kernel/fork.c | 3 -- > 5 files changed, 119 insertions(+), 21 deletions(-) > > -- > 2.43.5 > This applies cleanly to tip/perf/core with or without Jiri's patches ([0]). No need to rebase and resend, this is ready to go in. [0] https://lore.kernel.org/all/20241018202252.693462-1-jolsa@kernel.org/