From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0303CFA463 for ; Wed, 23 Oct 2024 20:03:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 597FA6B00A1; Wed, 23 Oct 2024 16:03:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 548446B00A2; Wed, 23 Oct 2024 16:03:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E80B6B00A3; Wed, 23 Oct 2024 16:03:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 15C9E6B00A1 for ; Wed, 23 Oct 2024 16:03:10 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 435EA1C5F6A for ; Wed, 23 Oct 2024 20:02:49 +0000 (UTC) X-FDA: 82705940562.18.4DB70B1 Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) by imf24.hostedemail.com (Postfix) with ESMTP id E46B518002D for ; Wed, 23 Oct 2024 20:03:04 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OphGpQXl; spf=pass (imf24.hostedemail.com: domain of andrii.nakryiko@gmail.com designates 209.85.215.179 as permitted sender) smtp.mailfrom=andrii.nakryiko@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729713635; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jP73v8SKOj9OMtC9FJHv9JWI9rQlYgH+l/YZtxHOFuU=; b=2NqAbIeL8zy69nNJ7r9ABCunM82wukx9CfZKbBewbHxF3e8QBvuOkMAS7QyImK8MMcvy/o YDVn7KWsdZRIKHdwjf414i6h8CpymHaIkpsmkXV9HCHQb/Fw1b6XK37rBBPDC551PtNTTT +TAMAhGAtf2zVTG2rW1P8dfro35pYVU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729713635; a=rsa-sha256; cv=none; b=2lO8nTeBnJ8XbKCwNm7IxEi2pMjRKHz0V6kLGQLTzeDMi6n/RrIz0QRD0KFbxfWV/4LOIU WjEQMvZ9nE8a+cqP66J7TUUW4ht5W2MBU45dm+lO95vtmPGp3xZJ5EmPUnFBZ3TWKuUbgl +xaK0gBFxFk3aHHQKND3WlXVToihViA= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OphGpQXl; spf=pass (imf24.hostedemail.com: domain of andrii.nakryiko@gmail.com designates 209.85.215.179 as permitted sender) smtp.mailfrom=andrii.nakryiko@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pg1-f179.google.com with SMTP id 41be03b00d2f7-7d4fa972cbeso110417a12.2 for ; Wed, 23 Oct 2024 13:03:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729713786; x=1730318586; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=jP73v8SKOj9OMtC9FJHv9JWI9rQlYgH+l/YZtxHOFuU=; b=OphGpQXlB8ivN9WMF2VcrhPoRxip7qzu9cXijKFskGNAJFQrKrEs0PHUr1gexaBJAA 43WuU0irPQB12DKm6GkDHYniRJ9QQbnOruMBFsT2WS6xLQWGGUGT2IFn/bp2Mpy8EolX 0Qla+mI+XlzXlJLaiKhCnoESxYEEgGLCGxd2+5xtAaKcyYF1Ou0Y/wG/OYAUZfCD9xt3 E+8GSNYA6pfA1qRr3JkB8Jri7HmTHcrugpOgkmJ1VjHAf/BkPYAV0ld6uJBO772aIc1y Q8T5jnAZye8MRyPus9gkr8tD1EyIHzfHeCk/jnQ9BffmvShe/BacA08EFqpV0BeOQJ/W XmJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729713786; x=1730318586; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jP73v8SKOj9OMtC9FJHv9JWI9rQlYgH+l/YZtxHOFuU=; b=CpxJtncS3jkhnOfNCqXPY+wC07zZQvyHtnmfmQKRoYxVpmgaPpGgqGjbmMEzvNj/dB 4FuKW5/lgLTD+T2St/5uuBGs2kJXo1uGY68K80jcRbmy0eXGUGZQRIu92DZWHoJZGppv lNsZ1rzeXOpAnaox4PDgAgxMWDN0JB9puAB+IkyS1guSHW4+t1tojNnxhW9v07kaDtBP /NTGWiHIU82C603Zn2Tx6vTZqM73RTZdHr3Oj/9F/dGlBMMFGhkdIVtw6EKMX+kOPr1p W8R/F2ZJ5ygEGWQAL3eq3CxMA/6ORQyAIydzmRRwE+zgpuDJp1aO+kwUfH9ppxN9P0FO U68A== X-Forwarded-Encrypted: i=1; AJvYcCVKCVq8Glj027PWhacMXo85B/sc+DIwPmAeCuVLq/jyqnRAK76A+RfnJoPAuVu9YiYFHyA0oO/+OQ==@kvack.org X-Gm-Message-State: AOJu0Ywh/MMIPPWOG9GCmtanXvW378CPnGcfKQVM0TPLe84ligwFSiDL YJPvYUV0Hfiwgx5VYLmsJzFWJuJ3ajURfKdW4C9FAf4UrE2uLk+/kSw+A0IOXilgh5C3C54gKlA IKgNAzc3aTtkV7ZMuOCt6h3EiVhs= X-Google-Smtp-Source: AGHT+IHwR2GYCMNX76w7QATFiO6JTAScsQkOxdJ6G3+ZLHIRRdkUSVzlfRDByZGqrvy1PYzkVT4QRkJbMz+6KJX32Cc= X-Received: by 2002:a05:6a21:1646:b0:1d7:2249:689 with SMTP id adf61e73a8af0-1d978b990ecmr4164984637.33.1729713786360; Wed, 23 Oct 2024 13:03:06 -0700 (PDT) MIME-Version: 1.0 References: <20241010205644.3831427-1-andrii@kernel.org> <20241010205644.3831427-5-andrii@kernel.org> <20241023192236.GB11151@noisy.programming.kicks-ass.net> In-Reply-To: <20241023192236.GB11151@noisy.programming.kicks-ass.net> From: Andrii Nakryiko Date: Wed, 23 Oct 2024 13:02:53 -0700 Message-ID: Subject: Re: [PATCH v3 tip/perf/core 4/4] uprobes: add speculative lockless VMA-to-inode-to-uprobe resolution To: Peter Zijlstra Cc: Andrii Nakryiko , linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org, oleg@redhat.com, rostedt@goodmis.org, mhiramat@kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, jolsa@kernel.org, paulmck@kernel.org, willy@infradead.org, surenb@google.com, akpm@linux-foundation.org, mjguzik@gmail.com, brauner@kernel.org, jannh@google.com, mhocko@kernel.org, vbabka@suse.cz, shakeel.butt@linux.dev, hannes@cmpxchg.org, Liam.Howlett@oracle.com, lorenzo.stoakes@oracle.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: E46B518002D X-Stat-Signature: r5zi4eor18kwn1oxmwr4gt98zxhqpshc X-HE-Tag: 1729713784-163518 X-HE-Meta: U2FsdGVkX19Wcjc8DVFXXZfX3bDOPCFOKzCHcVM3YEMHC2BfL3V8slB4y2OZa5ddpzDQUqQWla6VFh4A6GiG8LYSHVFCX2Dyzs8SDPu3I7RZnIKFN5bUyoHXnrp5FT8wCDWqovPvSRtf5svXbKBqWsh6k7vk7FTFOXwsuDe2axUosLs6x1Ej22ahdiewoWgniClSTKkHXDt2F4MXyThoe4EhFKe1RUlwiN1xQXkG7IFYEjsrUpMyYX4YoHlMzhH5bsfSWYVYbSUj798XVmryENPscq4Zs6hxY1WK5C5qof8K+YxuvFx+8wsmH8bTLFpLmCWhpiZtP0OaR1p+/AM0WvZSXCGgmJOeS5qgKo/WrwXd6FAGlxtwogSritgejnyp/Ln/l6QZdT3n95P/83sJBucfZ3OOKinGJf6JLoZTmjc7LMsN6X6wU2d5szJ5I8D8ad7y1viAqz8kK0o3AY2c33Bwu58Wq2E1qUNYGqvJLVcgBwttdzFgXYBus6lvwQU77j4zBtLPOP9UnQjsN1psYoY8J6a0BS7RhSGDvmN8ld3cjJnzn8nKurSxEHqfEC6FxVmsSqUuZoEOyrTvu4l3SHb+n/joJqkQh/LiiYUTMNUqlm269CZYHXQRMPac/GG+n7Z22VKlDs1KBfcptI80GcMobetIMyjyGb7j6bHV4mY9CEbAphGtV2s4/4UrAf8NY0ADHNrizl+v2Ow/2kN8b3aPRQgQKa+A2csClqkCcV5uoMIPU0tT3kk6UQqAp5L+GW3ws/zS2tP9jJhsgfrMfkoe/bgp2xC/l+asebuICsNJzV4wuzcuOrehyKaEoWFwAhrWibypSNavyDhYx+lzSTOSW4hgfrfDwJrhB+cfbSxxUwegOD8Z/FAHII4j2CyK4pH+X1a1uHMtVFcRT3dlMGRW/on/KlE91rdCwdRfL8FbuuSWvNsyxjKKN06tihSR4emOZACz1Z15spDE3i/ 8+HY1e6M 6gudkkCXSJQVcSxHiJCYnteRBgUOw9FT5grgEruG1dFFwD9r1dYfUI9YP7ig5Jczag3zjBLQyLLGLnlSs+aXth4dwyl7shBSCCRha+5WRFR49aM4yMv7J6CA+cQ9ToW37DW9QEll1xtJEqc9TwGSEkDSaOBKOCcNXq/idHXqwNBKnli4tsxbfuU5USNATSM7+T+pVcLecfrIRsYIh+JlhYIRdJN6aBoymoge8P1g0Q08MzE58ZIw7NHpSdmV3wmQSG0voL2Vn+opSBbXcY/XjbMIfT+lK2EkbvITxrtJFUHj+fC2d20+OpoPkaaTbd5bLME3bSJg6appFTJ6550C//W801aYQVZ9g2le/qI4GBNhVVUbXHaPaRHYkQSxTmZ+v0+XzgIah+9buQEAAYx+d0W5Lt+emLf4hdPE6W/I9TqrUfc6LvUnM+fAqig== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Oct 23, 2024 at 12:22=E2=80=AFPM Peter Zijlstra wrote: > > On Thu, Oct 10, 2024 at 01:56:44PM -0700, Andrii Nakryiko wrote: > > > Suggested-by: Matthew Wilcox > > I'm fairly sure I've suggested much the same :-) I'll add another Suggested-by, didn't mean to rob anyone of credits :) > > > Signed-off-by: Andrii Nakryiko > > --- > > kernel/events/uprobes.c | 50 +++++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 50 insertions(+) > > > > diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c > > index fa1024aad6c4..9dc6e78975c9 100644 > > --- a/kernel/events/uprobes.c > > +++ b/kernel/events/uprobes.c > > @@ -2047,6 +2047,52 @@ static int is_trap_at_addr(struct mm_struct *mm,= unsigned long vaddr) > > return is_trap_insn(&opcode); > > } > > > > +static struct uprobe *find_active_uprobe_speculative(unsigned long bp_= vaddr) > > +{ > > + struct mm_struct *mm =3D current->mm; > > + struct uprobe *uprobe =3D NULL; > > + struct vm_area_struct *vma; > > + struct file *vm_file; > > + struct inode *vm_inode; > > + unsigned long vm_pgoff, vm_start; > > + loff_t offset; > > + long seq; > > + > > + guard(rcu)(); > > + > > + if (!mmap_lock_speculation_start(mm, &seq)) > > + return NULL; > > So traditional seqcount assumed non-preemptible lock sides and would > spin-wait for the LSB to clear, but for PREEMPT_RT we added preemptible > seqcount support and that takes the lock to wait, which in this case is > exactly the same as returning NULL and doing the lookup holding > mmap_lock, so yeah. > yep, and on configurations with CONFIG_PER_VMA_LOCK=3Dn this will always return false > > + > > + vma =3D vma_lookup(mm, bp_vaddr); > > + if (!vma) > > + return NULL; > > + > > + /* vm_file memory can be reused for another instance of struct fi= le, > > Comment style nit. mechanical memory, sorry, missed this one > > > + * but can't be freed from under us, so it's safe to read fields = from > > + * it, even if the values are some garbage values; ultimately > > + * find_uprobe_rcu() + mmap_lock_speculation_end() check will ens= ure > > + * that whatever we speculatively found is correct > > + */ > > + vm_file =3D READ_ONCE(vma->vm_file); > > + if (!vm_file) > > + return NULL; > > + > > + vm_pgoff =3D data_race(vma->vm_pgoff); > > + vm_start =3D data_race(vma->vm_start); > > + vm_inode =3D data_race(vm_file->f_inode); > > So... seqcount has kcsan annotations other than data_race(). I suppose > this works, but it all feels like a bad copy with random changes. I'm not sure what this means... Do I need to change anything? Drop data_race()? Use READ_ONCE()? Do nothing? > > > + > > + offset =3D (loff_t)(vm_pgoff << PAGE_SHIFT) + (bp_vaddr - vm_star= t); > > + uprobe =3D find_uprobe_rcu(vm_inode, offset); > > + if (!uprobe) > > + return NULL; > > + > > + /* now double check that nothing about MM changed */ > > + if (!mmap_lock_speculation_end(mm, seq)) > > + return NULL; > > Typically seqcount does a re-try here. I'd like to keep it simple, we have fallback to locked version in case of a= race > > > + > > + return uprobe; > > +}