From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E0B3D44D55 for ; Wed, 6 Nov 2024 12:03:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E630D6B0088; Wed, 6 Nov 2024 07:03:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E132A6B00A8; Wed, 6 Nov 2024 07:03:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB4196B00A9; Wed, 6 Nov 2024 07:03:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id AA9B16B0088 for ; Wed, 6 Nov 2024 07:03:41 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6D44A1C7AEE for ; Wed, 6 Nov 2024 12:03:41 +0000 (UTC) X-FDA: 82755535044.05.96D270F Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) by imf22.hostedemail.com (Postfix) with ESMTP id 06FBEC002B for ; Wed, 6 Nov 2024 12:02:56 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf22.hostedemail.com: domain of breno.debian@gmail.com designates 209.85.208.54 as permitted sender) smtp.mailfrom=breno.debian@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730894535; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mAdKCHu4KzxDCkkQKBDhoW2UK3wGCXxAFsKHJYg1LAI=; b=56xe37XIrpJDZeiZZpw5zoZyjsriAdBgcue2qW9ADZIKJwiQ7kcpusr2s3jpeWIuuxSWcK a6lV0H7SBvgNfaPujiPa9oyj6vt0FcVfYKNrf5V9Q8XOJCJjnOf2kOJO8svMvoiPLvoYZV sbVqMZ+dPyahyM99zTv1/XkD8RhCyJs= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf22.hostedemail.com: domain of breno.debian@gmail.com designates 209.85.208.54 as permitted sender) smtp.mailfrom=breno.debian@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730894535; a=rsa-sha256; cv=none; b=NQ/nu3dCIy4djeFkoJYkqn5/quI8+U0jy4EImOnCTs5/z30TGOhWLUDTv/yqx1JbRz3YFn lUE005K8p+2d/sS13XpN6096vj6PIpwsYSd2FuXi3u06uTv2vbcM7p/7VwZjcv+yYzq9gg VqWl4qDyjuoK/a/sL1KX+Y0iYXq1CjA= Received: by mail-ed1-f54.google.com with SMTP id 4fb4d7f45d1cf-5cb6ca2a776so9934173a12.0 for ; Wed, 06 Nov 2024 04:03:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730894618; x=1731499418; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=mAdKCHu4KzxDCkkQKBDhoW2UK3wGCXxAFsKHJYg1LAI=; b=jrNAwfyoZyhUlF8YUJU8E9vOIzN+XSmiLAWWtCVzuzg6doqw8ddZc3kR3LBav2ex31 6IgnHJIcexFtdqkYI3yX2bqRkORUrkwZH8ppFtA97E7kSztb95J55+EPmwOr4/WOYYz2 9L8rqMMhjk/BdEJH4PCNUtQnEPRkOLn/HIYjRj6MssR3IlsDkJMCAVh9hBRVCNcoospn WAFCeBWHa0RePyO+oh+lzOGN/GwXIbqclIZjD6huBUZJK8fpBLpXceoWLdBK97po1hM/ n33s6jRBtYKyZFTgRWUOieHuRVOdwRc+W9/NmuV42LPVzZ+kdtT6xVYApd7IqCTcvCMi HTwQ== X-Forwarded-Encrypted: i=1; AJvYcCW7r/bzazYYvrdXWzkZYNUV1VnYlx4vxU7uDi9ftItJvQEAKpAExk0rLd3P+rIifIOPWvAxiwXb8A==@kvack.org X-Gm-Message-State: AOJu0YyLGPVrbGH91YixA4SsnROVEqbmOAgDvonRyDZm4WHq21QIKwVQ wqDRnbZ4Lefo8KSEtnYIZp0lAcHuIYC46rS0UvQa7tH+cXiMbCEd X-Google-Smtp-Source: AGHT+IGzKy/R2dqbY68xlQLh8qj6KcnlqGVzvzo5n76OGX3XV31gMGmie4AvwvKPTMNWMeaxOfq1Aw== X-Received: by 2002:a05:6402:2187:b0:5cb:6718:7326 with SMTP id 4fb4d7f45d1cf-5cd54a958f5mr18094994a12.21.1730894617844; Wed, 06 Nov 2024 04:03:37 -0800 (PST) Received: from gmail.com (fwdproxy-lla-003.fbsv.net. [2a03:2880:30ff:3::face:b00c]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5cee6aafc8dsm2713695a12.24.2024.11.06.04.03.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Nov 2024 04:03:37 -0800 (PST) Date: Wed, 6 Nov 2024 04:03:34 -0800 From: Breno Leitao To: Andrii Nakryiko Cc: linux-trace-kernel@vger.kernel.org, peterz@infradead.org, oleg@redhat.com, rostedt@goodmis.org, mhiramat@kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org, jolsa@kernel.org, paulmck@kernel.org, willy@infradead.org, surenb@google.com, akpm@linux-foundation.org, linux-mm@kvack.org Subject: Re: [PATCH v5 4/8] uprobes: travers uprobe's consumer list locklessly under SRCU protection Message-ID: <20241106-transparent-athletic-ammonite-586af8@leitao> References: <20240903174603.3554182-1-andrii@kernel.org> <20240903174603.3554182-5-andrii@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240903174603.3554182-5-andrii@kernel.org> X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 06FBEC002B X-Stat-Signature: jnnzsszhj1b5ar8sxctzwxupojwntufh X-HE-Tag: 1730894576-854177 X-HE-Meta: U2FsdGVkX1+8YSwiVytuSNLs7N0UkBkrSltxuaP1d7KzJTeOdYDW2bpaQ11Akef+doIzw0RyBd9ca/fgSHAfsLGheA6l2w9qlzsFh1QTZvtO7wsiVZSG2lqKbDKP1vHMQuf0RR7+Lza0Yl7nAPDtOZ0w1KE6hdmd14SSC65ldIwfqPGyPFTbC7R3wQ+kG13cs0mtllkMH+/umZt8nrmNjPkL/lLJR614Hqsi+pM71foE4r1VhG0eUVROONTJd0JG3y1xsEEoLbW3aPsZypXv4KBkFZxqV2jW4i0SfYQ4wCykttcJvPM7BJaJtreLV3RoCSlDAIbDAmUdWVL08deVo1cL5eIIqnO26bvIIvsngdrm1+9J24paXn5lxKuXhnL1hPQtoXtlaNc41jVxU4WEjXdso9wUOgfHstZVKraf1uZX+iDHFbGLEwajTeBFKC8hbQwAfTgQ4oXcc5YaanG+e0clQzAeLUNnWzFwXeAPkx8kzvQfbjMvCeb3ilYwz5djbrVbVk7kj7214IQIvaQG+Tgs2JWAVBjcXkmvZJZ2RYpBQVGcn8FNzSM44uV7ygPNiyDYBYDwkeFAg000X41McW1h6YZX+aWxXBfx0JaqKLp1W4TU9gCk3G7WeGFxtljCkYHN7yYbmpkS802UWu0pkSd8qHRQYtkim2oEqmDk25bVeaBmKe/Qy6AEAMNxk5CE+YlVOQtphf5l7fUIpBiazVX6eDVOMQ+3+G+/G09Dx7RFSmNTOCOfbGQonOxyCA5i7YSNhDpkJJjbxS8xUhEuXEQzlNJuw5/XSIimcVd5zOiIO3X84xX087MgBMZKJxKYLsJU27e6VjQHdB7yyygN9d3ilqhdVsTcGJrG608JyWkQuq8KLl2yCpKiowkancxyU2xr5dO/CFysYCYouZRvLChoGJcq/GU5/4GMgDCjYy6M5Hs8e3Lhvi2NTcsS/E8eCAxOjv1F+N2E0HHK7qN Ukgp3UyK duTJ9pSR9QOwYTv8TIQ7rdG5refQ6WpLjIWGzWarn3inAuHDRqVwKYsxHorNLE2bbM5ojp3JHRe1xFJG98J5tKcek7Ulepcn3CkpBZZnMi4eQ5kcDCWgOWem9XQTjLyVA1CjqqxPt7PwMAOVS54Ncv0fZJR2orxAXJoLJUoEJNsNWVsaN2jMUhiAqtgzRDk1K1pgrHjnLQw3CpZ9HD4QSjn9SImw0620yJXHoANUfVi12MZKJKzW8O5gwoFzZp71a6RWBslAewh3NVc4FPKYuBoRPwyX4DtkWBA+9rLLcvp+YXbsZsiJb5v46ib74usoFqny9veNBCTyJJ+p2+pD4fRCagYWQjpjlQphppQXVSf3DBiJy+Qmha+7XgagmfbhkVOHtdbUlYeFlp4Ll7Ht5drqG+aP6F/kyDBfqyNN1oWsjNDY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hello Andrii, On Tue, Sep 03, 2024 at 10:45:59AM -0700, Andrii Nakryiko wrote: > uprobe->register_rwsem is one of a few big bottlenecks to scalability of > uprobes, so we need to get rid of it to improve uprobe performance and > multi-CPU scalability. > > First, we turn uprobe's consumer list to a typical doubly-linked list > and utilize existing RCU-aware helpers for traversing such lists, as > well as adding and removing elements from it. > > For entry uprobes we already have SRCU protection active since before > uprobe lookup. For uretprobe we keep refcount, guaranteeing that uprobe > won't go away from under us, but we add SRCU protection around consumer > list traversal. I am seeing the following message in a kernel with RCU_PROVE_LOCKING: kernel/events/uprobes.c:937 RCU-list traversed without holding the required lock!! It seems the SRCU is not held, when coming from mmap_region -> uprobe_mmap. Here is the message I got in my debug kernel. (sorry for not decoding it, but, the stack trace is clear enough). WARNING: suspicious RCU usage 6.12.0-rc5-kbuilder-01152-gc688a96c432e #26 Tainted: G W E N ----------------------------- kernel/events/uprobes.c:938 RCU-list traversed without holding the required lock!! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 3 locks held by env/441330: #0: ffff00021c1bc508 (&mm->mmap_lock){++++}-{3:3}, at: vm_mmap_pgoff+0x84/0x1d0 #1: ffff800089f3ab48 (&uprobes_mmap_mutex[i]){+.+.}-{3:3}, at: uprobe_mmap+0x20c/0x548 #2: ffff0004e564c528 (&uprobe->consumer_rwsem){++++}-{3:3}, at: filter_chain+0x30/0xe8 stack backtrace: CPU: 4 UID: 34133 PID: 441330 Comm: env Kdump: loaded Tainted: G W E N 6.12.0-rc5-kbuilder-01152-gc688a96c432e #26 Tainted: [W]=WARN, [E]=UNSIGNED_MODULE, [N]=TEST Hardware name: Quanta S7GM 20S7GCU0010/S7G MB (CG1), BIOS 3D22 07/03/2024 Call trace: dump_backtrace+0x10c/0x198 show_stack+0x24/0x38 __dump_stack+0x28/0x38 dump_stack_lvl+0x74/0xa8 dump_stack+0x18/0x28 lockdep_rcu_suspicious+0x178/0x2c8 filter_chain+0xdc/0xe8 uprobe_mmap+0x2e0/0x548 mmap_region+0x510/0x988 do_mmap+0x444/0x528 vm_mmap_pgoff+0xf8/0x1d0 ksys_mmap_pgoff+0x184/0x2d8 That said, it seems we want to hold the SRCU, before reaching the filter_chain(). I hacked a bit, and adding the lock in uprobe_mmap() solves the problem, but, I might be missing something, since I am not familiar with this code. How does the following patch look like? commit 1bd7bcf03031ceca86fdddd8be2e5500497db29f Author: Breno Leitao Date: Mon Nov 4 06:53:31 2024 -0800 uprobes: Get SRCU lock before traverseing the list list_for_each_entry_srcu() is being called without holding the lock, which causes LOCKDEP (when enabled with RCU_PROVING) to complain such as: kernel/events/uprobes.c:937 RCU-list traversed without holding the required lock!! Get the SRCU uprobes_srcu lock before calling filter_chain(), which needs to have the SRCU lock hold, since it is going to call list_for_each_entry_srcu(). Signed-off-by: Breno Leitao Fixes: cc01bd044e6a ("uprobes: travers uprobe's consumer list locklessly under SRCU protection") diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 4b52cb2ae6d62..cc9d4ddeea9a6 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1391,6 +1391,7 @@ int uprobe_mmap(struct vm_area_struct *vma) struct list_head tmp_list; struct uprobe *uprobe, *u; struct inode *inode; + int srcu_idx; if (no_uprobe_events()) return 0; @@ -1409,6 +1410,7 @@ int uprobe_mmap(struct vm_area_struct *vma) mutex_lock(uprobes_mmap_hash(inode)); build_probe_list(inode, vma, vma->vm_start, vma->vm_end, &tmp_list); + srcu_idx = srcu_read_lock(&uprobes_srcu); /* * We can race with uprobe_unregister(), this uprobe can be already * removed. But in this case filter_chain() must return false, all @@ -1422,6 +1424,7 @@ int uprobe_mmap(struct vm_area_struct *vma) } put_uprobe(uprobe); } + srcu_read_unlock(&uprobes_srcu, srcu_idx); mutex_unlock(uprobes_mmap_hash(inode)); return 0;