From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D40FCCF886F for ; Sat, 5 Oct 2024 12:07:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 543426B036A; Sat, 5 Oct 2024 08:07:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F3BF6B036B; Sat, 5 Oct 2024 08:07:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 393AB6B036C; Sat, 5 Oct 2024 08:07:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 13AA26B036A for ; Sat, 5 Oct 2024 08:07:08 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 7B97D160219 for ; Sat, 5 Oct 2024 12:07:07 +0000 (UTC) X-FDA: 82639422894.21.3CBC450 Received: from smtpout.efficios.com (smtpout.efficios.com [167.114.26.122]) by imf22.hostedemail.com (Postfix) with ESMTP id AA526C0007 for ; Sat, 5 Oct 2024 12:07:05 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=efficios.com header.s=smtpout1 header.b=BZMj2elV; dmarc=pass (policy=none) header.from=efficios.com; spf=pass (imf22.hostedemail.com: domain of mathieu.desnoyers@efficios.com designates 167.114.26.122 as permitted sender) smtp.mailfrom=mathieu.desnoyers@efficios.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728129984; a=rsa-sha256; cv=none; b=iJyapB2qjKxBuTKS7tlJlvUq7zBbdJ6qIsQp6gVvvZhnWK2FJ89w58b7SZaHJ3dGw1dYPu jE6Y53DNne6SjqekSgySVW2HjffmsOAsZcS/1yNKo95vK66VHB4lpCiRlb9PagR1VK9oSA F/HLu768O+/gaDazBEcGV2+iBhJdKrA= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=efficios.com header.s=smtpout1 header.b=BZMj2elV; dmarc=pass (policy=none) header.from=efficios.com; spf=pass (imf22.hostedemail.com: domain of mathieu.desnoyers@efficios.com designates 167.114.26.122 as permitted sender) smtp.mailfrom=mathieu.desnoyers@efficios.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728129984; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EhyEwJ1oibzLrxjkZNBJH2Er68WJDAJz6jiqcmasd9M=; b=xbhFrS9P9DV3KRo4ML8DHXjZjm+od0MAgiT7t7uiLPyzUWTto9aQRRVJxCPN+dXygFaLK/ WKv3JOC0hh0XGODHlwEiiEmDvZZzfSLp5EX3HNsl9eV//QUWZva67Xr/oTS02sMinvAMyj x+sEpp16tvb/yZsuHa3yCBTbY+zJvv0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=efficios.com; s=smtpout1; t=1728130024; bh=hU5q9QUikjXmWkbzka9caDwMGlVJ8fnlLNqMoTuILEk=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=BZMj2elVuboGHieaGO34F9soADypE9CImibr9QZpx07zJPFd7FOpQgnvZo1V+nGmb dgb7CwaMa1NNo94MOmpv9HtSbEy1BFxxrpD/vwLXFm34FQk78S9bLzLq4Y/QnrVHtR CV3fPxRY3vdtx5NIV88S7E6wRlMjKDWIb+xwpke1BBt6owZwAGYYjfUR7bpgK+VLXa 9lbzJ1uUZDE0npSLEvBln7UfuFd4R3FHoqRYtVFt0dpTpvPtjGc36l0KQWWj+Bcrbc oqPFfjGHRtJ7m0B+EZfy20OMdoh2ajf+opXl+0PBbUmJFQqjwNISIIeD8WQ6sK3EOT daOVjRS1nRwHA== Received: from [IPV6:2606:6d00:100:4000:cacb:9855:de1f:ded2] (unknown [IPv6:2606:6d00:100:4000:cacb:9855:de1f:ded2]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4XLPKh4hQwzXwk; Sat, 5 Oct 2024 08:07:04 -0400 (EDT) Message-ID: <69c7e71a-2076-4fa7-90f3-534b52d74345@efficios.com> Date: Sat, 5 Oct 2024 08:05:05 -0400 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH v2 3/4] hp: Implement Hazard Pointers To: Joel Fernandes Cc: Boqun Feng , linux-kernel@vger.kernel.org, Linus Torvalds , Andrew Morton , Peter Zijlstra , Nicholas Piggin , Michael Ellerman , Greg Kroah-Hartman , Sebastian Andrzej Siewior , "Paul E. McKenney" , Will Deacon , Alan Stern , John Stultz , Neeraj Upadhyay , Frederic Weisbecker , Josh Triplett , Uladzislau Rezki , Steven Rostedt , Lai Jiangshan , Zqiang , Ingo Molnar , Waiman Long , Mark Rutland , Thomas Gleixner , Vlastimil Babka , maged.michael@gmail.com, Mateusz Guzik , Jonas Oberhauser , rcu@vger.kernel.org, linux-mm@kvack.org, lkmm@lists.linux.dev References: <20241004182734.1761555-1-mathieu.desnoyers@efficios.com> <20241004182734.1761555-4-mathieu.desnoyers@efficios.com> From: Mathieu Desnoyers Content-Language: en-US In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Queue-Id: AA526C0007 X-Rspamd-Server: rspam01 X-Stat-Signature: iaokwrqet5ywrfeu538xdyhm5ywjgzbz X-HE-Tag: 1728130025-238409 X-HE-Meta: U2FsdGVkX1+Uq0flfoaRiUnWUm3sxAqYFqgDsCGsxdvK9gfGc3AChRZG7SEFMMfkPMrEwb8jqPU9AbDK6YfUyqTdKHlGL1CatFRKiWGJ6MYdoNP/0QjG6Oj9TmQqwx1/zIIF7lPONV/5tptVeOhHquUWls0XT2fRi5mmJWJSU8spsyWnPZEVOJJQrtRb6sUoJAYlwMTC874cO4GSyHBmXAFEBYs6BvoaK9RlCzkXDw4+zam3SGGW9yjmednmK6t3JgEUmfkkk21WKvsfzc3cIxDF+aoPrZGrxddR2l+OJCgFoxpl1jpzoBXQ2BR7Kj7ItnLy1gLQfrNwmN4UzjmV1mwPInjfQbxb/h/HgpuzXwnBFFkq7kZyRbae4CXPT8VOMykgq28ktyfKorhM34x2OCPA649JBJDcbeJD/QklXOZHgmmK6FSLSnFn1cQBs4n3O2jD2itTD5QnLSG7BeJxw5FvIk3Ur3lgQXsFKlP2u5q+KYAy8zW+s8ss0w6fntmtZEVUQCCGtB+kwFnU3NsJQ+gi9nkmbIT+btyoQwP05CTNTZ6XdYUNsqeFoZDjxKP70kFbhEzTseIY+zNbWPv/XyWxO3w+yOe3a/+USIB7abymC/1soOd4WPCQscijT7kLhMqXZKU4tsuFCn0uWIjzkiPViEcgMGal61814J6AjWAHY0XaqCfYLfRr5Tn6FIE3wRiPbXrP5/KdERb+4xbm+1wsP88lEOXOUfP4xmFOBorJ2RaXltHH7jyVPXFbB6o5cI86crwquzjwVnmimj+VmBx2qxNbtLCe4RMcdbsuaFtFvWStQOgFO/HjpgDgS4++idVYRNbikfmmKEFDlVPUO3/4QhNSbEvATPRCxESH7tQ3LIkwmIfc49VXI32lVOsSqCEa3z0RjKxEC2ciOegTvk1MlVdCBhhxB0ZHOnjYJ/yi6BgAMj2I0zD9zO3c1IuMYeFq3AO0MLtpdi1ulvO 6AdevhSr u4upwcN+ULT96hfnSYcVqyt+46IFYNI9W9TpD2e6/meQLFuyUC475N+N/LXRsk/3T2Hxnr/XRvaqPCG4IhumRZb5+zIIYYNyVYJNzmcFQFEwvp0Hjm22HuUb/2l+MbByJpQEmly2NyI8VF92bWp+ymXPN0RRvk49izx8z8GSSbh4kFZwJemEIOyd1x13FOG2GW1miJppx1Nanl+FEOr7NRQAFgUY57sKoF5+uafar33bMPv+lE4dDxljB14JVFSbjTGwAKYzx8Be3XCsS2XyMCPDurWxbQ3gMPKj+ZbhEkIgak4d+Wb5PXWV7L1Ch+icxvK7HG9WyrSJ4Zq0MrgjkXdr1G/gBZy5cyHRhdb0Cr/hqUIf19rFfBkqhdkbHju3VYM6qEyDlOuoPZUXYFnQXiKAMZrxnA28zG9Mt4GJegMa7oggBmmVnuXhQJKXeYChQo/722JjSo0fGqhg533HL0EohbNGiTJonHY/5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024-10-04 23:25, Joel Fernandes wrote: > On Fri, Oct 4, 2024 at 2:29 PM Mathieu Desnoyers > wrote: >> >> This API provides existence guarantees of objects through Hazard >> Pointers (HP). This minimalist implementation is specific to use >> with preemption disabled, but can be extended further as needed. >> >> Each HP domain defines a fixed number of hazard pointer slots (nr_cpus) >> across the entire system. >> >> Its main benefit over RCU is that it allows fast reclaim of >> HP-protected pointers without needing to wait for a grace period. >> >> It also allows the hazard pointer scan to call a user-defined callback >> to retire a hazard pointer slot immediately if needed. This callback >> may, for instance, issue an IPI to the relevant CPU. >> >> There are a few possible use-cases for this in the Linux kernel: >> >> - Improve performance of mm_count by replacing lazy active mm by HP. >> - Guarantee object existence on pointer dereference to use refcount: >> - replace locking used for that purpose in some drivers, >> - replace RCU + inc_not_zero pattern, >> - rtmutex: Improve situations where locks need to be taken in >> reverse dependency chain order by guaranteeing existence of >> first and second locks in traversal order, allowing them to be >> locked in the correct order (which is reverse from traversal >> order) rather than try-lock+retry on nested lock. >> >> References: >> >> [1]: M. M. Michael, "Hazard pointers: safe memory reclamation for >> lock-free objects," in IEEE Transactions on Parallel and >> Distributed Systems, vol. 15, no. 6, pp. 491-504, June 2004 > [ ... ] >> --- >> Changes since v0: >> - Remove slot variable from hp_dereference_allocate(). >> --- >> include/linux/hp.h | 158 +++++++++++++++++++++++++++++++++++++++++++++ >> kernel/Makefile | 2 +- >> kernel/hp.c | 46 +++++++++++++ > > Just a housekeeping comment, ISTR Linus looking down on adding bodies > of C code to header files (like hp_dereference_allocate). I understand > maybe the rationale is that the functions included are inlined. But do > all of them have to be inlined? Such headers also hurt code browsing > capabilities in code browsers like clangd. clangd doesn't understand > header files because it can't independently compile them -- it uses > the compiler to generate and extract the AST for superior code > browsing/completion. > > Also have you looked at the benefits of inlining for hp.h? > hp_dereference_allocate() seems large enough that inlining may not > matter much, but I haven't compiled it and looked at the asm myself. Here is a comparison in userspace: * With "hp dereference allocate" inlined: test_hpref_benchmark (smp_mb) nr_reads 1994298193 nr_writes 22293162 nr_ops 2016591355 test_hpref_benchmark (barrier/membarrier) nr_reads 15208690879 nr_writes 1893785 nr_ops 15210584664 * With "hp dereference allocate" implemented as a function call: test_hpref_benchmark (smp_mb) nr_reads 1558924716 nr_writes 14261028 nr_ops 1573185744 test_hpref_benchmark (barrier/membarrier) nr_reads 5881131707 nr_writes 2005140 nr_ops 5883136847 So the overhead of the function call when using symmetric memory barriers between hp allocate/hp scan is a 20% slowdown. It's worse in the asymmetric barrier/membarrier case, introducing a 61% slowdown. Given that the overhead is noticeable, I am tempted to leave the hazard pointer allocate/retire as inline functions. About code browsers like clangd, I would recommend improving the tooling rather than alter the design of the code based on current tooling limitations. Thanks, Mathieu -- Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com