From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3E60CF9C69 for ; Sun, 22 Sep 2024 07:48:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C59486B007B; Sun, 22 Sep 2024 03:48:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE2496B0082; Sun, 22 Sep 2024 03:48:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A5BC66B0085; Sun, 22 Sep 2024 03:48:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 833396B007B for ; Sun, 22 Sep 2024 03:48:46 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 234E812019E for ; Sun, 22 Sep 2024 07:48:46 +0000 (UTC) X-FDA: 82591597452.03.8C7FEE6 Received: from smtpout.efficios.com (smtpout.efficios.com [167.114.26.122]) by imf23.hostedemail.com (Postfix) with ESMTP id 0439414000E for ; Sun, 22 Sep 2024 07:48:43 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=efficios.com header.s=smtpout1 header.b=ZdPaISEe; spf=pass (imf23.hostedemail.com: domain of mathieu.desnoyers@efficios.com designates 167.114.26.122 as permitted sender) smtp.mailfrom=mathieu.desnoyers@efficios.com; dmarc=pass (policy=none) header.from=efficios.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726991170; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kcW9rjGl6+smBF5xAjTvODeaDQ5dSYoawMKXDwlf3nw=; b=KCVg1yNMNZ8s9vM0E9eEY4jhIXvwpYDfIY0oxjBAn+HnL6ULTwTMfs+bP9X6+oKdECWW/S lmwiXM/iiITQEOr2o2MlSt/YSWDepThaZ1fly8EzT1wYuVZUNsG7VfjDgWJ8Kcj4BbLbZr m53ogLc73M06uL2DzngS2ke3plFcMEQ= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=efficios.com header.s=smtpout1 header.b=ZdPaISEe; spf=pass (imf23.hostedemail.com: domain of mathieu.desnoyers@efficios.com designates 167.114.26.122 as permitted sender) smtp.mailfrom=mathieu.desnoyers@efficios.com; dmarc=pass (policy=none) header.from=efficios.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726991170; a=rsa-sha256; cv=none; b=x+MJztVPB0mKjxLP9X6z1//TUfTmRBY7voLAXUIcPwQ1K5ZPD1Lxj5o1bBVU9pNldOpi5h 0+FE8KjH+VvIo6r/QClcXda6L7flutzWQsC2lSRoUcpFMtfYZ3+ObLhr7ABLJwyw5Wda/n 5KkASoedV0gSYfWMQPyLxCqo96cWa5Y= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=efficios.com; s=smtpout1; t=1726991322; bh=vj89NnEO3dY9ymXcJzbHblA4gmtlMtSWhJT/+5TqzD4=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=ZdPaISEemDKOOHayARAO5nS+VbU1ofrn5bcbePIQOVWcLhZ4HhVZRYpeGk9HY/TSa lG//+kwIWsQbu1p+8lDZM+S1yEGkOs7iLaEAlvO8z8D/d5U4pBf9L9OJxglypYg2Xy H4a4tgsyF5gv3FevVw9uug7Y6J372T4QT0N5a9cbXblTwYXLljxQxJPesROpXNvg1Z rsvBuN133yFnn/Qj13GkUfvxEjuQwNn8yowXKnJCT5nEup+H4rsGDcMEZcWL9z9SIg zSlSSMyRsJc8Pm09Hv+L+prx9QeQmFvGTMj6gkFtjXXoZCjimnuBKdt3fOaZsGd8m+ LMLteOSGKL61w== Received: from [172.17.2.162] (unknown [178.208.16.192]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4XBJCX3vk2z1LPp; Sun, 22 Sep 2024 03:48:40 -0400 (EDT) Message-ID: <8185fa8b-f6de-4bed-9fd1-73b72fc6d716@efficios.com> Date: Sun, 22 Sep 2024 09:47:56 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH 1/1] hpref: Hazard Pointers with Reference Counter To: Lai Jiangshan Cc: Boqun Feng , "Paul E. McKenney" , linux-kernel@vger.kernel.org, Will Deacon , Peter Zijlstra , Alan Stern , John Stultz , Linus Torvalds , Frederic Weisbecker , Joel Fernandes , Josh Triplett , Uladzislau Rezki , Steven Rostedt , Zqiang , Ingo Molnar , Waiman Long , Mark Rutland , Thomas Gleixner , Vlastimil Babka , maged.michael@gmail.com, Mateusz Guzik , rcu@vger.kernel.org, linux-mm@kvack.org, lkmm@lists.linux.dev, Neeraj Upadhyay References: <20240921164210.256278-1-mathieu.desnoyers@efficios.com> From: Mathieu Desnoyers Content-Language: en-US In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 0439414000E X-Stat-Signature: 9a47o31oyybabff8ji34wuwburdi4zbg X-Rspam-User: X-HE-Tag: 1726991323-201445 X-HE-Meta: U2FsdGVkX1+zpPlyvZC/JE8dlbLRQIILiCEg97JEjNnYHutsRnCaLCur5qyPKmuypRKneuDIEJtX9KDZfgrmh8wwtkdY6PTfpie2SDMka1vCGZEHlT09C4VTRMbSKnm0yz4SNl1hiubU4MrkOTIAIOdGVxlZs3GR69YsRPdbVevufYn7HUtuVDoncyq0SK8WvwExbVzuBHPHVnYlmyISlxfdRM+xx+E3btoBFB6ptP0Hx6iwMonQvnSgzApudvrmI2ZMWJM5rs5YmwvX32+oREsV7SbhA87qLLsLAMjnIjxIF79fvr/B+pLIsR//7m3q/1WVT0/kkYZQBMlTNxeAtXUkmzBzr6MdhvTU1jKC5eRUgejWN5pbBAHn3l8xG1lGtzapHIAWAE2zBJd1kv2Zpn2jxCWquaNAgYxEhW0P/wJBNbmat9sl8Efq8WWkIMFv/xxS3D8fNgYRUNNqzXPmQ2IT9Jkk6+JjOX7/FoWhaiKosz0gkebTsKFU6e7HTBbqkAUYvBezuMm/BDy0+1OEYNVX6MCs/esGin1LCDSx7MaJamyXQZcJLvk5w5GABc+Bx8KMZrjr5g3/9Q8GG0+BAxxQ2RBkc+SeA2haMkcDsRp+zU4qt/2WPXHxyOdLW64Gx6DJKYEa3WYzjJulzCb0VXSrb+U+kU62fZwi3kSaI0af+HrDYaAR3ZStuuVXrhZM4tg6M1F7ZaFkPMhalSKMPO+bBJJJsvHkNfv1DS7XSl8au1GLprRiAJsGwAMjWmlnjcH1UufKFVC4DL/Hl+FFThlRCZIrYj6jVeDJMXlZtp45qx932773b8vOK1WVmr5BMDQHX+Mi4EzdWQSxb8fUrPPc2mS5b98EPpUOvDqgDRNx+BN2zm0lSXSBp0KMmqmU2xiO5DwwIjzAGN3KamOZ12QtyzC9qYkmKsB3FHUeWs0xBo/3V7GUqPkr5iCL50OEE+bTm0C/WiCPDVe9Q3+ pFIKvQlj ELey5ZmkBRzHkD8BwOPG7q25Y8BTnQnoBca9Y5fBAAqFKDS190iIQkvNw4Jz7sFavANtoAZ9Zlrn/rwP7AdbkT6PbN/Np86kIwBjC/f2M5E3GjQpjFWD13q4JOjQWUfExAj4t/kn7VoQTVuPYUEZLSQ6whjAwDlwjq9oBiQtUjStzQw0AiM2SbyZM2S10gy0kTBLR7V5IwHP1mPwWrDk9VTJzW8vHe1prF4Ue/RwHy0o14a3QtN2d0wU8uJ0E+By61cZZeutemjPomxCZ2W7DJhKKdFwftq8ds04/ottcikJTocdXMVPLpN2Ouu8yaz8cz+GrKq9vhnIDjBMKrVaIhTw3FF9Pmwte9iEJw5RgkLItfg8k7wDkd6+tU3Dgt66WbXZlTNPstnP6bTMOnNDAvbV08+6u/c9lYad0o2e98WfmXjbbu1J5OOXHh6TtzrPhdMuTksCznMcKYzE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024-09-21 23:07, Lai Jiangshan wrote: >> +/* >> + * hpref_hp_get: Obtain a reference to a stable object, protected either >> + * by hazard pointer (fast-path) or using reference >> + * counter as fall-back. >> + */ >> +static inline >> +bool hpref_hp_get(struct hpref_node **node_p, struct hpref_ctx *ctx) >> +{ >> + int cpu = rseq_current_cpu_raw(); >> + struct hpref_percpu_slots *cpu_slots = rseq_percpu_ptr(hpref_percpu_slots, cpu); >> + struct hpref_slot *slot = &cpu_slots->slots[cpu_slots->current_slot]; >> + bool use_refcount = false; >> + struct hpref_node *node, *node2; >> + unsigned int next_slot; >> + >> +retry: >> + node = uatomic_load(node_p, CMM_RELAXED); >> + if (!node) >> + return false; >> + /* Use rseq to try setting current slot hp. Store B. */ >> + if (rseq_load_cbne_store__ptr(RSEQ_MO_RELAXED, RSEQ_PERCPU_CPU_ID, >> + (intptr_t *) &slot->node, (intptr_t) NULL, >> + (intptr_t) node, cpu)) { >> + slot = &cpu_slots->slots[HPREF_EMERGENCY_SLOT]; > > Can @cpu be possibly changed? if it can, it seems @cpu and @cpu_slots > should be updated first. Indeed, if migration happens between rseq_current_cpu_raw() and execution of rseq_load_cbne_store__ptr(), it will cause this second operation to fail. This in turn could cause the reader to retry for a long time (possibly forever) until it finally migrates back to the original CPU. As you suggest, we should re-load cpu and cpu_slots after failure here. More specifically, we should re-load those when the rseq c.s. fails with -1, which means it was abort or there was a cpu number mismatch. If the rseq c.s. returns 1, this means the slot did not contain NULL, so all we need to do is move over to the next slot. While applying your suggested change, I noticed that I can improve the fast-path by removing the notion of "current_slot" number, and thus remove increment of this hint from the fast path. The fast path will instead just scan all 8 slots trying to find a NULL one. This also lessens the odds that the fast-path must fallback to refcount in case of concurrent use. It provides a 50% performance improvement for the fast path with barrier/membarrier pairing. I've updated the https://github.com/compudj/userspace-rcu-dev/tree/hpref volatile feature branch with these changes. I'll give others some time to provide feedback on the overall idea before sending out an updated patch. Thanks for your feedback! Mathieu > >> + use_refcount = true; >> + /* >> + * This may busy-wait for another reader using the >> + * emergency slot to transition to refcount. >> + */ >> + caa_cpu_relax(); >> + goto retry; >> + } -- Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com