From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 689BEC25B10 for ; Fri, 10 May 2024 08:21:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE55D6B0092; Fri, 10 May 2024 04:21:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C96156B0093; Fri, 10 May 2024 04:21:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B35F56B0095; Fri, 10 May 2024 04:21:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 95ACE6B0092 for ; Fri, 10 May 2024 04:21:01 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 17455A260D for ; Fri, 10 May 2024 08:21:01 +0000 (UTC) X-FDA: 82101790722.21.0FD5348 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf14.hostedemail.com (Postfix) with ESMTP id 8AC2A10000D for ; Fri, 10 May 2024 08:20:58 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=gSCF4lyG; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="yGKA/5gv"; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=gSCF4lyG; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="yGKA/5gv"; dmarc=none; spf=pass (imf14.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715329259; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WR+T73eWryimsepHQm5MHZRCEyO39uHzdAjoVe4+aKU=; b=vwB7eu4NyDOB3gu69h3sJdJJAPs2yn0s9YfuUP3cehitvJrg8+VMknU3zm2gC08KN5FoOi lF3LORwXfRVcDWCIjlnk5HczWOeb+X/l2q+lgX/0KfqYOZ8pVXHf9/Jw064MUGcRRO4f8s hHlcL8+scuczDNC/j9w95J8+Hn2EKlA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715329259; a=rsa-sha256; cv=none; b=LhVA9wIgh1CmXHxUIggCy9fybHH7CkKIKgji3KchoizFwezqpyz86qkShphMnEZg8iXpI5 jiIEHMMngQuQlLmnZ+keZLXE8O9vufbqQ89jZI8clpPdzPRbxMGnppYcLL7RMDWoZXX5XO 26ToTQfNYJ7l/sOAKP8/JiepN+1H1Qo= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=gSCF4lyG; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="yGKA/5gv"; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=gSCF4lyG; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="yGKA/5gv"; dmarc=none; spf=pass (imf14.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9AC2D343E3; Fri, 10 May 2024 08:20:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1715329256; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=WR+T73eWryimsepHQm5MHZRCEyO39uHzdAjoVe4+aKU=; b=gSCF4lyGsY45v8g+7zH+yr2LaLGifQzorAfXiP/BtYst+rRsTzI7ywU1WgXe0sn61rCbBk 5dt4oSOOlWaU+xSj18jgVSy0EC+CNdR5+hnch0y+OqYOLjkDZb+tj8pObTYxtD0XpBuG+/ TeBymu2GAqFZsbQR1lx7sA7cG0j2cuQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1715329256; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=WR+T73eWryimsepHQm5MHZRCEyO39uHzdAjoVe4+aKU=; b=yGKA/5gv8KtUlzu5+NsiU1oCmjujD9R6vyZoWCer4bCW75ltn0fXLN27WRgebjgo8IgsJj FVi2btBSH5sTjqDQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1715329256; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=WR+T73eWryimsepHQm5MHZRCEyO39uHzdAjoVe4+aKU=; b=gSCF4lyGsY45v8g+7zH+yr2LaLGifQzorAfXiP/BtYst+rRsTzI7ywU1WgXe0sn61rCbBk 5dt4oSOOlWaU+xSj18jgVSy0EC+CNdR5+hnch0y+OqYOLjkDZb+tj8pObTYxtD0XpBuG+/ TeBymu2GAqFZsbQR1lx7sA7cG0j2cuQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1715329256; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=WR+T73eWryimsepHQm5MHZRCEyO39uHzdAjoVe4+aKU=; b=yGKA/5gv8KtUlzu5+NsiU1oCmjujD9R6vyZoWCer4bCW75ltn0fXLN27WRgebjgo8IgsJj FVi2btBSH5sTjqDQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 883641386E; Fri, 10 May 2024 08:20:56 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id MaTwIOjYPWb9IQAAD6G6ig (envelope-from ); Fri, 10 May 2024 08:20:56 +0000 Message-ID: <93840eb4-609d-49d3-b48a-9c26bfb5b8ec@suse.cz> Date: Fri, 10 May 2024 10:20:56 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 1/2] objpool: enable inlining objpool_push() and objpool_pop() operations Content-Language: en-US To: "wuqiang.matt" , Andrii Nakryiko , linux-trace-kernel@vger.kernel.org, rostedt@goodmis.org, mhiramat@kernel.org Cc: bpf@vger.kernel.org, "linux-mm@kvack.org" References: <20240424215214.3956041-1-andrii@kernel.org> <20240424215214.3956041-2-andrii@kernel.org> <0e8b7482-478e-4efc-ad5f-76d60cf02bfd@suse.cz> From: Vlastimil Babka Autocrypt: addr=vbabka@suse.cz; keydata= xsFNBFZdmxYBEADsw/SiUSjB0dM+vSh95UkgcHjzEVBlby/Fg+g42O7LAEkCYXi/vvq31JTB KxRWDHX0R2tgpFDXHnzZcQywawu8eSq0LxzxFNYMvtB7sV1pxYwej2qx9B75qW2plBs+7+YB 87tMFA+u+L4Z5xAzIimfLD5EKC56kJ1CsXlM8S/LHcmdD9Ctkn3trYDNnat0eoAcfPIP2OZ+ 9oe9IF/R28zmh0ifLXyJQQz5ofdj4bPf8ecEW0rhcqHfTD8k4yK0xxt3xW+6Exqp9n9bydiy tcSAw/TahjW6yrA+6JhSBv1v2tIm+itQc073zjSX8OFL51qQVzRFr7H2UQG33lw2QrvHRXqD Ot7ViKam7v0Ho9wEWiQOOZlHItOOXFphWb2yq3nzrKe45oWoSgkxKb97MVsQ+q2SYjJRBBH4 8qKhphADYxkIP6yut/eaj9ImvRUZZRi0DTc8xfnvHGTjKbJzC2xpFcY0DQbZzuwsIZ8OPJCc LM4S7mT25NE5kUTG/TKQCk922vRdGVMoLA7dIQrgXnRXtyT61sg8PG4wcfOnuWf8577aXP1x 6mzw3/jh3F+oSBHb/GcLC7mvWreJifUL2gEdssGfXhGWBo6zLS3qhgtwjay0Jl+kza1lo+Cv BB2T79D4WGdDuVa4eOrQ02TxqGN7G0Biz5ZLRSFzQSQwLn8fbwARAQABzSBWbGFzdGltaWwg QmFia2EgPHZiYWJrYUBzdXNlLmN6PsLBlAQTAQoAPgIbAwULCQgHAwUVCgkICwUWAgMBAAIe AQIXgBYhBKlA1DSZLC6OmRA9UCJPp+fMgqZkBQJkBREIBQkRadznAAoJECJPp+fMgqZkNxIQ ALZRqwdUGzqL2aeSavbum/VF/+td+nZfuH0xeWiO2w8mG0+nPd5j9ujYeHcUP1edE7uQrjOC Gs9sm8+W1xYnbClMJTsXiAV88D2btFUdU1mCXURAL9wWZ8Jsmz5ZH2V6AUszvNezsS/VIT87 AmTtj31TLDGwdxaZTSYLwAOOOtyqafOEq+gJB30RxTRE3h3G1zpO7OM9K6ysLdAlwAGYWgJJ V4JqGsQ/lyEtxxFpUCjb5Pztp7cQxhlkil0oBYHkudiG8j1U3DG8iC6rnB4yJaLphKx57NuQ PIY0Bccg+r9gIQ4XeSK2PQhdXdy3UWBr913ZQ9AI2usid3s5vabo4iBvpJNFLgUmxFnr73SJ KsRh/2OBsg1XXF/wRQGBO9vRuJUAbnaIVcmGOUogdBVS9Sun/Sy4GNA++KtFZK95U7J417/J Hub2xV6Ehc7UGW6fIvIQmzJ3zaTEfuriU1P8ayfddrAgZb25JnOW7L1zdYL8rXiezOyYZ8Fm ZyXjzWdO0RpxcUEp6GsJr11Bc4F3aae9OZtwtLL/jxc7y6pUugB00PodgnQ6CMcfR/HjXlae h2VS3zl9+tQWHu6s1R58t5BuMS2FNA58wU/IazImc/ZQA+slDBfhRDGYlExjg19UXWe/gMcl De3P1kxYPgZdGE2eZpRLIbt+rYnqQKy8UxlszsBNBFsZNTUBCACfQfpSsWJZyi+SHoRdVyX5 J6rI7okc4+b571a7RXD5UhS9dlVRVVAtrU9ANSLqPTQKGVxHrqD39XSw8hxK61pw8p90pg4G /N3iuWEvyt+t0SxDDkClnGsDyRhlUyEWYFEoBrrCizbmahOUwqkJbNMfzj5Y7n7OIJOxNRkB IBOjPdF26dMP69BwePQao1M8Acrrex9sAHYjQGyVmReRjVEtv9iG4DoTsnIR3amKVk6si4Ea X/mrapJqSCcBUVYUFH8M7bsm4CSxier5ofy8jTEa/CfvkqpKThTMCQPNZKY7hke5qEq1CBk2 wxhX48ZrJEFf1v3NuV3OimgsF2odzieNABEBAAHCwXwEGAEKACYCGwwWIQSpQNQ0mSwujpkQ PVAiT6fnzIKmZAUCZAUSmwUJDK5EZgAKCRAiT6fnzIKmZOJGEACOKABgo9wJXsbWhGWYO7mD 8R8mUyJHqbvaz+yTLnvRwfe/VwafFfDMx5GYVYzMY9TWpA8psFTKTUIIQmx2scYsRBUwm5VI EurRWKqENcDRjyo+ol59j0FViYysjQQeobXBDDE31t5SBg++veI6tXfpco/UiKEsDswL1WAr tEAZaruo7254TyH+gydURl2wJuzo/aZ7Y7PpqaODbYv727Dvm5eX64HCyyAH0s6sOCyGF5/p eIhrOn24oBf67KtdAN3H9JoFNUVTYJc1VJU3R1JtVdgwEdr+NEciEfYl0O19VpLE/PZxP4wX PWnhf5WjdoNI1Xec+RcJ5p/pSel0jnvBX8L2cmniYnmI883NhtGZsEWj++wyKiS4NranDFlA HdDM3b4lUth1pTtABKQ1YuTvehj7EfoWD3bv9kuGZGPrAeFNiHPdOT7DaXKeHpW9homgtBxj 8aX/UkSvEGJKUEbFL9cVa5tzyialGkSiZJNkWgeHe+jEcfRT6pJZOJidSCdzvJpbdJmm+eED w9XOLH1IIWh7RURU7G1iOfEfmImFeC3cbbS73LQEFGe1urxvIH5K/7vX+FkNcr9ujwWuPE9b 1C2o4i/yZPLXIVy387EjA6GZMqvQUFuSTs/GeBcv0NjIQi8867H3uLjz+mQy63fAitsDwLmR EP+ylKVEKb0Q2A== In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Action: no action X-Rspamd-Queue-Id: 8AC2A10000D X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: bzftjuk1guzry8znaxe69nc6dbhehfmb X-HE-Tag: 1715329258-471058 X-HE-Meta: U2FsdGVkX1+18YnsRX30SQA9MYvJIf0lrB7kK4HBK4y1k5vfJqg1wft/pel9Rum4gEBL5oHQGBsrc3qq3PEEvd6PdexGN7umJS0lStXDxF7DM5qtqNgb6PRafSESNhHVQwE6zzWvSoRqhA4WB0yQAU9YmHWzIIDNCHTxX89XAR+tJffZpNiK7QYwTPPs7QUVJyylT1L9XZR2S5GQ8WPcB6j9MuH1R2Ci0fc3cxhu1E2UB/oe57tcCR9paDdSk7UqgacUfJuxZ+P0h3Lhgo/TSlmJvJJZcAcGak7M9cRZPEILBfy58J6aMgKVZ0X2BSI8uDAIa2oI9yk3JlxyufeTAe9U6jg7F9bN7tpZazNNAC7ld8yK6D7Y/mncO1G3mYBI4IDOMndZwXGw08J6MzwhHA2dTtCYLQ86APJhVDd2gF1I4ayIYJog38tSPZvG2ed+mrrAzd0z4l34RWdLGPxLFUj/+0plrpfvRs9r2/Z8rI9Vbs0dbXarpp4FQuVKnyVfkVCLx+/Pek8vAYGTYTysTZLDN8XJk+1MrgiIH0pGtZIEDM2pri1QUqOz1b6O8TTWYSRcqBonCY2NwPsuIF373PfPxBSbrcv4AV+eTNTSabneTkeZ4SpI49npA8DG7frm4e0HUiLS+poH4Jo+UyVU/LbPl2pPO7IyjztgZcnIjV52AB2FHt3ynZ+oq7YR1r6N+z47Fs+V5TllXBRQVi8nCsxM29o1pTMAHL48vk+sjQnPnxXgpGEi/FqYtfHDA/ry5VDI3eCo5iGt1UxPlTmatnjHpsoueGukxyYhTG7UDT9Dayt8V//3waeWnHGO8tExyh2bCvCSsB/DbLc1eLQnXZLONDeBgZfdMnKs+VYt81ECW7VkJl9KhX7PvPMAWjfizcBiddjZvyEfc65H4nlC1PddxkTUNIk1/hj+PQseseu1RswNYbtaXkpjYTVWdVyV/6cxvW68RRB8I8xiMGp 4cPWt3TC pCbijLAM9GSvZpitC3X2WNxpD/T7uUn/Ht4jmrnQ7EwtuFmhiHeahl/HfuU0WHLMnsvm2DkwdBZ3d8trybCWN4URHPzx2SL6UHZTUlmFllpHRlrwf/QKeXqDbTGqR0vAYvsNa82Yt1+j+b6TM0phi7tp6nGBktYMxkl2cBQksGG593IEJV9/LeZApCKdHgHWumWNfaNg3qiqaEt+lDxt/0vxuwITB141XmV+EKaKkWVGWMS79CMNl8T+u4MVUO3lNGeVzfljsS9FjdIqvZ1i8PeW+dRd4UN5CMuXOkTBg0o4oej0QrJuVS776AXfAsCTHkPI5QPYp4aC3PKYzoUn8vXAddXP5gBvWKlht X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 5/10/24 9:59 AM, wuqiang.matt wrote: > On 2024/5/7 21:55, Vlastimil Babka wrote: >> >>> + } while (!try_cmpxchg_acquire(&slot->tail, &tail, tail + 1)); >>> + >>> + /* now the tail position is reserved for the given obj */ >>> + WRITE_ONCE(slot->entries[tail & slot->mask], obj); >>> + /* update sequence to make this obj available for pop() */ >>> + smp_store_release(&slot->last, tail + 1); >>> + >>> + return 0; >>> +} >>> >>> /** >>> * objpool_push() - reclaim the object and return back to objpool >>> @@ -134,7 +219,19 @@ void *objpool_pop(struct objpool_head *pool); >>> * return: 0 or error code (it fails only when user tries to push >>> * the same object multiple times or wrong "objects" into objpool) >>> */ >>> -int objpool_push(void *obj, struct objpool_head *pool); >>> +static inline int objpool_push(void *obj, struct objpool_head *pool) >>> +{ >>> + unsigned long flags; >>> + int rc; >>> + >>> + /* disable local irq to avoid preemption & interruption */ >>> + raw_local_irq_save(flags); >>> + rc = __objpool_try_add_slot(obj, pool, raw_smp_processor_id()); >> >> And IIUC, we could in theory objpool_pop() on one cpu, then later another >> cpu might do objpool_push() and cause the latter cpu's pool to go over >> capacity? Is there some implicit requirements of objpool users to take care >> of having matched cpu for pop and push? Are the current objpool users >> obeying this requirement? (I can see the selftests do, not sure about the >> actual users). >> Or am I missing something? Thanks. > > The objects are all pre-allocated along with creation of the new objpool > and the total number of objects never exceeds the capacity on local node. Aha, I see, the capacity of entries is enough to hold objects from all nodes in the most unfortunate case they all end up freed from a single cpu. > So objpool_push() would always find an available slot from the ring-array > for the given object to insert back. objpool_pop() would try looping all > the percpu slots until an object is found or whole objpool is empty. So it's correct, but seems rather wasteful to have the whole capacity for entries replicated on every cpu? It does make objpool_push() simple and fast, but as you say, objpool_pop() still has to search potentially all non-local percpu slots, with disabled irqs, which is far from ideal. And the "abort if the slot was already full" comment for objpool_try_add_slot() seems still misleading? Maybe that was your initial idea but changed later? > Currently kretprobe is the only actual usecase of objpool. > > I'm testing an updated objpool in our HIDS project for critical pathes, > which is widely deployed on servers inside my company. The new version > eliminates the raw_local_irq_save and raw_local_irq_restore pair of > objpool_push and gains up to 5% of performance boost. Mind Ccing me and linux-mm once you are posting that? Thanks, Vlastimil