From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86976D5D690 for ; Thu, 7 Nov 2024 21:53:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E903E6B00A0; Thu, 7 Nov 2024 16:53:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E415B6B00A4; Thu, 7 Nov 2024 16:53:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C92576B00A5; Thu, 7 Nov 2024 16:53:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A96FE6B00A0 for ; Thu, 7 Nov 2024 16:53:38 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 55B6E160394 for ; Thu, 7 Nov 2024 21:53:38 +0000 (UTC) X-FDA: 82760649678.08.4609BBA Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf10.hostedemail.com (Postfix) with ESMTP id 82B04C0014 for ; Thu, 7 Nov 2024 21:53:20 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="cbPWwZD/"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf10.hostedemail.com: domain of "SRS0=4mwd=SC=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=4mwd=SC=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731016356; a=rsa-sha256; cv=none; b=X/GBry5T5i+kfMTF7bjwRaYOhq9D0Ng2eZCAcF8NJnG/kwPszOp3rfQd+aKfzc0VstKOrM nfRSkJbg6feEIr7ktrOdBP+OtE5Lnk0cJIQhhvYXRcCBWDBM84XENMKcLhkhteQ3Qd5C+a Voj6xKh3i1sYZFppd/WBMFCJlcQ53jQ= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="cbPWwZD/"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf10.hostedemail.com: domain of "SRS0=4mwd=SC=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=4mwd=SC=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731016356; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cRm0isev75vj6KIAyinBLv/nOPwuVxeywRUyjldZpww=; b=jVhSy6z+qYR4A1jFydueq3+qGb7qNw50XxIU2Jb+0u18MlQV+xziW38pwWx6Nj/dsT2+58 aJ2zP+oKjncv6dSaR0tkWeBE/1hvP2iPnGo7iqOnOiOFX1/9CatiSDwKIWPyKZ9BztrBjt EVMl8A10J7wjPS59ToB1uBd8jYnIKKE= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id DE3775C543B; Thu, 7 Nov 2024 21:52:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3A300C4CECC; Thu, 7 Nov 2024 21:53:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1731016415; bh=Lmh54qhbt/PB9xXRo3Kmglf9cD770WGTE6ezwZrzet4=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To:From; b=cbPWwZD/DASoT3yZCLO/0zxf0SryQHF2dsHOqAw47siKhXDtmzJ/yo/uauQ8Jr4gK 1HoBvMxRhawjZB4j8Ds4I/4haA4iES6p3Ac8Juyk8WSUXBpCqlPgAzNJfBnsp0zARS ugT+xvkJG9Jcb6M9TbuMkvtRRNZQRGu7hHcpUSLiKRa+VqSgo3l+lrTBXbUtw0liHT u7poSeYUY+EwzlISldDOCARs6NyiJ95SUL/lLDoxBfyy89hWcY2tAPreQ5ZBzH1LZJ 2s3v0AjLdTjTkvvYiQjBzUBLVDYJ9qh5izkiF/FcJrgR3kva9FeuuPPk9xOqbO6/Fk h3gaqbAm6xcIw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id D6585CE0F1C; Thu, 7 Nov 2024 13:53:34 -0800 (PST) Date: Thu, 7 Nov 2024 13:53:34 -0800 From: "Paul E. McKenney" To: Boqun Feng Cc: Sebastian Andrzej Siewior , kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Marco Elver , Peter Zijlstra , Tomas Gleixner , Vlastimil Babka , akpm@linux-foundation.org, cl@linux.com, iamjoonsoo.kim@lge.com, longman@redhat.com, penberg@kernel.org, rientjes@google.com, sfr@canb.auug.org.au Subject: Re: [PATCH v2 3/3] scftorture: Use a lock-less list to free memory. Message-ID: <8714bdf6-e257-446c-855f-0f4e65e2921e@paulmck-laptop> Reply-To: paulmck@kernel.org References: <20241107111821.3417762-1-bigeasy@linutronix.de> <20241107111821.3417762-4-bigeasy@linutronix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Queue-Id: 82B04C0014 X-Rspamd-Server: rspam01 X-Stat-Signature: to57am3w9c1qummzg5axtz91ek9nyrjq X-HE-Tag: 1731016400-276882 X-HE-Meta: U2FsdGVkX1/Ijbs0wfmkX3HIS7DzheXpSe5soxtptZ6K+u5AvmhFxnJ+P6kUH4RdvD8Flmw4l8jsZHd1Cm+E0/359mpXBDGasezCnERjkC5n1Q+1LRBOYcOhADh3avHNgG0GEbckLZsqr3Uelmpy5Mwwe7JDU3QkDjWGZE2IHGSW/E/DwAFC4ta/1BgdnkhYBKKd2lNmruvhnIc+GRD1kYWbU4nVejxIkwGaMr6LZYJcBoHLxS3XnDCEIM6Ow5gwyBwIjxFWrfPIHCM3BWBBkAWD0x8+l5J+9rEmyUU3Rg3QOUUBe2sq0h9/CmCDwllJzR7N3FR1as2lOa99em5oBy90WNdbpU8EEmG8a1ODFV60ACtizjAUacSKsCfvlscI9QfQDxSQBfiprMj5LfBHovfp24ucaAvh2RRTx1nGCWRIDh4JkkrTnNqCxKBWCt91um1ywLzhHIAq70BhYfsc3kJGVGami1jY6nbqAHvrajrhjqJUKrtJJdMUu814LtP64X7G8Bd1ERY1s6fgn9g3VQ9XIBOKrG3IFcVz1nybXrSqxGJrxmsYLemKDjlh9zd5FJoMOIg7BR9djESAR3saMFtNU91ptfX6Jjw2Wn6If59JmV4ee14d0GCWze3dQeSS6i0/SNkzNY/YVU1E4X64H0c2FbPEqEB/6SWfrXfRCMuuncwu78U4kGV5db6GBlfwtKTVKz4c5oDEaXXcyWrqqMB2BjfltWV1Qp8ruLOrFlDZN8zoMCrfV5hbDlLreMIgDNNf0Dy/WOOYod60XsJ0gxcf+up9Fx2ErJFz4vLb4fyY9c3AODJoYqXPFNPbmg9ue3YOGWbOXGSg3pu6WU9T3Pvzu9eY49/wlts0mq8qcXWPM/zwnZcpMJuVJ9qZTdyScYzMklOx6KjdYpNWNSmGMylGlqpHzx9NevU6Y+OIn7ZYbdReKYIUAXCer00Yuba0m4Mbj25ZrER+MjYdMV8 nNtXazKt qZ2cjuAFmLnAy/QIWVfjG7T1SNxHtIpGKXukkxQHke6b5JBy90N9IQegOemAIoa86qNgn+q4tYP/KfJWV46b++CjgiDtvI40mq/6CTwk4hklUX48yxA0UgZBr6hm7SHGfBi0HO3qBMRraAgpxQJmqSUTo7Zv0BC8LYFFrCz2zhGwdhwPE7CVRjvVu0VVpQhn9r2GJPrBYaU+Sj9s+C/EATuUgKjYEAtm6gElb2Yu4U+9UqS+tPWckTh2CDXbGipuj+3u6ECj2v1zUwr8C7WrBs60L592L47tqbTTuwtmldeVo5V1VZYJ9EVwoH1YJPWFPGb22Ldif1G6uTbeYMkiZOyGAvBa5hOYAzx4zeZlOJmk6HPjL2AJh69VLZvuiawCJnrDgCcNsZLYdDqcuBGePBldBhdukoe1rgNXfS9QltJGV8f1puQ3WsjmZb3VTn2wj5GpvWRHQewyOzzRHCYyn8Rt8wzgbVGKliYzdhnwAUBQgemzmoUPtc9UaNNjsdjkZsknD9z0ZxNOpgJZba1koLPDTOE+DpalCFftsNANW6xHHbzmQjsMdjYA4uWeiYtnunZ0MDRghC647Ochp4EdqKdnP94ycw/mZ9wLjkMRHzH6VLcvOYS9z02LtAQ/o/nfhBJVAfMeKesiFdz2CimzwoQg5HvlUtF7+arTMQ9ACuBNPAEYjDc0pAeIjSDganhsWvIOJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Nov 07, 2024 at 12:45:25PM -0800, Boqun Feng wrote: > On Thu, Nov 07, 2024 at 12:13:08PM +0100, Sebastian Andrzej Siewior wrote: > > scf_handler() is used as a SMP function call. This function is always > > invoked in IRQ-context even with forced-threading enabled. This function > > frees memory which not allowed on PREEMPT_RT because the locking > > underneath is using sleeping locks. > > > > Add a per-CPU scf_free_pool where each SMP functions adds its memory to > > be freed. This memory is then freed by scftorture_invoker() on each > > iteration. On the majority of invocations the number of items is less > > than five. If the thread sleeps/ gets delayed the number exceed 350 but > > did not reach 400 in testing. These were the spikes during testing. > > The bulk free of 64 pointers at once should improve the give-back if the > > list grows. The list size is ~1.3 items per invocations. > > > > Having one global scf_free_pool with one cleaning thread let the list > > grow to over 10.000 items with 32 CPUs (again, spikes not the average) > > especially if the CPU went to sleep. The per-CPU part looks like a good > > compromise. > > > > Reported-by: "Paul E. McKenney" > > Closes: https://lore.kernel.org/lkml/41619255-cdc2-4573-a360-7794fc3614f7@paulmck-laptop/ > > Signed-off-by: Sebastian Andrzej Siewior > > --- > > kernel/scftorture.c | 39 +++++++++++++++++++++++++++++++++++---- > > 1 file changed, 35 insertions(+), 4 deletions(-) > > > > diff --git a/kernel/scftorture.c b/kernel/scftorture.c > > index 555b3b10621fe..1268a91af5d88 100644 > > --- a/kernel/scftorture.c > > +++ b/kernel/scftorture.c > > @@ -97,6 +97,7 @@ struct scf_statistics { > > static struct scf_statistics *scf_stats_p; > > static struct task_struct *scf_torture_stats_task; > > static DEFINE_PER_CPU(long long, scf_invoked_count); > > +static DEFINE_PER_CPU(struct llist_head, scf_free_pool); > > > > // Data for random primitive selection > > #define SCF_PRIM_RESCHED 0 > > @@ -133,6 +134,7 @@ struct scf_check { > > bool scfc_wait; > > bool scfc_rpc; > > struct completion scfc_completion; > > + struct llist_node scf_node; > > }; > > > > // Use to wait for all threads to start. > > @@ -148,6 +150,31 @@ static DEFINE_TORTURE_RANDOM_PERCPU(scf_torture_rand); > > > > extern void resched_cpu(int cpu); // An alternative IPI vector. > > > > +static void scf_add_to_free_list(struct scf_check *scfcp) > > +{ > > + struct llist_head *pool; > > + unsigned int cpu; > > + > > + cpu = raw_smp_processor_id() % nthreads; > > + pool = &per_cpu(scf_free_pool, cpu); > > + llist_add(&scfcp->scf_node, pool); > > +} > > + > > +static void scf_cleanup_free_list(unsigned int cpu) > > +{ > > + struct llist_head *pool; > > + struct llist_node *node; > > + struct scf_check *scfcp; > > + > > + pool = &per_cpu(scf_free_pool, cpu); > > + node = llist_del_all(pool); > > + while (node) { > > + scfcp = llist_entry(node, struct scf_check, scf_node); > > + node = node->next; > > + kfree(scfcp); > > + } > > +} > > + > > // Print torture statistics. Caller must ensure serialization. > > static void scf_torture_stats_print(void) > > { > > @@ -296,7 +323,7 @@ static void scf_handler(void *scfc_in) > > if (scfcp->scfc_rpc) > > complete(&scfcp->scfc_completion); > > } else { > > - kfree(scfcp); > > + scf_add_to_free_list(scfcp); > > } > > } > > > > @@ -363,7 +390,7 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra > > scfp->n_single_wait_ofl++; > > else > > scfp->n_single_ofl++; > > - kfree(scfcp); > > + scf_add_to_free_list(scfcp); > > scfcp = NULL; > > } > > break; > > @@ -391,7 +418,7 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra > > preempt_disable(); > > } else { > > scfp->n_single_rpc_ofl++; > > - kfree(scfcp); > > + scf_add_to_free_list(scfcp); > > scfcp = NULL; > > } > > break; > > @@ -428,7 +455,7 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra > > pr_warn("%s: Memory-ordering failure, scfs_prim: %d.\n", __func__, scfsp->scfs_prim); > > atomic_inc(&n_mb_out_errs); // Leak rather than trash! > > } else { > > - kfree(scfcp); > > + scf_add_to_free_list(scfcp); > > } > > barrier(); // Prevent race-reduction compiler optimizations. > > } > > @@ -479,6 +506,8 @@ static int scftorture_invoker(void *arg) > > VERBOSE_SCFTORTOUT("scftorture_invoker %d started", scfp->cpu); > > > > do { > > + scf_cleanup_free_list(cpu); > > + > > scftorture_invoke_one(scfp, &rand); > > while (cpu_is_offline(cpu) && !torture_must_stop()) { > > schedule_timeout_interruptible(HZ / 5); > > @@ -538,6 +567,8 @@ static void scf_torture_cleanup(void) > > > > end: > > torture_cleanup_end(); > > + for (i = 0; i < nthreads; i++) > > This needs to be: > > for (i = 0; i < nr_cpu_ids; i++) > > because nthreads can be larger than nr_cpu_ids, and it'll access a > out-of-bound percpu section. I clearly did not test thoroughly enough. Good catch!!! Thanx, Paul > Regards, > Boqun > > > + scf_cleanup_free_list(i); > > } > > > > static int __init scf_torture_init(void) > > -- > > 2.45.2 > >