From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8952D5D681 for ; Thu, 7 Nov 2024 18:31:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 342106B0088; Thu, 7 Nov 2024 13:31:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2F1E56B0089; Thu, 7 Nov 2024 13:31:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B9986B008A; Thu, 7 Nov 2024 13:31:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id EE1AB6B0088 for ; Thu, 7 Nov 2024 13:31:08 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 9E0041A071D for ; Thu, 7 Nov 2024 18:31:08 +0000 (UTC) X-FDA: 82760138538.17.4A96F46 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf26.hostedemail.com (Postfix) with ESMTP id 81907140025 for ; Thu, 7 Nov 2024 18:30:39 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=o1vzfRD0; spf=pass (imf26.hostedemail.com: domain of "SRS0=4mwd=SC=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=4mwd=SC=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org"; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731004216; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hM76lvTyXMM6D75BpOC6JBGUh4jsFp0x+8KJFC2o35g=; b=CYUz6nXVSYnLjHA1sYqaiz3plj4efOVR3Re+Cs+MEcyFhYLT7Gqz0HANlvsXRAqzIuNTYk sLNEJ2YvV5pZ5WNTT2telWpbiCMvlTlxZkZdp1Eb+syn0WpvwtsD8jDAsGBGnzLhhnFQF9 50kDhr6UJqCRmXld+KUn+S/GsgXxNgA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731004216; a=rsa-sha256; cv=none; b=p/9V4lIU+TVHGh99DrU0QmnslluQB3+BUXNhxA5FzCOpDEqXbNBcwS1ZuNr3KmkdWK210u iTuh8ezXsuOrMWTk24ocYyaSvnP17dYZk1BF2y6Y5Oy3NI0nX27HlSd1za1Dr1BGpPiB9R w4Gmhcv7a0HrVi1zMSjlYxq4ZMQVpSU= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=o1vzfRD0; spf=pass (imf26.hostedemail.com: domain of "SRS0=4mwd=SC=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=4mwd=SC=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org"; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id DE3E65C4957; Thu, 7 Nov 2024 18:30:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9B5E0C4CECC; Thu, 7 Nov 2024 18:31:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1731004264; bh=ZYRVY8sNunfS28Hfdf4uEMKjhHqSaZEaY12DFnb70wU=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To:From; b=o1vzfRD0XXs4iz5vG5R9iWZ/cEnogz6sOsXkw8/ECOKv7hykf+iFQqbEOs8UcnBAb Lcz/UMSMPLAP3qWuWjtYQJZ9hgUZaJEZzciasbJoWGA5EEgnhBGuIcL/mPtMKHzIoy hl02RJQ/MalHZVuReXg3BgJMDmlq6xZVBeTJRUGi5qQcdaOccxsOmsERJcaGiLYExf 52c5P+lyEd+xGxelXnkG1k5brlnqtCLDcuhxRYAiSrC7CvllIVolJrsK5bmt5lbHaY ouIo8PaUSlybQm+tQEOtiUjwPekhQjUmT0kLS6ua9OYTUj6oSe88OKxJjIPY9nbSo6 7r5aNLOdV5Z2A== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 46D3ECE0886; Thu, 7 Nov 2024 10:31:04 -0800 (PST) Date: Thu, 7 Nov 2024 10:31:04 -0800 From: "Paul E. McKenney" To: Sebastian Andrzej Siewior Cc: kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Boqun Feng , Marco Elver , Peter Zijlstra , Tomas Gleixner , Vlastimil Babka , akpm@linux-foundation.org, cl@linux.com, iamjoonsoo.kim@lge.com, longman@redhat.com, penberg@kernel.org, rientjes@google.com, sfr@canb.auug.org.au Subject: Re: [PATCH v2 3/3] scftorture: Use a lock-less list to free memory. Message-ID: Reply-To: paulmck@kernel.org References: <20241107111821.3417762-1-bigeasy@linutronix.de> <20241107111821.3417762-4-bigeasy@linutronix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20241107111821.3417762-4-bigeasy@linutronix.de> X-Stat-Signature: tyef5decy8nsgp5dhbq4fwsjmxpxysqf X-Rspam-User: X-Rspamd-Queue-Id: 81907140025 X-Rspamd-Server: rspam02 X-HE-Tag: 1731004239-170800 X-HE-Meta: U2FsdGVkX1+j4ySs2L9U2y0/hbt4zJh0ZDZVZM77GfL+2BbqWZnZgmsW2K/L34ECYS54A9GKOuqV5Od449xg2XdtrcCuN65B+5YYWppKbOvRWFZlVKINIs0HAb9o4X2do6FfsHFjhZpZmEGx0apy323wdduGn5nl7QT01EUUOf9Y6Y7i10hdJd1FaG1inTlQS6hrgV9p/G2riehybj+Rqn1mU0QOL4tBAj0B7D872me9xkvC/r/lYzTowXQM3rqy2VPGMI2+5p996+xTDL2Gd328qRdhFZXUh6jyXhrlEh1GnWxtgvYMFkQxtEK9G8dsc7glP9JsTgim/1h3yngsg2d0rQfwLmWcH/lCn0/kovhRwJIaa94k3xsQ+8bmUVcMi2o3nu1nf9ZmkkJDDOGjUgtnEWSeU9jXK6R8zkBFljCMuJwsTEGXIUhdBH8O7V2/TSlGmgObWWsE5xOj4/1RaJ0a2Dfrrli6A5cEO86dtVgp7CWzbYTba+siA1d4Oq6yzBRGkF+0nqtp3g+/mzxTx8C0CvRvdU9FDLINkyuyPpLh+fPtSj5BKMBvXnSFrbvGC0zSa8Vm1SwZK4lc0Hz4lJBPIrTfRRBPWLCDnTqd/MC+m7j8DlePEMXkdiFLSo0zfxNieEnq51jjn1dl/2lSlK3OpprMdFQ9eguTVAUkCI/FsR28d9tayFM0SxU1sQ3gEb8xOfulw7bmFtkAc8vCraKHDybugabcxQ8Wk4TQ48SwaR8AV0nXKg5StcOm0UJZTlKGqKOtVsBTA5zo7sAzAgU14WnzsLPPeV6vs0g0usgPhFeInUCRgtkSiBijyuvt7Uf86ToaEVNPh1LNSfZcxgmns3IvctpcVxClYGjIsijtgcgriPfGRFWNzh2TzHBNU4KW1CwP/sVzRR65SRK6SsVdJ2GYzVfayr4Mdt4Ta127r1/e9c17gXFj2rEPwZxMLUqXDfdvaFCvFC7H+3i K6fKt3ql fYVVo506JaZIDU0zrGe7gRyuUEkl56yxy6VMCNoGBJonk7hzOO2cquUMW/JTv/viOkyj89je/f8LGzQ0TM61eHDiSsmUjZNjvKrdn/cWyzeXYslZ4rwZYrLCnW4p7SwWpt3eLpzFFu/w2Il/uYwG7DJJmYnrncpNE99H9MzWti//CVEDdgA3k0By7SNpuumutwjdHspPihXZCYRPLl+cs4SQD5A0n6T0D8LyO8+Ix5fKjvxZwSfk8SgdMCH39KYf5kNkzfGq6hziQUIHaMD6IbDpOb5M18OYy+WUINpnexQtVaxqDT6yWQroLFC6ThzKup9XkckIo0Mihpuvv6Df13bMkPcMiAL0foEkiPr8QIxdUOK1xqCNWMuIlTufcUI0O5dxfbc8KSfUL3VHCQzkeJ8ctdYp6eR/X8l/pfEj89NPcY3RMzQ5BxKOHWo9NwSBAJ8j1LUuSR3zuSTaTwtNs3BDLOAAYJ/1N+wCswSq2Q+EeuUb15oT+Y2vquf6eFWPAQA6KPUrnYelUux0JY3gGWhUpy9QJT0deyg8MnswSlEScaD8N8HJBBnEwkZHVROcne7Xy/QDNqPB2ZLSiCuTv0JHlU6FjNdPdiHKhRDgsqJh3cEHGq15AD52LZZApP7MMRF3WZN1ncjVxzJCL9w5xrQ4CAM/tankrjKtrVVq0Tf48saTL8EgMbHLu49SgsrKIruz5G7hj7cNA+/ogS22JDW+DsF3xmwqyuFd74vX2iuRqVl0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Nov 07, 2024 at 12:13:08PM +0100, Sebastian Andrzej Siewior wrote: > scf_handler() is used as a SMP function call. This function is always > invoked in IRQ-context even with forced-threading enabled. This function > frees memory which not allowed on PREEMPT_RT because the locking > underneath is using sleeping locks. > > Add a per-CPU scf_free_pool where each SMP functions adds its memory to > be freed. This memory is then freed by scftorture_invoker() on each > iteration. On the majority of invocations the number of items is less > than five. If the thread sleeps/ gets delayed the number exceed 350 but > did not reach 400 in testing. These were the spikes during testing. > The bulk free of 64 pointers at once should improve the give-back if the > list grows. The list size is ~1.3 items per invocations. > > Having one global scf_free_pool with one cleaning thread let the list > grow to over 10.000 items with 32 CPUs (again, spikes not the average) > especially if the CPU went to sleep. The per-CPU part looks like a good > compromise. > > Reported-by: "Paul E. McKenney" > Closes: https://lore.kernel.org/lkml/41619255-cdc2-4573-a360-7794fc3614f7@paulmck-laptop/ > Signed-off-by: Sebastian Andrzej Siewior Nice!!! One nit at the end below. > --- > kernel/scftorture.c | 39 +++++++++++++++++++++++++++++++++++---- > 1 file changed, 35 insertions(+), 4 deletions(-) > > diff --git a/kernel/scftorture.c b/kernel/scftorture.c > index 555b3b10621fe..1268a91af5d88 100644 > --- a/kernel/scftorture.c > +++ b/kernel/scftorture.c > @@ -97,6 +97,7 @@ struct scf_statistics { > static struct scf_statistics *scf_stats_p; > static struct task_struct *scf_torture_stats_task; > static DEFINE_PER_CPU(long long, scf_invoked_count); > +static DEFINE_PER_CPU(struct llist_head, scf_free_pool); > > // Data for random primitive selection > #define SCF_PRIM_RESCHED 0 > @@ -133,6 +134,7 @@ struct scf_check { > bool scfc_wait; > bool scfc_rpc; > struct completion scfc_completion; > + struct llist_node scf_node; > }; > > // Use to wait for all threads to start. > @@ -148,6 +150,31 @@ static DEFINE_TORTURE_RANDOM_PERCPU(scf_torture_rand); > > extern void resched_cpu(int cpu); // An alternative IPI vector. > > +static void scf_add_to_free_list(struct scf_check *scfcp) > +{ > + struct llist_head *pool; > + unsigned int cpu; > + > + cpu = raw_smp_processor_id() % nthreads; > + pool = &per_cpu(scf_free_pool, cpu); > + llist_add(&scfcp->scf_node, pool); > +} > + > +static void scf_cleanup_free_list(unsigned int cpu) > +{ > + struct llist_head *pool; > + struct llist_node *node; > + struct scf_check *scfcp; > + > + pool = &per_cpu(scf_free_pool, cpu); > + node = llist_del_all(pool); > + while (node) { > + scfcp = llist_entry(node, struct scf_check, scf_node); > + node = node->next; > + kfree(scfcp); > + } > +} > + > // Print torture statistics. Caller must ensure serialization. > static void scf_torture_stats_print(void) > { > @@ -296,7 +323,7 @@ static void scf_handler(void *scfc_in) > if (scfcp->scfc_rpc) > complete(&scfcp->scfc_completion); > } else { > - kfree(scfcp); > + scf_add_to_free_list(scfcp); > } > } > > @@ -363,7 +390,7 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra > scfp->n_single_wait_ofl++; > else > scfp->n_single_ofl++; > - kfree(scfcp); > + scf_add_to_free_list(scfcp); > scfcp = NULL; > } > break; > @@ -391,7 +418,7 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra > preempt_disable(); > } else { > scfp->n_single_rpc_ofl++; > - kfree(scfcp); > + scf_add_to_free_list(scfcp); > scfcp = NULL; > } > break; > @@ -428,7 +455,7 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra > pr_warn("%s: Memory-ordering failure, scfs_prim: %d.\n", __func__, scfsp->scfs_prim); > atomic_inc(&n_mb_out_errs); // Leak rather than trash! > } else { > - kfree(scfcp); > + scf_add_to_free_list(scfcp); > } > barrier(); // Prevent race-reduction compiler optimizations. > } > @@ -479,6 +506,8 @@ static int scftorture_invoker(void *arg) > VERBOSE_SCFTORTOUT("scftorture_invoker %d started", scfp->cpu); > > do { > + scf_cleanup_free_list(cpu); > + > scftorture_invoke_one(scfp, &rand); > while (cpu_is_offline(cpu) && !torture_must_stop()) { > schedule_timeout_interruptible(HZ / 5); > @@ -538,6 +567,8 @@ static void scf_torture_cleanup(void) > > end: > torture_cleanup_end(); > + for (i = 0; i < nthreads; i++) > + scf_cleanup_free_list(i); It would be better for this to precede the call to torture_cleanup_end(). As soon as torture_cleanup_end() is invoked, in theory, another torture test might start. Yes, in practice, this would only matter if the next module was again scftorture and you aren't supposed to modprobe a given module until after the prior rmmod has completed, which would prevent this scf_cleanup_free_list() from interacting with the incoming instance of scftorture. But why even allow the possibility? Thanx, Paul > } > > static int __init scf_torture_init(void) > -- > 2.45.2 >