From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77763D5D689 for ; Thu, 7 Nov 2024 20:45:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D228D6B00A3; Thu, 7 Nov 2024 15:45:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CD1626B00A5; Thu, 7 Nov 2024 15:45:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4B386B00A6; Thu, 7 Nov 2024 15:45:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 911D56B00A3 for ; Thu, 7 Nov 2024 15:45:31 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4C698A8006 for ; Thu, 7 Nov 2024 20:45:31 +0000 (UTC) X-FDA: 82760478024.19.065989F Received: from mail-yb1-f170.google.com (mail-yb1-f170.google.com [209.85.219.170]) by imf01.hostedemail.com (Postfix) with ESMTP id F13C140008 for ; Thu, 7 Nov 2024 20:45:00 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HWSIKwcs; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of boqun.feng@gmail.com designates 209.85.219.170 as permitted sender) smtp.mailfrom=boqun.feng@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731012158; a=rsa-sha256; cv=none; b=pCw+ZBqd/bFwjnC6RskXvx3sPDGjx9nBaZVZL0EoaCRLyl8kbNbDFPNW37MPkCmdh56aEV +Mg77D40D59Tw2m22Z6AEklhTtjRGXVQ0/mQpGovMCooqR5b848gMJ+KxT6/GpJ3X4nxOs O/f6A892VdEJV4jlbGiJ2A5DaRKe1x0= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HWSIKwcs; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of boqun.feng@gmail.com designates 209.85.219.170 as permitted sender) smtp.mailfrom=boqun.feng@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731012158; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ASBXo4Z2Zrw1kV9v2x3uChH0kNYmBXHhzCm3vG2Xcz8=; b=sswnNKhkEyJGuwyEkIsAScol7vA3nkj+6widD1/Flilw2DlI8EnaZMqcB62+bWL/XD0Bee YQWiIwwcsowP+TbO+EbbJ6Ef1Yn9vv3lMChtzAW4gWE9MEUuPZ9WvADGXkr3LB3kgsEmQz GRqOEV0TEXJ5dKLNMiGxKRZbBODmA44= Received: by mail-yb1-f170.google.com with SMTP id 3f1490d57ef6-e30d0d84d23so1305663276.3 for ; Thu, 07 Nov 2024 12:45:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731012328; x=1731617128; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:feedback-id:from:to:cc:subject:date :message-id:reply-to; bh=ASBXo4Z2Zrw1kV9v2x3uChH0kNYmBXHhzCm3vG2Xcz8=; b=HWSIKwcsOZpS1qqVBo1K/HBNVRFeVMSqypG1ZvdVsWOj27WWyM1wCSOV37dCVYfwKs fQ9AmkGK/p6exABWS/qJ7+uo7aTAwxVXMJB/hqbAzJRow4SuQDqHOfcV09fCZYAtt8pb JH7zLp1+UHki9VLgEf9QK2JpdVz3Fls5ogDxrDkPbRWfC0EOOqpSzFes3Ttb/qoKW3OZ W2NobjUPGKOHLGHLc//hUUWLkjXs9VLo00XcBFFPb8/VA/ujbagYd7ff9EyF7oMLqQms XxOP4YPQlxc+HxdmOIENEIuU22g7bfpkOa59WXTjmg3GhaUXvVnD4ZRfO2Xcy0aX54g2 TtXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731012328; x=1731617128; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:feedback-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ASBXo4Z2Zrw1kV9v2x3uChH0kNYmBXHhzCm3vG2Xcz8=; b=XhcY4drqNE54HMI+eq8fp7Ir6/8QtgzMZWlNeV05A6rA1g0CZneEuaLbS5nLn9wkpw xSTI52s26PFBayL2kE9FARngpyHAZxgzzQ76vA0+hp+Cb+jCHY4XHOE8JuEuzB9h4Ze2 j8e+WyQDpC2i6g6uz6275GSMM2/mLyFUNTyu+7BU6ymoNsrEi25gGPXRqBazvqNt5RVw oRuvPc7H3B51On/0sTZcHSxjKczwPkdTuPgBNESexs7zG/33c/6PaE0zB47O4zsJEjpw VU4n+9J6saDYwTIq3S63LrrhbAsGM3x/rR5e2cuGQH/2e3DIHK0mVlQIWfAWZlT2tRwI jqJw== X-Forwarded-Encrypted: i=1; AJvYcCXPyDxFXyuoqzuk+ti+3P8rx0bLd3+f9E307x7/0S00ozSHbZgtPKMZlfG/1kEOV9Ho+vm3feD5lw==@kvack.org X-Gm-Message-State: AOJu0YwbdJkIEFILZ+rkiBVpQatZq6wA2hwIhYSc4Nb/UEA3wEv+dIfP YC8O2OsTN0vqf2iu9RmYTrQROGLvbXmETX4oGGy5BgnDBS3ck3BW X-Google-Smtp-Source: AGHT+IE6noozYvbqtiN1Ng44Z7wlpCMOXuJB4I0BDFpByRSZ3rjYle85/BW+mx0dMKqPe71bag4ulg== X-Received: by 2002:a05:6902:f84:b0:e30:cd90:b631 with SMTP id 3f1490d57ef6-e337f8f0c61mr412352276.33.1731012327942; Thu, 07 Nov 2024 12:45:27 -0800 (PST) Received: from fauth-a1-smtp.messagingengine.com (fauth-a1-smtp.messagingengine.com. [103.168.172.200]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-462ff5e14f9sm11750031cf.86.2024.11.07.12.45.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Nov 2024 12:45:27 -0800 (PST) Received: from phl-compute-12.internal (phl-compute-12.phl.internal [10.202.2.52]) by mailfauth.phl.internal (Postfix) with ESMTP id C929D1200043; Thu, 7 Nov 2024 15:45:26 -0500 (EST) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-12.internal (MEProxy); Thu, 07 Nov 2024 15:45:26 -0500 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrtdeggddufeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhepfffhvfevuffkfhggtggujgesthdtredttddtvden ucfhrhhomhepuehoqhhunhcuhfgvnhhguceosghoqhhunhdrfhgvnhhgsehgmhgrihhlrd gtohhmqeenucggtffrrghtthgvrhhnpefhtedvgfdtueekvdekieetieetjeeihedvteeh uddujedvkedtkeefgedvvdehtdenucffohhmrghinhepkhgvrhhnvghlrdhorhhgnecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepsghoqhhunhdo mhgvshhmthhprghuthhhphgvrhhsohhnrghlihhthidqieelvdeghedtieegqddujeejke ehheehvddqsghoqhhunhdrfhgvnhhgpeepghhmrghilhdrtghomhesfhhigihmvgdrnhgr mhgvpdhnsggprhgtphhtthhopedujedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtoh epsghighgvrghshieslhhinhhuthhrohhnihigrdguvgdprhgtphhtthhopehkrghsrghn qdguvghvsehgohhoghhlvghgrhhouhhpshdrtghomhdprhgtphhtthhopehlihhnuhigqd hkvghrnhgvlhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehlihhnuhig qdhmmheskhhvrggtkhdrohhrghdprhgtphhtthhopehprghulhhmtghksehkvghrnhgvlh drohhrghdprhgtphhtthhopegvlhhvvghrsehgohhoghhlvgdrtghomhdprhgtphhtthho pehpvghtvghriiesihhnfhhrrgguvggrugdrohhrghdprhgtphhtthhopehtghhlgieslh hinhhuthhrohhnihigrdguvgdprhgtphhtthhopehvsggrsghkrgesshhushgvrdgtii X-ME-Proxy: Feedback-ID: iad51458e:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 7 Nov 2024 15:45:26 -0500 (EST) Date: Thu, 7 Nov 2024 12:45:25 -0800 From: Boqun Feng To: Sebastian Andrzej Siewior Cc: kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, "Paul E. McKenney" , Marco Elver , Peter Zijlstra , Tomas Gleixner , Vlastimil Babka , akpm@linux-foundation.org, cl@linux.com, iamjoonsoo.kim@lge.com, longman@redhat.com, penberg@kernel.org, rientjes@google.com, sfr@canb.auug.org.au Subject: Re: [PATCH v2 3/3] scftorture: Use a lock-less list to free memory. Message-ID: References: <20241107111821.3417762-1-bigeasy@linutronix.de> <20241107111821.3417762-4-bigeasy@linutronix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20241107111821.3417762-4-bigeasy@linutronix.de> X-Stat-Signature: 1ogdgxsocnuxb6p81zke9tak8datn5j5 X-Rspamd-Queue-Id: F13C140008 X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1731012300-546312 X-HE-Meta: U2FsdGVkX19dGmnZ2O+yDJujEz7gGTLmY/ZLrn9MXh9cbeZszmsXFtG3QatW95phe1qBYmPB2JJyh/bcDftxtOTusYYh7eqUp+GQuDwqHl8mnfxGKHhsUEU0U5+aL7uGU3rtoQHJ4d8NrA8x3IPOPhL6XUpRzvr8cTl2jFGVvqYRBSHQv3B1nZzjKUNtIkvdJMrcnSI5/vRhFb4rEqvnJECi7SEaMRLSrZbi/Z52WIbmLSPPo2ZwNhulompoQI+AxNg9sCsSgNt7lKA/A6Y7CsUNSgC4eeofdhD2VwKzJSueT1x2OE7o8gQYhW+BHBjGyQJDwiLmoZqs2Trw9+kPdOML+LI1/WUoEwFk7KGoZFt3XoRvOtz0hkEH3yrPqJGywz11aQEtbERklt7pmVfmHEeC9/0uabAdnLxkPv2FwCXZzEaJ9r1lLiYZjDUZ6rAtg08VsNm3zTv2QrWXHxxL1eRLRsdx0/bu2DbY3n1wt8S6mR53jKXfLQIOQyJrEWTHkNbfAHQbFnjNp2+YJ/EwkfgWGOeWlIIdBljuRav59ryUCuxP6EENZ5rlLqpqXWwIRLHEeb2vG+NK0zIozmYcxi5uWN6DBjkhBq/KCEcwx8svaEE9VTLDx+IsPJpcvHAFerQ5qnTEkcKct8iJm58xoMi8R2c80EPTcijY8LYZHpLYNYMFVdIKsPEFOX7J91d+TO8F2cXPLBKJ4R//CjJ6fhkHlGF6h42Zln7wumRxbvryTLksGpB7VnxG08NtbhKbGemagAgpZSWVYdiWLM3L9k2RbKN4RmIlGPGHrSFreF69++viTNJvlsq4jhm/0uokI1Wt1TgElH4R/1UOuWAFDbJ+xpP5PhGZuXrzsapJXzGmkFYYqoqIWGugKMxUa95xvMaKjLXGudgaa2rDSHzLeL11HMAg87JHvPagmzoueNKPcnIWJXtJy58m9DyTdvvj4qmD0WAMyX7feSF/iTx TE0cLdEN VCQI3WtVVtgzNI4w97H9SQfHZqY6hfrHyCrsw46VUGbZR/1MI1lIrR6x5UaAbYrFP3Qwflbg6MVEnVdEfQM+saqLtKKL0qIMt/M7R1X+1egayJL+VwU0bJ+RochQe9eiyk8Fef0jLtASaS/trjfDj3f6Di7CmMdaXhGvLL7Zc7HKGSMiiRWkMSRuIxsttyyKzYoFVo//BviwWGgYY30+B3K+POgoPh2DXrdY+XSnFf8TOgtkGr0BxBoo9HjqX2i0AAwQMtFGN/JziwGdM2gQ5vK0FKK0QJPNlaKcOUwBma5CDPNhki5b0+ux2AupSUF3SqseEiuJKCgvKo1RXp8HUGeqfXYk6DO4VXDO0n6BzmStoY9G6LegW7mPYtA7PGdkN+YBU0h8K8oXYhhqNVQK2X3fDZA5AX1aefYTB0QTyQvldqnXJHfsQigJlmbWvpqdYOSgo X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Nov 07, 2024 at 12:13:08PM +0100, Sebastian Andrzej Siewior wrote: > scf_handler() is used as a SMP function call. This function is always > invoked in IRQ-context even with forced-threading enabled. This function > frees memory which not allowed on PREEMPT_RT because the locking > underneath is using sleeping locks. > > Add a per-CPU scf_free_pool where each SMP functions adds its memory to > be freed. This memory is then freed by scftorture_invoker() on each > iteration. On the majority of invocations the number of items is less > than five. If the thread sleeps/ gets delayed the number exceed 350 but > did not reach 400 in testing. These were the spikes during testing. > The bulk free of 64 pointers at once should improve the give-back if the > list grows. The list size is ~1.3 items per invocations. > > Having one global scf_free_pool with one cleaning thread let the list > grow to over 10.000 items with 32 CPUs (again, spikes not the average) > especially if the CPU went to sleep. The per-CPU part looks like a good > compromise. > > Reported-by: "Paul E. McKenney" > Closes: https://lore.kernel.org/lkml/41619255-cdc2-4573-a360-7794fc3614f7@paulmck-laptop/ > Signed-off-by: Sebastian Andrzej Siewior > --- > kernel/scftorture.c | 39 +++++++++++++++++++++++++++++++++++---- > 1 file changed, 35 insertions(+), 4 deletions(-) > > diff --git a/kernel/scftorture.c b/kernel/scftorture.c > index 555b3b10621fe..1268a91af5d88 100644 > --- a/kernel/scftorture.c > +++ b/kernel/scftorture.c > @@ -97,6 +97,7 @@ struct scf_statistics { > static struct scf_statistics *scf_stats_p; > static struct task_struct *scf_torture_stats_task; > static DEFINE_PER_CPU(long long, scf_invoked_count); > +static DEFINE_PER_CPU(struct llist_head, scf_free_pool); > > // Data for random primitive selection > #define SCF_PRIM_RESCHED 0 > @@ -133,6 +134,7 @@ struct scf_check { > bool scfc_wait; > bool scfc_rpc; > struct completion scfc_completion; > + struct llist_node scf_node; > }; > > // Use to wait for all threads to start. > @@ -148,6 +150,31 @@ static DEFINE_TORTURE_RANDOM_PERCPU(scf_torture_rand); > > extern void resched_cpu(int cpu); // An alternative IPI vector. > > +static void scf_add_to_free_list(struct scf_check *scfcp) > +{ > + struct llist_head *pool; > + unsigned int cpu; > + > + cpu = raw_smp_processor_id() % nthreads; > + pool = &per_cpu(scf_free_pool, cpu); > + llist_add(&scfcp->scf_node, pool); > +} > + > +static void scf_cleanup_free_list(unsigned int cpu) > +{ > + struct llist_head *pool; > + struct llist_node *node; > + struct scf_check *scfcp; > + > + pool = &per_cpu(scf_free_pool, cpu); > + node = llist_del_all(pool); > + while (node) { > + scfcp = llist_entry(node, struct scf_check, scf_node); > + node = node->next; > + kfree(scfcp); > + } > +} > + > // Print torture statistics. Caller must ensure serialization. > static void scf_torture_stats_print(void) > { > @@ -296,7 +323,7 @@ static void scf_handler(void *scfc_in) > if (scfcp->scfc_rpc) > complete(&scfcp->scfc_completion); > } else { > - kfree(scfcp); > + scf_add_to_free_list(scfcp); > } > } > > @@ -363,7 +390,7 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra > scfp->n_single_wait_ofl++; > else > scfp->n_single_ofl++; > - kfree(scfcp); > + scf_add_to_free_list(scfcp); > scfcp = NULL; > } > break; > @@ -391,7 +418,7 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra > preempt_disable(); > } else { > scfp->n_single_rpc_ofl++; > - kfree(scfcp); > + scf_add_to_free_list(scfcp); > scfcp = NULL; > } > break; > @@ -428,7 +455,7 @@ static void scftorture_invoke_one(struct scf_statistics *scfp, struct torture_ra > pr_warn("%s: Memory-ordering failure, scfs_prim: %d.\n", __func__, scfsp->scfs_prim); > atomic_inc(&n_mb_out_errs); // Leak rather than trash! > } else { > - kfree(scfcp); > + scf_add_to_free_list(scfcp); > } > barrier(); // Prevent race-reduction compiler optimizations. > } > @@ -479,6 +506,8 @@ static int scftorture_invoker(void *arg) > VERBOSE_SCFTORTOUT("scftorture_invoker %d started", scfp->cpu); > > do { > + scf_cleanup_free_list(cpu); > + > scftorture_invoke_one(scfp, &rand); > while (cpu_is_offline(cpu) && !torture_must_stop()) { > schedule_timeout_interruptible(HZ / 5); > @@ -538,6 +567,8 @@ static void scf_torture_cleanup(void) > > end: > torture_cleanup_end(); > + for (i = 0; i < nthreads; i++) This needs to be: for (i = 0; i < nr_cpu_ids; i++) because nthreads can be larger than nr_cpu_ids, and it'll access a out-of-bound percpu section. Regards, Boqun > + scf_cleanup_free_list(i); > } > > static int __init scf_torture_init(void) > -- > 2.45.2 >