From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45E32C47DDF for ; Fri, 26 Jan 2024 22:23:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 85BB26B008C; Fri, 26 Jan 2024 17:23:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 80C066B0093; Fri, 26 Jan 2024 17:23:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D3C16B0098; Fri, 26 Jan 2024 17:23:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5E9596B008C for ; Fri, 26 Jan 2024 17:23:37 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id EF6F51408C7 for ; Fri, 26 Jan 2024 22:23:36 +0000 (UTC) X-FDA: 81722890032.23.F701C1B Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf06.hostedemail.com (Postfix) with ESMTP id 5AA3D18000D for ; Fri, 26 Jan 2024 22:23:34 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=NgmwWkzG; spf=none (imf06.hostedemail.com: domain of tim.c.chen@linux.intel.com has no SPF policy when checking 198.175.65.11) smtp.mailfrom=tim.c.chen@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706307815; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zuY7lL/Oqsss+h4z+5Mo9FLHeC0t4uNcpKK2zUdAWwQ=; b=Hu8mkZF4n8sF37t2Huvy03999kjgxfJr80aEd5iHSFO56OkG8OEX5eK7htJqH9YqshnbvG AaBN8zXcHhqsmEB4c7KYXYKT1EYj8LP+gzzw25puiKovFMNBdxdRBbxjU9GhlsnmsBpfG4 evg+1zp9Q41T3tVzcPJefcmgpp6PH3o= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=NgmwWkzG; spf=none (imf06.hostedemail.com: domain of tim.c.chen@linux.intel.com has no SPF policy when checking 198.175.65.11) smtp.mailfrom=tim.c.chen@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706307815; a=rsa-sha256; cv=none; b=19rni3IDGVNEA1AJgpS2IlPgk37N/iiYDiYsV/tXN+U1UALHrrNKl0EcTJSNndg4GbZONk WyZ1fei+4ptgmpUWTUUNdyghl7IK8Aq3ZZDyMad+yQSeFbp2V+2SLg69ei5UhatnNEoVBn jzky+tQ5TK3+Ybq/+hy3UE8V0nY/QJo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706307814; x=1737843814; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=snNBb4ZA1iemTzKjgqNBLfmnTn5JS2jZunb5rdWoIpE=; b=NgmwWkzGJQjWDD5vYLnyF9yqIQRTMKAdihyKmHRlxonl4OJ7WPRUeO/s CBUIWEqNpfnOg3kN9l2yVaucX0w3wGrDUm4K6Al72ueXBrynMYPEM8QWl W99fr6UrTR7twthp3Wvw6qv8e2qOelXGOev2/b/6LyIxMunJs3ghKsxzk XFi8+Ypn05YF1JLdV7fcVyVr4p5vjTFwdB19x2JDoOoU0EQq7jsLTK7HY M1iMlxR/A1FmmF/1jvMz2tbAuQVkR0jZBNttKT5jlw0ji85G6q3WZrRHe hTVss11uwV5tFooMNDRYYyOJr8RcDKSFSz4wXTPF6/Oyof10NZ6vjv8nt A==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9251535" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9251535" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 14:23:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="906436829" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="906436829" Received: from dlberger-mobl.amr.corp.intel.com (HELO [10.209.52.191]) ([10.209.52.191]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 14:23:31 -0800 Message-ID: <7379f03364a7291770f9622b91c4d84e26352b20.camel@linux.intel.com> Subject: Re: [PATCH v5 3/7] padata: dispatch works on different nodes From: Tim Chen To: Gang Li , David Hildenbrand , David Rientjes , Mike Kravetz , Muchun Song , Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, ligang.bdlg@bytedance.com Date: Fri, 26 Jan 2024 14:23:30 -0800 In-Reply-To: <20240126152411.1238072-4-gang.li@linux.dev> References: <20240126152411.1238072-1-gang.li@linux.dev> <20240126152411.1238072-4-gang.li@linux.dev> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.44.4 (3.44.4-2.fc36) MIME-Version: 1.0 X-Rspamd-Queue-Id: 5AA3D18000D X-Rspam-User: X-Stat-Signature: hocmntx8454u94pm1je1wcjqf7wbq4ci X-Rspamd-Server: rspam01 X-HE-Tag: 1706307814-887155 X-HE-Meta: U2FsdGVkX19B5t4MOizVDnmfwD+QaGwUT+4TBeRWdcxzJRzosEokzmDIGzPxCo/qf+BoFLOJm7W9jJ4zti0OMF+o4BgBC1viZjCZqy8rautb1RGIkC/Aid+/Ts3mkXULk/J/Tvz0qhuKJxU65OGDK/iIoDjdQwGtjyekPOA9IIddpqUL6dV1BKfd4awriAMAq/XcTBoO0ByzqhP5e1H13s+aT6bVeR3jNgqL14IDRnS+l5W5Ihb2ZoPREzS20f5JPc3ZOQ0/xsQytNz4djhBVMLQSTqX7Of7O4l8Axjd0o0MN+IYYYfBWCkB3YCwVl1QAYGhOigAU5OvCkc3PyZqy9l9kXg61kwzXiIXebn4cF4RsWq2c22PZTPTVjxG8ZZ+xUS2UpYn3nC5TwU1ewD2rhnltT80qzrUuc7BX873Tpb2aXjrYqQsi1/mnwM+zdWCpi/eZmGlK3HY/3OwgfqiwilLBmNPpPqUMTZargse/sZbWg7VOe9cflZXo88lTFAyUC1pJFxvCw/Ti59kBAargzuE3NL78VnznjFSAZRhoZ+Uz5LHBm1r7YLZpOtyS7cpEQdxIJ1mNLya91OiTXXxOX889vWaX0exAXVOSwW7txAJXahatxqMQTMZsyXfpDzmaVEegbCW8OIEhubgh7QUg6ZoSky8H+OetX7OAHVszuyUmfCJZC0lhFzaJAVa57YG73YFiKR6rPDIkFoHSxdaG0hd/NwZ7Fj82tpvlA83PAnCYTPqfQhbIvw1DMWVvXkTNR4RWGCSC6uCpbZEMKtcNBPrut67NNBYIXSEmyfI1Ujiuh698nsfqfVx6yNlJamKmcQOOi6UTPl1HqqUKQ0EIKtul6kLAASXgQ2eigExoVhPmQxWQ2sS90o67Cf4Y37di0ksTaEtvcifBCBTsbEKTV9fJi9SmMPjmOejqMo/Fs5SeQYq3/xtDjNuw5euxqbt3EfGqn8IGfTydGbKWPv tVo0aZzY vQq6+IYK4oKVWQmJWs39CgG3bhFVorYJejlMDrE9UPYBWwv6ay6lSmAhiVm9mS22de6inyjOcIqxLCcmUp/X2uHFoP+hTFwVYqxaXmlVdHgwa1nV25G/5TLm8nVwQcnwFxqkQG+Wx055aMl3wJpyyepEmqIYOcGmjY7lVcvCOQVXfUS+4hpaHBkI8e8tCHIRH7HzDU8iL/PvylZRbbVn3wwDQQJc6d8/JfJMSSk6+OH8pgEA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, 2024-01-26 at 23:24 +0800, Gang Li wrote: > When a group of tasks that access different nodes are scheduled on the > same node, they may encounter bandwidth bottlenecks and access latency. >=20 > Thus, numa_aware flag is introduced here, allowing tasks to be > distributed across different nodes to fully utilize the advantage of > multi-node systems. >=20 > Signed-off-by: Gang Li > Tested-by: David Rientjes > Reviewed-by: Muchun Song Reviewed-by: Tim Chen > --- > include/linux/padata.h | 2 ++ > kernel/padata.c | 14 ++++++++++++-- > mm/mm_init.c | 1 + > 3 files changed, 15 insertions(+), 2 deletions(-) >=20 > diff --git a/include/linux/padata.h b/include/linux/padata.h > index 495b16b6b4d72..8f418711351bc 100644 > --- a/include/linux/padata.h > +++ b/include/linux/padata.h > @@ -137,6 +137,7 @@ struct padata_shell { > * appropriate for one worker thread to do at once. > * @max_threads: Max threads to use for the job, actual number may be le= ss > * depending on task size and minimum chunk size. > + * @numa_aware: Distribute jobs to different nodes with CPU in a round r= obin fashion. > */ > struct padata_mt_job { > void (*thread_fn)(unsigned long start, unsigned long end, void *arg); > @@ -146,6 +147,7 @@ struct padata_mt_job { > unsigned long align; > unsigned long min_chunk; > int max_threads; > + bool numa_aware; > }; > =20 > /** > diff --git a/kernel/padata.c b/kernel/padata.c > index 179fb1518070c..e3f639ff16707 100644 > --- a/kernel/padata.c > +++ b/kernel/padata.c > @@ -485,7 +485,8 @@ void __init padata_do_multithreaded(struct padata_mt_= job *job) > struct padata_work my_work, *pw; > struct padata_mt_job_state ps; > LIST_HEAD(works); > - int nworks; > + int nworks, nid; > + static atomic_t last_used_nid __initdata; > =20 > if (job->size =3D=3D 0) > return; > @@ -517,7 +518,16 @@ void __init padata_do_multithreaded(struct padata_mt= _job *job) > ps.chunk_size =3D roundup(ps.chunk_size, job->align); > =20 > list_for_each_entry(pw, &works, pw_list) > - queue_work(system_unbound_wq, &pw->pw_work); > + if (job->numa_aware) { > + int old_node =3D atomic_read(&last_used_nid); > + > + do { > + nid =3D next_node_in(old_node, node_states[N_CPU]); > + } while (!atomic_try_cmpxchg(&last_used_nid, &old_node, nid)); > + queue_work_node(nid, system_unbound_wq, &pw->pw_work); > + } else { > + queue_work(system_unbound_wq, &pw->pw_work); > + } > =20 > /* Use the current thread, which saves starting a workqueue worker. */ > padata_work_init(&my_work, padata_mt_helper, &ps, PADATA_WORK_ONSTACK); > diff --git a/mm/mm_init.c b/mm/mm_init.c > index 2c19f5515e36c..549e76af8f82a 100644 > --- a/mm/mm_init.c > +++ b/mm/mm_init.c > @@ -2231,6 +2231,7 @@ static int __init deferred_init_memmap(void *data) > .align =3D PAGES_PER_SECTION, > .min_chunk =3D PAGES_PER_SECTION, > .max_threads =3D max_threads, > + .numa_aware =3D false, > }; > =20 > padata_do_multithreaded(&job);