From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 872B8C3DA4A for ; Mon, 5 Aug 2024 14:29:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F1C8F6B0082; Mon, 5 Aug 2024 10:29:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ECC7C6B0088; Mon, 5 Aug 2024 10:29:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE2B56B0089; Mon, 5 Aug 2024 10:29:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C0FF56B0082 for ; Mon, 5 Aug 2024 10:29:06 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2B09381C3A for ; Mon, 5 Aug 2024 14:29:06 +0000 (UTC) X-FDA: 82418423892.07.C30B4C9 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf09.hostedemail.com (Postfix) with ESMTP id 1789E14003D for ; Mon, 5 Aug 2024 14:29:02 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=MDFUOX5u; spf=pass (imf09.hostedemail.com: domain of frederic@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722868083; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=i8tJoj1TDT6cwgYEHfX/teFRfQM3kS9JxI2Q1V3uoQw=; b=A4ZQV34cdNMSMEaqJi0WdrdJQAbgdwbB6lSI7Kvl5ySkNrPLL0UGlkD62HXbF5DICUk73D Kw9bVewkg3w403dgxhWk8/feUlui0saJ4JLhw590Y8zTavyCX5pwrEOV2H/KDSGy7cgEA/ 2WcXdP9KB1dvHyHiBmgfnY+eFRNPgeo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722868083; a=rsa-sha256; cv=none; b=Fpm95FM3DC0UfeM+sfd2MdyZF41U5B3OjNFi78bD6cxTXNHA3J6leV8gz1mVu/GW3gbUZX WkmXGktSA3O7papS0zZeZqbkLF9q8gLkJvMgckv8GBeRfM4e8Zu/lBd0A/KzTyVmqHXVcn usNZ8wsY5yhudZ1nnu6rChLZUcbjjH4= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=MDFUOX5u; spf=pass (imf09.hostedemail.com: domain of frederic@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 94B05CE0A15; Mon, 5 Aug 2024 14:28:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8C29DC32782; Mon, 5 Aug 2024 14:28:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722868138; bh=2MUO96snC0kehze5bCKp0JkvmV/H1MvGnp8mPHmGAiM=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=MDFUOX5ueN5MXxhCuXCSKKZN2EmVbjnuR4T/bPKzEDT8PuaM24GmAKtFyUAu1r7At hB0zwQE05FfKARiaBNJNOOkoZ+gjR/WmyXaJQN9OqKOriCWmKElyz2PYCrofdUtZqp ndK2lR1EefMAdGqAQKzqsxpaYh4tLG3pXqqUlhzzdA4OxD372mV+VWFIJvmukb+Fom WkHEAOy0MmcmFv0yx3nNEcRycM4cs2iKbO7pMgGLVRGTebDArVSf0jxtzRfMJKA1Q8 ZPV23z8PYWXrDiHWyG4xD1vZx5R0Ig+6IVZE1vjQJprsM/EwTZlM0S4itG4RoSZ8dc vkltUEmEoVGtg== Date: Mon, 5 Aug 2024 16:28:55 +0200 From: Frederic Weisbecker To: Vlastimil Babka Cc: LKML , Andrew Morton , Kees Cook , Peter Zijlstra , Thomas Gleixner , Michal Hocko , linux-mm@kvack.org, "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Boqun Feng , Zqiang , rcu@vger.kernel.org Subject: Re: [RFC PATCH 12/20] kthread: Implement preferred affinity Message-ID: References: <20240726215701.19459-1-frederic@kernel.org> <20240726215701.19459-13-frederic@kernel.org> <4e9d1f6d-9cd8-493c-9440-b46a99f1c8af@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4e9d1f6d-9cd8-493c-9440-b46a99f1c8af@suse.cz> X-Rspamd-Queue-Id: 1789E14003D X-Stat-Signature: bfneoo4bdmacndywstukyjnu49hp4jyi X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1722868142-103815 X-HE-Meta: U2FsdGVkX1+uw9bRPa57TZbe+4bGSRtFO7+j0Ir2G5TLag8s2B8I+VZrjq4frEN5dVfnPDJx209ZLubBkEwW+hLwoqntbg7q6eiRHBOsDTXT96WtwMdvUfp0I47jMqAK2RT99GNqvCJL9qCUwn47d5GkDdAHI0mEtPwSF4KZYaZalzKhRfVyKpA3DwoXmcVDPqNLtO9bU9v6R+g3fcsFcL4NHjot03zcwPAixmmlkrZxqQPJj8YRxSzuq3LeKbEFD89TP0tMJ2RrIMXEIP0dFBBj92IquM6pAtM2aHS1llJSzRHADZyvTHoMO6o9r4jFveQrdBy3TLePNPzuyD7K6M8JScUovpBOskyf7zMfzeqROnP0MvXlJfTIafZ3v+XbJGIWw5QsbYLsAkJOSDPT219uBCA+6N3PExkEu/hJV/qjbZIzLdRCONO5LNoI30fJCzdgGpcMtb5DYspNtphYTZw36Yr5W7KmLgy4SZtOZTNEXqLmlMMca5F5PoDWplCkzdBkKjornW7+VScrWKzErdcj9GGD0lfO8zWoCIgtAnSDQ9/Y9IQcMvInDjq6RwrYBPN6LpM2zGh7xne0WTh+DSj7MYem+E25ueyuIuzs/iOasftrS5oEuGY6sIaTD9bLmhIt4XKo4++DWWuP+ez9Umil6PTkbM8w4obHhqqLjtJhjIZwAYorky4Zh4+Pk5qP+5+I2cFMMwf/A5RSe2BR0Erzv99PNyFTZySwgr7oggWxzPWhDxTo8/2hfs66TIdOrvo5KYQ9r/gj0jShJ8sZtJgDxsVE5aV+XYYWX+UqSNGPCIZfUpAVU8qktZzbw8xSUE8necPGlnrtybPlb8OXqJ1Ldv0OpoSwub3qu1+zunwXOUrwTcAlHeXi66i2psGVx6u7DwHHDSdyvuUqcJ+V+fsfI0f2xhZmrcpWYKWqsrnyLuE4qOwybNePYAeN1BsEHTT4Xxhrx5r1w7yDAF+ R5bVoDN4 9+Z4WGh+wwkXCAXH+c3C9Yqo4GqNS60G3RI60Gu53vaAFT+tLiwKkHT03wAPZJ62ajDuB8EmH+6LOFMP+fCRbYb1ONYXkR6cO0NMaxVgfEE9ssTCLCloZ3WvpQ5LApXQsbfGQBVdjZ5DPApGYUYeRrPJqyr94pa5E/CUiptpRo0sdWJ3W9HRpjxhFq8kCnaRnEWJu2FDDMrNqpnQo4qW6XXao7mvet74lvGAyNfPbp/2KTjy5IGxjc/1Dlcvcs5lFV3oF4Tw9Z5gOCH5+Z2YREqQUNQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Le Tue, Jul 30, 2024 at 05:49:51PM +0200, Vlastimil Babka a écrit : > Acked-by: Vlastimil Babka > > Nit: > > > +int kthread_affine_preferred(struct task_struct *p, const struct cpumask *mask) > > +{ > > + struct kthread *kthread = to_kthread(p); > > + cpumask_var_t affinity; > > + unsigned long flags; > > + int ret; > > + > > + if (!wait_task_inactive(p, TASK_UNINTERRUPTIBLE) || kthread->started) { > > + WARN_ON(1); > > + return -EINVAL; > > + } > > + > > Should we also fail if kthread->preferred_affinity already exist? In > case somebody calls this twice. Good point! > > Also for some of the use cases (kswapd, kcompactd) it would make sense > to be able to add cpus of a node as they are onlined. Which seems we > didn't do, except some corner case handling in kcompactd, but maybe we > should? I wonder if the current implementation of onlining a completely > new node with cpus does the right thing as a result of the individual > onlining operations, or we end up with being affined to a single cpu (or > none). > > But that would need some kind of kthread_affine_preferred_update() > implementation? So you mean that the "for_each_node_state()" loop in kcompactd doesn't handle all possible nodes but only those online when it's called? Or am I confused? If all users of preferred affinity were to use NUMA nodes, it could be a good idea to do a flavour of kernel/smpboot.c which would handle per-node kthreads instead of per-cpu kthreads. I initially thought about that. It would have handled all the lifecycle of those kthreads, including creation, against hotplug. Unfortunately RCU doesn't rely on per-NUMA nodes but rather use its own tree. If there be more users of real per NUMA nodes kthreads than kswapd and kcompactd, of course that would be much worth considering. Thanks.