From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE06BC35FEB for ; Tue, 17 Sep 2024 06:58:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 49AAA6B0088; Tue, 17 Sep 2024 02:58:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 44A5F6B0089; Tue, 17 Sep 2024 02:58:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2EA3D6B008A; Tue, 17 Sep 2024 02:58:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0BA296B0088 for ; Tue, 17 Sep 2024 02:58:49 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 47FAEA019C for ; Tue, 17 Sep 2024 06:58:48 +0000 (UTC) X-FDA: 82573327536.02.4B62B86 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf28.hostedemail.com (Postfix) with ESMTP id 0211CC000D for ; Tue, 17 Sep 2024 06:58:45 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=2CVDQJMu; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=iwb8hpMg; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=2CVDQJMu; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=iwb8hpMg; dmarc=none; spf=pass (imf28.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726556317; a=rsa-sha256; cv=none; b=XokR2XYnzoWnHks/4v6z0E2H0HafuHF5gH5q0NNQUUaa/0WLLXsZsv9YYi6rwsnXti4riy 71hNz7atLTtDnPm1jyp5WPzWluPC+pIy1IUK9Tt/aDU5V2aSgPhyrmAK6SgpT0zmEuo2dK 2tiaVgSlWaY/usnOuN+nHGoPSvddojA= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=2CVDQJMu; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=iwb8hpMg; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=2CVDQJMu; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=iwb8hpMg; dmarc=none; spf=pass (imf28.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726556317; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eb6/2/53r1amqAN8+Pxosb0NBJtbXKF+syEPphgBDmA=; b=K4MajXm1n0aNvGUE6b6bNCIHmso+b/SQGPJH1AM0VI8G6Cm4Xunx0eaRLYkeRI5S9xsM3J eD1lG9VSZEkW8Nl6I2TXg7Y1TnTjGhvGa5P+YHuA2SnzWvwivfZvB5aw+3IsiM1NAQIjwF bQwXrNSmHIW27iHJhRK1sexpqzmf2sw= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 0D4452203A; Tue, 17 Sep 2024 06:58:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1726556324; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eb6/2/53r1amqAN8+Pxosb0NBJtbXKF+syEPphgBDmA=; b=2CVDQJMuRbly79vOqT2OVsObIe8NtK43I+OXPZ/Aaqs5/JYl5fFwC6q48YJyUqkh3WTW6v 1BHS2gzNOeT1uMray2CB1Db9FJvF1lWruQ5zF05ChrDzm2ar907l1eBtIcxtTUByedFOlQ P/r39fsJ6kbolu0wgO37ZPEG00ZRo7I= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1726556324; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eb6/2/53r1amqAN8+Pxosb0NBJtbXKF+syEPphgBDmA=; b=iwb8hpMgHE++RS/frFPzcd/x/4+48zRpv+0xEHyhe2C9ppCXCVwaB1zKjXzNOB0bgCUv+M ZUqtpa8Lby5XFRDA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1726556324; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eb6/2/53r1amqAN8+Pxosb0NBJtbXKF+syEPphgBDmA=; b=2CVDQJMuRbly79vOqT2OVsObIe8NtK43I+OXPZ/Aaqs5/JYl5fFwC6q48YJyUqkh3WTW6v 1BHS2gzNOeT1uMray2CB1Db9FJvF1lWruQ5zF05ChrDzm2ar907l1eBtIcxtTUByedFOlQ P/r39fsJ6kbolu0wgO37ZPEG00ZRo7I= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1726556324; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eb6/2/53r1amqAN8+Pxosb0NBJtbXKF+syEPphgBDmA=; b=iwb8hpMgHE++RS/frFPzcd/x/4+48zRpv+0xEHyhe2C9ppCXCVwaB1zKjXzNOB0bgCUv+M ZUqtpa8Lby5XFRDA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DDD32139CE; Tue, 17 Sep 2024 06:58:43 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id /CimNaMo6WbwNQAAD6G6ig (envelope-from ); Tue, 17 Sep 2024 06:58:43 +0000 Message-ID: <4b107fec-e391-4680-9457-b282310b4454@suse.cz> Date: Tue, 17 Sep 2024 09:01:08 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 12/19] kthread: Default affine kthread to its preferred NUMA node To: Michal Hocko , Frederic Weisbecker Cc: LKML , Andrew Morton , Kees Cook , Peter Zijlstra , Thomas Gleixner , linux-mm@kvack.org, "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Boqun Feng , Zqiang , rcu@vger.kernel.org References: <20240916224925.20540-1-frederic@kernel.org> <20240916224925.20540-13-frederic@kernel.org> Content-Language: en-US From: Vlastimil Babka In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Action: no action X-Rspam-User: X-Stat-Signature: e63xch3ic86jdehotctyiu5558spmbqt X-Rspamd-Queue-Id: 0211CC000D X-Rspamd-Server: rspam02 X-HE-Tag: 1726556325-941683 X-HE-Meta: U2FsdGVkX1+y/W60sI/rCz2Arvkn22L3h4UlloVIibp7v2esKqiLSabtO21lPrz64ERXTI2tHyf4c+lZK5C2Gw2/p+KT1H8KcSp2CHR8IjjU7YJQUoTx7JNK5Ad4eJG4cJCBUqaC7C3WeaXNsyPJPkz+/UNQzkJlkdmbzTmL56l6uiFQNtJCh37vtG1g9qsDaK+5PwJ7Nh6ZWq3+2ODz/A2fzq3PkoU+wDrTSH7EqlEJGSrWPBUh2lFbmi9L1EJlKjlTonvzRnb9tehEQCvM0EpjcQxHPJ86aHNuCQe4FOiE9G/7Hm6Mn9EgHoh6VXkl78mx8IAB5gLElnVukODNSLbKJQqbF/QvyVSEace75pijaPJnMJVZn28CAKtgkpe7YEcuOevlMrAsvzeLkpuDEodKuFSptnqztFfTstxT5inKzm6FgYRt/vL1ty+kM9ODyqV4deSW6deWZz1A8kOGBex996M0Q5In+y09PHLkrvDg3sqtIjVfRzIFKv/ZyF0cFp9w493BT5ueEfsWH46GGjywX8A0aw/hli4yO/AZHo9L9fEPZEknxFotKM2yyuSjdV2Kk8Y1JS3iOXFw7P6nlBC9lIzufN2t3R+miTJpyBVCdubL5j6F6sSLYA5MrCJyO9udcfOuEp8npIPDDcnG2GHNe0smPWyktc1bOwcG9B+kx1Cxfy6D4ljKcYOcmsmu9QeXBekC4dPYeonOsXFf6aR22K4fL4ZExsxoaOLw/GhLlFbwSLDBu7dxewlAvvI6kLvP6AvaSCwV9czzuHkzb1FBHKclHUy1I7SWBsuBL8N4P/ic10yCicRM823AJdqreY5q3cw0Jw3rUNuncddkETghNO1JD/50SMFog9/DLj0vsL3ugi5p99aHfX5e6GR40hasYeZfrUKMyBJkhCbZuPgV6WepZnn2n17ivh3oyxP1mChcMT/QBVer/JH+mDa7uVbMecSe+eIeTVLePIs dxU8aYX0 L84p0N/Mm/Btuzo73Qz7s8ElR68nD/8Es+QHMDm3LiRf5kJ9U30AHb9i7c9lEgj7IQnPVtImyjw6vfgINThh1rpYHBaWSNW/mIwcPwOkNR2CizqDZUR8BalSWvoFhhCLO+a2e+hk55wuQJn5l337UuXl86guXcOvmjh8hmTc6FdX9spwpW9SXegvoF2dk/tu/S+Z4uwa8NAsB8q+uzYFQiOOC4CR9NP8nVO2kQ8ULhTmn2mnZU5mbu12h2gM2rNHV/fjN4SCcaA0cgWXm2gQaqiB9OT3XoFcvkVwG366jumpzy40EJXky9Ki8SaNwvAYvnVhefWIlLLrUCA0cOA/tssyC4w+WFOjxhjmS/1YZvitv7mE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 9/17/24 8:26 AM, Michal Hocko wrote: > On Tue 17-09-24 00:49:16, Frederic Weisbecker wrote: >> Kthreads attached to a preferred NUMA node for their task structure >> allocation can also be assumed to run preferrably within that same node. >> >> A more precise affinity is usually notified by calling >> kthread_create_on_cpu() or kthread_bind[_mask]() before the first wakeup. >> >> For the others, a default affinity to the node is desired and sometimes >> implemented with more or less success when it comes to deal with hotplug >> events and nohz_full / CPU Isolation interactions: >> >> - kcompactd is affine to its node and handles hotplug but not CPU Isolation >> - kswapd is affine to its node and ignores hotplug and CPU Isolation >> - A bunch of drivers create their kthreads on a specific node and >> don't take care about affining further. >> >> Handle that default node affinity preference at the generic level >> instead, provided a kthread is created on an actual node and doesn't >> apply any specific affinity such as a given CPU or a custom cpumask to >> bind to before its first wake-up. > > Makes sense. > >> This generic handling is aware of CPU hotplug events and CPU isolation >> such that: >> >> * When a housekeeping CPU goes up and is part of the node of a given >> kthread, it is added to its applied affinity set (and >> possibly the default last resort online housekeeping set is removed >> from the set). >> >> * When a housekeeping CPU goes down while it was part of the node of a >> kthread, it is removed from the kthread's applied >> affinity. The last resort is to affine the kthread to all online >> housekeeping CPUs. > > But I am not really sure about this part. Sure it makes sense to set the > affinity to exclude isolated CPUs but why do we care about hotplug > events at all. Let's say we offline all cpus from a given node (or > that all but isolated cpus are offline - is this even > realistic/reasonable usecase?). Wouldn't scheduler ignore the kthread's > affinity in such a case? In other words how is that different from > tasksetting an userspace task to a cpu that goes offline? We still do > allow such a task to run, right? We just do not care about affinity > anymore. AFAIU it handles better the situation where all houskeeping cpus from the preferred node go down, then it affines to houskeeping cpus from any node vs any cpu including isolated ones. Yes it's probably a scenario that's not recommendable, but someone might do it anyway...