From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 566BDC35FEA for ; Tue, 17 Sep 2024 07:12:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B36336B0088; Tue, 17 Sep 2024 03:12:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AE6316B0089; Tue, 17 Sep 2024 03:12:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9AD5E6B008A; Tue, 17 Sep 2024 03:12:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7E43F6B0088 for ; Tue, 17 Sep 2024 03:12:20 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id B95AD8016D for ; Tue, 17 Sep 2024 07:12:19 +0000 (UTC) X-FDA: 82573361598.04.C88AE8F Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf18.hostedemail.com (Postfix) with ESMTP id 5B9911C0008 for ; Tue, 17 Sep 2024 07:12:17 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=VGqV8L9M; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=LknOXeBH; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=VGqV8L9M; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=LknOXeBH; dmarc=none; spf=pass (imf18.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726557015; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=N617NhoUHi8bylefi8DLZm5EoXdh0UsbJwfB5ib7reg=; b=FEzlln7WCIJG7dQ1A7wQM4tSABPETnj3XcMKvtEwpTB1bAq33w/AgXbhyL4968VjwQMz2n 4eBIiGEDUvhMJEaJBOCAIgjsNZW3dYe+u7g9cAwvwGbinyTliEPtyUXiTYH5yA5s3QRT3r lZ9zuBlE7RblThTIXihP+EmF+GBZcEY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726557015; a=rsa-sha256; cv=none; b=PWW7hPeyTTkm/v9TpNnv5snUhtaQtF0AfzYms6OGfBINuIJSAF9/NYkvsOxvg8dZ9J2Kgu lBK+8CF3pK/Mv58sba+tsQ9at7KIUu/s4mU/EtSWUOkCKOdgJREOWEw5Vs/wNSuKLfBg69 oOz44THgYvsKfvjYQ6BOulUbBATGOf8= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=VGqV8L9M; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=LknOXeBH; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=VGqV8L9M; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=LknOXeBH; dmarc=none; spf=pass (imf18.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id ABA071FF95; Tue, 17 Sep 2024 07:12:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1726557135; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N617NhoUHi8bylefi8DLZm5EoXdh0UsbJwfB5ib7reg=; b=VGqV8L9MtPvCaAyOEoVIBsOjaFs1htU2JdsdMKYffeGa7wZEQs0uKPcEMDkJqFpWKKrihV 0R7EZ+rfVyOwCkWTXoJBRjQ3kyWTrFLayzLvbNEpbegNRhBUK+2lZ1y83vHH+xEuc33h7s 4UxeZ/i9Bqcaw2BI+vSRb581ARCkuCA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1726557135; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N617NhoUHi8bylefi8DLZm5EoXdh0UsbJwfB5ib7reg=; b=LknOXeBHiovcaWj9EeUjg6JJkwb7OCb/1kg+OwZz6z+06b8LhSdyHKD9gPXbk/zwjTp1Ou CB8BnX/TbFCMntBg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1726557135; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N617NhoUHi8bylefi8DLZm5EoXdh0UsbJwfB5ib7reg=; b=VGqV8L9MtPvCaAyOEoVIBsOjaFs1htU2JdsdMKYffeGa7wZEQs0uKPcEMDkJqFpWKKrihV 0R7EZ+rfVyOwCkWTXoJBRjQ3kyWTrFLayzLvbNEpbegNRhBUK+2lZ1y83vHH+xEuc33h7s 4UxeZ/i9Bqcaw2BI+vSRb581ARCkuCA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1726557135; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N617NhoUHi8bylefi8DLZm5EoXdh0UsbJwfB5ib7reg=; b=LknOXeBHiovcaWj9EeUjg6JJkwb7OCb/1kg+OwZz6z+06b8LhSdyHKD9gPXbk/zwjTp1Ou CB8BnX/TbFCMntBg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 86511139CE; Tue, 17 Sep 2024 07:12:15 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id qgxLIM8r6WakOQAAD6G6ig (envelope-from ); Tue, 17 Sep 2024 07:12:15 +0000 Message-ID: <3f86acec-8aa0-4448-843f-509a182b5459@suse.cz> Date: Tue, 17 Sep 2024 09:14:40 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 12/19] kthread: Default affine kthread to its preferred NUMA node To: Michal Hocko Cc: Frederic Weisbecker , LKML , Andrew Morton , Kees Cook , Peter Zijlstra , Thomas Gleixner , linux-mm@kvack.org, "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Boqun Feng , Zqiang , rcu@vger.kernel.org References: <20240916224925.20540-1-frederic@kernel.org> <20240916224925.20540-13-frederic@kernel.org> <4b107fec-e391-4680-9457-b282310b4454@suse.cz> Content-Language: en-US From: Vlastimil Babka In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5B9911C0008 X-Stat-Signature: 5w6kf4nstzf6nt3oqwxp7z4k9mybyrrg X-Rspam-User: X-HE-Tag: 1726557137-76004 X-HE-Meta: U2FsdGVkX19lnuQ/wUhUYhS/0JA9hKEb9nDCaEdBomabl4OO2P4Dq68sJy/ohhK8W1t8gGvwbIc5kTelnHyMYDqwxuZXx4cDlV48KL9fnpXcZVpyFSRq7Wyt6Z4ot0nxzgQxVBK/dbsU7tAseHKQST7MypUWlJKBr7yfmE4YBpiJb6W5qX93c5GKMRxGvRsVaXl/2rEcmFyoyI6paJT9q355mZohXnfjFC06gIJdflrRtdADIt24aCMTL8uuGNhTij0PLIxHn6XU5yaxXX6CNIfpOkzEC9lzsef14wR6JGucZraMAvlipgspNg2y536sh2622txj64mCPmJYHDkfLezEE6mw+vKMUpePsokRO6e6OYlWUe6jtzSEDtj9mGteWbyTZu8cFmEmEe8XhxjWa+VScvJH8Hmrf6j9hOmnzDAv004lWUGo61Q1S2D4hZ1BCTY2oDQqpA4FDYbn33AvoUB0EinOC1nqsDEOtja3l/t+FFm/j+7BVOInHFENwuEkZQqZzmwV3ii7tDG3LaMH4b9ElDgdT/CEbfX8dqSARuIIgaI8lBDquXtH7gVGzZ/CNqfxL7MmByhDo1pew5t4ogxQ1ahh5Y25J+jn+n1+WsfFxLQmDOdSHeZ6i+phH5SAVEWQapzPDefevAN3/AfnAPPmHEnCO49x5RYmdLud/1rP1OyCRw0cRqbvBq0eTsqTA7zGEE2bwiwaVL7QgPKVvbXVlK/mxRss6/PFrfgxqgE8THjU/QUhMLXvYq8oUXwtbYJ24+7uDwNXWJ2oRUphHY8qdlW6AiZIuUjGbSxIdoxeI513shBON3e2qJ/FmyNEVhIylQK0bzajdNXhwGMFC0+I9ADD4ZIR7LXC/z9jxx1GhrgoLTHDOYPgq9xNe8E4ePTV7AbKGpIxrsb+GBRCSsEL5tz6yPKXRZjCkxs5A1zdEa+iM3k+po+ZUZVpR0wWaEjnKSyCIGPwwEN6JVF OFRFI0L8 IeHRA8/TVLGEWYtHBh/l+VLWc1eCzRbn6A6SXKeZ3vyu4L2IbhIOiVTJazQCt+Uy6VyGNabzIN3Wf3XknLWkheLK3ppK/nIC7OrlJFzI/EjhJ6muW98hjQKgzjj3vxuSZm7nPEOBXbGC6QLvC5SLnDzNGgPf+AFEj7lWn34SX8bke08jZ+UiEyML21oE8Kk2+C+Jxnrvjpp/YxxXP7Yf55jAInY4uVm4Fgegwr+bwM29IxJeyLoxqNGGddsX1Oev4ddDHonS3geRmF8EP9kvJ7SaN3uQzekL14Ogj4J6Agx1zP2GRrWAmJd5m+FEjjD3xRdgSc5ebGJpjq/qBmse5dvBk0jHuuKzpZcvCFcLRh8itvkI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 9/17/24 9:05 AM, Michal Hocko wrote: > On Tue 17-09-24 09:01:08, Vlastimil Babka wrote: >> On 9/17/24 8:26 AM, Michal Hocko wrote: >>> On Tue 17-09-24 00:49:16, Frederic Weisbecker wrote: >>>> Kthreads attached to a preferred NUMA node for their task structure >>>> allocation can also be assumed to run preferrably within that same node. >>>> >>>> A more precise affinity is usually notified by calling >>>> kthread_create_on_cpu() or kthread_bind[_mask]() before the first wakeup. >>>> >>>> For the others, a default affinity to the node is desired and sometimes >>>> implemented with more or less success when it comes to deal with hotplug >>>> events and nohz_full / CPU Isolation interactions: >>>> >>>> - kcompactd is affine to its node and handles hotplug but not CPU Isolation >>>> - kswapd is affine to its node and ignores hotplug and CPU Isolation >>>> - A bunch of drivers create their kthreads on a specific node and >>>> don't take care about affining further. >>>> >>>> Handle that default node affinity preference at the generic level >>>> instead, provided a kthread is created on an actual node and doesn't >>>> apply any specific affinity such as a given CPU or a custom cpumask to >>>> bind to before its first wake-up. >>> >>> Makes sense. >>> >>>> This generic handling is aware of CPU hotplug events and CPU isolation >>>> such that: >>>> >>>> * When a housekeeping CPU goes up and is part of the node of a given >>>> kthread, it is added to its applied affinity set (and >>>> possibly the default last resort online housekeeping set is removed >>>> from the set). >>>> >>>> * When a housekeeping CPU goes down while it was part of the node of a >>>> kthread, it is removed from the kthread's applied >>>> affinity. The last resort is to affine the kthread to all online >>>> housekeeping CPUs. >>> >>> But I am not really sure about this part. Sure it makes sense to set the >>> affinity to exclude isolated CPUs but why do we care about hotplug >>> events at all. Let's say we offline all cpus from a given node (or >>> that all but isolated cpus are offline - is this even >>> realistic/reasonable usecase?). Wouldn't scheduler ignore the kthread's >>> affinity in such a case? In other words how is that different from >>> tasksetting an userspace task to a cpu that goes offline? We still do >>> allow such a task to run, right? We just do not care about affinity >>> anymore. >> >> AFAIU it handles better the situation where all houskeeping cpus from >> the preferred node go down, then it affines to houskeeping cpus from any >> node vs any cpu including isolated ones. > > Doesn't that happen automagically? Or can it end up on a random > isolated cpu? Good question, perhaps it can and there's no automagic, as I see code like: + /* Make sure the kthread never gets re-affined globally */ + set_cpus_allowed_ptr(current, housekeeping_cpumask(HK_TYPE_KTHREAD));