From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D912C636CC for ; Tue, 7 Feb 2023 12:16:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 890076B009B; Tue, 7 Feb 2023 07:16:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 83FC76B009D; Tue, 7 Feb 2023 07:16:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 72F0D6B009F; Tue, 7 Feb 2023 07:16:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 62C5D6B009B for ; Tue, 7 Feb 2023 07:16:54 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2BC6E12017E for ; Tue, 7 Feb 2023 12:16:54 +0000 (UTC) X-FDA: 80440394748.16.00367DC Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf19.hostedemail.com (Postfix) with ESMTP id 7235F1A0019 for ; Tue, 7 Feb 2023 12:16:52 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=BNl1ztfy; spf=pass (imf19.hostedemail.com: domain of frederic@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675772212; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Sj3Fc1nJRFHhEGQ9G4K5URqZKMMl9fg6ZZuCb5oJm2s=; b=K+kfzGPtd7q0SeTFLsaLPN6suDm+3grMw2fAL5DrHUzykCTJV9ZX0ll6hJxc6Obv855ozt PzyA+sAoq7ICLyOjAGORNLN3wCqOdnU590xxqBn3VlaKr3sJNTtljxaC1RB9RVUK0P+r/Z X/vrougE66Ue9wgBWWQcKHpOPxK/Cxo= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=BNl1ztfy; spf=pass (imf19.hostedemail.com: domain of frederic@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675772212; a=rsa-sha256; cv=none; b=5xSbKoPDPg3FK/iUFb5e4CAiMB5vRlaeIac4mhZ0GbZ8VRX3HOPJNHmonHIh7EzL/RzADV P7o4HML5fxA+fEruD/vyCJSzcJffHyUHBzX8YvTqoFpgNPMSgZhcgAma7dLo95ZGF8sWyP 95upNqiHwZiVAde09qmrlmK/vqeAMJo= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 827166135A; Tue, 7 Feb 2023 12:16:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F03FC433AA; Tue, 7 Feb 2023 12:16:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1675772210; bh=2JXiT/NDBiJUjJkNFjkNT8IoqvYyB9wj9pr5FrsMNEo=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=BNl1ztfy4YwyRJx/Nz+Yol0AJxjcjfDDMG2DxzVQoqNwGsBar8AZF6/RTi8x0IzRA Ae7c3GBPt5/qN0tVSJnsAyQdKrMP5/xXUnjSF56Y0zXteDiYcmc8ibaGwTKLI55leZ 51P4LXuWGWv59+2n9H/jZYqDAm5KAMrKQy/O1/d6bf7ZP5lqUUNjTocb/hm/G6qxxL HSIlCvzdoi3SjdSq8LpeO4y9z+RN1Sa+52YrjIi+eOjRZFdIA879OYH103BVdMKleo rCpqVoHihTVP4ehCvUIYd8ibub1lgD4oU6TG1GjNBmOF2tlV/hrgSQfcF5BSMcX8Rx Z0FiDMsBm+JbA== Date: Tue, 7 Feb 2023 13:16:47 +0100 From: Frederic Weisbecker To: Waiman Long Cc: LKML , Roman Gushchin , Michal Hocko , Marcelo Tosatti , Leonardo , Johannes Weiner , Shakeel Butt , Muchun Song , Andrew Morton , Peter Zijlstra , Thomas Gleixner , cgroups@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH 2/2] sched/isolation: Add cpu_is_isolated() API Message-ID: References: <20230203232409.163847-1-frederic@kernel.org> <20230203232409.163847-3-frederic@kernel.org> <0f388863-9498-e61e-e2dc-965654544489@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <0f388863-9498-e61e-e2dc-965654544489@redhat.com> X-Rspamd-Queue-Id: 7235F1A0019 X-Stat-Signature: 7rdug1qsntg3j14frf3sw8fjssjs3k3h X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1675772212-755676 X-HE-Meta: U2FsdGVkX1+qSklozBxz2m1F6sA/9gJf/xpp8UmQEW2eSSd1DeiHMoXy3Y83lJO3OKZ6GAPsBuKopL1LO+3DAwfJ9oH/6IiKtK2bCHL2RbWQnEFFUQ0rpTEPwkAjRGyJoezOajLaOmz17l+iHT4pyOVzSo9IGd4HAUZOAyOs0zyS/e3LKTT/lRppJ05ajrCiFaRc/W96FY3mxMhw1jMCvLL6ytFpbWDPedZS4fDlaso4i4mB8zOoHg8tPitbYWgKHIA6jqWwIfnE5y2aixQ8Z8vX1y0V9iIKoiZ7URE6PDedn81TX3WRXVLXaGgHwlZSyObUOCWIVRs7N/hYAQwJktgmv8Z8TE+BjOvz2o3qlpO1BbTQA8wEDBWNVuAXC8oN/+kwbqBSfLj9k4O5ctB+oOOgIRVT9gHMbFLdIQLmByUgnPycQU8Xj+oVUXshD4ANNurqAjQMEkljxlgYQccg6Ka4BeksrC+Lt7AIXcV6z+tQNHhwev1gvxqAFrV6Kx+VSLoEAAat2pjFwQ60cQCEMdHw31nb3JBYF8bwn5Y4BlpFv2WNjEu9Rl18etmagtFFjclOEV0Jcs2EC7zIDqW510Jj9L2zCgIvLQyDiCTS2qCK72i4Cm+t5FLNVzFlyMgv+6C/8WHb/3BfMGDoTWGpG47Zn1/2QlbN8WSN8G6qdJnWyudYaUh/glLM4eqvvB/JuSnuLK02eXDq7Xn6JL4JQh6/Xri7Oqql+yyCb1rn570oOB17J+bY9i1VCjQBn/UGVnmd9tXqdW1NnS/7xM2Qs6rA59T33ZZFemgF/1Ad8YVptjEA9yCwW6QxEpEKmmEpvPM5WE1xAPgPep1VNRY5xtJBq/YKbQIpunXlnmzXlhvz40mw8OUcn8xRWIJgCkfstlT/+OHbfPJQKoK3LsRgW4eX7ndnpSl3y4vOIxBwn+wdNORkhizY3EBj8pOkkOu6cshQ/El8CrPCNO3rJoU 6d5CXDPB zon3kJUIAHG6p2E3W4o2f1/5PBoY3aIP1a017BLlalrYI5jZykRQUlUMbFpQoI4wnkeqngFGXe+wooSdS37EUF0BYgxs1iZtMEcy2jDxn0KTmPN5oZr7e6k71bdMPEo0zis+eXB5QzGs9VG7rLHGnwA32eMeNqamcep3SoB+Sgz2uX4gAE3PjSnnPIwI9hLpCsi57lJao19Yayx+byoeRV4IbpR2YwVAtclkELIMa5m95ikxMqte2yHuJxX0mqpr8W6optdcJskKQ10GVjMPEaIW37fkJGnFih2Tq4CvAQqqugC8AyaEN7EdQUJ0Il7sVLspV+6MXlGQmLVic66fuSd8x80FxgmWH8ChQdoT3APXaKmF871P+m2+BDjoDdh87+/p7XBsuGXeaNc09lSxMpq69GTU6n64EyPZeVgPX4Zw4yZ9XDGQ8rQYeb2A/kOpi87SevbmUkKZfDoqbxzkbaAM4Wg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Feb 03, 2023 at 10:53:46PM -0500, Waiman Long wrote: > On 2/3/23 18:24, Frederic Weisbecker wrote: > > Provide this new API to check if a CPU has been isolated either through > > isolcpus= or nohz_full= kernel parameter. > > > > It aims at avoiding kernel load deemed to be safely spared on CPUs > > running sensitive workload that can't bear any disturbance, such as > > pcp cache draining. > > > > Suggested-by: Michal Hocko > > Signed-off-by: Frederic Weisbecker > > --- > > include/linux/sched/isolation.h | 6 ++++++ > > 1 file changed, 6 insertions(+) > > > > diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h > > index b645cc81fe01..088672f08469 100644 > > --- a/include/linux/sched/isolation.h > > +++ b/include/linux/sched/isolation.h > > @@ -53,4 +53,10 @@ static inline bool housekeeping_cpu(int cpu, enum hk_type type) > > return true; > > } > > +static inline bool cpu_is_isolated(int cpu) > > +{ > > + return !housekeeping_test_cpu(cpu, HK_TYPE_DOMAIN) || > > + !housekeeping_test_cpu(cpu, HK_TYPE_KERNEL_NOISE); > > +} > > + > > #endif /* _LINUX_SCHED_ISOLATION_H */ > > CPUs in an isolated cpuset partition is similar to HK_TYPE_DOMAIN CPUs as > load balancing is disabled. I can add an API to access the cpumask and add > to this API. However, that list is dynamic as it can be changed at run time. > Will that be a problem? Or should that be used separately? So that's what I intended first but the dynamic part of cpuset made me postpone that to better days. But yes ideally it should look like: static inline bool cpu_is_isolated(int cpu) { return !housekeeping_test_cpu(cpu, HK_TYPE_KERNEL_NOISE) || on_null_domain(cpu_rq(cpu)); } And there should be a hook in something like detach_destroy_domains() to flush the pcp cache when a CPU is attached to a NULL domain. All that with proper RCU synchronization: UPDATE READER ------ ------ rcu_assign_pointer(cpu_rq(cpu)->sd, NULL); rcu_read_lock(); synchronize_rcu(); if (!cpu_is_isolated(cpu)) stock = &per_cpu(memcg_stock, cpu); schedule_work_on(cpu, &stock->work); flush_work(&stock->work); rcu_read_unlock() Thanks.