From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EB4BC74A5B for ; Wed, 29 Mar 2023 14:31:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8246E28000A; Wed, 29 Mar 2023 10:31:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7D4A1280008; Wed, 29 Mar 2023 10:31:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 69BB828000A; Wed, 29 Mar 2023 10:31:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 57C92280008 for ; Wed, 29 Mar 2023 10:31:52 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 176D6C0C32 for ; Wed, 29 Mar 2023 14:31:52 +0000 (UTC) X-FDA: 80622174864.23.8AE333D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf12.hostedemail.com (Postfix) with ESMTP id CCA4F40028 for ; Wed, 29 Mar 2023 14:31:49 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=HmL3AhSI; spf=pass (imf12.hostedemail.com: domain of mtosatti@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=mtosatti@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680100309; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=k8U3rACF49U6AOCOedOjgERqK5/T1G/mMgCJwPeJvHA=; b=53x71KfbEALbxzRpsjJCZBkRstq3CDWQhaqEaw8HRFZcpSCr1cFlbO7HdM37rrlEUFrHf3 0zQdy3xKemNV4na8QfKXAe7JPFmmeu/oyJbQmjT0Bn1VKosv/f+R2tOaywcGi2AwfKz6AF uVQoK+hH54ItwT2kYFDCtdC2KxODNyI= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=HmL3AhSI; spf=pass (imf12.hostedemail.com: domain of mtosatti@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=mtosatti@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680100309; a=rsa-sha256; cv=none; b=eEOxeX5OzRVUW1Rw3t64ss6Da6YQZe11EjNJA4vNMs0PU/u+mdbC2MRa1VQR08/A9ZsxDX 5ZjkfIj4uGLxKNK5x0jI8D5hp1j0gPlplX5HkgaaYj1d+YYbJ9W8QhIX/NCT0rJdXunX6M FGlSyuSA7d++4FV/srqQRJ2spWbQK+E= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680100309; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k8U3rACF49U6AOCOedOjgERqK5/T1G/mMgCJwPeJvHA=; b=HmL3AhSIdKVLqSepJSBbsHt5kXukaR1+45p2za/3Rk/MCD7CgPx7cv2XHTcJyQ926bQC00 a7vxSNg3OSr0AHCJEJfDyQ+4u7XCsgagI34fN/MQaN1ys6K0wUuLtr4UchmxMUvFIGEqLx 9oNYnPctrmHt2wC5hQoGp1eOOapIKJQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-215-tqUh_sqXPgCPuULRl0SPWg-1; Wed, 29 Mar 2023 10:31:43 -0400 X-MC-Unique: tqUh_sqXPgCPuULRl0SPWg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B40C5185A791; Wed, 29 Mar 2023 14:31:42 +0000 (UTC) Received: from tpad.localdomain (ovpn-112-2.gru2.redhat.com [10.97.112.2]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7024D404DC5C; Wed, 29 Mar 2023 14:31:42 +0000 (UTC) Received: by tpad.localdomain (Postfix, from userid 1000) id A5DD04015571D; Wed, 29 Mar 2023 11:20:21 -0300 (-03) Date: Wed, 29 Mar 2023 11:20:21 -0300 From: Marcelo Tosatti To: Michal Hocko Cc: Frederic Weisbecker , Frederic Weisbecker , Andrew Morton , Leonardo Bras , Peter Zijlstra , Thomas Gleixner , Johannes Weiner , Roman Gushchin , Shakeel Butt , Muchun Song , LKML , linux-mm@kvack.org Subject: Re: [PATCH 1/2] sched/isolation: Add cpu_is_isolated() API Message-ID: References: <20230317134448.11082-1-mhocko@kernel.org> <20230317134448.11082-2-mhocko@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: 4edmmmd7nusn8tkwowiri4zb8yx5yfaz X-Rspamd-Queue-Id: CCA4F40028 X-HE-Tag: 1680100309-578148 X-HE-Meta: U2FsdGVkX182MQWe/uGV3akIMb5iU1jOOOQmXV60s9aLsuFu+FCaFXhyZW76ZhXdn75iF0Oau0MX1QwnSFwV6kAZTC/HpwRZbexKhq8DPYMgrAsuPcJ785k9xAhrXsMikHoQz/26FJu69qck1WLjgefxF0BQqIkLWbf2PoKEDYeofQDJYrkTk5nRWx70jH7fYEfwhA9Ho4bf7M90R9uO2QXwqc9RV1wFRZ9r6iM97rEyFW2nvJPmPtaWolmUQfCt5eg1VlKLDmA77WS7u4fE0kGvQMhWi7vX7n2rXYDBwRhQ5QQJmWN3Mc5YpJgZ3kSqmt8jl18Aai/tiu1FmPdtjFwiVsa05EE5y49UCk4uLCcAvf9mrEJL0KHoTiXFyLY8O81PxbrOe9GrGRn1qZqr5mympOgTR0BdZ4eGLsgmI4dNGqyav0Gtz+AtEEZrSxTlOaHCAgYg4YrBXLhNXTE4sT86sRrJs9Oemv4dxFVkDZEMiiZt9pz5wijRRTe0oGayPD6n6YUhFc5eNbzAl5D6J/8HL+awDlLI02Z8P+xKmrwhuaKPaSWU3Ingg154Qsg3um4RjQuv4BA8aPZalwaKfT3EFsnL7Wy0BH4LYLisRSjskr6OHryjQlA7DK4C4of1Y8iLErtVGAn/t8gEwaym5FkSnGSp2geOOAHVNrB/bgzmzPinPw8FgEvpLbb7zjNJXEnPdwBugn/WJoTrvLIuSUgicmTlUCYyArvimnSm2Xsxo3c/swmbwcDm7Gu6/DYtd4jejsj7KAbZ77NA1y3le6aIAt4pycrw92SXXlP0fC1SIQ/mGjO1dk5l6E226UT+uT8y+FkflvULSozbozJAKAKUzQi2Kpg2h7BuyK0O+oCr+bMDTRsmkE2evJFxe4VNlfvpXHZPEj3lB8F20lbtbaYj3nzZwOCgnfaD8Yls60PIcXfI4fJQjZXFQ/sHaL+Dmg6VywAfBcquTDNAXyU ZvWdHiEQ Q4+PlEFFiTCnFTXYaYztFdTnANZ4/hkz0A6oKYYTJ/YSx3hlRTW1JvUWV34/VgCWTejvO3j3tpSYRFBiOutaIVB6euFY8L4GqLOO8cZ1hcyL4ERSRC37mfj8hiOrC5Z7wWx3psge3iKOfTIEVTASYbr8Sq1od3ZDnBtavqpBR1EIg4o+pi9X0yYAny/+Phozgv4syW89Y+wdSqCcRpTqUKy8qf8P7GCmNdF6CQtMpZZ/stMn66IwIFNYPiPAXHACBxJxj1z+AafppbY157yIS/rU2UcTkoWb/ZINoiYO4fP6j4+mCX2tcQ/IJ9g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Mar 28, 2023 at 01:48:02PM +0200, Michal Hocko wrote: > On Mon 27-03-23 07:24:54, Marcelo Tosatti wrote: > > On Fri, Mar 24, 2023 at 11:35:35PM +0100, Frederic Weisbecker wrote: > > > Le Sat, Mar 18, 2023 at 09:04:38AM +0100, Michal Hocko a écrit : > > > > On Fri 17-03-23 15:35:05, Marcelo Tosatti wrote: > [...] > > > > > Actually introducing cpu_is_isolated() seems fine, but it can call > > > > > housekeeping_test_cpu(cpu, HK_TYPE_TICK) AFAICS. > > > > > > > > This is not really my area. Frederic, could you have a look please? > > > > > > The point is to have a function that tells if either nohz_full= or > > > isolcpus=[domain] has been passed for the given CPU. > > > > > > Because I assumed that both would be interested in avoiding that flush > > > noise, wouldn't it be the case? > > > > Yes, that is the case. But as a note: for the two main types of > > configuration performed (one uses isolcpus=[domain] and the other > > cgroups, for isolating processes) nohz_full= is always set. > > > > So just testing for nohz_full= would be sufficient (which perhaps would > > make the code simpler). > > I do not see any mention about that assumption under Documentation/. Documentation/admin-guide/kernel-per-CPU-kthreads.rst SCHED_SOFTIRQ ------------- Do all of the following: 1. Avoid sending scheduler IPIs to the CPU to be de-jittered, for example, ensure that at most one runnable kthread is present on that CPU. If a thread that expects to run on the de-jittered CPU awakens, the scheduler will send an IPI that can result in a subsequent SCHED_SOFTIRQ. 2. CONFIG_NO_HZ_FULL=y and ensure that the CPU to be de-jittered is marked as an adaptive-ticks CPU using the "nohz_full=" boot parameter. This reduces the number of scheduler-clock interrupts that the de-jittered CPU receives, minimizing its chances of being selected to do the load balancing work that runs in SCHED_SOFTIRQ context. > Is this a best practice documented anywhere or it just happens to be > the case with workloads you deal with? Option 2. However Frederic seems interested in matching the exported toggles with the known use-cases classes. For example, for this guide: http://www.comfilewiki.co.kr/en/doku.php?id=comfilepi:improving_real-time_performance:index Using nohz_full= would be a benefit (and its not being currently set, perhaps due to not knowing all the options?). http://www.comfilewiki.co.kr/en/doku.php?id=comfilepi:improving_real-time_performance:index AFAIU the workloads for which disabling nohz_full= is a benefit are those where the switching between nohz full mode and sched tick enabled mode and vice-versa (which involve programming the local timer) happens often and is therefore avoidable? For example switching between 1 runnable task and more than 1 runnable task (and vice versa).