From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6C8ACA0EC3 for ; Tue, 12 Sep 2023 07:38:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0DEFC6B009F; Tue, 12 Sep 2023 03:38:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 08F2C6B00A1; Tue, 12 Sep 2023 03:38:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E98B76B00A8; Tue, 12 Sep 2023 03:38:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D8EA56B009F for ; Tue, 12 Sep 2023 03:38:53 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B47D4A0630 for ; Tue, 12 Sep 2023 07:38:53 +0000 (UTC) X-FDA: 81227143746.27.114C2DB Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by imf28.hostedemail.com (Postfix) with ESMTP id D009CC0017 for ; Tue, 12 Sep 2023 07:38:51 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=XEKl48I4; spf=pass (imf28.hostedemail.com: domain of mingo.kernel.org@gmail.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=mingo.kernel.org@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694504331; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+iqrvcXY/4bAbdcHH6fGo/1fRQ+e4TYFsEq51IoZPeM=; b=ZOjfB7B362K6T0MfIsAGCxChL8w5aIJ4HLxaYlB8XmE64TaV7d/sUD0u9rHrQ+hO0d7opu +PljGkiAQcHIdW2AGI8wiSMBrm/txJZRIm8IwnzIWRstHmx7VnQj3xJxG8jLpVLyiNsTiX zmh4+XHbQHhzaQI1F2QxsOX6Rm0UjpY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694504331; a=rsa-sha256; cv=none; b=yk6H93paXRDHSDYly2AafNlK9409LnpapwN57lneChAJ0ZoBLyPQGYrVaWAwnOTWOiH1vH Ss6ABuKqruDWMh+MmYTfcL0z2UbzYwHBISceeiHcDWI90SlLeL1tSbs1pl/ox8e52Ulte0 CnKTa7gBvA/fhRComMuye5eiTJmQtf8= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=XEKl48I4; spf=pass (imf28.hostedemail.com: domain of mingo.kernel.org@gmail.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=mingo.kernel.org@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-307d58b3efbso4812463f8f.0 for ; Tue, 12 Sep 2023 00:38:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694504330; x=1695109130; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:from:to:cc:subject:date:message-id :reply-to; bh=+iqrvcXY/4bAbdcHH6fGo/1fRQ+e4TYFsEq51IoZPeM=; b=XEKl48I4mVHqk22BCnb8Oq403UzNqllqOmayKYeVP+ohonz8DlgS8M+fSs1aiPm7XV 1pHcZs9gD4KfkN7Eul4CIzudZTowOoLBuZohSHrg82f7pAtk6H/uZB++KXNDu2GPLykL OoIDuEpM0zwzRIEdCzHZA0+ZKEWThIjCKXx92D1r+xh/18jQBp3I1fwfdGHKK5asiWkc U70IXeqirJGKoxmy3lfNFOsbSB70t5QSwFfjG3I4rSfVuXuOJzUjJ6VG9F1xrqjnyq4L sPhXegO6sszcUII+0ETR7h/mUqenYG5DWPlWbLpbPwxLGvwdbfUr4mGZQRKmsuD3QIKw s4hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694504330; x=1695109130; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+iqrvcXY/4bAbdcHH6fGo/1fRQ+e4TYFsEq51IoZPeM=; b=Pqnae4Dldi4I8532sXRbQadPCxcilw7LRn34I5uGXgd3hlXQTMfMdBx+R5FntRU7ym k93fYNPVWIaQiyb2TEPVNe7pEP9lmNCzpyoOmzQ6SMz60CACzc0lp241R7O4fAOyjaHM EWKGHrt7spAy0KSDpH0HcKudLRzCcI6OMXx4nSNoWEULBmiivf/Ibq+MeRHNDAd1Z3u7 NIIbKbujQLG+/c2ESb155o6d8gE8PQ338DmIuIpxPbjbhk8wmnFLmd6lRadiBc9s5hXy oKYTV3IbKMGTRfrEzHx2KDB8VP9utje1zBDQptU66a0fjwfEk/eex2YoBG7nDQfoprFt PpKA== X-Gm-Message-State: AOJu0YwnGEesd8+JVhJ5tyEZbzdypu/aPUPv/w8XJLbsExyLN6hZTMNB 3kkzidb/0ZI9FOvKEYL6sfvLPHCB6KE= X-Google-Smtp-Source: AGHT+IHt+8osXiQG7XmgmjaxhjKlPhc7MjegoQlIytXX/ed0M1Al0SFk5YGedvsmE/r/IJWs5M2LSw== X-Received: by 2002:a5d:4b4c:0:b0:317:3deb:a899 with SMTP id w12-20020a5d4b4c000000b003173deba899mr9235784wrs.1.1694504329990; Tue, 12 Sep 2023 00:38:49 -0700 (PDT) Received: from gmail.com (1F2EF048.nat.pool.telekom.hu. [31.46.240.72]) by smtp.gmail.com with ESMTPSA id j12-20020adfe50c000000b003176aa612b1sm12058797wrm.38.2023.09.12.00.38.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:38:49 -0700 (PDT) Date: Tue, 12 Sep 2023 09:38:46 +0200 From: Ingo Molnar To: Peter Zijlstra Cc: Linus Torvalds , Steven Rostedt , Ankur Arora , linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, tglx@linutronix.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com Subject: Re: [PATCH v2 7/9] sched: define TIF_ALLOW_RESCHED Message-ID: References: <20230830184958.2333078-8-ankur.a.arora@oracle.com> <20230908070258.GA19320@noisy.programming.kicks-ass.net> <87zg1v3xxh.fsf@oracle.com> <87edj64rj1.fsf@oracle.com> <20230911124856.453fba22@gandalf.local.home> <20230912072022.GA35261@noisy.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230912072022.GA35261@noisy.programming.kicks-ass.net> X-Rspamd-Queue-Id: D009CC0017 X-Rspam-User: X-Stat-Signature: fgusubik6d8km6ybme9cqph3mk16k9ds X-Rspamd-Server: rspam03 X-HE-Tag: 1694504331-887005 X-HE-Meta: U2FsdGVkX1/6tSbT+rEyKizby5M6rBpAGLo0prMZgLZvt+8YLwYh06T0MkgcmnRYUBJ1p7lWUPBNdhwt9iHBWYaVT0c3ZYDoKyZl0HOY4AzxVECeM6H2M+f1ANwE7K1T8dTOr1vwd0Ov1gr1VQI8PIaOjitezOmh7px42zQIIXBrwb7de1mMNiLEfJeLMTd+wc8TyKrf3xzAULy1PGbQyvFTZcnLvRVmkv448nka0yymx9Ks3OWfLDG0oHoIHHvd2S4UnQH0SKI61Kj/oFvVv1CAMNqfpTbAlsdOyAHcaQPkC4CeaVpZdnTGd4uItPUOuvu5yr3syYDNpGwmegwFJln2rY2i3oxkyVYdjj2sfOgYH91YKWGP1dTJ2C4Ss//z6hIYo418eVZzrsXVkKPFXC1nd70OD7vwPpyugBxauxi+5dYo36F01kBk10bQpYw0+2u8DpBkErPd7g77ux0QqG6hynfPsyAREJh7u8duvHaNuw5HAGHAKzOn/SyJoIOsp1q1GK88kfWOd+ziI38W+mlvfmnO82lKMqgweWXK7aGFvRPUz6ZGAyDdxPMzKLxWloXlBzepG9C14m5esGkiS8efnyZT+dIkylqVYrxHoEK1T+yTNEOuScukhRipJJZGH309JeTksG+z8rj+cFZ2wDheY4dJFKY+Ct/GDDBRRc0FauvKV9A/XAtxWgvgLLHh2dg8tvbHL2VgpRBQ3/XSofglCZgO5AzJQdeqRuzlv0s8xwygbXsL6XHSLZNTPixRlegx6+SIniYUarAvPMFv5fKW6Kug+ruTWFKQ5cjgXG3bwehgM4JH0JeLfx4n2R5GKJsZEe4sgM1aHEWTw0WonuwVHVlqrYsxmiPAMEk6jtqq2YctSiChEAUK4XlGjGNIbd2LzMtg2c2TmFYzHsK84UdsXvokj5ZQR7Be3eNzXUlaIZOJDB1ckMR5BMQl9t/4STH+zGdl8XdS+E7a4Wc g9Lq94Gl G1kARy+7mDmgY2iIJgcXoe8CdqswIQSMiZiZ/TlYXKiR9iZv8HVPiaYKWMlH+sy/KOIBoSr20cRZLFGZyMT0GMJBb4/8Mt8HhEsJgdQE9jtphV3Y2wwitwvDCemL9YaurWlD2q4IdO5cPBRdgoRB+s5uA5iWPTMIUqBZG0BbQ63sL//33afTOjrzzpHSo2D662aBXgkCbwXIh0xc7ln79rsuuI0jtP7tv9NjcjlNr5ubyuaKDwOCoXYqmBhNGu8hBuVprGPJigCV+gP5Rd4FONmYnE7eX+ugVknbU0kXCMFtkYClpX45vYFAuy96/SoGsgAs/Kj42FS/NABr+d/vMxQeDS6VrucUuzK2uyOyX9wEpmPTVQTDQ3HdoHm5DAHsbfP3Kypz/7SORmu3ANqqPMerpKy0ZgI8PtKW8ROESkyXLXDaGrtUE0Xwp+HYHWEAncep01ieOW0elN55rx7sK2MhDPXJW+BBUz4LCTP0WWaviHqzOQ5Y2Hhsa7HERqZcYKeBLzioxer98RE8WpKnhfsXgR0K0xEvqQuaOFAM4M/U8Fm8VV2dpy62viHmrKSWhtBnLoxhky2KpW37yDNlJkP38txvRmtytzkKnyAg/fqt8m30MtbqBhfjZITC7bl6tgThgQ3rTHRwqnh6DAA1D7W3uM5ByXW9xePOQb7ixwMvJ0qNfHRJA2D6/7w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: * Peter Zijlstra wrote: > On Mon, Sep 11, 2023 at 02:16:18PM -0700, Linus Torvalds wrote: > > On Mon, 11 Sept 2023 at 13:50, Linus Torvalds > > wrote: > > > > > > Except we've actually been *adding* to this whole mess, rather than > > > removing it. So we have actively *expanded* on that preemption choice > > > with PREEMPT_DYNAMIC. > > > > Actually, that config option makes no sense. > > > > It makes the sched_cond() behavior conditional with a static call. > > > > But all the *real* overhead is still there and unconditional (ie all > > the preempt count updates and the "did it go down to zero and we need > > to check" code). > > > > That just seems stupid. It seems to have all the overhead of a > > preemptible kernel, just not doing the preemption. > > > > So I must be mis-reading this, or just missing something important. > > > > The real cost seems to be > > > > PREEMPT_BUILD -> PREEMPTION -> PREEMPT_COUNT > > > > and PREEMPT vs PREEMPT_DYNAMIC makes no difference to that, since both > > will end up with that, and thus both cases will have all the spinlock > > preempt count stuff. > > > > There must be some non-preempt_count cost that people worry about. > > > > Or maybe I'm just mis-reading the Kconfig stuff entirely. That's > > possible, because this seems *so* pointless to me. > > > > Somebody please hit me with a clue-bat to the noggin. > > Well, I was about to reply to your previous email explaining this, but > this one time I did read more email.. > > Yes, PREEMPT_DYNAMIC has all the preempt count twiddling and only nops > out the schedule()/cond_resched() calls where appropriate. > > This work was done by a distro (SuSE) and if they're willing to ship this > I'm thinking the overheads are acceptable to them. > > For a significant number of workloads the real overhead is the extra > preepmtions themselves more than the counting -- but yes, the counting is > measurable, but probably in the noise compared to other some of the other > horrible things we have done the past years. > > Anyway, if distros are fine shipping with PREEMPT_DYNAMIC, then yes, > deleting the other options are definitely an option. Yes, so my understanding is that distros generally worry more about macro-overhead, for example material changes to a random subset of key benchmarks that specific enterprise customers care about, and distros are not nearly as sensitive about micro-overhead that preempt_count() maintenance causes. PREEMPT_DYNAMIC is basically a reflection of that: the desire to have only a single kernel image, but a boot-time toggle to differentiate between desktop and server loads and have CONFIG_PREEMPT (desktop) but also PREEMPT_VOLUNTARY behavior (server). There's also the view that PREEMPT kernels are a bit more QA-friendly, because atomic code sequences are much better defined & enforced via kernel warnings. Without preempt_count we only have irqs-off warnings, that are only a small fraction of all critical sections in the kernel. Ideally we'd be able to patch out most of the preempt_count maintenance overhead too - OTOH these days it's little more than noise on most CPUs, considering the kind of horrible security-workaround overhead we have on almost all x86 CPU types ... :-/ Thanks, Ingo