From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FD4CEB64DC for ; Thu, 6 Jul 2023 11:40:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B11758D0002; Thu, 6 Jul 2023 07:40:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AC1968D0001; Thu, 6 Jul 2023 07:40:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 962578D0002; Thu, 6 Jul 2023 07:40:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 86C308D0001 for ; Thu, 6 Jul 2023 07:40:21 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 320311C8513 for ; Thu, 6 Jul 2023 11:40:21 +0000 (UTC) X-FDA: 80980993842.22.D9BA305 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf18.hostedemail.com (Postfix) with ESMTP id 4A2381C001D for ; Thu, 6 Jul 2023 11:40:19 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jpkdlCXW; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf18.hostedemail.com: domain of frederic@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=frederic@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688643619; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=b0+MHrEGpMQY63e2IdAsucrZt8WsgNhC9vU+EzRi/1U=; b=QXCMvf7cuj1fJb9sR1qo4spDgtJATdqeuwq0wK1gsI+3pkRyOGCEDncNJAsYtKHyvdvPxq n6uoxsEEdg9/5f7bGPVeS98BtFmfy14MHv23KGWyKmF45CoTeNVTt8pvzwmDrPuDrUC18l PmF1t/+zUNs2gNUFDKnATo8RIzLFF5U= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jpkdlCXW; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf18.hostedemail.com: domain of frederic@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=frederic@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688643619; a=rsa-sha256; cv=none; b=2dS9W4AhPbBZqkQHaoePpgRUoWuf/lEGYQWiOIGhA6v4PoCImMJIKu9iwhodTrSTAtk0JO YurNRmynsifQEJrT9xnttmSrjDzkzbAvxvmGsfCa/YsZzR/1YX5HDJvGZgpidWealaICfc 1wuyACoBf6dxuy8kJlNhJnEgeIstrVk= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0FBC06190B; Thu, 6 Jul 2023 11:40:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 50F67C433C7; Thu, 6 Jul 2023 11:40:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1688643617; bh=h08GLHzvzPag2+5puOztYp01UnHlh2rRD22/d6waft8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=jpkdlCXWe7GWqMfmwmsb2ud2sF5O4+fD2WXQSRpi2ETowI8YWAiKUY2wLjsaVNJpX +RxS2fLk+ORVul2Nyn0W4KqpCYtF0uwRkzgfC1Th8q/05Gki9oc5hbhpOztdJmf1tc dK/szrpOsSLV2AWvcUKZ5PIBfWIXCeUiDu8SVmx6qB7nXGA9312BveqiC/GGmrBff3 /YM1im7HcTF7G+laaX0i/fws9kOhYMTb9Wt4fSNCWYw7hFJE8UNr07UVKdCiPRmjj8 VL2y/hVlHIW1hO3bgyapVeY+dyvbFX4rWbKMljPZv2Y3v5ZMX+Y8ojxKKyrKJRxf+1 mwP544/PX36qw== Date: Thu, 6 Jul 2023 13:40:14 +0200 From: Frederic Weisbecker To: Valentin Schneider Cc: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, Nicolas Saenz Julienne , Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , "Paul E. McKenney" , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , Thomas =?iso-8859-1?Q?Wei=DFschuh?= , Juri Lelli , Daniel Bristot de Oliveira , Marcelo Tosatti , Yair Podemsky Subject: Re: [RFC PATCH 11/14] context-tracking: Introduce work deferral infrastructure Message-ID: References: <20230705181256.3539027-1-vschneid@redhat.com> <20230705181256.3539027-12-vschneid@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4A2381C001D X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ssfmwify4bymkcqgrqbw16ybnanqbue6 X-HE-Tag: 1688643619-656939 X-HE-Meta: U2FsdGVkX19fz/qU3mhpGwrGWBL1DQ7vr6GGbKULUs1gqpoXoU90mPzshrC0lGb11w65rHYug4QonFC3XoXndXpCQVBYPoMVttJnHOX7cVjbxjSRiWS4C5dCixOFURz1Frq69QrU2oFCt3hPZfWBr62soz7ALU7/1d0gpEUpCZQLBpYfjRyny7rzFZ/TIB6h4fcxSJlm/kb8ZuY5xgWgBVWZMJ3R+sgCkeOdyN/QzHqHDLbx/vNCyzduUSjJJ2kIAVYcKghKX0GjhrDIavZFsREGZWGJ3xVhMooh5Leq2Rob5RyEPZfa83KlnuA9/6OB34lG2RC4P0MIn1o2nge/ClUZ/HxfEQSehWqXvgfbJYx/8EXWkLGsnx/EdcuKfIPXJm5zuuVNqINKmXUcXl+L0Cn/KqDpIDnwulxMhrdOyTUcvxsk5EYm7nSbUgJal9qvNXdNiEbX6PT4WLELxaq1vaTK1J0JA3KLD/48d2ylyaczW+Sc54mykwE5IQWCSFKdDshl0MSKKJIrg8poh/oSqpgijHS0pj4fXWNejNEV0RaUwmykrP5FTcYiFQvm8JKCbE/URklXd05UHAkNDcG5gj1kZ1WUkns7lE9vbtWek4AHEGhjVeSKxWu6uybG3+neR/hIVyz9sGRG+p8Tgu177k6tZTyeyUBb9RU8oJzoyz6faE/+7vWCNpknq2n1s0VCx+F+cYi1KMFZPZvligHzI63huOZLdQqTTd3LLeTjAM86wmRE5PyPYx/MovxXEI3vYcn+CHMSFylp7vp8Dn7iNb5YGW3cE34bDldNhmw+f1yavVmBY9wKtXD8oTf175F0rA975HPa2DEpqy/9qYR3q7mtPSZlFs2kf4y18NGTBWhhWIylf/4xbpYCFZHWnjeo1P+O0DaADx+0kgFc2lBKVydPzFZaMm8/n21GhAobi6vEM6ErbBk6+Coy+guP6Yw6RF77nbpip29X/pjQtJN UGwA+B6I S2T6j7vfT1g6Dhlpy+TFt9Kka/cw5UoDwf2w4i08Hl5DAH6xv9zKo0+eja6uOBIGG0B+dWRJIsk+8/m20g+kwAC3m6NGaOpjcMjw0Qv1iokfo+fVjc+VM7WbMhQ1oPDpaYyah69zcFX1QCU2D5sUi+ayZc0eO5vnO2+i4oqexCIfG5ZQ7c6p3Kd2mhYWPbaY1Oa/msvcsLq/sjq0c9G7so5xx+OJBj6CkvNNLn6smV1q1zyol1dLKs9ApvqnAqlRMJhR9tLis2zhcpjhfNwzsC4YLgjgDWV3S2jwX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Jul 06, 2023 at 12:30:46PM +0100, Valentin Schneider wrote: > >> + ret = atomic_try_cmpxchg(&ct->work, &old_work, old_work | work); > >> + > >> + preempt_enable(); > >> + return ret; > >> +} > > [...] > >> @@ -100,14 +158,19 @@ static noinstr void ct_kernel_exit_state(int offset) > >> */ > >> static noinstr void ct_kernel_enter_state(int offset) > >> { > >> + struct context_tracking *ct = this_cpu_ptr(&context_tracking); > >> int seq; > >> + unsigned int work; > >> > >> + work = ct_work_fetch(ct); > > > > So this adds another fully ordered operation on user <-> kernel transition. > > How many such IPIs can we expect? > > > > Despite having spent quite a lot of time on that question, I think I still > only have a hunch. > > Poking around RHEL systems, I'd say 99% of the problematic IPIs are > instruction patching and TLB flushes. > > Staring at the code, there's quite a lot of smp_calls for which it's hard > to say whether the target CPUs can actually be isolated or not (e.g. the > CPU comes from a cpumask shoved in a struct that was built using data from > another struct of uncertain origins), but then again some of them don't > need to hook into context_tracking. > > Long story short: I /think/ we can consider that number to be fairly small, > but there could be more lurking in the shadows. I guess it will still be time to reconsider the design if we ever reach such size. > > > If this is just about a dozen, can we stuff them in the state like in the > > following? We can potentially add more of them especially on 64 bits we could > > afford 30 different works, this is just shrinking the RCU extended quiescent > > state counter space. Worst case that can happen is that RCU misses 65535 > > idle/user <-> kernel transitions and delays a grace period... > > > > I'm trying to grok how this impacts RCU, IIUC most of RCU mostly cares about the > even/odd-ness of the thing, and rcu_gp_fqs() cares about the actual value > but only to check if it has changed over time (rcu_dynticks_in_eqs_since() > only does a !=). > > I'm rephrasing here to make sure I get it - is it then that the worst case > here is 2^(dynticks_counter_size) transitions happen between saving the > dynticks snapshot and checking it again, so RCU waits some more? That's my understanding as well but I have to defer on Paul to make sure I'm not overlooking something. Thanks.