From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E969C4332F for ; Thu, 9 Nov 2023 23:47:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 90C1D280017; Thu, 9 Nov 2023 18:47:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8BC0E280016; Thu, 9 Nov 2023 18:47:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 783E5280017; Thu, 9 Nov 2023 18:47:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 66C50280016 for ; Thu, 9 Nov 2023 18:47:44 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 3A7B2C0C4F for ; Thu, 9 Nov 2023 23:47:44 +0000 (UTC) X-FDA: 81440055648.21.E64A766 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf26.hostedemail.com (Postfix) with ESMTP id 5D3DF14000E for ; Thu, 9 Nov 2023 23:47:42 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=l+A9XrnO; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf26.hostedemail.com: domain of jpoimboe@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=jpoimboe@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699573662; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bLVKVcexzcm3WvuxP86nvcqHDQY8v5kKvoNWpbyti98=; b=DKYa6lIUEFICHR3M0/FXs70S+6lm4Haozg9xb19dKI0nBwQtWZ9mYlQRgNjpaqy5Jr4eWH GOvzuWd8chxyr04xYQYIMA0VEVALdpN7G21aj1Ik7BdFlt+K9mr2Fik/Mfvq9sbj4KlW7x Pd0BD2CE1mowbQHYdZ6C0D+sXNpBmBY= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=l+A9XrnO; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf26.hostedemail.com: domain of jpoimboe@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=jpoimboe@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699573662; a=rsa-sha256; cv=none; b=0oJ+rVVHzKxiAfsNYM7HRHRgyJo0AxgvP+GNKFdrYAaq75/iAcLFwJrRCrz+su4G6lH+dx KJE9EE+vpnoLwCpBiY3GnvKXrLIWhGXHYLdQl3HgzRLVGbdh53HL8gkFGQ11UBuKciWaH+ Ed4S0oSdoHgm2PaELPheD/pSWUS/yqY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id 3C6CCB82205; Thu, 9 Nov 2023 23:47:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D7AB5C433C7; Thu, 9 Nov 2023 23:47:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1699573659; bh=Zkq/7iZ3H80Jnb4PLSdJyK3YLpUxFO5+W/MaihT41SQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=l+A9XrnOk6oY72C2nTFJMUmZJRnnID0D70pM5Z5vYztBmNoq0v6Ll/AqmLukM0jrl VuVhTeIC7r2OCuTeCvMA6Br1s3A1E0B32L92RdnTOQoOv74OZbA00oFHpaZjXtNdIY QoRvBAdICqqupiVYwsCpU70B6pgg1qrY++l0aXl+yYhSvw9jMWnZILIZb+OFx1thxA UMOTUwAH93hCZvZUS5jtAtJg7DYUaGjgi9oPFJewcgyDPD1ry+e+QWu/1z4xi+4jlC POYaBbAPgrTV/h6P9YgGbPghCI+t+RIheW1G1NXGDv1PWdBITeq2BOB/R8NryYYKf5 k1xv/uSlnX++Q== Date: Thu, 9 Nov 2023 15:47:36 -0800 From: Josh Poimboeuf To: Ankur Arora Cc: Steven Rostedt , linux-kernel@vger.kernel.org, tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, paulmck@kernel.org, linux-mm@kvack.org, x86@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, jgross@suse.com, andrew.cooper3@citrix.com, mingo@kernel.org, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, David.Laight@ACULAB.COM, richard@nod.at, mjguzik@gmail.com, Jiri Kosina , Miroslav Benes , Petr Mladek , Joe Lawrence , live-patching@vger.kernel.org Subject: Re: [RFC PATCH 07/86] Revert "livepatch,sched: Add livepatch task switching to cond_resched()" Message-ID: <20231109234736.4kik62ys47ey23ju@treble> References: <20231107215742.363031-1-ankur.a.arora@oracle.com> <20231107215742.363031-8-ankur.a.arora@oracle.com> <20231107181609.7e9e9dcc@gandalf.local.home> <20231109172637.ayue3jexgdxd53tu@treble> <20231109123147.2bb11809@gandalf.local.home> <20231109175118.olggitpaltz47n3b@treble> <87o7g2bkxj.fsf@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <87o7g2bkxj.fsf@oracle.com> X-Rspam-User: X-Stat-Signature: 9i5abmggg5816dccgp5nmdn1mrk9twbe X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5D3DF14000E X-HE-Tag: 1699573662-355546 X-HE-Meta: U2FsdGVkX18cRLUekH2Aaw9JvNOV3PB/jBZ+jRsc/rxjYdWu0bswsNf233ja7CWMoYh2m91rzHaqMqeFTSC40M072ccbpR+Z+d9Cwo+WAGnnmZHecAW8ex3kQesDc/8CYhuJp16e0taDURo6BIYWogeiI/aBX5yLo30Rj4WU+kPaJOAbTVaiwtFnsKVCZ0/tqKCdldAgsMQZTTAUxGrVLhMng61o1D97g64a+BHoxUU0TlLHACzSeXW9dbR8RegeVXYChAZ/iqQQRxBM0tNX80KOIvpIVJhoRVfVCgw+i9VbhJ5cfoJ5pQjFVYSWoJa6Z2kxPMDxNR8aH9JTOh4nkc6UZqczTszkzYluWLsc4QAZvlZ610olPmy5cCwI0hjO3V1m6LA8qZqwrWSwf5wixGNPsXaF1tCivS7SWstKh4qLi9U5h0UE/jO0eaU+YkdMpfDYWVapQ9Iw+Pv0yEUCrVTO/fP6A9xtVPG9odqbXsaZlz8SOlhwJMrcfAL75c15+encvusNncF84tIwVqZXdabjefDmqibK5nEunbTt+jeV0fCAtkui31Nfn6D3+OnUH0ga6AkjCi8AFsptgbIEvkwNZJZw+Ewjyhi2ZjUZXezQsMY4NOQhzc080aLx9aWLBoVnDOoZ/vKRY0KlMwzoHzoaUdze9Gyk6jRZrSYsFGljHwFd7HUp/9BZHXyXUenrH155NMTUFYaDxff7AuhHWmKlaxaaS987AASlarXIJcsYcNgA45KuvdQjeKoM9qS8STYC+6DHNJCTEGtyyRe4vvO0hKDlRz0JB0sGDHmyj75u+hnxFbr1nDtWls/Zw/LdmG55Jf5/kkiIUqWr+h/7v+Jn8CMdT8l/F7DppZDct1Hd+ttOhTKiHvYR6tQ8TzHN516K5viP5UPe6UHD5dAhq/+tz4a47jW1D90KReOdLsEimOFVF7Y9wx78+UgI9bkz4w0B5Vi0dpMVSUZCAcg du4YhBam 4/maKw70o879OjPYUM6muR0/MXaTpcCSzy+SONtY7igZChpnlsLS+kf1M7dkaQgWt0M/WoRvloQpibMHXpF2Z3lQkTIlIlfrz1m1z7arsBob7GZ3L8mYFAyM1ZXo8ZHamyfudQS5oCOMNi4OWIFApL+BgdxOOfJLsoa0ixL1/vyBExTbAi4oPXbxhv8Aj/9GghRyU7DCzLOvuC8N/Rw1VYgl1XxlybkCQHkeLpEnOJz1V3WY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Nov 09, 2023 at 02:50:48PM -0800, Ankur Arora wrote: > >> I guess I'm not fully understanding what the cond rescheds are for. But > >> would an IPI to all CPUs setting NEED_RESCHED, fix it? > > Yeah. We could just temporarily toggle to full preemption, when > NEED_RESCHED_LAZY is always upgraded to NEED_RESCHED which will > then send IPIs. > > > If all livepatch arches had the ORC unwinder, yes. > > > > The problem is that frame pointer (and similar) unwinders can't reliably > > unwind past an interrupt frame. > > Ah, I wonder if we could just disable the preempt_schedule_irq() path > temporarily? Hooking into schedule() alongside something like this: > > @@ -379,7 +379,7 @@ noinstr irqentry_state_t irqentry_enter(struct pt_regs *regs) > > void irqentry_exit_cond_resched(void) > { > - if (!preempt_count()) { > + if (klp_cond_resched_disable() && !preempt_count()) { > > The problem would be tasks that don't go through any preemptible > sections. Let me back up a bit and explain what klp is trying to do. When a livepatch is applied, klp needs to unwind all the tasks, preferably within a reasonable amount of time. We can't unwind task A from task B while task A is running, since task A could be changing the stack during the unwind. So task A needs to be blocked or asleep. The only exception to that is if the unwind happens in the context of task A itself. The problem we were seeing was CPU-bound kthreads (e.g., vhost_worker) not getting patched within a reasonable amount of time. We fixed it by hooking the klp unwind into cond_resched() so it can unwind from the task itself. It only worked because we had a non-preempted hook (because non-ORC unwinders can't unwind reliably through preemption) which called klp to unwind from the context of the task. Without something to hook into, we have a problem. We could of course hook into schedule(), but if the kthread never calls schedule() from a non-preempted context then it still doesn't help. -- Josh