From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A5EEC5B549 for ; Fri, 30 May 2025 09:36:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 23C206B00D1; Fri, 30 May 2025 05:36:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1EC2C6B00D2; Fri, 30 May 2025 05:36:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B3836B00D3; Fri, 30 May 2025 05:36:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DBA1B6B00D1 for ; Fri, 30 May 2025 05:36:39 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 81661B30DF for ; Fri, 30 May 2025 09:36:39 +0000 (UTC) X-FDA: 83499069318.18.C46B898 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) by imf07.hostedemail.com (Postfix) with ESMTP id 9A1984000C for ; Fri, 30 May 2025 09:36:37 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=FykPvIO6; spf=pass (imf07.hostedemail.com: domain of libo.gcs85@bytedance.com designates 209.85.216.51 as permitted sender) smtp.mailfrom=libo.gcs85@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748597797; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PQ/D1sjhfYHEwDTh2b+Qyte458RPTLSFlphr3fZumJQ=; b=hYkzAvfTWHTMcJhpRqL3Jo9J8I6oii5XRzeC00xqcRH1etXH9+Aujszk8+xVFa5zDnvSIp QaR9vMwaDHoUGtgYSKRNV5hBigQazCF2WztSmi3asRCaiLN29IdP/dKy+57GqLlK4oCBB0 YswywW8wVITxQ+0+0V/7qkEN/c7gm5s= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=FykPvIO6; spf=pass (imf07.hostedemail.com: domain of libo.gcs85@bytedance.com designates 209.85.216.51 as permitted sender) smtp.mailfrom=libo.gcs85@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748597797; a=rsa-sha256; cv=none; b=fT+rMq0vFA1zhBsJPzAkRQJt+cA7toO0i6pQ5vN/AdS8QmSfnuAp/t5S4oevBChGR5nj2h KPIKc0MFFpQqmAWE/InpTAGThXYarIDQpDN00jLhpVYEOgO5tJalOHg2KWNy5WowlQmFwf 1alq9L2f/SXjwxMD5SeZVnx7iFndsXM= Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-311d5fdf1f0so1705325a91.1 for ; Fri, 30 May 2025 02:36:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1748597796; x=1749202596; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PQ/D1sjhfYHEwDTh2b+Qyte458RPTLSFlphr3fZumJQ=; b=FykPvIO64mv8oav4C0VuogMEha4IqOJIUG5c+f87voikZm093DY7VPUVebhJFkf299 A13y3XyxV02qtw/OXgoDvZCDvgUy5NdhI+C3E1pFPKL0JN6NejCoai4J7OFgIKmGLxHx 81hRIZImCbo+5L56j+k8rE54x3nVrLn3mgkcKUSz3WQzruhi4SMH7jxa6elGGTSl2w6u EfAQ5BSP/DPBiauyBbZQwO0er5QO3L8wWKiKMTOsg+pYOEQLF81pHTscmo/FnbhhUAL+ GMtblzCfRUfEwHlQvW0pcdjMJmX3SgYHOqbv33ymkh0qDgwXw0ffVjMl291wf0w+9yvC lSOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748597796; x=1749202596; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PQ/D1sjhfYHEwDTh2b+Qyte458RPTLSFlphr3fZumJQ=; b=g+suTQqjUR8r1Jcha1JsLzwWsBInfrKV0eqapBd3ae0MKdcQkrVVRuDPdtPCn5UCj9 KAKMuy1GTfa982P//xd1E62jiJVltyA1MCfAJNn+wYlRjLtVoE1M+bVSndzEnlc5pPOh OKLfDLuFsqoQdfXdrR57pxixJ3VVMibe+Ihet1En+JavUbDQ/wc/HFX84EaM2yznL6Rn 2CCju687wz7V2FotIBm+FRQTjXFNw4zYmwGvZddwJSaAFyZPEj0SU0LUyNY5IPdHPlgN Gk3uOyzrVrXV/uPuor+bQvrlTHUHgWBE+uj2z+VcxolPSW63OOdlNXhQEHYwWzi70Mmr 5XYQ== X-Forwarded-Encrypted: i=1; AJvYcCUDeF5b38n2JO2J9bEhhqrD9h/2zFbrjnQR1psL2imdxhhXrAdTv5KFgiC2f7uZb6cCH1F8HDI7Hw==@kvack.org X-Gm-Message-State: AOJu0Yzo0tuzHGYHECNB2Ji8vl5fEoCq5AU/skmI6jlb+UWXTDJWxux7 Y0bB701NGZnw++5w79ORj0EEaNbfA9/akl/uKtdII6rgL4hA6S+htxe3qwtO44Hh9rw= X-Gm-Gg: ASbGnctpCNZp86i6HYB46+6AKFsSAru3kgblWFSDkDLmrOUQfNiP5WDWviTqpqo60em XahpzzdTSxLDfGe44MEDhhw18Fmk5PwdZdPkbcop/y91qqe78NOY5tDv4HRUz/jNjJnnsdbgvGl 7dpFDA8BEG3OZfyiCLD2XkdZLPZvC6Um16XSOGtmRtjjMSk0btNIciQF7iMadv7OkNz7PjzhlnB QDieaX7GIb19slFvokbrQKmN4bwZfjPJUHvFTpKICE0ptKqo9VwadTg05KmHymcWUG8YBk4eLG/ vr+DOeLJ6eTQetLgwjTvz38C0fd2y3DBm/82JfQYWdEaX71Yv3/36lvyD5K/MhYipfjJ0PoPR8O vbHNCMY/Ixu4VLNuxN2pn X-Google-Smtp-Source: AGHT+IE1J4YJ6wHA6b9CUB/dnKICEd+OlMDbXuOs/bJjQ/NWJ3rv4Cxh7kWsPUyPIoUfIb7mzNdLsw== X-Received: by 2002:a17:90b:4f4d:b0:312:1cd7:b337 with SMTP id 98e67ed59e1d1-3125034a47amr1876977a91.5.1748597796457; Fri, 30 May 2025 02:36:36 -0700 (PDT) Received: from FQ627FTG20.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3124e29f7b8sm838724a91.2.2025.05.30.02.36.21 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 30 May 2025 02:36:36 -0700 (PDT) From: Bo Li To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, luto@kernel.org, kees@kernel.org, akpm@linux-foundation.org, david@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, peterz@infradead.org Cc: dietmar.eggemann@arm.com, hpa@zytor.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, jannh@google.com, pfalcato@suse.de, riel@surriel.com, harry.yoo@oracle.com, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, yinhongbo@bytedance.com, dengliang.1214@bytedance.com, xieyongji@bytedance.com, chaiwen.cc@bytedance.com, songmuchun@bytedance.com, yuanzhu@bytedance.com, chengguozhu@bytedance.com, sunjiadong.lff@bytedance.com, Bo Li Subject: [RFC v2 32/35] RPAL: fix unknown nmi on AMD CPU Date: Fri, 30 May 2025 17:28:00 +0800 Message-Id: X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Queue-Id: 9A1984000C X-Rspamd-Server: rspam09 X-Stat-Signature: t9ju3uo7xg58nqm6b7hsutbtrqe8anoj X-HE-Tag: 1748597797-437711 X-HE-Meta: U2FsdGVkX1/Mc9Nv/sIdQgGLWMs1sDEsI8Mv6kZT63HMMcF+BEA4VQv9QsrUJWptunuqt/rhaCSMwLVBfb3rqsUQE6jhYcl4SD+HB5kVydBbMDKimPGjissL7kbxc9O6bapisbZVJwFJhyPbrFmwoyc/iecW5IGkn6aDUUkD8WWTIhAV3c0DqHvyhZbv7xGg12MdFn8utP6GzVWzHJChex5dpx1kiZPFcPpvWt+LyV4JljTNcvBSu9gos5sbeXacc4WiJsG6v+eQ9x6RTo2ZP7rkDaJNq7Z1llvWcDQGpb/mfYYzlfydL20wZz4EtGeCV5NHI5PL2zZQGRGFXKu1A0DkxMhPD30krK+9wN55iaBPIQJXBX8Jr5ZdZBzGUWauDaRLGIv1M7iEXbjjxIooRt0umDloS2p8whzYyLTpSxwtVnVL8xv8h3sw467wwGy7D+4shh4lRzsS/Z/cBQ82YQ4nbbVCBGTfQbDPOiihsCUDsPzfnADDcTaYn99qc1+KQRepB3kRASBAK+QUcOEL+64Li6N82I9NXa8PMsU/iy2LjXDBUT52MftlKD0ZxHMeK0iIc8xFl8/sMz0wxYiZ9oxcGF6/pYBXEEO+2bGsicXMw/4AdPXAYg4ClsuyHHCuaTG2bFYoVct0Upm1Heie8FnQh3x54pGSCH63fsCiyUHvoEPKFQogE3LbkLJPffHQiU2sKfM1lijqWsx9A226uwya8HktEvMilXP8h6YSq2D2/z31wJKIsmw6Hp0Pw0gz53278Klq13SBB/J7X0pj1sQJgyMWanza76A55WN/ge/+mqL1rCBy+9B0rwK4TePKgZUZ2CDfyUjEj9VsRpc8/wOkIv9l7sUW5rkTMFqcmXduzZskJIoOZZrJ/bG2R/LdXYKIdZWrdkPKvETPmTkonGOzwl0CyA5cdiaa70QkeJkjF4PcmAkNXGdQ7ipDnRsaG+HDwzvnO4jgDOu2cV7 rmnnB2Me Hh7ET2K0kUpezXsHBbk94GpmCN6ryXIIkFYwHfKrMUCMnese7Ok8uQqZdmPSpd0VG8BoGQH9MI7+9wx44vOTjJX6dsrHFU4ybEIyZMVgcavxIR9FyETntqfvyZwWeND7WMJaoGd4uBJN3M8G5DfyYqf8j/dkShB99rWREMXE+jMsu8qgt+uDyx/cqmcG71tegxyoATY6yEA1AjyzGPfNq1HU4Lit7CTv+xrmhTDlInbZzdjD6iJFm7WyoQAGomEkpg+49cTQ9UZW95B0v0Tjlqg9Uky22HOchINtverPGuDY70r/3BxWc3JYuhg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In Lazy switch, the function event_sched_out() will be called. This function deletes the perf event of the task being scheduled out, causing the active_mask in cpu_hw_events to be cleared. In AMD's NMI handler, if the bit corresponding to active_mask is not set, the CPU will not handle the NMI event, ultimately triggering an unknown NMI error. Additionally, event_sched_out() may call amd_pmu_wait_on_overflow(), leading to a busy wait of up to 50us during lazy switch. This patch adds two per_cpu variables. rpal_nmi_handle is set when an NMI occurs. When encountering an unknown NMI, this NMI is skipped. rpal_nmi is set before lazy switch and cleared after lazy switch, preventing the busy wait caused by amd_pmu_wait_on_overflow(). Signed-off-by: Bo Li --- arch/x86/events/amd/core.c | 14 ++++++++++++++ arch/x86/kernel/nmi.c | 20 ++++++++++++++++++++ arch/x86/rpal/core.c | 17 ++++++++++++++++- 3 files changed, 50 insertions(+), 1 deletion(-) diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index b20661b8621d..633a9ac4e77c 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -719,6 +719,10 @@ static void amd_pmu_wait_on_overflow(int idx) } } +#ifdef CONFIG_RPAL +DEFINE_PER_CPU(bool, rpal_nmi); +#endif + static void amd_pmu_check_overflow(void) { struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); @@ -732,6 +736,11 @@ static void amd_pmu_check_overflow(void) if (in_nmi()) return; +#ifdef CONFIG_RPAL + if (this_cpu_read(rpal_nmi)) + return; +#endif + /* * Check each counter for overflow and wait for it to be reset by the * NMI if it has overflowed. This relies on the fact that all active @@ -807,6 +816,11 @@ static void amd_pmu_disable_event(struct perf_event *event) if (in_nmi()) return; +#ifdef CONFIG_RPAL + if (this_cpu_read(rpal_nmi)) + return; +#endif + amd_pmu_wait_on_overflow(event->hw.idx); } diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index be93ec7255bf..dd72b6d1c7f9 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -351,12 +351,23 @@ NOKPROBE_SYMBOL(unknown_nmi_error); static DEFINE_PER_CPU(bool, swallow_nmi); static DEFINE_PER_CPU(unsigned long, last_nmi_rip); +#ifdef CONFIG_RPAL +DEFINE_PER_CPU(bool, rpal_nmi_handle); +#endif static noinstr void default_do_nmi(struct pt_regs *regs) { unsigned char reason = 0; int handled; bool b2b = false; +#ifdef CONFIG_RPAL + bool rpal_handle = false; + + if (__this_cpu_read(rpal_nmi_handle)) { + __this_cpu_write(rpal_nmi_handle, false); + rpal_handle = true; + } +#endif /* * Back-to-back NMIs are detected by comparing the RIP of the @@ -471,6 +482,15 @@ static noinstr void default_do_nmi(struct pt_regs *regs) */ if (b2b && __this_cpu_read(swallow_nmi)) __this_cpu_add(nmi_stats.swallow, 1); +#ifdef CONFIG_RPAL + /* + * Lazy switch may clear the bit in active_mask, causing + * nmi event not handled. This will lead to unknown nmi, + * try to avoid this. + */ + else if (rpal_handle) + goto out; +#endif else unknown_nmi_error(reason, regs); diff --git a/arch/x86/rpal/core.c b/arch/x86/rpal/core.c index 6a22b9faa100..92281b557a6c 100644 --- a/arch/x86/rpal/core.c +++ b/arch/x86/rpal/core.c @@ -376,11 +376,26 @@ rpal_exception_context_switch(struct pt_regs *regs) return next; } +DECLARE_PER_CPU(bool, rpal_nmi_handle); +DECLARE_PER_CPU(bool, rpal_nmi); __visible struct task_struct *rpal_nmi_context_switch(struct pt_regs *regs) { struct task_struct *next; - next = rpal_kernel_context_switch(regs); + if (rpal_test_current_thread_flag(RPAL_LAZY_SWITCHED_BIT)) + rpal_update_fsbase(regs); + + next = rpal_misidentify(); + if (unlikely(next != NULL)) { + next = rpal_fix_critical_section(next, regs); + if (next) { + __this_cpu_write(rpal_nmi_handle, true); + /* avoid wait in amd_pmu_check_overflow */ + __this_cpu_write(rpal_nmi, true); + next = rpal_do_kernel_context_switch(next, regs); + __this_cpu_write(rpal_nmi, false); + } + } return next; } -- 2.20.1