From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D81EC83F0A for ; Wed, 9 Jul 2025 19:53:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C444C6B0161; Wed, 9 Jul 2025 15:53:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C1C516B0162; Wed, 9 Jul 2025 15:53:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B31E16B0163; Wed, 9 Jul 2025 15:53:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A39946B0161 for ; Wed, 9 Jul 2025 15:53:21 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 60FDFC01C1 for ; Wed, 9 Jul 2025 19:53:21 +0000 (UTC) X-FDA: 83645775402.02.079A6E2 Received: from common.ash.relay.mailchannels.net (common.ash.relay.mailchannels.net [23.83.222.38]) by imf16.hostedemail.com (Postfix) with ESMTP id B3F91180010 for ; Wed, 9 Jul 2025 19:53:18 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=stgolabs.net header.s=dreamhost header.b=MNAPTSqG; arc=pass ("mailchannels.net:s=arc-2022:i=1"); spf=pass (imf16.hostedemail.com: domain of dave@stgolabs.net designates 23.83.222.38 as permitted sender) smtp.mailfrom=dave@stgolabs.net; dmarc=none ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752090799; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=T7+gk4yndEAXxrNbTlbrYpYmnRnH6YglAKEGyZm6Jvk=; b=0pEiCz+UEnVzlCLUDhf61zNZ9ykyJTx0jr5x+XtGqNyEbwgXxn5ujk8ODpvgBbRtJ2RRAJ I2jPCDh8r0m7KfktW8/q3fhGCRaduBsuIWYC3btUeOguQZdr6sNeq5RnEabqT0gIVchpsR l8Y2cSNTAMT72noe9/P+3/j4TrUXeBA= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1752090799; a=rsa-sha256; cv=pass; b=g4ruiW7p0v41fFmbVspyMNuSIaTffYcJGQOVMo2haRBmntP3T844YxOMjlmjVDN6MbN4+W 48EmlpRa6fgvzGD6OFJsuAdVV1DGegDaaCy3FkmZnyRuBnJzUFpB1/yOgpFRbEcB1mEUqZ y4P+rbTs/ukzcKNkSYiAFCNEkDu210k= ARC-Authentication-Results: i=2; imf16.hostedemail.com; dkim=pass header.d=stgolabs.net header.s=dreamhost header.b=MNAPTSqG; arc=pass ("mailchannels.net:s=arc-2022:i=1"); spf=pass (imf16.hostedemail.com: domain of dave@stgolabs.net designates 23.83.222.38 as permitted sender) smtp.mailfrom=dave@stgolabs.net; dmarc=none X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id E1BD5902FB6; Wed, 9 Jul 2025 19:53:16 +0000 (UTC) Received: from pdx1-sub0-mail-a220.dreamhost.com (trex-green-6.trex.outbound.svc.cluster.local [100.111.53.65]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 2D743903243; Wed, 9 Jul 2025 19:53:15 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1752090795; a=rsa-sha256; cv=none; b=0ZOBVuLcMnJG4nUtPqunO2ilkmQD8f6QmXJRaRw4oHqljM3hvIMnSWB8YJiBgbXEmRnmm3 9FHFvmYSzrylACfjluli6jZnuKHo7pt5uiVWWHJbPo4D9ZxeHPAqgEE05/Ga0JaiFenTi4 S6frqWHQ723eDPoVyDM8M5YEVH3BwbTkvfBvmN1b0H1lrK3MjT9dwe0BKJKBiwBlHbD07P 5lLaw34A29zdXuAYuhWcOuYuARMUWy51gVMuvwEMzC0+t3LEngCV1k/yXLCSEEY4g9V7I4 W9De6I8grFZFnq48XKZzatgU78yE9cDujSr6io8cpiT7hFvX1YLcaWi12yMeJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1752090795; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=T7+gk4yndEAXxrNbTlbrYpYmnRnH6YglAKEGyZm6Jvk=; b=kNo7zS5ttuqZZC68w8PCWFd/otpMIgUDnCgvNzHQHfGXsf697u5vAXiGGa/agEViQDhEmy /YWFofPSzqpvoMBV5etpeoD6qEuj7YKycLhDlhlOAr3NZiF3H1T0/gCI/qbEU04bnqRG+3 RlGfu1zAK6d88GXjqwXdKr1XSMcz129JUt8VcD8QtYGi57rP4UDG5P3JPGQi3O60IJ5Nva zbPu3Dokog5yZLr+9ZuiabYQGs78Qb6b9arUXC4lmMyJyKuDf+4vZaX2SYJ47TUaeYlWZH rOn+YV3RMsJLS+xb21ggOaf4K/HzVuqDjCH9Cs70lfcNBj1f6LR3JvFUOUeFBw== ARC-Authentication-Results: i=1; rspamd-5c976dc8b-dj4rr; auth=pass smtp.auth=dreamhost smtp.mailfrom=dave@stgolabs.net X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|dave@stgolabs.net X-MailChannels-Auth-Id: dreamhost X-Eyes-Callous: 2336bfa936ee4816_1752090795798_3380218129 X-MC-Loop-Signature: 1752090795797:4037420958 X-MC-Ingress-Time: 1752090795797 Received: from pdx1-sub0-mail-a220.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.111.53.65 (trex/7.1.3); Wed, 09 Jul 2025 19:53:15 +0000 Received: from offworld (syn-076-167-199-067.res.spectrum.com [76.167.199.67]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: dave@stgolabs.net) by pdx1-sub0-mail-a220.dreamhost.com (Postfix) with ESMTPSA id 4bcpYj4DQfz7F; Wed, 9 Jul 2025 12:53:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stgolabs.net; s=dreamhost; t=1752090795; bh=T7+gk4yndEAXxrNbTlbrYpYmnRnH6YglAKEGyZm6Jvk=; h=Date:From:To:Cc:Subject:Content-Type; b=MNAPTSqGqIUV9YlJMB4z55PsBA59V9i/q1iBdovNWtX3x01yj3XRlx+GvEjTbvzfk 5GLEebuebBRx/675NKzuPV28KiJosUny0+rcXQeU57XGYYZdwoPbbE+HJfmB/+W1No /TFPyyAtIVjnTCt4A/hJh9MSRHeHNQtRKYZydYXcrOouHE/kuIpOoAPSWEXNJaSA5T kPLrEBn2l7Zbg8xkzmPMBkbUzkdY5+uxyHN/UzgRHAlpDGacLNTd/TsAbSkSaTOkaM OtQmt/8ATb2ksJmH4qNJk4ffjrJy0J5a+X2XAyPkfLFZmRVRy65ftxxj1HszhLJdn5 W8oZdfHmKn8sA== Date: Wed, 9 Jul 2025 12:53:10 -0700 From: Davidlohr Bueso To: "H. Peter Anvin" Cc: Peter Zijlstra , Jonathan Cameron , Catalin Marinas , james.morse@arm.com, linux-cxl@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-acpi@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, gregkh@linuxfoundation.org, Will Deacon , Dan Williams , Yicong Yang , linuxarm@huawei.com, Yushan Wang , Lorenzo Pieralisi , Mark Rutland , Dave Hansen , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, Andy Lutomirski , Adam Manzanares Subject: Re: [PATCH v2 0/8] Cache coherency management subsystem Message-ID: <20250709195310.7ofioxl5vpqs6rib@offworld> Mail-Followup-To: "H. Peter Anvin" , Peter Zijlstra , Jonathan Cameron , Catalin Marinas , james.morse@arm.com, linux-cxl@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-acpi@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, gregkh@linuxfoundation.org, Will Deacon , Dan Williams , Yicong Yang , linuxarm@huawei.com, Yushan Wang , Lorenzo Pieralisi , Mark Rutland , Dave Hansen , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, Andy Lutomirski , Adam Manzanares References: <20250624154805.66985-1-Jonathan.Cameron@huawei.com> <20250625085204.GC1613200@noisy.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20220429 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B3F91180010 X-Stat-Signature: 5ehtiurgpipbyukafk8ey1ni4hy8rfwp X-HE-Tag: 1752090798-77274 X-HE-Meta: U2FsdGVkX18tV++ibvFZdbX/82AvoExZun9cnDKIptbZpVNPtFacp/SaaSOB+Zuv5bsQ5nwAIxsCRl/orEFnait/AOmBoI1/JyFQ5mQp2eibO+3PacyKZjMlK/HF31ggg9UkyUboulIjfJDqNL611QZw+Bpp4M5EyKnXqmTQsFerzjNcD97w8kJG7angHMp2FgXv40LsZFMVQ1s2dCRetL1flSY5nGbeToDSiv1b/mFYgEmDDOU2fsH8cQGh4cr2ZrY4EJGQh8cl+70WOafYHCWHB95C0dZOAQt1PZbf9X0mRGsgN0VvFVQSUlnNwOiUNyKi+OznADUQd7skbXGBEaprSjyG+yEJKw09DpTXJpcWco0g914ht4RkV+IgjSuU6GFW+qgusDENffSfML4eBihfVgYh5OLKjSAvhYSZNu/8WvEv6LrXM2JuGK+LLdYMjKlcfmHsRzo7gqzFJC/vHa2tB7iDs3JlbQtUTj+mZP9135Qjqccs5CJFu3qWXU68jHU70or7KNFnM0PCFw4hb8HI3Vodo/weYLQqGM+rl7rUhZmeO5cvoifCginw8oKim0n7RvLkIKhWewrwiq0oeniSC4w3CllHmr7YP6BkLsEGZMb4jwVNr5QgDWwhM3P9GsQXwACfIkP+D3ctDhB3yWIPM/YbDzETmFsN1UdGRlkUhcvnvxE3brnwn/SJl7HG1ZF1SiolJBT316YjJKp8sVjdBw0TlqXIaybGJSzhJQYlcX24YyRVNXlE4x6QlB2jFZXCXfh/MxtJ+1V8MRP2dcv4vN4jt/JyCQDH6ICyocRgIqiytQQRD3eKqQ+RYagE+7iLgV5a/p1qGIrLlUC/SuPk1AQFYFDzAtEo8ER9WxWL1RE+ofhdXwjErH5xI1kjS9K8hPdkRYwIYBhvYwTsgXDiBehQpYAS5C0ZExicocpRmUF4oHe8jjSO6fuQjZKB02nQiUbmWr996f335eZ t5I4L7DO 23zXtlxJDiIVehiHLzJQ/8RGr1Q2vZvgqubGo7UQdK2FNsHvKaagrWu3nT1zjQjscjNtzuJYh1Fijwrj503/8/44vFPeGqjG6Q91lGJZRArqJg4wi3HJtjPsyckau9HhfqffGwx9+lFzNWLrKT0gK7xe9VZq4cMpYdZTGXhOR8y3TtUzFQJxvF7gGwuxwRNtNsDsTierrblXI3z5iFFB/JjOieh1peAkVpbN+Pqb8g7y0jTnUUWbUXDFX0TSkkxWVD1iHdBE1N3PIeVGsdMBEndhN+MjOB/YTqcPGPy+f1RgtVV5n3ai/8XMjXJTyGkvbklpEcOVPu3rPQMrAuXVxU0q7+Ym4B4dR+CzZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, 25 Jun 2025, H. Peter Anvin wrote: >On June 25, 2025 1:52:04 AM PDT, Peter Zijlstra wrote: >>On Tue, Jun 24, 2025 at 04:47:56PM +0100, Jonathan Cameron wrote: >> >>> On x86 there is the much loved WBINVD instruction that causes a write back >>> and invalidate of all caches in the system. It is expensive but it is >> >>Expensive is not the only problem. It actively interferes with things >>like Cache-Allocation-Technology (RDT-CAT for the intel folks). Doing >>WBINVD utterly destroys the cache subsystem for everybody on the >>machine. >> >>> necessary in a few corner cases. >> >>Don't we have things like CLFLUSH/CLFLUSHOPT/CLWB exactly so that we can >>avoid doing dumb things like WBINVD ?!? >> >>> These are cases where the contents of >>> Physical Memory may change without any writes from the host. Whilst there >>> are a few reasons this might happen, the one I care about here is when >>> we are adding or removing mappings on CXL. So typically going from >>> there being actual memory at a host Physical Address to nothing there >>> (reads as zero, writes dropped) or visa-versa. >> >>> The >>> thing that makes it very hard to handle with CPU flushes is that the >>> instructions are normally VA based and not guaranteed to reach beyond >>> the Point of Coherence or similar. You might be able to (ab)use >>> various flush operations intended to ensure persistence memory but >>> in general they don't work either. >> >>Urgh so this. Dan, Dave, are we getting new instructions to deal with >>this? I'm really not keen on having WBINVD in active use. >> > >WBINVD is the nuclear weapon to use when you have lost all notion of where the problematic data can be, and amounts to a full reset of the cache system. > >WBINVD can block interrupts for many *milliseconds*, system wide, and so is really only useful for once-per-boot type events, like MTRR initialization. Correct, and cpu_cache_invalidate_memregion() was introduced exactly with these constraints in mind, and the current x86 is the worse case scenario. As Jonathan pointed out, ranged optimizations only improve what is already there. Thanks, Davidlohr