From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37F91C36002 for ; Mon, 24 Mar 2025 12:00:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 92769280002; Mon, 24 Mar 2025 08:00:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D75E280001; Mon, 24 Mar 2025 08:00:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 79E89280002; Mon, 24 Mar 2025 08:00:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5F865280001 for ; Mon, 24 Mar 2025 08:00:54 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C76D21206A9 for ; Mon, 24 Mar 2025 12:00:54 +0000 (UTC) X-FDA: 83256303228.06.F904F8A Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by imf22.hostedemail.com (Postfix) with ESMTP id BC93EC001E for ; Mon, 24 Mar 2025 12:00:51 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf22.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742817652; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H+zfiOXaWFuYml4P6ucIN7B4jL6P43WgSqr8mP5K7kk=; b=O28afejkmSw5osqRQq4MKmCMGsGZzo8xuJikqLMK+sEpzRWIWP9LxUpbAAeiIg0RpQc+oI 2g1yn09af0VVC7FkChgfZhVYgzBFWfe5KMK2krmoQqeC/jIq9uvsbNcAJq5sBtBSgT2Wxx u4lbjR+6mTUxJqI3Db+rxkDjq3tabmE= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf22.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742817652; a=rsa-sha256; cv=none; b=RBoayshSt208SzYVT/Kv2UUGkLSOtYj+nuvRBhGg2Ns0muHnSMZU/aMqLswCJwieHbQHEf DRfrN8+HxbvTdczGnnq3hvO5AKrnbnAckvGKxD9mIaWi+F2eOeW4wvAuAiMbMy3wqKf90W e1/9RycUcKfehDpmMIiVMrIHRbIFtt4= Received: from mail.maildlp.com (unknown [172.18.186.31]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4ZLs415Lnbz6M4ly; Mon, 24 Mar 2025 19:57:21 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id 48F591400D4; Mon, 24 Mar 2025 20:00:47 +0800 (CST) Received: from localhost (10.48.158.58) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 24 Mar 2025 13:00:44 +0100 Date: Mon, 24 Mar 2025 12:00:40 +0000 From: Jonathan Cameron To: Conor Dooley CC: , , , Yicong Yang , , , , Yushan Wang , , , Lorenzo Pieralisi , Mark Rutland , "Catalin Marinas" , Will Deacon , "Dan Williams" Subject: Re: [RFC PATCH 0/6] Cache coherency management subsystem Message-ID: <20250324120040.00003d95@huawei.com> In-Reply-To: <20250321-failing-squatted-37a88909bde2@spud> References: <20250320174118.39173-1-Jonathan.Cameron@huawei.com> <20250321-failing-squatted-37a88909bde2@spud> X-Mailer: Claws Mail 4.3.0 (GTK 3.24.42; x86_64-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.48.158.58] X-ClientProxiedBy: lhrpeml500003.china.huawei.com (7.191.162.67) To frapeml500008.china.huawei.com (7.182.85.71) X-Rspamd-Queue-Id: BC93EC001E X-Rspamd-Server: rspam05 X-Rspam-User: X-Stat-Signature: hbetcsake1hjax6bxfm7zdxuh9g6ar8r X-HE-Tag: 1742817651-324964 X-HE-Meta: U2FsdGVkX185chOgAJd5EgG1Vg5dbTrM0crmvWuH2pokPY4rwnq97lMfC8n9lENAiuPbFFkD+8OjPz9A31LhZ++cd4p8ydIBKaYF/nCE/g12DAvxHvEjMXs7V8kHUP2dliYvYM+c9P1w34MVzYgKk4zO3mvYDIJctRlLIqs6VMxbPEcNjNdwgmvRXzZacCu+KXTHViA7wEDM6j4VhCw67KrKKEMHew6+oWEqRpWl0Al5s3v8T2epmT+D6Yup6CGm4hLMMotN4oWhZ/c90gAhuaeLDZQF0Ph1M73sFUM7Ibi4s3pQxCUyZbUmfKb/OtcTCblI2w7FARv4zN1KRSd1Gt8URKaUrXUrgwaMBBLhHXCC3Lwn1tfHnbHtFJ2Sqoa6PMcahGKZYf7LnOeQlnxWMK8BbNvuBbuqyg3fmSxs0rIX6ZuLAa7aZaBC37tewJJ588kL1GyxD5AreGT2hELyTcD0Cp/SFMNtGdXG5kdix1giUAS9cVj7hJWuTWH0CHlSLX3I0P0Mw4yoIrt4cJafMzS5+uYr2oIh2Qr4A3pX2J0PaV3S0qYSqxMyBzW4u2tqy6tJE5YvfoQpX/Eh+BbohgOar9Hvr7PGnsJVapwMN+NLDqQ6tJEwANzFt1gR+NpNEm/8R2sWIxit4K0y9HPZtSqAnApGlLLVC/TiqITVMXJioe8KRyA6QzQPW7vu4ClM+0Wprgr+yWibPMoEw2rrxO8VjkUUtySfBYpXPS2YWfIll6o0D3dJK+lZw2Dha9PGzhf9nrG5oe30q8OBxUWkJkXLSXrWQKhcdLRofxzvROxuG0h11jLBZcEFl179cByBjKBT801etsCQiTNb1WU+r0JpwvZBJ2ZDd07ITjNasL+lRKw+PA7mMToArtu1FxO4ZkfXREM/36MbrDfdob2mwaLLp1WGwHYtvGSmjBD023LzJWsjeWLF81qdepz+leF+FP7jra4Ons5Rc5z9T+6 9MMfbFnH yZmCVnHIo8s20peOaYcAOunElRUG8IFGho2CC+OKgK1vQuYoeTdwNecPDcB5ECHCW5YkkDYgVzsQGjjMm72dLypJUNZDO889pZIazyc3t2aKUxG0aDmTQN5aENXV8YDJi5KhEc7jURvMJqNKb8rrE57WiMn4nLE5lMKVHF+pNGdDkducelmXkXfpE8dJKa68yEu68BbOd+saDJKo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, 21 Mar 2025 22:32:15 +0000 Conor Dooley wrote: > On Thu, Mar 20, 2025 at 05:41:12PM +0000, Jonathan Cameron wrote: > > Note that I've only a vague idea of who will care about this > > so please do +CC others as needed. > > > > On x86 there is the much loved WBINVD instruction that causes a write back > > and invalidate of all caches in the system. It is expensive but it is > > necessary in a few corner cases. These are cases where the contents of > > Physical Memory may change without any writes from the host. Whilst there > > are a few reasons this might happen, the one I care about here is when > > we are adding or removing mappings on CXL. So typically going from > > there being actual memory at a host Physical Address to nothing there > > (reads as zero, writes dropped) or visa-versa. That involves the > > reprogramming of address decoders (HDM Decoders); in the near future > > it may also include the device offering dynamic capacity extents. The > > thing that makes it very hard to handle with CPU flushes is that the > > instructions are normally VA based and not guaranteed to reach beyond > > the Point of Coherence or similar. You might be able to (ab)use > > various flush operations intended to ensure persistence memory but > > in general they don't work either. > > > > So on other architectures such as ARM64 we have no instruction similar to > > WBINVD but we may have device interfaces in the system that provide a way > > to ensure a PA range undergoes the write back and invalidate action. This > > RFC is to find a way to support those cache maintenance device interfaces. > > The ones I know about are much more flexible than WBINVD, allowing > > invalidation of particular PA ranges, or a much richer set of flush types > > (not supported yet as not needed for upstream use cases). > > > > To illustrate how a solution might work, I've taken both a HiSilicon > > design (slight quirk as registers overlap with existing PMU driver) > > and more controversially a firmware interface proposal from ARM > > (wrapped up in made up ACPI) that was dropped from the released spec > > but for which the alpha spec is still available. > > > > Why drivers/cache? > > - Mainly because it exists and smells like a reasonable place. > > - Conor, you are maintainer for this currently do you mind us putting this > > stuff in there? > > drivers/cache was just something to put the cache controller drivers we > have on RISC-V that implement the various arch_dma*() callbacks in > non-standard ways that made more sense than drivers/soc/ > since the controllers are IP provided by CPU vendors. There's only > two drivers here now, but I am aware of another two non-standard CMO > mechanisms if the silicon with them so there'll likely be more in the > future :) I'm only really maintainer of it to avoid it being another > thing for Palmer to look after :) I suspected as much :) > > I've only skimmed this for now, but I think it is reasonable to put them > here. Maybe my skim is showing, but it would not surprise me to see a > driver providing both non-standard arch_dma*() callbacks as well as > dealing with CXL mappings via this new class on RISC-V in the future.. Absolutely. The use of an ARM callback was just a place holder for now (Greg pointed that one out as well as I forgot to mention it in the patch description!) I think this will turn out to be at least some subset of implementations for other architectures unless they decide to go the route of an instruction (like x86). > Either way, I think it'd probably be a good idea to add ?you? as a > co-maintainer if the directory is going to be used for your proposed > interface/drivers, for what I hope is an obvious reason! Sure. That would make sense. Jonathan >