[PATCH v6 0/7] Cache coherency management subsystem

[PATCH v6 0/7] Cache coherency management subsystem

[Jonathan Cameron]

v6: Split the Kconfig menu to differentiate the new type of driver this set
    adds from existing riscv cache maintenance drivers for interaction with
    non coherent DMA master. This provides a good resolution to concerns
    around mixing different types of driver, whilst avoiding yet another
    drivers/* directory (Thanks to Arnd Bergmann) 

Support system level interfaces for cache maintenance as found on some
ARM64 systems. It is expected that systems using other CPU architectures
(such as RiscV) that support CXL memory and allow for native OS flows
will also use this. This is needed for correct functionality during
various forms of memory hotplug (e.g. CXL). Typical hardware has MMIO
interface found via ACPI DSDT. A system will often contain multiple
hardware instances.

Includes parameter changes to cpu_cache_invalidate_memregion() but no
functional changes for architectures that already support this call.

Merge plan. Through Conor's drivers/cache tree which routes through
the SoC tree.
  *  Andrew Morton has expressed he is fine with the MM related changes
     going via another appropriate tree.
  *  CXL maintainers expressed that they don't consider it appropriate
     to go through theit tree.
  *  The tiny touching of Arm specific code has an ack from Catalin.

v5: Changes called out in individual patches.
Comment and patch description updates to make the following clearer.
  - Difference from cache-coherence operations for non-coherent DMA.
    Longer term it may make sense to share infrastructure but for now
    we have two parallel systems as there would be near zero overlap
    in code or functionality.
  - Why multiple agent handling is necessary and what that means for
    the HiSilicon HHAs in our systems + the driver inclued in this set.
 
v4: (Small changes called out in each patch)
- Drop the ACPI driver. It has done it's job as a second implementation
  to help with generality testing. I have heard zero interest in actually
  doing the specification work needed to make that official. Easy to bring
  back if needed in future. I have it locally still as a second test
  case.
- Add a cpu_cache_invalidate_all() helper for the 0,-1 case that is used
  to indicate everything should be flushed as no fine grained range info
  available.
- Simplify the necessary symbols to be selected by architectures by
  making CONFIG_GENERIC_CPU_CACHE_MAINTENANCE select
  ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
- Avoid naming mentioning devices as there is no struct device.
- Use a kref so as to have something on which a _put() operation makes
  sense avoiding rather confusing freeing of an internal structure pointer
  that was seen in v3.
- Gather tags given.
- Various minor things like typos, header tweaks etc.
Thanks to all who reviewed v3.

On current ARM64 systems (and likely other architectures) the
implementation of cache flushing need for actions such as CXL memory
hotplug e.g. cpu_cache_invalidate_memregion(), is performed by system
components outside of the CPU, controlled via either firmware or MMIO
interfaces.

These control units run the necessary coherency protocol operations to
cause the write backs and cache flushes to occur asynchronously. They allow
filtering by PA range to reduce disruption to the system. Systems
supporting this interface must be designed to ensure that, when complete,
all cache lines in the range are in invalid state or clean state
(prefetches may have raced with the invalidation). This must include
memory-side caches and other non architectural caches beyond the Point
of Coherence (ARM terminology) such that writes will reach memory even
after OS programmable address decoders are modified (for CXL this is
any HDM decoders that aren't locked). Software will guarantee that no
writes to these memory ranges race with this operation. Whilst this is
subtly different from write backs must reach the physical memory that
difference probably doesn't matter to those reading this series.

The often distributed nature of the relevant coherency management units
(e.g. due to interleaving) requires the appropriate commands to be issued
to multiple (potentially heterogeneous) units. To enable this a
registration framework is provided to which drivers may register a set
of callbacks. Upon a request for a cache maintenance operation the
framework iterates over all registered callback sets, calling first a
command to write back and invalidate, and then optionally a command to wait
for completion. Filtering on relevance if a give request is left to the
individual drivers.

In this version only one driver is included. This is the HiSilicon Hydra
Home Agent driver which controls hardware found on some of our relevant
server SoCs. Also available (I can post if anyone is interested)
is an ACPI driver based on a firmware interface that was in a public
PSCI specification alpha version

QEMU emulation code at
http://gitlab.com/jic23/qemu cxl-2025-03-20 

Notes:
- I don't particularly like defining 'generic' infrastructure with so few
  implementations. If anyone can point me at docs for another one or two,
  or confirm that they think this is fine that would be great!
  The converse to this is I don't want to wait longer for those to surface
  given the necessity to support this one platform that I do know about!

Jonathan Cameron (4):
  memregion: Drop unused IORES_DESC_* parameter from
    cpu_cache_invalidate_memregion()
  arm64: Select GENERIC_CPU_CACHE_MAINTENANCE
  MAINTAINERS: Add Jonathan Cameron to drivers/cache and add
    lib/cache_maint.c + header
  cache: Make top level Kconfig menu a boolean dependent on RISCV

Yicong Yang (2):
  memregion: Support fine grained invalidate by
    cpu_cache_invalidate_memregion()
  lib: Support ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION

Yushan Wang (1):
  cache: Support cache maintenance for HiSilicon SoC Hydra Home Agent

 MAINTAINERS                     |   3 +
 arch/arm64/Kconfig              |   1 +
 arch/x86/mm/pat/set_memory.c    |   2 +-
 drivers/cache/Kconfig           |  37 +++++-
 drivers/cache/Makefile          |   2 +
 drivers/cache/hisi_soc_hha.c    | 194 ++++++++++++++++++++++++++++++++
 drivers/cxl/core/region.c       |   5 +-
 drivers/nvdimm/region.c         |   2 +-
 drivers/nvdimm/region_devs.c    |   2 +-
 include/linux/cache_coherency.h |  61 ++++++++++
 include/linux/memregion.h       |  16 ++-
 lib/Kconfig                     |   4 +
 lib/Makefile                    |   2 +
 lib/cache_maint.c               | 138 +++++++++++++++++++++++
 14 files changed, 457 insertions(+), 12 deletions(-)
 create mode 100644 drivers/cache/hisi_soc_hha.c
 create mode 100644 include/linux/cache_coherency.h
 create mode 100644 lib/cache_maint.c

-- 
2.48.1

[PATCH v6 1/7] memregion: Drop unused IORES_DESC_* parameter from cpu_cache_invalidate_memregion()

[Jonathan Cameron]

The res_desc parameter was originally introduced for documentation purposes
and with the idea that with HDM-DB CXL invalidation could be triggered from
the device. That has not come to pass and the continued existence of the
option is confusing when we add a range in the following patch which might
not be a strict subset of the res_desc. So avoid that confusion by dropping
the parameter.

Link: https://lore.kernel.org/linux-mm/686eedb25ed02_24471002e@dwillia2-xfh.jf.intel.com.notmuch/
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Suggested-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>

---
v6: No change.
v5: No change.
v4: Dan's tag (thanks!)
V3: New patch.
    As Dan calls out in the linked mail, an alternative might be to lookup
    the ranges and enforce the descriptor but his expressed preference
    was for dropping the parameter.
---
 arch/x86/mm/pat/set_memory.c | 2 +-
 drivers/cxl/core/region.c    | 2 +-
 drivers/nvdimm/region.c      | 2 +-
 drivers/nvdimm/region_devs.c | 2 +-
 include/linux/memregion.h    | 7 +++----
 5 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index d2d54b8c4dbb..0cfee2544ad4 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -368,7 +368,7 @@ bool cpu_cache_has_invalidate_memregion(void)
 }
 EXPORT_SYMBOL_NS_GPL(cpu_cache_has_invalidate_memregion, "DEVMEM");
 
-int cpu_cache_invalidate_memregion(int res_desc)
+int cpu_cache_invalidate_memregion(void)
 {
 	if (WARN_ON_ONCE(!cpu_cache_has_invalidate_memregion()))
 		return -ENXIO;
diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index e14c1d305b22..36489cb086f3 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -236,7 +236,7 @@ static int cxl_region_invalidate_memregion(struct cxl_region *cxlr)
 		return -ENXIO;
 	}
 
-	cpu_cache_invalidate_memregion(IORES_DESC_CXL);
+	cpu_cache_invalidate_memregion();
 	return 0;
 }
 
diff --git a/drivers/nvdimm/region.c b/drivers/nvdimm/region.c
index cd9b52040d7b..47e263ecedf7 100644
--- a/drivers/nvdimm/region.c
+++ b/drivers/nvdimm/region.c
@@ -110,7 +110,7 @@ static void nd_region_remove(struct device *dev)
 	 * here is ok.
 	 */
 	if (cpu_cache_has_invalidate_memregion())
-		cpu_cache_invalidate_memregion(IORES_DESC_PERSISTENT_MEMORY);
+		cpu_cache_invalidate_memregion();
 }
 
 static int child_notify(struct device *dev, void *data)
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index a5ceaf5db595..c375b11aea6d 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/drivers/nvdimm/region_devs.c
@@ -90,7 +90,7 @@ static int nd_region_invalidate_memregion(struct nd_region *nd_region)
 		}
 	}
 
-	cpu_cache_invalidate_memregion(IORES_DESC_PERSISTENT_MEMORY);
+	cpu_cache_invalidate_memregion();
 out:
 	for (i = 0; i < nd_region->ndr_mappings; i++) {
 		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
diff --git a/include/linux/memregion.h b/include/linux/memregion.h
index c01321467789..945646bde825 100644
--- a/include/linux/memregion.h
+++ b/include/linux/memregion.h
@@ -26,8 +26,7 @@ static inline void memregion_free(int id)
 
 /**
  * cpu_cache_invalidate_memregion - drop any CPU cached data for
- *     memregions described by @res_desc
- * @res_desc: one of the IORES_DESC_* types
+ *     memregion
  *
  * Perform cache maintenance after a memory event / operation that
  * changes the contents of physical memory in a cache-incoherent manner.
@@ -46,7 +45,7 @@ static inline void memregion_free(int id)
  * the cache maintenance.
  */
 #ifdef CONFIG_ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
-int cpu_cache_invalidate_memregion(int res_desc);
+int cpu_cache_invalidate_memregion(void);
 bool cpu_cache_has_invalidate_memregion(void);
 #else
 static inline bool cpu_cache_has_invalidate_memregion(void)
@@ -54,7 +53,7 @@ static inline bool cpu_cache_has_invalidate_memregion(void)
 	return false;
 }
 
-static inline int cpu_cache_invalidate_memregion(int res_desc)
+static inline int cpu_cache_invalidate_memregion(void)
 {
 	WARN_ON_ONCE("CPU cache invalidation required");
 	return -ENXIO;
-- 
2.48.1

[PATCH v6 2/7] memregion: Support fine grained invalidate by cpu_cache_invalidate_memregion()

[Jonathan Cameron]

From: Yicong Yang <yangyicong@hisilicon.com>

Extend cpu_cache_invalidate_memregion() to support invalidating a
particular range of memory by introducing start and length parameters.
Control of types of invalidation is left for when use cases turn up. For
now everything is Clean and Invalidate.

Where the range is unknown, use the provided cpu_cache_invalidate_all()
helper to act as documentation of intent in a fashion that is clearer than
passing (0, -1) to cpu_cache_invalidate_memregion().

Signed-off-by: Yicong Yang <yangyicong@hisilicon.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Davidlohr Bueso <dave@stgolabs.net>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>

---
v6: No change
v5: Tiny tweaks to patch description for readability.
v4: Add cpu_cache_invalidate_all() helper for the (0, -1) case that
    applies when we don't have the invalidate range so just want to
    invalidate all caches. - (Thanks to Dan Williams for this suggestion).
v3: Rebase on top of previous patch that removed the IO_RESDESC_*
    parameter.
---
 arch/x86/mm/pat/set_memory.c |  2 +-
 drivers/cxl/core/region.c    |  5 ++++-
 drivers/nvdimm/region.c      |  2 +-
 drivers/nvdimm/region_devs.c |  2 +-
 include/linux/memregion.h    | 13 +++++++++++--
 5 files changed, 18 insertions(+), 6 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index 0cfee2544ad4..05e7704f0128 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -368,7 +368,7 @@ bool cpu_cache_has_invalidate_memregion(void)
 }
 EXPORT_SYMBOL_NS_GPL(cpu_cache_has_invalidate_memregion, "DEVMEM");
 
-int cpu_cache_invalidate_memregion(void)
+int cpu_cache_invalidate_memregion(phys_addr_t start, size_t len)
 {
 	if (WARN_ON_ONCE(!cpu_cache_has_invalidate_memregion()))
 		return -ENXIO;
diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index 36489cb086f3..7d0f6f07352f 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -236,7 +236,10 @@ static int cxl_region_invalidate_memregion(struct cxl_region *cxlr)
 		return -ENXIO;
 	}
 
-	cpu_cache_invalidate_memregion();
+	if (!cxlr->params.res)
+		return -ENXIO;
+	cpu_cache_invalidate_memregion(cxlr->params.res->start,
+				       resource_size(cxlr->params.res));
 	return 0;
 }
 
diff --git a/drivers/nvdimm/region.c b/drivers/nvdimm/region.c
index 47e263ecedf7..53567f3ed427 100644
--- a/drivers/nvdimm/region.c
+++ b/drivers/nvdimm/region.c
@@ -110,7 +110,7 @@ static void nd_region_remove(struct device *dev)
 	 * here is ok.
 	 */
 	if (cpu_cache_has_invalidate_memregion())
-		cpu_cache_invalidate_memregion();
+		cpu_cache_invalidate_all();
 }
 
 static int child_notify(struct device *dev, void *data)
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index c375b11aea6d..1220530a23b6 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/drivers/nvdimm/region_devs.c
@@ -90,7 +90,7 @@ static int nd_region_invalidate_memregion(struct nd_region *nd_region)
 		}
 	}
 
-	cpu_cache_invalidate_memregion();
+	cpu_cache_invalidate_all();
 out:
 	for (i = 0; i < nd_region->ndr_mappings; i++) {
 		struct nd_mapping *nd_mapping = &nd_region->mapping[i];
diff --git a/include/linux/memregion.h b/include/linux/memregion.h
index 945646bde825..a55f62cc5266 100644
--- a/include/linux/memregion.h
+++ b/include/linux/memregion.h
@@ -27,6 +27,9 @@ static inline void memregion_free(int id)
 /**
  * cpu_cache_invalidate_memregion - drop any CPU cached data for
  *     memregion
+ * @start: start physical address of the target memory region.
+ * @len: length of the target memory region. -1 for all the regions of
+ *       the target type.
  *
  * Perform cache maintenance after a memory event / operation that
  * changes the contents of physical memory in a cache-incoherent manner.
@@ -45,7 +48,7 @@ static inline void memregion_free(int id)
  * the cache maintenance.
  */
 #ifdef CONFIG_ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
-int cpu_cache_invalidate_memregion(void);
+int cpu_cache_invalidate_memregion(phys_addr_t start, size_t len);
 bool cpu_cache_has_invalidate_memregion(void);
 #else
 static inline bool cpu_cache_has_invalidate_memregion(void)
@@ -53,10 +56,16 @@ static inline bool cpu_cache_has_invalidate_memregion(void)
 	return false;
 }
 
-static inline int cpu_cache_invalidate_memregion(void)
+static inline int cpu_cache_invalidate_memregion(phys_addr_t start, size_t len)
 {
 	WARN_ON_ONCE("CPU cache invalidation required");
 	return -ENXIO;
 }
 #endif
+
+static inline int cpu_cache_invalidate_all(void)
+{
+	return cpu_cache_invalidate_memregion(0, -1);
+}
+
 #endif /* _MEMREGION_H_ */
-- 
2.48.1

[PATCH v6 3/7] lib: Support ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION

[Jonathan Cameron]

From: Yicong Yang <yangyicong@hisilicon.com>

ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION provides the mechanism for
invalidating certain memory regions in a cache-incoherent manner. Currently
this is used by NVDIMM and CXL memory drivers in cases where it is
necessary to flush all data from caches by physical address range.
The operations in question are effectively memory hotplug, where stale
data might otherwise remain in the caches.

This is separate from the invalidates done to enable use of non-coherent
DMA masters, primarily in terms of when it is needed (not related to DMA
mappings) and how deep the flush must push data. The flushes done for
non-coherent DMA only need to reach the Point of Coherence of a single host
(which is often nearer CPUs and DMA masters than the physical storage).
This operation must push the data out of non architectural caches
(memory-side caches, write buffers etc) and typically all the way to the
memory device.

In some architectures these operations are supported by system components
that may become available only later in boot as they are either present
on a discoverable bus, or via a firmware description of an MMIO interface
(e.g. ACPI DSDT). Provide a framework to handle this case.

Architectures can opt in for this support via
CONFIG_GENERIC_CPU_CACHE_MAINTENANCE

Add a registration framework. Each driver provides an ops structure and
the first op is Write Back and Invalidate by PA Range. The driver may
over invalidate.

For systems that can perform this operation asynchronously an optional
completion check operation is also provided. If present that must be called
to ensure that the action has finished. This provides a considerable
performance advantage if multiple agents are involved in the maintenance
operation.

When multiple agents are present in the system each should register with
this framework and the core code will issue the invalidate to all of them
before checking for completion on each. This is done to avoid need for
filtering in the core code which can become complex when interleave,
potentially across different cache coherency hardware is going on, so it
is easier to tell everyone and let those who don't care do nothing.

Signed-off-by: Yicong Yang <yangyicong@hisilicon.com>
Co-developed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>

---
v6: No change.
v5: Picked up ack from Conor. Thanks!
    Add discussion of difference from the existing operations for
    working with non-coherent DMA (Arnd)
    Expand a little on asynchronous nature and why it matters for
    some systems.
---
 include/linux/cache_coherency.h |  61 ++++++++++++++
 lib/Kconfig                     |   4 +
 lib/Makefile                    |   2 +
 lib/cache_maint.c               | 138 ++++++++++++++++++++++++++++++++
 4 files changed, 205 insertions(+)

diff --git a/include/linux/cache_coherency.h b/include/linux/cache_coherency.h
new file mode 100644
index 000000000000..cc81c5733e31
--- /dev/null
+++ b/include/linux/cache_coherency.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Cache coherency maintenance operation device drivers
+ *
+ * Copyright Huawei 2025
+ */
+#ifndef _LINUX_CACHE_COHERENCY_H_
+#define _LINUX_CACHE_COHERENCY_H_
+
+#include <linux/list.h>
+#include <linux/kref.h>
+#include <linux/types.h>
+
+struct cc_inval_params {
+	phys_addr_t addr;
+	size_t size;
+};
+
+struct cache_coherency_ops_inst;
+
+struct cache_coherency_ops {
+	int (*wbinv)(struct cache_coherency_ops_inst *cci,
+		     struct cc_inval_params *invp);
+	int (*done)(struct cache_coherency_ops_inst *cci);
+};
+
+struct cache_coherency_ops_inst {
+	struct kref kref;
+	struct list_head node;
+	const struct cache_coherency_ops *ops;
+};
+
+int cache_coherency_ops_instance_register(struct cache_coherency_ops_inst *cci);
+void cache_coherency_ops_instance_unregister(struct cache_coherency_ops_inst *cci);
+
+struct cache_coherency_ops_inst *
+_cache_coherency_ops_instance_alloc(const struct cache_coherency_ops *ops,
+				    size_t size);
+/**
+ * cache_coherency_ops_instance_alloc - Allocate cache coherency ops instance
+ * @ops: Cache maintenance operations
+ * @drv_struct: structure that contains the struct cache_coherency_ops_inst
+ * @member: Name of the struct cache_coherency_ops_inst member in @drv_struct.
+ *
+ * This allocates a driver specific structure and initializes the
+ * cache_coherency_ops_inst embedded in the drv_struct. Upon success the
+ * pointer must be freed via cache_coherency_ops_instance_put().
+ *
+ * Returns a &drv_struct * on success, %NULL on error.
+ */
+#define cache_coherency_ops_instance_alloc(ops, drv_struct, member)	    \
+	({								    \
+		static_assert(__same_type(struct cache_coherency_ops_inst,  \
+					  ((drv_struct *)NULL)->member));   \
+		static_assert(offsetof(drv_struct, member) == 0);	    \
+		(drv_struct *)_cache_coherency_ops_instance_alloc(ops,	    \
+			sizeof(drv_struct));				    \
+	})
+void cache_coherency_ops_instance_put(struct cache_coherency_ops_inst *cci);
+
+#endif
diff --git a/lib/Kconfig b/lib/Kconfig
index e629449dd2a3..e11136d188ae 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -542,6 +542,10 @@ config MEMREGION
 config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
 	bool
 
+config GENERIC_CPU_CACHE_MAINTENANCE
+	bool
+	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
+
 config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
 	bool
 
diff --git a/lib/Makefile b/lib/Makefile
index 1ab2c4be3b66..aaf677cf4527 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -127,6 +127,8 @@ obj-$(CONFIG_HAS_IOMEM) += iomap_copy.o devres.o
 obj-$(CONFIG_CHECK_SIGNATURE) += check_signature.o
 obj-$(CONFIG_DEBUG_LOCKING_API_SELFTESTS) += locking-selftest.o
 
+obj-$(CONFIG_GENERIC_CPU_CACHE_MAINTENANCE) += cache_maint.o
+
 lib-y += logic_pio.o
 
 lib-$(CONFIG_INDIRECT_IOMEM) += logic_iomem.o
diff --git a/lib/cache_maint.c b/lib/cache_maint.c
new file mode 100644
index 000000000000..9256a9ffc34c
--- /dev/null
+++ b/lib/cache_maint.c
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Generic support for Memory System Cache Maintenance operations.
+ *
+ * Coherency maintenance drivers register with this simple framework that will
+ * iterate over each registered instance to first kick off invalidation and
+ * then to wait until it is complete.
+ *
+ * If no implementations are registered yet cpu_cache_has_invalidate_memregion()
+ * will return false. If this runs concurrently with unregistration then a
+ * race exists but this is no worse than the case where the operations instance
+ * responsible for a given memory region has not yet registered.
+ */
+#include <linux/cache_coherency.h>
+#include <linux/cleanup.h>
+#include <linux/container_of.h>
+#include <linux/export.h>
+#include <linux/kref.h>
+#include <linux/list.h>
+#include <linux/memregion.h>
+#include <linux/module.h>
+#include <linux/rwsem.h>
+#include <linux/slab.h>
+
+static LIST_HEAD(cache_ops_instance_list);
+static DECLARE_RWSEM(cache_ops_instance_list_lock);
+
+static void __cache_coherency_ops_instance_free(struct kref *kref)
+{
+	struct cache_coherency_ops_inst *cci =
+		container_of(kref, struct cache_coherency_ops_inst, kref);
+	kfree(cci);
+}
+
+void cache_coherency_ops_instance_put(struct cache_coherency_ops_inst *cci)
+{
+	kref_put(&cci->kref, __cache_coherency_ops_instance_free);
+}
+EXPORT_SYMBOL_GPL(cache_coherency_ops_instance_put);
+
+static int cache_inval_one(struct cache_coherency_ops_inst *cci, void *data)
+{
+	if (!cci->ops)
+		return -EINVAL;
+
+	return cci->ops->wbinv(cci, data);
+}
+
+static int cache_inval_done_one(struct cache_coherency_ops_inst *cci)
+{
+	if (!cci->ops)
+		return -EINVAL;
+
+	if (!cci->ops->done)
+		return 0;
+
+	return cci->ops->done(cci);
+}
+
+static int cache_invalidate_memregion(phys_addr_t addr, size_t size)
+{
+	int ret;
+	struct cache_coherency_ops_inst *cci;
+	struct cc_inval_params params = {
+		.addr = addr,
+		.size = size,
+	};
+
+	guard(rwsem_read)(&cache_ops_instance_list_lock);
+	list_for_each_entry(cci, &cache_ops_instance_list, node) {
+		ret = cache_inval_one(cci, &params);
+		if (ret)
+			return ret;
+	}
+	list_for_each_entry(cci, &cache_ops_instance_list, node) {
+		ret = cache_inval_done_one(cci);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+struct cache_coherency_ops_inst *
+_cache_coherency_ops_instance_alloc(const struct cache_coherency_ops *ops,
+				    size_t size)
+{
+	struct cache_coherency_ops_inst *cci;
+
+	if (!ops || !ops->wbinv)
+		return NULL;
+
+	cci = kzalloc(size, GFP_KERNEL);
+	if (!cci)
+		return NULL;
+
+	cci->ops = ops;
+	INIT_LIST_HEAD(&cci->node);
+	kref_init(&cci->kref);
+
+	return cci;
+}
+EXPORT_SYMBOL_NS_GPL(_cache_coherency_ops_instance_alloc, "CACHE_COHERENCY");
+
+int cache_coherency_ops_instance_register(struct cache_coherency_ops_inst *cci)
+{
+	guard(rwsem_write)(&cache_ops_instance_list_lock);
+	list_add(&cci->node, &cache_ops_instance_list);
+
+	return 0;
+}
+EXPORT_SYMBOL_NS_GPL(cache_coherency_ops_instance_register, "CACHE_COHERENCY");
+
+void cache_coherency_ops_instance_unregister(struct cache_coherency_ops_inst *cci)
+{
+	guard(rwsem_write)(&cache_ops_instance_list_lock);
+	list_del(&cci->node);
+}
+EXPORT_SYMBOL_NS_GPL(cache_coherency_ops_instance_unregister, "CACHE_COHERENCY");
+
+int cpu_cache_invalidate_memregion(phys_addr_t start, size_t len)
+{
+	return cache_invalidate_memregion(start, len);
+}
+EXPORT_SYMBOL_NS_GPL(cpu_cache_invalidate_memregion, "DEVMEM");
+
+/*
+ * Used for optimization / debug purposes only as removal can race
+ *
+ * Machines that do not support invalidation, e.g. VMs, will not have any
+ * operations instance to register and so this will always return false.
+ */
+bool cpu_cache_has_invalidate_memregion(void)
+{
+	guard(rwsem_read)(&cache_ops_instance_list_lock);
+	return !list_empty(&cache_ops_instance_list);
+}
+EXPORT_SYMBOL_NS_GPL(cpu_cache_has_invalidate_memregion, "DEVMEM");
-- 
2.48.1

Re: [PATCH v6 3/7] lib: Support ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION

[Randy Dunlap]

Hi,

On 11/17/25 2:47 AM, Jonathan Cameron wrote:
> diff --git a/lib/Kconfig b/lib/Kconfig
> index e629449dd2a3..e11136d188ae 100644
> --- a/lib/Kconfig
> +++ b/lib/Kconfig
> @@ -542,6 +542,10 @@ config MEMREGION
>  config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
>  	bool
>  
> +config GENERIC_CPU_CACHE_MAINTENANCE
> +	bool
> +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> +
>  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
>  	bool

Architectures and/or platforms select ARCH_HAS_*.

With this change above, it becomes the only entry in
lib/Kconfig that does "select ARCH_HAS_anytning".

so I think this is wrong, back*wards.

-- 
~Randy

Re: [PATCH v6 3/7] lib: Support ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 2882 bytes --]

On Mon, Nov 17, 2025 at 10:51:11AM -0800, Randy Dunlap wrote:
> Hi,
> 
> On 11/17/25 2:47 AM, Jonathan Cameron wrote:
> > diff --git a/lib/Kconfig b/lib/Kconfig
> > index e629449dd2a3..e11136d188ae 100644
> > --- a/lib/Kconfig
> > +++ b/lib/Kconfig
> > @@ -542,6 +542,10 @@ config MEMREGION
> >  config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> >  	bool
> >  
> > +config GENERIC_CPU_CACHE_MAINTENANCE
> > +	bool
> > +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > +
> >  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
> >  	bool
> 
> Architectures and/or platforms select ARCH_HAS_*.
> 
> With this change above, it becomes the only entry in
> lib/Kconfig that does "select ARCH_HAS_anytning".
> 
> so I think this is wrong, back*wards.

Maybe it is backwards, but I feel like this way is more logical. ARM64
has memregion invalidation only because this generic approach is
enabled, so the arch selects what it needs to get the support.
Alternatively, something like
| diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
| index 5f7f63d24931..75b2507f7eb2 100644
| --- a/arch/arm64/Kconfig
| +++ b/arch/arm64/Kconfig
| @@ -21,6 +21,7 @@ config ARM64
|  	select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
|  	select ARCH_HAS_CACHE_LINE_SIZE
|  	select ARCH_HAS_CC_PLATFORM
| +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
|  	select ARCH_HAS_CURRENT_STACK_POINTER
|  	select ARCH_HAS_DEBUG_VIRTUAL
|  	select ARCH_HAS_DEBUG_VM_PGTABLE
| @@ -146,7 +147,6 @@ config ARM64
|  	select GENERIC_ARCH_TOPOLOGY
|  	select GENERIC_CLOCKEVENTS_BROADCAST
|  	select GENERIC_CPU_AUTOPROBE
| -	select GENERIC_CPU_CACHE_MAINTENANCE
|  	select GENERIC_CPU_DEVICES
|  	select GENERIC_CPU_VULNERABILITIES
|  	select GENERIC_EARLY_IOREMAP
| diff --git a/lib/Kconfig b/lib/Kconfig
| index 09aec4a1e13f..ac223e627bc5 100644
| --- a/lib/Kconfig
| +++ b/lib/Kconfig
| @@ -544,8 +544,9 @@ config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
|  	bool
|  
|  config GENERIC_CPU_CACHE_MAINTENANCE
| -	bool
| -	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
| +	def_bool y
| +	depends on ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
| +	depends on ARM64
|  
|  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
|  	bool
implies (to me at least) that arm64 has memregion invalidation as an
architectural feature and that the GENERIC_CPU_CACHE_MAINTENANCE option
is a just common cross-arch code, like generic entry etc, rather than
being the option gating the drivers that provide the feature in the
first place.

I didn't really care which way it went, and was gonna post something to
squash and avoid another revision, but I found the resultant Kconfig
setup to be make less sense to me than what came before. If the switched
around version is less likely to be problematic etc, then sure, but I
amn't convinced by switching it at a first glance.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v6 3/7] lib: Support ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION

[Jonathan Cameron]

On Tue, 18 Nov 2025 00:13:07 +0000
Conor Dooley <conor@kernel.org> wrote:

> On Mon, Nov 17, 2025 at 10:51:11AM -0800, Randy Dunlap wrote:
> > Hi,
> > 
> > On 11/17/25 2:47 AM, Jonathan Cameron wrote:  
> > > diff --git a/lib/Kconfig b/lib/Kconfig
> > > index e629449dd2a3..e11136d188ae 100644
> > > --- a/lib/Kconfig
> > > +++ b/lib/Kconfig
> > > @@ -542,6 +542,10 @@ config MEMREGION
> > >  config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > >  	bool
> > >  
> > > +config GENERIC_CPU_CACHE_MAINTENANCE
> > > +	bool
> > > +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > > +
> > >  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
> > >  	bool  
> > 
> > Architectures and/or platforms select ARCH_HAS_*.
> > 
> > With this change above, it becomes the only entry in
> > lib/Kconfig that does "select ARCH_HAS_anytning".
> > 
> > so I think this is wrong, back*wards.  
> 
> Maybe it is backwards, but I feel like this way is more logical. ARM64
> has memregion invalidation only because this generic approach is
> enabled, so the arch selects what it needs to get the support.

Exactly this. Catalin requested this form in response to an earlier
version where arm64 Kconfig just had both selects for pretty much that
reason. This is expected to be used on a subset of architectures.
It is similar to things like GENERIC_ARCH_NUMA in this respect (though the
arch_numa_init() etc in there are called only from other arch code
so no ARCH_HAS_ symbols are associated with them).

> Alternatively, something like

I'm fine with this solution if Randy prefers it.

Thanks for your help with this.

Jonathan

> | diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> | index 5f7f63d24931..75b2507f7eb2 100644
> | --- a/arch/arm64/Kconfig
> | +++ b/arch/arm64/Kconfig
> | @@ -21,6 +21,7 @@ config ARM64
> |  	select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
> |  	select ARCH_HAS_CACHE_LINE_SIZE
> |  	select ARCH_HAS_CC_PLATFORM
> | +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> |  	select ARCH_HAS_CURRENT_STACK_POINTER
> |  	select ARCH_HAS_DEBUG_VIRTUAL
> |  	select ARCH_HAS_DEBUG_VM_PGTABLE
> | @@ -146,7 +147,6 @@ config ARM64
> |  	select GENERIC_ARCH_TOPOLOGY
> |  	select GENERIC_CLOCKEVENTS_BROADCAST
> |  	select GENERIC_CPU_AUTOPROBE
> | -	select GENERIC_CPU_CACHE_MAINTENANCE
> |  	select GENERIC_CPU_DEVICES
> |  	select GENERIC_CPU_VULNERABILITIES
> |  	select GENERIC_EARLY_IOREMAP
> | diff --git a/lib/Kconfig b/lib/Kconfig
> | index 09aec4a1e13f..ac223e627bc5 100644
> | --- a/lib/Kconfig
> | +++ b/lib/Kconfig
> | @@ -544,8 +544,9 @@ config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> |  	bool
> |  
> |  config GENERIC_CPU_CACHE_MAINTENANCE
> | -	bool
> | -	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> | +	def_bool y
> | +	depends on ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> | +	depends on ARM64
> |  
> |  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
> |  	bool
> implies (to me at least) that arm64 has memregion invalidation as an
> architectural feature and that the GENERIC_CPU_CACHE_MAINTENANCE option
> is a just common cross-arch code, like generic entry etc, rather than
> being the option gating the drivers that provide the feature in the
> first place.
> 
> I didn't really care which way it went, and was gonna post something to
> squash and avoid another revision, but I found the resultant Kconfig
> setup to be make less sense to me than what came before. If the switched
> around version is less likely to be problematic etc, then sure, but I
> amn't convinced by switching it at a first glance.
> 

Re: [PATCH v6 3/7] lib: Support ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION

[Randy Dunlap]

On 11/18/25 1:30 AM, Jonathan Cameron wrote:
> On Tue, 18 Nov 2025 00:13:07 +0000
> Conor Dooley <conor@kernel.org> wrote:
> 
>> On Mon, Nov 17, 2025 at 10:51:11AM -0800, Randy Dunlap wrote:
>>> Hi,
>>>
>>> On 11/17/25 2:47 AM, Jonathan Cameron wrote:  
>>>> diff --git a/lib/Kconfig b/lib/Kconfig
>>>> index e629449dd2a3..e11136d188ae 100644
>>>> --- a/lib/Kconfig
>>>> +++ b/lib/Kconfig
>>>> @@ -542,6 +542,10 @@ config MEMREGION
>>>>  config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
>>>>  	bool
>>>>  
>>>> +config GENERIC_CPU_CACHE_MAINTENANCE
>>>> +	bool
>>>> +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
>>>> +
>>>>  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
>>>>  	bool  
>>>
>>> Architectures and/or platforms select ARCH_HAS_*.
>>>
>>> With this change above, it becomes the only entry in
>>> lib/Kconfig that does "select ARCH_HAS_anytning".
>>>
>>> so I think this is wrong, back*wards.  
>>
>> Maybe it is backwards, but I feel like this way is more logical. ARM64
>> has memregion invalidation only because this generic approach is
>> enabled, so the arch selects what it needs to get the support.
> 
> Exactly this. Catalin requested this form in response to an earlier
> version where arm64 Kconfig just had both selects for pretty much that
> reason. This is expected to be used on a subset of architectures.
> It is similar to things like GENERIC_ARCH_NUMA in this respect (though the
> arch_numa_init() etc in there are called only from other arch code
> so no ARCH_HAS_ symbols are associated with them).
> 
>> Alternatively, something like
> 
> I'm fine with this solution if Randy prefers it.

I do much prefer this alternative.

> Thanks for your help with this.

Thanks for listening.


>> | diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> | index 5f7f63d24931..75b2507f7eb2 100644
>> | --- a/arch/arm64/Kconfig
>> | +++ b/arch/arm64/Kconfig
>> | @@ -21,6 +21,7 @@ config ARM64
>> |  	select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
>> |  	select ARCH_HAS_CACHE_LINE_SIZE
>> |  	select ARCH_HAS_CC_PLATFORM
>> | +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
>> |  	select ARCH_HAS_CURRENT_STACK_POINTER
>> |  	select ARCH_HAS_DEBUG_VIRTUAL
>> |  	select ARCH_HAS_DEBUG_VM_PGTABLE
>> | @@ -146,7 +147,6 @@ config ARM64
>> |  	select GENERIC_ARCH_TOPOLOGY
>> |  	select GENERIC_CLOCKEVENTS_BROADCAST
>> |  	select GENERIC_CPU_AUTOPROBE
>> | -	select GENERIC_CPU_CACHE_MAINTENANCE
>> |  	select GENERIC_CPU_DEVICES
>> |  	select GENERIC_CPU_VULNERABILITIES
>> |  	select GENERIC_EARLY_IOREMAP
>> | diff --git a/lib/Kconfig b/lib/Kconfig
>> | index 09aec4a1e13f..ac223e627bc5 100644
>> | --- a/lib/Kconfig
>> | +++ b/lib/Kconfig
>> | @@ -544,8 +544,9 @@ config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
>> |  	bool
>> |  
>> |  config GENERIC_CPU_CACHE_MAINTENANCE
>> | -	bool
>> | -	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
>> | +	def_bool y
>> | +	depends on ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
>> | +	depends on ARM64
>> |  
>> |  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
>> |  	bool
>> implies (to me at least) that arm64 has memregion invalidation as an
>> architectural feature and that the GENERIC_CPU_CACHE_MAINTENANCE option
>> is a just common cross-arch code, like generic entry etc, rather than
>> being the option gating the drivers that provide the feature in the
>> first place.
>>
>> I didn't really care which way it went, and was gonna post something to
>> squash and avoid another revision, but I found the resultant Kconfig
>> setup to be make less sense to me than what came before. If the switched
>> around version is less likely to be problematic etc, then sure, but I
>> amn't convinced by switching it at a first glance.


-- 
~Randy

Re: [PATCH v6 3/7] lib: Support ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION

[Jonathan Cameron]

On Tue, 18 Nov 2025 17:18:31 -0800
Randy Dunlap <rdunlap@infradead.org> wrote:

> On 11/18/25 1:30 AM, Jonathan Cameron wrote:
> > On Tue, 18 Nov 2025 00:13:07 +0000
> > Conor Dooley <conor@kernel.org> wrote:
> >   
> >> On Mon, Nov 17, 2025 at 10:51:11AM -0800, Randy Dunlap wrote:  
> >>> Hi,
> >>>
> >>> On 11/17/25 2:47 AM, Jonathan Cameron wrote:    
> >>>> diff --git a/lib/Kconfig b/lib/Kconfig
> >>>> index e629449dd2a3..e11136d188ae 100644
> >>>> --- a/lib/Kconfig
> >>>> +++ b/lib/Kconfig
> >>>> @@ -542,6 +542,10 @@ config MEMREGION
> >>>>  config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> >>>>  	bool
> >>>>  
> >>>> +config GENERIC_CPU_CACHE_MAINTENANCE
> >>>> +	bool
> >>>> +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> >>>> +
> >>>>  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
> >>>>  	bool    
> >>>
> >>> Architectures and/or platforms select ARCH_HAS_*.
> >>>
> >>> With this change above, it becomes the only entry in
> >>> lib/Kconfig that does "select ARCH_HAS_anytning".
> >>>
> >>> so I think this is wrong, back*wards.    
> >>
> >> Maybe it is backwards, but I feel like this way is more logical. ARM64
> >> has memregion invalidation only because this generic approach is
> >> enabled, so the arch selects what it needs to get the support.  
> > 
> > Exactly this. Catalin requested this form in response to an earlier
> > version where arm64 Kconfig just had both selects for pretty much that
> > reason. This is expected to be used on a subset of architectures.
> > It is similar to things like GENERIC_ARCH_NUMA in this respect (though the
> > arch_numa_init() etc in there are called only from other arch code
> > so no ARCH_HAS_ symbols are associated with them).
> >   
> >> Alternatively, something like  
> > 
> > I'm fine with this solution if Randy prefers it.  
> 
> I do much prefer this alternative.
> 
> > Thanks for your help with this.  
> 
> Thanks for listening.

Conor,

Given it is your proposed solution, I'm guessing you'll either spin a patch
on top or squash it into original.  If you spin a patch for this.

Acked-by: Jonathan Cameron <jonathan.cameron@huawei.com>

Thanks again!

Jonathan

> 
> 
> >> | diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> >> | index 5f7f63d24931..75b2507f7eb2 100644
> >> | --- a/arch/arm64/Kconfig
> >> | +++ b/arch/arm64/Kconfig
> >> | @@ -21,6 +21,7 @@ config ARM64
> >> |  	select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
> >> |  	select ARCH_HAS_CACHE_LINE_SIZE
> >> |  	select ARCH_HAS_CC_PLATFORM
> >> | +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> >> |  	select ARCH_HAS_CURRENT_STACK_POINTER
> >> |  	select ARCH_HAS_DEBUG_VIRTUAL
> >> |  	select ARCH_HAS_DEBUG_VM_PGTABLE
> >> | @@ -146,7 +147,6 @@ config ARM64
> >> |  	select GENERIC_ARCH_TOPOLOGY
> >> |  	select GENERIC_CLOCKEVENTS_BROADCAST
> >> |  	select GENERIC_CPU_AUTOPROBE
> >> | -	select GENERIC_CPU_CACHE_MAINTENANCE
> >> |  	select GENERIC_CPU_DEVICES
> >> |  	select GENERIC_CPU_VULNERABILITIES
> >> |  	select GENERIC_EARLY_IOREMAP
> >> | diff --git a/lib/Kconfig b/lib/Kconfig
> >> | index 09aec4a1e13f..ac223e627bc5 100644
> >> | --- a/lib/Kconfig
> >> | +++ b/lib/Kconfig
> >> | @@ -544,8 +544,9 @@ config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> >> |  	bool
> >> |  
> >> |  config GENERIC_CPU_CACHE_MAINTENANCE
> >> | -	bool
> >> | -	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> >> | +	def_bool y
> >> | +	depends on ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> >> | +	depends on ARM64
> >> |  
> >> |  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
> >> |  	bool
> >> implies (to me at least) that arm64 has memregion invalidation as an
> >> architectural feature and that the GENERIC_CPU_CACHE_MAINTENANCE option
> >> is a just common cross-arch code, like generic entry etc, rather than
> >> being the option gating the drivers that provide the feature in the
> >> first place.
> >>
> >> I didn't really care which way it went, and was gonna post something to
> >> squash and avoid another revision, but I found the resultant Kconfig
> >> setup to be make less sense to me than what came before. If the switched
> >> around version is less likely to be problematic etc, then sure, but I
> >> amn't convinced by switching it at a first glance.  
> 
> 

Re: [PATCH v6 3/7] lib: Support ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 4739 bytes --]

On Wed, Nov 19, 2025 at 09:42:55AM +0000, Jonathan Cameron wrote:
> On Tue, 18 Nov 2025 17:18:31 -0800
> Randy Dunlap <rdunlap@infradead.org> wrote:
> 
> > On 11/18/25 1:30 AM, Jonathan Cameron wrote:
> > > On Tue, 18 Nov 2025 00:13:07 +0000
> > > Conor Dooley <conor@kernel.org> wrote:
> > >   
> > >> On Mon, Nov 17, 2025 at 10:51:11AM -0800, Randy Dunlap wrote:  
> > >>> Hi,
> > >>>
> > >>> On 11/17/25 2:47 AM, Jonathan Cameron wrote:    
> > >>>> diff --git a/lib/Kconfig b/lib/Kconfig
> > >>>> index e629449dd2a3..e11136d188ae 100644
> > >>>> --- a/lib/Kconfig
> > >>>> +++ b/lib/Kconfig
> > >>>> @@ -542,6 +542,10 @@ config MEMREGION
> > >>>>  config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > >>>>  	bool
> > >>>>  
> > >>>> +config GENERIC_CPU_CACHE_MAINTENANCE
> > >>>> +	bool
> > >>>> +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > >>>> +
> > >>>>  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
> > >>>>  	bool    
> > >>>
> > >>> Architectures and/or platforms select ARCH_HAS_*.
> > >>>
> > >>> With this change above, it becomes the only entry in
> > >>> lib/Kconfig that does "select ARCH_HAS_anytning".
> > >>>
> > >>> so I think this is wrong, back*wards.    
> > >>
> > >> Maybe it is backwards, but I feel like this way is more logical. ARM64
> > >> has memregion invalidation only because this generic approach is
> > >> enabled, so the arch selects what it needs to get the support.  
> > > 
> > > Exactly this. Catalin requested this form in response to an earlier
> > > version where arm64 Kconfig just had both selects for pretty much that
> > > reason. This is expected to be used on a subset of architectures.
> > > It is similar to things like GENERIC_ARCH_NUMA in this respect (though the
> > > arch_numa_init() etc in there are called only from other arch code
> > > so no ARCH_HAS_ symbols are associated with them).
> > >   
> > >> Alternatively, something like  
> > > 
> > > I'm fine with this solution if Randy prefers it.  
> > 
> > I do much prefer this alternative.
> > 
> > > Thanks for your help with this.  
> > 
> > Thanks for listening.
> 
> Conor,
> 
> Given it is your proposed solution, I'm guessing you'll either spin a patch
> on top or squash it into original.  If you spin a patch for this.
> 
> Acked-by: Jonathan Cameron <jonathan.cameron@huawei.com>

New patch I think, since you say Catalin specifically asked for the
current setup.

> 
> Thanks again!
> 
> Jonathan
> 
> > 
> > 
> > >> | diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > >> | index 5f7f63d24931..75b2507f7eb2 100644
> > >> | --- a/arch/arm64/Kconfig
> > >> | +++ b/arch/arm64/Kconfig
> > >> | @@ -21,6 +21,7 @@ config ARM64
> > >> |  	select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
> > >> |  	select ARCH_HAS_CACHE_LINE_SIZE
> > >> |  	select ARCH_HAS_CC_PLATFORM
> > >> | +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > >> |  	select ARCH_HAS_CURRENT_STACK_POINTER
> > >> |  	select ARCH_HAS_DEBUG_VIRTUAL
> > >> |  	select ARCH_HAS_DEBUG_VM_PGTABLE
> > >> | @@ -146,7 +147,6 @@ config ARM64
> > >> |  	select GENERIC_ARCH_TOPOLOGY
> > >> |  	select GENERIC_CLOCKEVENTS_BROADCAST
> > >> |  	select GENERIC_CPU_AUTOPROBE
> > >> | -	select GENERIC_CPU_CACHE_MAINTENANCE
> > >> |  	select GENERIC_CPU_DEVICES
> > >> |  	select GENERIC_CPU_VULNERABILITIES
> > >> |  	select GENERIC_EARLY_IOREMAP
> > >> | diff --git a/lib/Kconfig b/lib/Kconfig
> > >> | index 09aec4a1e13f..ac223e627bc5 100644
> > >> | --- a/lib/Kconfig
> > >> | +++ b/lib/Kconfig
> > >> | @@ -544,8 +544,9 @@ config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > >> |  	bool
> > >> |  
> > >> |  config GENERIC_CPU_CACHE_MAINTENANCE
> > >> | -	bool
> > >> | -	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > >> | +	def_bool y
> > >> | +	depends on ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > >> | +	depends on ARM64
> > >> |  
> > >> |  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
> > >> |  	bool
> > >> implies (to me at least) that arm64 has memregion invalidation as an
> > >> architectural feature and that the GENERIC_CPU_CACHE_MAINTENANCE option
> > >> is a just common cross-arch code, like generic entry etc, rather than
> > >> being the option gating the drivers that provide the feature in the
> > >> first place.
> > >>
> > >> I didn't really care which way it went, and was gonna post something to
> > >> squash and avoid another revision, but I found the resultant Kconfig
> > >> setup to be make less sense to me than what came before. If the switched
> > >> around version is less likely to be problematic etc, then sure, but I
> > >> amn't convinced by switching it at a first glance.  
> > 
> > 
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v6 3/7] lib: Support ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION

[Jonathan Cameron]

On Wed, 19 Nov 2025 18:44:43 +0000
Conor Dooley <conor@kernel.org> wrote:

> On Wed, Nov 19, 2025 at 09:42:55AM +0000, Jonathan Cameron wrote:
> > On Tue, 18 Nov 2025 17:18:31 -0800
> > Randy Dunlap <rdunlap@infradead.org> wrote:
> >   
> > > On 11/18/25 1:30 AM, Jonathan Cameron wrote:  
> > > > On Tue, 18 Nov 2025 00:13:07 +0000
> > > > Conor Dooley <conor@kernel.org> wrote:
> > > >     
> > > >> On Mon, Nov 17, 2025 at 10:51:11AM -0800, Randy Dunlap wrote:    
> > > >>> Hi,
> > > >>>
> > > >>> On 11/17/25 2:47 AM, Jonathan Cameron wrote:      
> > > >>>> diff --git a/lib/Kconfig b/lib/Kconfig
> > > >>>> index e629449dd2a3..e11136d188ae 100644
> > > >>>> --- a/lib/Kconfig
> > > >>>> +++ b/lib/Kconfig
> > > >>>> @@ -542,6 +542,10 @@ config MEMREGION
> > > >>>>  config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > > >>>>  	bool
> > > >>>>  
> > > >>>> +config GENERIC_CPU_CACHE_MAINTENANCE
> > > >>>> +	bool
> > > >>>> +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > > >>>> +
> > > >>>>  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
> > > >>>>  	bool      
> > > >>>
> > > >>> Architectures and/or platforms select ARCH_HAS_*.
> > > >>>
> > > >>> With this change above, it becomes the only entry in
> > > >>> lib/Kconfig that does "select ARCH_HAS_anytning".
> > > >>>
> > > >>> so I think this is wrong, back*wards.      
> > > >>
> > > >> Maybe it is backwards, but I feel like this way is more logical. ARM64
> > > >> has memregion invalidation only because this generic approach is
> > > >> enabled, so the arch selects what it needs to get the support.    
> > > > 
> > > > Exactly this. Catalin requested this form in response to an earlier
> > > > version where arm64 Kconfig just had both selects for pretty much that
> > > > reason. This is expected to be used on a subset of architectures.
> > > > It is similar to things like GENERIC_ARCH_NUMA in this respect (though the
> > > > arch_numa_init() etc in there are called only from other arch code
> > > > so no ARCH_HAS_ symbols are associated with them).
> > > >     
> > > >> Alternatively, something like    
> > > > 
> > > > I'm fine with this solution if Randy prefers it.    
> > > 
> > > I do much prefer this alternative.
> > >   
> > > > Thanks for your help with this.    
> > > 
> > > Thanks for listening.  
> > 
> > Conor,
> > 
> > Given it is your proposed solution, I'm guessing you'll either spin a patch
> > on top or squash it into original.  If you spin a patch for this.
> > 
> > Acked-by: Jonathan Cameron <jonathan.cameron@huawei.com>  
> 
> New patch I think, since you say Catalin specifically asked for the
> current setup.

Whilst Catalin proposed the form used, the basis was about having two
entrees for effectively the same thing in arm64/Kconfig and this
still has the single entry property so should be fine.

Jonathan

> 
> > 
> > Thanks again!
> > 
> > Jonathan
> >   
> > > 
> > >   
> > > >> | diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > > >> | index 5f7f63d24931..75b2507f7eb2 100644
> > > >> | --- a/arch/arm64/Kconfig
> > > >> | +++ b/arch/arm64/Kconfig
> > > >> | @@ -21,6 +21,7 @@ config ARM64
> > > >> |  	select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
> > > >> |  	select ARCH_HAS_CACHE_LINE_SIZE
> > > >> |  	select ARCH_HAS_CC_PLATFORM
> > > >> | +	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > > >> |  	select ARCH_HAS_CURRENT_STACK_POINTER
> > > >> |  	select ARCH_HAS_DEBUG_VIRTUAL
> > > >> |  	select ARCH_HAS_DEBUG_VM_PGTABLE
> > > >> | @@ -146,7 +147,6 @@ config ARM64
> > > >> |  	select GENERIC_ARCH_TOPOLOGY
> > > >> |  	select GENERIC_CLOCKEVENTS_BROADCAST
> > > >> |  	select GENERIC_CPU_AUTOPROBE
> > > >> | -	select GENERIC_CPU_CACHE_MAINTENANCE
> > > >> |  	select GENERIC_CPU_DEVICES
> > > >> |  	select GENERIC_CPU_VULNERABILITIES
> > > >> |  	select GENERIC_EARLY_IOREMAP
> > > >> | diff --git a/lib/Kconfig b/lib/Kconfig
> > > >> | index 09aec4a1e13f..ac223e627bc5 100644
> > > >> | --- a/lib/Kconfig
> > > >> | +++ b/lib/Kconfig
> > > >> | @@ -544,8 +544,9 @@ config ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > > >> |  	bool
> > > >> |  
> > > >> |  config GENERIC_CPU_CACHE_MAINTENANCE
> > > >> | -	bool
> > > >> | -	select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > > >> | +	def_bool y
> > > >> | +	depends on ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
> > > >> | +	depends on ARM64
> > > >> |  
> > > >> |  config ARCH_HAS_MEMREMAP_COMPAT_ALIGN
> > > >> |  	bool
> > > >> implies (to me at least) that arm64 has memregion invalidation as an
> > > >> architectural feature and that the GENERIC_CPU_CACHE_MAINTENANCE option
> > > >> is a just common cross-arch code, like generic entry etc, rather than
> > > >> being the option gating the drivers that provide the feature in the
> > > >> first place.
> > > >>
> > > >> I didn't really care which way it went, and was gonna post something to
> > > >> squash and avoid another revision, but I found the resultant Kconfig
> > > >> setup to be make less sense to me than what came before. If the switched
> > > >> around version is less likely to be problematic etc, then sure, but I
> > > >> amn't convinced by switching it at a first glance.    
> > > 
> > >   
> >   
> 

[PATCH v6 4/7] arm64: Select GENERIC_CPU_CACHE_MAINTENANCE

[Jonathan Cameron]

The generic CPU cache maintenance framework provides a way to register
drivers for devices implementing the underlying support for
cpu_cache_has_invalidate_memregion(). Enable it for arm64 by selecting
GENERIC_CPU_CACHE_MAINTENANCE which provides the implementation for,
and in turn selects, ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
---
v6: No change.
v5: No change.
v4: Drop select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION as that
    is now selected by GENERIC_CPU_CACHE_MAINTENANCE (Catalin Marinas)
    Picked up tag from Catalin. (thanks!)
---
 arch/arm64/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 6663ffd23f25..893e0af0bc51 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -149,6 +149,7 @@ config ARM64
 	select GENERIC_ARCH_TOPOLOGY
 	select GENERIC_CLOCKEVENTS_BROADCAST
 	select GENERIC_CPU_AUTOPROBE
+	select GENERIC_CPU_CACHE_MAINTENANCE
 	select GENERIC_CPU_DEVICES
 	select GENERIC_CPU_VULNERABILITIES
 	select GENERIC_EARLY_IOREMAP
-- 
2.48.1

[PATCH v6 5/7] MAINTAINERS: Add Jonathan Cameron to drivers/cache and add lib/cache_maint.c + header

[Jonathan Cameron]

Seems unfair to inflict the cache-coherency drivers on Conor with out also
stepping up as a second maintainer for drivers/cache.

Include the library support for cache-coherency maintenance drivers to the
existing entry.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
---
v6: No change.
v5: No change.
v4: Tag from Conor (thanks!)  Updated commit message to make it
    reflect the added files.
---
 MAINTAINERS | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 46126ce2f968..b517f2703615 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -24417,10 +24417,13 @@ F:	drivers/staging/
 
 STANDALONE CACHE CONTROLLER DRIVERS
 M:	Conor Dooley <conor@kernel.org>
+M:	Jonathan Cameron <jonathan.cameron@huawei.com>
 S:	Maintained
 T:	git https://git.kernel.org/pub/scm/linux/kernel/git/conor/linux.git/
 F:	Documentation/devicetree/bindings/cache/
 F:	drivers/cache
+F:	include/cache_coherency.h
+F:	lib/cache_maint.c
 
 STARFIRE/DURALAN NETWORK DRIVER
 M:	Ion Badulescu <ionut@badula.org>
-- 
2.48.1

[PATCH v6 6/7] cache: Make top level Kconfig menu a boolean dependent on RISCV

[Jonathan Cameron]

The next patch will add a new type of cache maintenance driver responsible
for flushing deeper than is necessary for non coherent DMA (current
use case of drivers/cache drivers), as needed when performing operations
such as memory hotplug and security unlocking of persistent memory. The two
types of operation are similar enough to share a drivers/cache directory
and MAINTAINERS but are otherwise currently unrelated.

To avoid confusion have two separate menus. Each has dependencies that are
implemented by making them boolean symbols, here CACHEMAINT_FOR_DMA
which is dependent on RISCV as all driver are currently for platforms of
that architecture. Set new symbol default to y to avoid breaking existing
configs. This has no affect on actual code built, just visibility of the
menu.

Suggested-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
v6: New patch as suggested by Arnd.
---
 drivers/cache/Kconfig | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/drivers/cache/Kconfig b/drivers/cache/Kconfig
index db51386c663a..59a79df4c0ce 100644
--- a/drivers/cache/Kconfig
+++ b/drivers/cache/Kconfig
@@ -1,9 +1,17 @@
 # SPDX-License-Identifier: GPL-2.0
-menu "Cache Drivers"
+
+menuconfig CACHEMAINT_FOR_DMA
+	bool "Cache management for noncoherent DMA"
+	depends on RISCV
+	default y
+	help
+	  These drivers implement support for noncoherent DMA master devices
+	  on platforms that lack the standard CPU interfaces for this.
+
+if CACHEMAINT_FOR_DMA
 
 config AX45MP_L2_CACHE
 	bool "Andes Technology AX45MP L2 Cache controller"
-	depends on RISCV
 	select RISCV_NONSTANDARD_CACHE_OPS
 	help
 	  Support for the L2 cache controller on Andes Technology AX45MP platforms.
@@ -16,7 +24,6 @@ config SIFIVE_CCACHE
 
 config STARFIVE_STARLINK_CACHE
 	bool "StarFive StarLink Cache controller"
-	depends on RISCV
 	depends on ARCH_STARFIVE
 	depends on 64BIT
 	select RISCV_DMA_NONCOHERENT
@@ -24,4 +31,4 @@ config STARFIVE_STARLINK_CACHE
 	help
 	  Support for the StarLink cache controller IP from StarFive.
 
-endmenu
+endif #CACHEMAINT_FOR_DMA
-- 
2.48.1

[PATCH v6 7/7] cache: Support cache maintenance for HiSilicon SoC Hydra Home Agent

[Jonathan Cameron]

From: Yushan Wang <wangyushan12@huawei.com>

Hydra Home Agent is a device used to maintain cache coherency. Add support
for explicit cache maintenance operations using it. A system has multiple
of these agents. Whilst only one agent is responsible for a given cache
line, interleave means that for a range operation, responsibility for the
cache lines making up the range will typically be spread across multiple
instances.

Put this driver on a new Kconfig menu under drivers/cache. The short
description as memory hotplug like operations is intended to cover
the somewhat complex set of cases where this unit applies and differentiate
it clearly from typical non coherent DMA flows.

Co-developed-by: Yicong Yang <yangyicong@hisilicon.com>
Signed-off-by: Yicong Yang <yangyicong@hisilicon.com>
Signed-off-by: Yushan Wang <wangyushan12@huawei.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>

---
v6: New menu as suggested by Arnd. The description and naming may want
    updating as more drivers are upstreamed and perhaps the usecase
    based description is not sufficient. (Thanks Arnd!)
v5: Drop stale comment on devm_ioremap_resource() as it is no longer used.
    Also drop mention from patch description (Conor)

    Add some overview comments to top of driver and improve comment on
    searching for a cacheline to be explicit about what happens if it is
    not found (as not in scope for device, or happens not to be in any
    caches). (Conor)
    Also update the commit message to make it clear there are always
    multiple instances of this unit in a system (Conor)

v4: Update for naming changes around device / instance.
    Switch to kref put based freeing via helper.
---
 drivers/cache/Kconfig        |  22 ++++
 drivers/cache/Makefile       |   2 +
 drivers/cache/hisi_soc_hha.c | 194 +++++++++++++++++++++++++++++++++++
 3 files changed, 218 insertions(+)

diff --git a/drivers/cache/Kconfig b/drivers/cache/Kconfig
index 59a79df4c0ce..1518449d47b5 100644
--- a/drivers/cache/Kconfig
+++ b/drivers/cache/Kconfig
@@ -32,3 +32,25 @@ config STARFIVE_STARLINK_CACHE
 	  Support for the StarLink cache controller IP from StarFive.
 
 endif #CACHEMAINT_FOR_DMA
+
+menuconfig CACHEMAINT_FOR_HOTPLUG
+	bool "Cache management for memory hot plug like operations"
+	depends on GENERIC_CPU_CACHE_MAINTENANCE
+	help
+	  These drivers implement cache management for flows where it is necessary
+	  to flush data from all host caches.
+
+if CACHEMAINT_FOR_HOTPLUG
+
+config HISI_SOC_HHA
+	tristate "HiSilicon Hydra Home Agent (HHA) device driver"
+	depends on (ARM64 && ACPI) || COMPILE_TEST
+	help
+	  The Hydra Home Agent (HHA) is responsible for cache coherency
+	  on the SoC. This drivers enables the cache maintenance functions of
+	  the HHA.
+
+	  This driver can be built as a module. If so, the module will be
+	  called hisi_soc_hha.
+
+endif #CACHEMAINT_FOR_HOTPLUG
diff --git a/drivers/cache/Makefile b/drivers/cache/Makefile
index 55c5e851034d..b3362b15d6c1 100644
--- a/drivers/cache/Makefile
+++ b/drivers/cache/Makefile
@@ -3,3 +3,5 @@
 obj-$(CONFIG_AX45MP_L2_CACHE)		+= ax45mp_cache.o
 obj-$(CONFIG_SIFIVE_CCACHE)		+= sifive_ccache.o
 obj-$(CONFIG_STARFIVE_STARLINK_CACHE)	+= starfive_starlink_cache.o
+
+obj-$(CONFIG_HISI_SOC_HHA)		+= hisi_soc_hha.o
diff --git a/drivers/cache/hisi_soc_hha.c b/drivers/cache/hisi_soc_hha.c
new file mode 100644
index 000000000000..25ff0f5ae79b
--- /dev/null
+++ b/drivers/cache/hisi_soc_hha.c
@@ -0,0 +1,194 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Driver for HiSilicon Hydra Home Agent (HHA).
+ *
+ * Copyright (c) 2025 HiSilicon Technologies Co., Ltd.
+ * Author: Yicong Yang <yangyicong@hisilicon.com>
+ *         Yushan Wang <wangyushan12@huawei.com>
+ *
+ * A system typically contains multiple HHAs. Each is responsible for a subset
+ * of the physical addresses in the system, but interleave can make the mapping
+ * from a particular cache line to a responsible HHA complex. As such no
+ * filtering is done in the driver, with the hardware being responsible for
+ * responding with success for even if it was not responsible for any addresses
+ * in the range on which the operation was requested.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/cache_coherency.h>
+#include <linux/dev_printk.h>
+#include <linux/init.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/kernel.h>
+#include <linux/memregion.h>
+#include <linux/module.h>
+#include <linux/mod_devicetable.h>
+#include <linux/mutex.h>
+#include <linux/platform_device.h>
+
+#define HISI_HHA_CTRL		0x5004
+#define   HISI_HHA_CTRL_EN	BIT(0)
+#define   HISI_HHA_CTRL_RANGE	BIT(1)
+#define   HISI_HHA_CTRL_TYPE	GENMASK(3, 2)
+#define HISI_HHA_START_L	0x5008
+#define HISI_HHA_START_H	0x500c
+#define HISI_HHA_LEN_L		0x5010
+#define HISI_HHA_LEN_H		0x5014
+
+/* The maintain operation performs in a 128 Byte granularity */
+#define HISI_HHA_MAINT_ALIGN	128
+
+#define HISI_HHA_POLL_GAP_US		10
+#define HISI_HHA_POLL_TIMEOUT_US	50000
+
+struct hisi_soc_hha {
+	/* Must be first element */
+	struct cache_coherency_ops_inst cci;
+	/* Locks HHA instance to forbid overlapping access. */
+	struct mutex lock;
+	void __iomem *base;
+};
+
+static bool hisi_hha_cache_maintain_wait_finished(struct hisi_soc_hha *soc_hha)
+{
+	u32 val;
+
+	return !readl_poll_timeout_atomic(soc_hha->base + HISI_HHA_CTRL, val,
+					  !(val & HISI_HHA_CTRL_EN),
+					  HISI_HHA_POLL_GAP_US,
+					  HISI_HHA_POLL_TIMEOUT_US);
+}
+
+static int hisi_soc_hha_wbinv(struct cache_coherency_ops_inst *cci,
+			struct cc_inval_params *invp)
+{
+	struct hisi_soc_hha *soc_hha =
+		container_of(cci, struct hisi_soc_hha, cci);
+	phys_addr_t top, addr = invp->addr;
+	size_t size = invp->size;
+	u32 reg;
+
+	if (!size)
+		return -EINVAL;
+
+	addr = ALIGN_DOWN(addr, HISI_HHA_MAINT_ALIGN);
+	top = ALIGN(addr + size, HISI_HHA_MAINT_ALIGN);
+	size = top - addr;
+
+	guard(mutex)(&soc_hha->lock);
+
+	if (!hisi_hha_cache_maintain_wait_finished(soc_hha))
+		return -EBUSY;
+
+	/*
+	 * Hardware will search for addresses ranging [addr, addr + size - 1],
+	 * last byte included, and perform maintenance in 128 byte granules
+	 * on those cachelines which contain the addresses. If a given instance
+	 * is either not responsible for a cacheline or that cacheline is not
+	 * currently present then the search will fail, no operation will be
+	 * necessary and the device will report success.
+	 */
+	size -= 1;
+
+	writel(lower_32_bits(addr), soc_hha->base + HISI_HHA_START_L);
+	writel(upper_32_bits(addr), soc_hha->base + HISI_HHA_START_H);
+	writel(lower_32_bits(size), soc_hha->base + HISI_HHA_LEN_L);
+	writel(upper_32_bits(size), soc_hha->base + HISI_HHA_LEN_H);
+
+	reg = FIELD_PREP(HISI_HHA_CTRL_TYPE, 1); /* Clean Invalid */
+	reg |= HISI_HHA_CTRL_RANGE | HISI_HHA_CTRL_EN;
+	writel(reg, soc_hha->base + HISI_HHA_CTRL);
+
+	return 0;
+}
+
+static int hisi_soc_hha_done(struct cache_coherency_ops_inst *cci)
+{
+	struct hisi_soc_hha *soc_hha =
+		container_of(cci, struct hisi_soc_hha, cci);
+
+	guard(mutex)(&soc_hha->lock);
+	if (!hisi_hha_cache_maintain_wait_finished(soc_hha))
+		return -ETIMEDOUT;
+
+	return 0;
+}
+
+static const struct cache_coherency_ops hha_ops = {
+	.wbinv = hisi_soc_hha_wbinv,
+	.done = hisi_soc_hha_done,
+};
+
+static int hisi_soc_hha_probe(struct platform_device *pdev)
+{
+	struct hisi_soc_hha *soc_hha;
+	struct resource *mem;
+	int ret;
+
+	soc_hha = cache_coherency_ops_instance_alloc(&hha_ops,
+						     struct hisi_soc_hha, cci);
+	if (!soc_hha)
+		return -ENOMEM;
+
+	platform_set_drvdata(pdev, soc_hha);
+
+	mutex_init(&soc_hha->lock);
+
+	mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+	if (!mem) {
+		ret = -ENOMEM;
+		goto err_free_cci;
+	}
+
+	soc_hha->base = ioremap(mem->start, resource_size(mem));
+	if (!soc_hha->base) {
+		ret = dev_err_probe(&pdev->dev, -ENOMEM,
+				    "failed to remap io memory");
+		goto err_free_cci;
+	}
+
+	ret = cache_coherency_ops_instance_register(&soc_hha->cci);
+	if (ret)
+		goto err_iounmap;
+
+	return 0;
+
+err_iounmap:
+	iounmap(soc_hha->base);
+err_free_cci:
+	cache_coherency_ops_instance_put(&soc_hha->cci);
+	return ret;
+}
+
+static void hisi_soc_hha_remove(struct platform_device *pdev)
+{
+	struct hisi_soc_hha *soc_hha = platform_get_drvdata(pdev);
+
+	cache_coherency_ops_instance_unregister(&soc_hha->cci);
+	iounmap(soc_hha->base);
+	cache_coherency_ops_instance_put(&soc_hha->cci);
+}
+
+static const struct acpi_device_id hisi_soc_hha_ids[] = {
+	{ "HISI0511", },
+	{ }
+};
+MODULE_DEVICE_TABLE(acpi, hisi_soc_hha_ids);
+
+static struct platform_driver hisi_soc_hha_driver = {
+	.driver = {
+		.name = "hisi_soc_hha",
+		.acpi_match_table = hisi_soc_hha_ids,
+	},
+	.probe = hisi_soc_hha_probe,
+	.remove = hisi_soc_hha_remove,
+};
+
+module_platform_driver(hisi_soc_hha_driver);
+
+MODULE_IMPORT_NS("CACHE_COHERENCY");
+MODULE_DESCRIPTION("HiSilicon Hydra Home Agent driver supporting cache maintenance");
+MODULE_AUTHOR("Yicong Yang <yangyicong@hisilicon.com>");
+MODULE_AUTHOR("Yushan Wang <wangyushan12@huawei.com>");
+MODULE_LICENSE("GPL");
-- 
2.48.1