From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F91FC7115A for ; Thu, 19 Jun 2025 10:11:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B8FD6B00A7; Thu, 19 Jun 2025 06:11:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 96A786B00B3; Thu, 19 Jun 2025 06:11:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 832906B00B5; Thu, 19 Jun 2025 06:11:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6F7A96B00A7 for ; Thu, 19 Jun 2025 06:11:20 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id C8F7680A8B for ; Thu, 19 Jun 2025 10:11:19 +0000 (UTC) X-FDA: 83571732678.22.B4E2502 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by imf13.hostedemail.com (Postfix) with ESMTP id 05E0020007 for ; Thu, 19 Jun 2025 10:11:16 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750327878; a=rsa-sha256; cv=none; b=JaqE15XxKciph9U0iKEiiQLIuj6NiYtF6B2+z4D+J9ztNNGORTstqR/UtmDgSBLS7QCq05 y8GAa8QMmnDdMUSI0Zcyi1Mbz6AjRzM8S+k5zHHtZEtW6vuMy2Y1OZR5xadqw0P2Bl+PZI ftA5njFUwRgP5w4svrSj/lUk+89p4NA= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750327878; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nELm232dDm4tAnudTCpWfY09mL5kRGkhcYowEz1XUUQ=; b=pbQ5AQAwwv7uyXxZ8DHJ1I3eJk5mS415fEjxbRXanwUvBM9qwuZhQNuCE06idwByM5LZjF PxtxmI0o6DM8gugg4CA38kHvg6QgNzPZDmP2p5FU+AcGvpuXbHXPceXtOIpscGP8wTGDfc ltEDmkjXwSAycu7ZiTzIe5gEFUmNjFo= Received: from mail.maildlp.com (unknown [172.18.186.231]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4bNGTw5Ys8z6L5R7; Thu, 19 Jun 2025 18:06:28 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id B7F281404C6; Thu, 19 Jun 2025 18:11:11 +0800 (CST) Received: from localhost (10.122.19.247) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Thu, 19 Jun 2025 12:11:10 +0200 Date: Thu, 19 Jun 2025 11:11:09 +0100 From: Jonathan Cameron To: Yuquan Wang CC: , , , , , , Yicong Yang , Niyas Sait , , Vandana Salve , Davidlohr Bueso , Dave Jiang , Alison Schofield , Ira Weiny , Dan Williams , Alexander Shishkin , Peter Zijlstra , Ingo Molnar , "Arnaldo Carvalho de Melo" , Mark Rutland , Gregory Price , Huang Ying Subject: Re: [RFC PATCH 1/4] cxl: Register devices for CXL Hotness Monitoring Units (CHMU) Message-ID: <20250619111109.0000640e@huawei.com> In-Reply-To: References: <20241121101845.1815660-1-Jonathan.Cameron@huawei.com> <20241121101845.1815660-2-Jonathan.Cameron@huawei.com> X-Mailer: Claws Mail 4.3.0 (GTK 3.24.42; x86_64-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.122.19.247] X-ClientProxiedBy: lhrpeml500009.china.huawei.com (7.191.174.84) To frapeml500008.china.huawei.com (7.182.85.71) X-Stat-Signature: gj4dfscsqyr7qa4eqdpxcq7skqp9o5na X-Rspamd-Queue-Id: 05E0020007 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1750327876-609381 X-HE-Meta: U2FsdGVkX19BgWG/C7WErKOAdAZ/5STgvcjCA929dCbBJphMtYTStXLtyaD0jIfNzBrG+zxKu1JwGfEbu28tZ0mcg54gnVJcb1IyMRYuXDb0Su2rKG/PipzBYuC8p6rVQ9XzCsaf4fSh79cVNToQECxNVitZc+oaFDlkG0BFVYMPOVaYXcjKpysjlKI2ptmlv5ZFqvfZUrlCzCiulevrR8LzHdDtC4uxLvODDSLR1+VZBQc4GhNk9r103JNIfDNLKcEPNrglNAhm9ib1QfcZXwNCXhylu2zkSB5CjO6nQCkaI3UFy/nO1t1mJ3o67oBXk9M5mZub4oFmsvOdwxhAGXmgVBnTC3jBJ2woZ5ZGME6zWeOVcWQndVdu15yenAO9IrlKSBKLKlgPR2l+xFxbZXIGWHqJWuI5UuSmtv0NNFufv7zWZLIeKqTDvlnud6AovzD6qb1neUz+0VImca62RaLzIu4UAzQHeXOWwwbKLuJ2O+OKeivCccB07vkE1W4D25HAMmExivEequuC7+mMd42a0NnB/biBtXd/n97yHzW0UFAXVbQAxvjGpvIuaiVzuN5lrRB3ok6MD0/idiFfTIpfyMQ2JzMfY1GVsOZ+jaj56SWdT/yoTXSJUrJkJHox+VO29/+cvrmcFDDqADw5YYwYxC4EZQdIYwbD1sB0xO0CuWgItHinJr2ROGOEVK3RA9w5ZRtSd1GMqyxW+hDuVBE1A2WpMVuHW9dSaB7YXmaXC5Fk+pDRVZ2isXmHxOEjAgGbChB3XMrxq/xxwq+lfC7TxG9HONdGMbd71iaUFQchpXzukEWKufna3MEP5QqBTbm7LXxwkdmuRFFZurojV7BYNuqKn8alvKBmWoXsCM00TM+xayzUzE8XPg3poN+ujyVKYcMpmXxXMoZ6pM9sKx6ySkM+RjMI+Eeruvvpz/6MzLkrEqWSloh35rXnL0XYoIqYmY0l7Kc23ozcDe+ wVKxxk8/ vGZVpUOhCWDjlSoeK7b+BxbYSmftnvQGGqKw7AFZ35v3S17vmll66EbOtllouf0TVd42sD79BjzUj+wl/YmNIAa7kgDDZdcmbA6vcuw/up532vVq8hN/PdkCVyckhi0aRBL9xdwUALSmmOY2k7KbToOrfWuHUGgxwjEhBmGC6DQWfZQYRnLY2kYt/q4o7xhDfI3Vgp7tS+9lQUPXSi71WC3M2GrJtsqLUqc3Of99SDollEcJjxmFgalAXa7u0jfUsS9r0WdkDoqbT51RY0lYmrfphPYelPTaaKzK7hSRkXZpbYilmxpVlxFjTYQ87VEQI55MkHZlgR+xXNDJ66aN30aivAb2KM6ma0x3tVhMNyKqKou8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 19 Jun 2025 09:47:00 +0800 Yuquan Wang wrote: > On Thu, Nov 21, 2024 at 10:18:42AM +0000, Jonathan Cameron wrote: > > Basic registration using similar approach to how the CPMUs > > are registered. > >=20 > > Signed-off-by: Jonathan Cameron > > --- > > drivers/cxl/core/Makefile | 1 + > > drivers/cxl/core/hmu.c | 64 +++++++++++++++++++++++++++++++++++++++ > > drivers/cxl/core/regs.c | 14 +++++++++ > > drivers/cxl/cxl.h | 4 +++ > > drivers/cxl/cxlpci.h | 1 + > > drivers/cxl/hmu.h | 23 ++++++++++++++ > > drivers/cxl/pci.c | 26 +++++++++++++++- > > 7 files changed, 132 insertions(+), 1 deletion(-) > >=20 > > diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile > > index 9259bcc6773c..d060abb773ae 100644 > > --- a/drivers/cxl/core/Makefile > > +++ b/drivers/cxl/core/Makefile > > @@ -12,6 +12,7 @@ cxl_core-y +=3D memdev.o > > cxl_core-y +=3D mbox.o > > cxl_core-y +=3D pci.o > > cxl_core-y +=3D hdm.o > > +cxl_core-y +=3D hmu.o > > cxl_core-y +=3D pmu.o > > cxl_core-y +=3D cdat.o > > cxl_core-$(CONFIG_TRACING) +=3D trace.o > > diff --git a/drivers/cxl/core/hmu.c b/drivers/cxl/core/hmu.c > > new file mode 100644 > > index 000000000000..3ee938bb6c05 > > --- /dev/null > > +++ b/drivers/cxl/core/hmu.c > > @@ -0,0 +1,64 @@ > > +// SPDX-License-Identifier: GPL-2.0-only > > +/* Copyright(c) 2024 Huawei. All rights reserved. */ > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include "core.h" > > + > > +static void cxl_hmu_release(struct device *dev) > > +{ > > + struct cxl_hmu *hmu =3D to_cxl_hmu(dev); > > + > > + kfree(hmu); > > +} > > + > > +const struct device_type cxl_hmu_type =3D { > > + .name =3D "cxl_hmu", > > + .release =3D cxl_hmu_release, > > +}; > > + > > +static void remove_dev(void *dev) > > +{ > > + device_unregister(dev); > > +} > > + > > +int devm_cxl_hmu_add(struct device *parent, struct cxl_hmu_regs *regs, > > + int assoc_id, int index) > > +{ > > + struct cxl_hmu *hmu; > > + struct device *dev; > > + int rc; > > + > > + hmu =3D kzalloc(sizeof(*hmu), GFP_KERNEL); > > + if (!hmu) > > + return -ENOMEM; > > + > > + hmu->assoc_id =3D assoc_id; > > + hmu->index =3D index; > > + hmu->base =3D regs->hmu; > > + dev =3D &hmu->dev; > > + device_initialize(dev); > > + device_set_pm_not_required(dev); > > + dev->parent =3D parent; > > + dev->bus =3D &cxl_bus_type; > > + dev->type =3D &cxl_hmu_type; > > + rc =3D dev_set_name(dev, "hmu_mem%d.%d", assoc_id, index); > > + if (rc) > > + goto err; > > + > > + rc =3D device_add(dev); > > + if (rc) > > + goto err; > > + > > + return devm_add_action_or_reset(parent, remove_dev, dev); > > + > > +err: > > + put_device(&hmu->dev); > > + return rc; > > +} > > +EXPORT_SYMBOL_NS_GPL(devm_cxl_hmu_add, CXL); > > + > > diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c > > index e1082e749c69..c12afaa6ef98 100644 > > --- a/drivers/cxl/core/regs.c > > +++ b/drivers/cxl/core/regs.c > > @@ -401,6 +401,20 @@ int cxl_map_pmu_regs(struct cxl_register_map *map,= struct cxl_pmu_regs *regs) > > } > > EXPORT_SYMBOL_NS_GPL(cxl_map_pmu_regs, CXL); > > =20 > > +int cxl_map_hmu_regs(struct cxl_register_map *map, struct cxl_hmu_regs= *regs) > > +{ > > + struct device *dev =3D map->host; > > + resource_size_t phys_addr; > > + > > + phys_addr =3D map->resource; > > + regs->hmu =3D devm_cxl_iomap_block(dev, phys_addr, map->max_size); =20 > I applied CHMU patch on 6.15.0 kernel and I tried to boot the virt with > one cxl root port and one device (jic23/cxl-2025-06-10), then the dmesg s= hows > "Failed to request region 0x10210000-0x1023ffff". I guess it is caused by= the > 'map->max_size'(0x30000) is large and the resource has been allocated by = CPMU regs. > I tried to change it to 0x10000, the hmu_mem0.0 could be created as norma= l. Ah. I was meaning to post an updated version of this series just to fix the bug you've hit here but forgot to do so! Sorry about that. I need to figure out if there is a more elegant way to do this but in meant= ime here is what I'm carrying on top of these posted series. It's a fairly horr= ible bit of layering as we need the generic code to know to poke around inside a= temporary mapping just to get the size that it should iomap. =46rom fadffb32ed302dfb6dec4497214e1d3b39450f8b Mon Sep 17 00:00:00 2001 From: Jonathan Cameron Date: Thu, 17 Apr 2025 11:02:07 +0100 Subject: [PATCH] Fix up sizing of CHMU issue --- drivers/cxl/core/regs.c | 16 ++++++++++++++-- drivers/cxl/cxlpci.h | 7 +++++++ drivers/cxl/hmu.c | 6 ------ 3 files changed, 21 insertions(+), 8 deletions(-) diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index 925870c4e494..5ae31696bf8b 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -416,10 +416,22 @@ EXPORT_SYMBOL_NS_GPL(cxl_map_pmu_regs, "CXL"); int cxl_map_hmu_regs(struct cxl_register_map *map, struct cxl_hmu_regs *re= gs) { struct device *dev =3D map->host; - resource_size_t phys_addr; + u64 __iomem *poke; + u64 common_cap[2]; + resource_size_t phys_addr, phys_size; =20 phys_addr =3D map->resource; - regs->hmu =3D devm_cxl_iomap_block(dev, phys_addr, map->max_size); + /* Finding out the size of a CHMU means poking around inside */ + poke =3D ioremap(phys_addr, sizeof(common_cap)); + if (!poke) { + return -ENOMEM; + } + common_cap[0] =3D le64_to_cpu(readq(poke)); + common_cap[1] =3D le64_to_cpu(readq(poke + 1)); + iounmap(poke); + phys_size =3D FIELD_GET(CHMU_COMMON_CAP0_NUMINST_MSK, common_cap[0]) * + FIELD_GET(CHMU_COMMON_CAP1_INSTLEN_MSK, common_cap[1]) + 0x10; + regs->hmu =3D devm_cxl_iomap_block(dev, phys_addr, phys_size); if (!regs->hmu) return -ENOMEM; =20 diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h index f7b902eab288..a91407292aea 100644 --- a/drivers/cxl/cxlpci.h +++ b/drivers/cxl/cxlpci.h @@ -77,6 +77,13 @@ enum cxl_regloc_type { CXL_REGLOC_RBI_TYPES }; =20 +/* A few CHMU registers are needed to establish size */ +#define CHMU_COMMON_CAP0_REG 0x00 +#define CHMU_COMMON_CAP0_VER_MSK GENMASK(3, 0) +#define CHMU_COMMON_CAP0_NUMINST_MSK GENMASK(15, 8) +#define CHMU_COMMON_CAP1_REG 0x08 +#define CHMU_COMMON_CAP1_INSTLEN_MSK GENMASK(15, 0) + /* * Table Access DOE, CDAT Read Entry Response * diff --git a/drivers/cxl/hmu.c b/drivers/cxl/hmu.c index 1a7a0f60a6ad..a1953e8750c8 100644 --- a/drivers/cxl/hmu.c +++ b/drivers/cxl/hmu.c @@ -27,12 +27,6 @@ #include "cxl.h" #include "hmu.h" =20 -#define CHMU_COMMON_CAP0_REG 0x00 -#define CHMU_COMMON_CAP0_VER_MSK GENMASK(3, 0) -#define CHMU_COMMON_CAP0_NUMINST_MSK GENMASK(15, 8) -#define CHMU_COMMON_CAP1_REG 0x08 -#define CHMU_COMMON_CAP1_INSTLEN_MSK GENMASK(15, 0) - /* Register offsets within instance */ #define CHMU_INST0_CAP0_REG 0x00 #define CHMU_INST0_CAP0_MSI_N_MSK GENMASK(3, 0) --=20 2.48.1 > > + if (!regs->hmu) > > + return -ENOMEM; > > + > > + return 0; > > +} > > +EXPORT_SYMBOL_NS_GPL(cxl_map_hmu_regs, CXL); > > + > > static int cxl_map_regblock(struct cxl_register_map *map) > > { > > struct device *host =3D map->host; > > diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h > > index 5406e3ab3d4a..8172bc1f7a8d 100644 > > --- a/drivers/cxl/cxl.h > > +++ b/drivers/cxl/cxl.h > > @@ -227,6 +227,9 @@ struct cxl_regs { > > struct_group_tagged(cxl_pmu_regs, pmu_regs, > > void __iomem *pmu; > > ); > > + struct_group_tagged(cxl_hmu_regs, hmu_regs, > > + void __iomem *hmu; > > + ); > > =20 > > /* > > * RCH downstream port specific RAS register > > @@ -292,6 +295,7 @@ int cxl_map_component_regs(const struct cxl_registe= r_map *map, > > unsigned long map_mask); > > int cxl_map_device_regs(const struct cxl_register_map *map, > > struct cxl_device_regs *regs); > > +int cxl_map_hmu_regs(struct cxl_register_map *map, struct cxl_hmu_regs= *regs); > > int cxl_map_pmu_regs(struct cxl_register_map *map, struct cxl_pmu_regs= *regs); > > =20 > > enum cxl_regloc_type; > > diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h > > index 4da07727ab9c..71f5e9620137 100644 > > --- a/drivers/cxl/cxlpci.h > > +++ b/drivers/cxl/cxlpci.h > > @@ -67,6 +67,7 @@ enum cxl_regloc_type { > > CXL_REGLOC_RBI_VIRT, > > CXL_REGLOC_RBI_MEMDEV, > > CXL_REGLOC_RBI_PMU, > > + CXL_REGLOC_RBI_HMU, > > CXL_REGLOC_RBI_TYPES > > }; > > =20 > > diff --git a/drivers/cxl/hmu.h b/drivers/cxl/hmu.h > > new file mode 100644 > > index 000000000000..c4798ed9764b > > --- /dev/null > > +++ b/drivers/cxl/hmu.h > > @@ -0,0 +1,23 @@ > > +/* SPDX-License-Identifier: GPL-2.0-only */ > > +/* > > + * Copyright(c) 2024 Huawei > > + * CXL Specification rev 3.2 Setion 8.2.8 (CHMU Register Interface) > > + */ > > +#ifndef CXL_HMU_H > > +#define CXL_HMU_H > > +#include > > + > > +#define CXL_HMU_REGMAP_SIZE 0xe00 /* Table 8-32 CXL 3.0 specification = */ > > +struct cxl_hmu { > > + struct device dev; > > + void __iomem *base; > > + int assoc_id; > > + int index; > > +}; > > + > > +#define to_cxl_hmu(dev) container_of(dev, struct cxl_hmu, dev) > > +struct cxl_hmu_regs; > > +int devm_cxl_hmu_add(struct device *parent, struct cxl_hmu_regs *regs, > > + int assoc_id, int idx); > > + > > +#endif > > diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c > > index 188412d45e0d..e89ea9d3f007 100644 > > --- a/drivers/cxl/pci.c > > +++ b/drivers/cxl/pci.c > > @@ -15,6 +15,7 @@ > > #include "cxlmem.h" > > #include "cxlpci.h" > > #include "cxl.h" > > +#include "hmu.h" > > #include "pmu.h" > > =20 > > /** > > @@ -814,7 +815,7 @@ static int cxl_pci_probe(struct pci_dev *pdev, cons= t struct pci_device_id *id) > > struct cxl_dev_state *cxlds; > > struct cxl_register_map map; > > struct cxl_memdev *cxlmd; > > - int i, rc, pmu_count; > > + int i, rc, hmu_count, pmu_count; > > bool irq_avail; > > =20 > > /* > > @@ -938,6 +939,29 @@ static int cxl_pci_probe(struct pci_dev *pdev, con= st struct pci_device_id *id) > > } > > } > > =20 > > + hmu_count =3D cxl_count_regblock(pdev, CXL_REGLOC_RBI_HMU); > > + for (i =3D 0; i < hmu_count; i++) { > > + struct cxl_hmu_regs hmu_regs; > > + > > + rc =3D cxl_find_regblock_instance(pdev, CXL_REGLOC_RBI_HMU, &map, i); > > + if (rc) { > > + dev_dbg(&pdev->dev, "Could not find HMU regblock\n"); > > + break; > > + } > > + > > + rc =3D cxl_map_hmu_regs(&map, &hmu_regs); > > + if (rc) { > > + dev_dbg(&pdev->dev, "Could not map HMU regs\n"); > > + break; > > + } > > + > > + rc =3D devm_cxl_hmu_add(cxlds->dev, &hmu_regs, cxlmd->id, i); > > + if (rc) { > > + dev_dbg(&pdev->dev, "Could not add HMU instance\n"); > > + break; > > + } > > + } > > + > > rc =3D cxl_event_config(host_bridge, mds, irq_avail); > > if (rc) > > return rc; > > --=20 > > 2.43.0 > > =20 >=20 >=20