* RE: [lkp@intel.com: [linux-next:master 13449/13717] drivers/vdpa/mlx5/net/mlx5_vnet.c:2896:3: error: implicit declaration of function 'mlx5_msix_free' is invalid in C99]
2023-04-23 10:50 ` Michael S. Tsirkin
@ 2023-04-23 11:42 ` Eli Cohen
0 siblings, 0 replies; 4+ messages in thread
From: Eli Cohen @ 2023-04-23 11:42 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Dragos Tatulea, llvm, oe-kbuild-all,
Linux Memory Management List, Saeed Mahameed
[-- Attachment #1: Type: text/plain, Size: 7002 bytes --]
Michael,
You applied the vdpa patch without rebasing on net-next where the related mlx5_core is available:
git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git main
I wonder though why the code is still not in Linus' tree.
Moreover, I am waiting for Saeed to send out another mlx5_core patch (attached as 0001-net-mlx5_core-Modify-mlx5_msix_alloc-free-to-not-req.patch):
After which I want to send the vdpa patch: 0002-vdpa-mlx5-Support-interrupt-bypassing.patch
> -----Original Message-----
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Sunday, 23 April 2023 13:50
> To: Eli Cohen <elic@nvidia.com>
> Cc: Dragos Tatulea <dtatulea@nvidia.com>; llvm@lists.linux.dev; oe-kbuild-
> all@lists.linux.dev; Linux Memory Management List <linux-mm@kvack.org>;
> Saeed Mahameed <saeedm@nvidia.com>
> Subject: Re: [lkp@intel.com: [linux-next:master 13449/13717]
> drivers/vdpa/mlx5/net/mlx5_vnet.c:2896:3: error: implicit declaration of
> function 'mlx5_msix_free' is invalid in C99]
>
> Sorry got confused. Referring to the robot warning here.
>
> On Sun, Apr 23, 2023 at 07:29:45AM +0000, Eli Cohen wrote:
> > Are you referring to kernel robot warning or to updating MAINTAINERS?
> > I was not the maintainer of anything so not sure what you mean here.
> >
> > > -----Original Message-----
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Sunday, 23 April 2023 10:11
> > > To: Dragos Tatulea <dtatulea@nvidia.com>; Eli Cohen <elic@nvidia.com>
> > > Cc: llvm@lists.linux.dev; oe-kbuild-all@lists.linux.dev; Linux Memory
> > > Management List <linux-mm@kvack.org>; Michael S. Tsirkin
> > > <mst@redhat.com>; Saeed Mahameed <saeedm@nvidia.com>
> > > Subject: [lkp@intel.com: [linux-next:master 13449/13717]
> > > drivers/vdpa/mlx5/net/mlx5_vnet.c:2896:3: error: implicit declaration of
> > > function 'mlx5_msix_free' is invalid in C99]
> > >
> > > CC Dragos.
> > > Does one of you want to post a MAINTAINERS patch btw?
> > >
> > >
> > > ----- Forwarded message from kernel test robot <lkp@intel.com> -----
> > >
> > > From: kernel test robot <lkp@intel.com>
> > > Date: Sun, 23 Apr 2023 07:29:38 +0800
> > > To: Eli Cohen <elic@nvidia.com>
> > > Cc: llvm@lists.linux.dev, oe-kbuild-all@lists.linux.dev, Linux Memory
> > > Management List <linux-mm@kvack.org>, "Michael S. Tsirkin"
> > > <mst@redhat.com>, Saeed Mahameed <saeedm@nvidia.com>
> > > Subject: [linux-next:master 13449/13717]
> > > drivers/vdpa/mlx5/net/mlx5_vnet.c:2896:3: error: implicit declaration of
> > > function
> > > 'mlx5_msix_free' is invalid in C99
> > > Message-ID: <202304230755.9jjF6ZJr-lkp@intel.com>
> > >
> > > tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> > > master
> > > head: d3e1ee0e67e7603d36f4fa2fec6b881c01aabe89
> > > commit: b9ca712f33f59e977592cb639c2e00a02f45a146 [13449/13717]
> > > vdpa/mlx5: Support interrupt bypassing
> > > config: x86_64-allmodconfig (https://download.01.org/0day-
> > > ci/archive/20230423/202304230755.9jjF6ZJr-lkp@intel.com/config)
> > > compiler: clang version 14.0.6 (https://github.com/llvm/llvm-project
> > > f28c006a5895fc0e329fe15fead81e37457cb1d1)
> > > reproduce (this is a W=1 build):
> > > wget https://raw.githubusercontent.com/intel/lkp-
> > > tests/master/sbin/make.cross -O ~/bin/make.cross
> > > chmod +x ~/bin/make.cross
> > > # https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-
> > > next.git/commit/?id=b9ca712f33f59e977592cb639c2e00a02f45a146
> > > git remote add linux-next
> > > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> > > git fetch --no-tags linux-next master
> > > git checkout b9ca712f33f59e977592cb639c2e00a02f45a146
> > > # save the config file
> > > mkdir build_dir && cp config build_dir/.config
> > > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang
> make.cross
> > > W=1 O=build_dir ARCH=x86_64 olddefconfig
> > > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang
> make.cross
> > > W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash arch/x86/xen/
> > > drivers/power/supply/ drivers/vdpa/mlx5/
> > >
> > > If you fix the issue, kindly add following tag where applicable
> > > | Reported-by: kernel test robot <lkp@intel.com>
> > > | Link: https://lore.kernel.org/oe-kbuild-all/202304230755.9jjF6ZJr-
> > > lkp@intel.com/
> > >
> > > Note: the linux-next/master HEAD
> > > d3e1ee0e67e7603d36f4fa2fec6b881c01aabe89 builds fine.
> > > It may have been fixed somewhere.
> > >
> > > All errors (new ones prefixed by >>):
> > >
> > > >> drivers/vdpa/mlx5/net/mlx5_vnet.c:2896:3: error: implicit declaration
> of
> > > function 'mlx5_msix_free' is invalid in C99 [-Werror,-Wimplicit-function-
> > > declaration]
> > > mlx5_msix_free(ndev->mvdev.mdev, ent->map);
> > > ^
> > > drivers/vdpa/mlx5/net/mlx5_vnet.c:2896:3: note: did you mean
> > > 'mlx5_db_free'?
> > > include/linux/mlx5/driver.h:1083:6: note: 'mlx5_db_free' declared here
> > > void mlx5_db_free(struct mlx5_core_dev *dev, struct mlx5_db *db);
> > > ^
> > > >> drivers/vdpa/mlx5/net/mlx5_vnet.c:3259:14: error: implicit declaration
> of
> > > function 'mlx5_msix_alloc' is invalid in C99 [-Werror,-Wimplicit-function-
> > > declaration]
> > > ent->map = mlx5_msix_alloc(ndev->mvdev.mdev, int_handler,
> > > NULL, ent->name);
> > > ^
> > > drivers/vdpa/mlx5/net/mlx5_vnet.c:3259:14: note: did you mean
> > > 'mlx5_db_alloc'?
> > > include/linux/mlx5/driver.h:1078:19: note: 'mlx5_db_alloc' declared
> here
> > > static inline int mlx5_db_alloc(struct mlx5_core_dev *dev, struct
> mlx5_db
> > > *db)
> > > ^
> > > >> drivers/vdpa/mlx5/net/mlx5_vnet.c:3259:12: error: assigning to 'struct
> > > msi_map' from incompatible type 'int'
> > > ent->map = mlx5_msix_alloc(ndev->mvdev.mdev, int_handler,
> > > NULL, ent->name);
> > > ^
> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > 3 errors generated.
> > >
> > >
> > > vim +/mlx5_msix_free +2896 drivers/vdpa/mlx5/net/mlx5_vnet.c
> > >
> > > 2885
> > > 2886 static void free_irqs(struct mlx5_vdpa_net *ndev)
> > > 2887 {
> > > 2888 struct mlx5_vdpa_irq_pool_entry *ent;
> > > 2889 int i;
> > > 2890
> > > 2891 if (!msix_mode_supported(&ndev->mvdev))
> > > 2892 return;
> > > 2893
> > > 2894 for (i = ndev->irqp.num_ent - 1; i >= 0; i--) {
> > > 2895 ent = ndev->irqp.entries + i;
> > > > 2896 mlx5_msix_free(ndev->mvdev.mdev, ent-
> >map);
> > > 2897 }
> > > 2898 kfree(ndev->irqp.entries);
> > > 2899 }
> > > 2900
> > >
> > > --
> > > 0-DAY CI Kernel Test Service
> > > https://github.com/intel/lkp-tests
> > >
> > > ----- End forwarded message -----
> >
[-- Attachment #2: 0001-net-mlx5_core-Modify-mlx5_msix_alloc-free-to-not-req.patch --]
[-- Type: application/octet-stream, Size: 2924 bytes --]
From 20068dcf37dbc3fdfe30706ded4d5c18f32f22c4 Mon Sep 17 00:00:00 2001
From: Eli Cohen <elic@nvidia.com>
Date: Wed, 19 Apr 2023 16:55:41 +0300
Subject: [PATCH 1/2] net/mlx5_core: Modify mlx5_msix_alloc/free to not
request/free irq
Leave that task to the caller so it can control whether it wants to add
private data to be passed to the handler.
Signed-off-by: Eli Cohen <elic@nvidia.com>
---
.../net/ethernet/mellanox/mlx5/core/pci_irq.c | 21 ++-----------------
include/linux/mlx5/driver.h | 4 +---
2 files changed, 3 insertions(+), 22 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c
index e12e528c09f5..0c7d4a0bf19d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c
@@ -486,21 +486,16 @@ struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, u16 vecidx,
/**
* mlx5_msix_alloc - allocate msix interrupt
* @dev: mlx5 device from which to request
- * @handler: interrupt handler
* @affdesc: affinity descriptor
- * @name: interrupt name
*
* Returns: struct msi_map with result encoded.
* Note: the caller must make sure to release the irq by calling
* mlx5_msix_free() if shutdown was initiated.
*/
struct msi_map mlx5_msix_alloc(struct mlx5_core_dev *dev,
- irqreturn_t (*handler)(int, void *),
- const struct irq_affinity_desc *affdesc,
- const char *name)
+ const struct irq_affinity_desc *affdesc)
{
struct msi_map map;
- int err;
if (!dev->pdev) {
map.virq = 0;
@@ -508,18 +503,7 @@ struct msi_map mlx5_msix_alloc(struct mlx5_core_dev *dev,
return map;
}
- map = pci_msix_alloc_irq_at(dev->pdev, MSI_ANY_INDEX, affdesc);
- if (!map.virq)
- return map;
-
- err = request_irq(map.virq, handler, 0, name, NULL);
- if (err) {
- mlx5_core_warn(dev, "err %d\n", err);
- pci_msix_free_irq(dev->pdev, map);
- map.virq = 0;
- map.index = -ENOMEM;
- }
- return map;
+ return pci_msix_alloc_irq_at(dev->pdev, MSI_ANY_INDEX, affdesc);
}
EXPORT_SYMBOL(mlx5_msix_alloc);
@@ -530,7 +514,6 @@ EXPORT_SYMBOL(mlx5_msix_alloc);
*/
void mlx5_msix_free(struct mlx5_core_dev *dev, struct msi_map map)
{
- free_irq(map.virq, NULL);
pci_msix_free_irq(dev->pdev, map);
}
EXPORT_SYMBOL(mlx5_msix_free);
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index 135a3c8d8237..7fe437a6c443 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -1313,9 +1313,7 @@ enum {
};
struct msi_map mlx5_msix_alloc(struct mlx5_core_dev *dev,
- irqreturn_t (*handler)(int, void *),
- const struct irq_affinity_desc *affdesc,
- const char *name);
+ const struct irq_affinity_desc *affdesc);
void mlx5_msix_free(struct mlx5_core_dev *dev, struct msi_map map);
#endif /* MLX5_DRIVER_H */
--
2.39.2
[-- Attachment #3: 0002-vdpa-mlx5-Support-interrupt-bypassing.patch --]
[-- Type: application/octet-stream, Size: 10310 bytes --]
From 23695859d32bb69907bdd33beaba4fb2a70d1e2e Mon Sep 17 00:00:00 2001
From: Eli Cohen <elic@nvidia.com>
Date: Thu, 5 Jan 2023 15:09:13 +0200
Subject: [PATCH 2/2] vdpa/mlx5: Support interrupt bypassing
Add support for generation of interrupts from the device directly to the
VM to the VCPU thus avoiding the overhead on the host CPU.
When supported, the driver will attempt to allocate vectors for each
data virtqueue. If a vector for a virtqueue cannot be provided it will
use the QP mode where notifications go through the driver.
In addition, we add a shutdown callback to make sure allocated
interrupts are released in case of shutdown to allow clean shutdown.
Issue: 2211199
Change-Id: I7633ee56ca2861a8d0f4935bb29109c8e80f352c
Signed-off-by: Eli Cohen <elic@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/vdpa/mlx5/net/mlx5_vnet.c | 157 ++++++++++++++++++++++++++++--
drivers/vdpa/mlx5/net/mlx5_vnet.h | 15 +++
2 files changed, 163 insertions(+), 9 deletions(-)
diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
index e29e32b306ad..81a6f23f42b3 100644
--- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
+++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
@@ -83,6 +83,7 @@ struct mlx5_vq_restore_info {
u64 driver_addr;
u16 avail_index;
u16 used_index;
+ struct msi_map map;
bool ready;
bool restore;
};
@@ -118,6 +119,7 @@ struct mlx5_vdpa_virtqueue {
u16 avail_idx;
u16 used_idx;
int fw_state;
+ struct msi_map map;
/* keep last in the struct */
struct mlx5_vq_restore_info ri;
@@ -808,6 +810,13 @@ static bool counters_supported(const struct mlx5_vdpa_dev *mvdev)
BIT_ULL(MLX5_OBJ_TYPE_VIRTIO_Q_COUNTERS);
}
+static bool msix_mode_supported(struct mlx5_vdpa_dev *mvdev)
+{
+ return MLX5_CAP_DEV_VDPA_EMULATION(mvdev->mdev, event_mode) &
+ (1 << MLX5_VIRTIO_Q_EVENT_MODE_MSIX_MODE) &&
+ pci_msix_can_alloc_dyn(mvdev->mdev->pdev);
+}
+
static int create_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtqueue *mvq)
{
int inlen = MLX5_ST_SZ_BYTES(create_virtio_net_q_in);
@@ -849,9 +858,15 @@ static int create_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtque
if (vq_is_tx(mvq->index))
MLX5_SET(virtio_net_q_object, obj_context, tisn_or_qpn, ndev->res.tisn);
- MLX5_SET(virtio_q, vq_ctx, event_mode, MLX5_VIRTIO_Q_EVENT_MODE_QP_MODE);
+ if (mvq->map.virq) {
+ MLX5_SET(virtio_q, vq_ctx, event_mode, MLX5_VIRTIO_Q_EVENT_MODE_MSIX_MODE);
+ MLX5_SET(virtio_q, vq_ctx, event_qpn_or_msix, mvq->map.index);
+ } else {
+ MLX5_SET(virtio_q, vq_ctx, event_mode, MLX5_VIRTIO_Q_EVENT_MODE_QP_MODE);
+ MLX5_SET(virtio_q, vq_ctx, event_qpn_or_msix, mvq->fwqp.mqp.qpn);
+ }
+
MLX5_SET(virtio_q, vq_ctx, queue_index, mvq->index);
- MLX5_SET(virtio_q, vq_ctx, event_qpn_or_msix, mvq->fwqp.mqp.qpn);
MLX5_SET(virtio_q, vq_ctx, queue_size, mvq->num_ent);
MLX5_SET(virtio_q, vq_ctx, virtio_version_1_0,
!!(ndev->mvdev.actual_features & BIT_ULL(VIRTIO_F_VERSION_1)));
@@ -1194,6 +1209,56 @@ static void counter_set_dealloc(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_vir
mlx5_vdpa_warn(&ndev->mvdev, "dealloc counter set 0x%x\n", mvq->counter_set_id);
}
+static irqreturn_t int_handler(int irq, void *nh)
+{
+ struct vdpa_callback *cb = nh;
+
+ pr_info("ELI %s-%s-%d: irqn %d, callback %px, priv %px\n", current->comm, __func__, __LINE__, irq, cb->callback, cb->private);
+ if (cb->callback)
+ return cb->callback(cb->private);
+
+ return IRQ_HANDLED;
+}
+
+static void alloc_vector(struct mlx5_vdpa_net *ndev,
+ struct mlx5_vdpa_virtqueue *mvq)
+{
+ struct mlx5_vdpa_irq_pool *irqp = &ndev->irqp;
+ struct mlx5_vdpa_irq_pool_entry *ent;
+ int err;
+ int i;
+
+ for (i = 0; i < irqp->num_ent; i++) {
+ ent = &irqp->entries[i];
+ if (!ent->used) {
+ snprintf(ent->name, MLX5_VDPA_IRQ_NAME_LEN, "%s-vq-%d",
+ dev_name(&ndev->mvdev.vdev.dev), mvq->index);
+ ent->dev_id = &ndev->event_cbs[mvq->index];
+ err = request_irq(ent->map.virq, int_handler, 0, ent->name, ent->dev_id);
+ if (err)
+ return;
+
+ ent->used = true;
+ mvq->map = ent->map;
+ return;
+ }
+ }
+}
+
+static void dealloc_vector(struct mlx5_vdpa_net *ndev,
+ struct mlx5_vdpa_virtqueue *mvq)
+{
+ struct mlx5_vdpa_irq_pool *irqp = &ndev->irqp;
+ int i;
+
+ for (i = 0; i < irqp->num_ent; i++)
+ if (mvq->map.virq == irqp->entries[i].map.virq) {
+ free_irq(mvq->map.virq, irqp->entries[i].dev_id);
+ irqp->entries[i].used = false;
+ return;
+ }
+}
+
static int setup_vq(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtqueue *mvq)
{
u16 idx = mvq->index;
@@ -1223,27 +1288,31 @@ static int setup_vq(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtqueue *mvq)
err = counter_set_alloc(ndev, mvq);
if (err)
- goto err_counter;
+ goto err_connect;
+ alloc_vector(ndev, mvq);
err = create_virtqueue(ndev, mvq);
if (err)
- goto err_connect;
+ goto err_vq;
if (mvq->ready) {
err = modify_virtqueue(ndev, mvq, MLX5_VIRTIO_NET_Q_OBJECT_STATE_RDY);
if (err) {
mlx5_vdpa_warn(&ndev->mvdev, "failed to modify to ready vq idx %d(%d)\n",
idx, err);
- goto err_connect;
+ goto err_modify;
}
}
mvq->initialized = true;
return 0;
-err_connect:
+err_modify:
+ destroy_virtqueue(ndev, mvq);
+err_vq:
+ dealloc_vector(ndev, mvq);
counter_set_dealloc(ndev, mvq);
-err_counter:
+err_connect:
qp_destroy(ndev, &mvq->vqqp);
err_vqqp:
qp_destroy(ndev, &mvq->fwqp);
@@ -1288,6 +1357,7 @@ static void teardown_vq(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtqueue *
suspend_vq(ndev, mvq);
destroy_virtqueue(ndev, mvq);
+ dealloc_vector(ndev, mvq);
counter_set_dealloc(ndev, mvq);
qp_destroy(ndev, &mvq->vqqp);
qp_destroy(ndev, &mvq->fwqp);
@@ -2505,6 +2575,7 @@ static int save_channel_info(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtqu
ri->desc_addr = mvq->desc_addr;
ri->device_addr = mvq->device_addr;
ri->driver_addr = mvq->driver_addr;
+ ri->map = mvq->map;
ri->restore = true;
return 0;
}
@@ -2549,6 +2620,7 @@ static void restore_channels_info(struct mlx5_vdpa_net *ndev)
mvq->desc_addr = ri->desc_addr;
mvq->device_addr = ri->device_addr;
mvq->driver_addr = ri->driver_addr;
+ mvq->map = ri->map;
}
}
@@ -2833,6 +2905,21 @@ static struct device *mlx5_get_vq_dma_dev(struct vdpa_device *vdev, u16 idx)
return mvdev->vdev.dma_dev;
}
+static void free_irqs(struct mlx5_vdpa_net *ndev)
+{
+ struct mlx5_vdpa_irq_pool_entry *ent;
+ int i;
+
+ if (!msix_mode_supported(&ndev->mvdev))
+ return;
+
+ for (i = ndev->irqp.num_ent - 1; i >= 0; i--) {
+ ent = ndev->irqp.entries + i;
+ mlx5_msix_free(ndev->mvdev.mdev, ent->map);
+ }
+ kfree(ndev->irqp.entries);
+}
+
static void mlx5_vdpa_free(struct vdpa_device *vdev)
{
struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev);
@@ -2848,6 +2935,7 @@ static void mlx5_vdpa_free(struct vdpa_device *vdev)
mlx5_mpfs_del_mac(pfmdev, ndev->config.mac);
}
mlx5_vdpa_free_resources(&ndev->mvdev);
+ free_irqs(ndev);
kfree(ndev->event_cbs);
kfree(ndev->vqs);
}
@@ -2876,9 +2964,23 @@ static struct vdpa_notification_area mlx5_get_vq_notification(struct vdpa_device
return ret;
}
-static int mlx5_get_vq_irq(struct vdpa_device *vdv, u16 idx)
+static int mlx5_get_vq_irq(struct vdpa_device *vdev, u16 idx)
{
- return -EOPNOTSUPP;
+ struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev);
+ struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev);
+ struct mlx5_vdpa_virtqueue *mvq;
+
+ if (!is_index_valid(mvdev, idx))
+ return -EINVAL;
+
+ if (is_ctrl_vq_idx(mvdev, idx))
+ return -EOPNOTSUPP;
+
+ mvq = &ndev->vqs[idx];
+ if (!mvq->map.virq)
+ return -EOPNOTSUPP;
+
+ return mvq->map.virq;
}
static u64 mlx5_vdpa_get_driver_features(struct vdpa_device *vdev)
@@ -3155,6 +3257,30 @@ static int config_func_mtu(struct mlx5_core_dev *mdev, u16 mtu)
return err;
}
+static void allocate_irqs(struct mlx5_vdpa_net *ndev)
+{
+ struct mlx5_vdpa_irq_pool_entry *ent;
+ int i;
+
+ if (!msix_mode_supported(&ndev->mvdev))
+ return;
+
+ ndev->irqp.entries = kcalloc(ndev->mvdev.max_vqs, sizeof(*ndev->irqp.entries), GFP_KERNEL);
+ if (!ndev->irqp.entries)
+ return;
+
+ for (i = 0; i < ndev->mvdev.max_vqs; i++) {
+ ent = ndev->irqp.entries + i;
+ snprintf(ent->name, MLX5_VDPA_IRQ_NAME_LEN, "%s-vq-%d",
+ dev_name(&ndev->mvdev.vdev.dev), i);
+ ent->map = mlx5_msix_alloc(ndev->mvdev.mdev, NULL);
+ if (!ent->map.virq)
+ return;
+
+ ndev->irqp.num_ent++;
+ }
+}
+
static int mlx5_vdpa_dev_add(struct vdpa_mgmt_dev *v_mdev, const char *name,
const struct vdpa_dev_set_config *add_config)
{
@@ -3233,6 +3359,7 @@ static int mlx5_vdpa_dev_add(struct vdpa_mgmt_dev *v_mdev, const char *name,
}
init_mvqs(ndev);
+ allocate_irqs(ndev);
init_rwsem(&ndev->reslock);
config = &ndev->config;
@@ -3413,6 +3540,17 @@ static void mlx5v_remove(struct auxiliary_device *adev)
kfree(mgtdev);
}
+static void mlx5v_shutdown(struct auxiliary_device *auxdev)
+{
+ struct mlx5_vdpa_mgmtdev *mgtdev;
+ struct mlx5_vdpa_net *ndev;
+
+ mgtdev = auxiliary_get_drvdata(auxdev);
+ ndev = mgtdev->ndev;
+
+ free_irqs(ndev);
+}
+
static const struct auxiliary_device_id mlx5v_id_table[] = {
{ .name = MLX5_ADEV_NAME ".vnet", },
{},
@@ -3424,6 +3562,7 @@ static struct auxiliary_driver mlx5v_driver = {
.name = "vnet",
.probe = mlx5v_probe,
.remove = mlx5v_remove,
+ .shutdown = mlx5v_shutdown,
.id_table = mlx5v_id_table,
};
diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.h b/drivers/vdpa/mlx5/net/mlx5_vnet.h
index c90a89e1de4d..36c44d9fdd16 100644
--- a/drivers/vdpa/mlx5/net/mlx5_vnet.h
+++ b/drivers/vdpa/mlx5/net/mlx5_vnet.h
@@ -26,6 +26,20 @@ static inline u16 key2vid(u64 key)
return (u16)(key >> 48) & 0xfff;
}
+#define MLX5_VDPA_IRQ_NAME_LEN 32
+
+struct mlx5_vdpa_irq_pool_entry {
+ struct msi_map map;
+ bool used;
+ char name[MLX5_VDPA_IRQ_NAME_LEN];
+ void *dev_id;
+};
+
+struct mlx5_vdpa_irq_pool {
+ int num_ent;
+ struct mlx5_vdpa_irq_pool_entry *entries;
+};
+
struct mlx5_vdpa_net {
struct mlx5_vdpa_dev mvdev;
struct mlx5_vdpa_net_resources res;
@@ -49,6 +63,7 @@ struct mlx5_vdpa_net {
struct vdpa_callback config_cb;
struct mlx5_vdpa_wq_ent cvq_ent;
struct hlist_head macvlan_hash[MLX5V_MACVLAN_SIZE];
+ struct mlx5_vdpa_irq_pool irqp;
struct dentry *debugfs;
};
--
2.39.2
^ permalink raw reply [flat|nested] 4+ messages in thread