linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Martin Oliveira <martin.oliveira@eideticom.com>
To: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-pci@vger.kernel.org, linux-mm@kvack.org
Cc: Jason Gunthorpe <jgg@ziepe.ca>, Leon Romanovsky <leon@kernel.org>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Logan Gunthorpe <logang@deltatee.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Tejun Heo <tj@kernel.org>,
	"Rafael J. Wysocki" <rafael@kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Mike Marciniszyn <mike.marciniszyn@intel.com>,
	Michael Guralnik <michaelgur@nvidia.com>,
	Martin Oliveira <martin.oliveira@eideticom.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Ard Biesheuvel <ardb@kernel.org>,
	Valentine Sinitsyn <valesini@yandex-team.ru>,
	Lukas Wunner <lukas@wunner.de>
Subject: [PATCH 0/6] Enable P2PDMA in Userspace RDMA
Date: Wed,  5 Jun 2024 13:29:28 -0600	[thread overview]
Message-ID: <20240605192934.742369-1-martin.oliveira@eideticom.com> (raw)

This patch series enables P2PDMA memory to be used in userspace RDMA
transfers. With this series, P2PDMA memory mmaped into userspace (ie.
only NVMe CMBs, at the moment) can then be used with ibv_reg_mr() (or
similar) interfaces. This can be tested by passing a sysfs p2pmem
allocator to the --mmap flag of the perftest tools.

This requires addressing three issues:

* Stop exporting the P2PDMA VMAs with page_mkwrite which is incompatible
with FOLL_LONGTERM

* Fix folio_fast_pin_allowed() path to take into account ZONE_DEVICE pages.

* Remove the restriction on FOLL_LONGTREM with FOLL_PCI_P2PDMA which was
initially put in place due to excessive caution with assuming P2PDMA
would have similar problems to fsdax with unmap_mapping_range(). Seeing
P2PDMA only uses unmap_mapping_range() on device unbind and immediately
waits for all page reference counts to go to zero after calling it, it
is actually believed to be safe from reuse and user access faults. See
[1] for more discussion.

This was tested using a Mellanox ConnectX-6 SmartNIC (MT28908 Family),
using the mlx5_core driver, as well as an NVMe CMB.

Thanks,
Martin

[1]: https://lore.kernel.org/linux-mm/87cypuvh2i.fsf@nvdebian.thelocal/T/

Martin Oliveira (6):
  kernfs: create vm_operations_struct without page_mkwrite()
  sysfs: add mmap_allocates parameter to struct bin_attribute
  PCI/P2PDMA: create VMA without page_mkwrite() operator
  mm/gup: handle ZONE_DEVICE pages in folio_fast_pin_allowed()
  mm/gup: allow FOLL_LONGTERM & FOLL_PCI_P2PDMA
  RDMA/umem: add support for P2P RDMA

 drivers/infiniband/core/umem.c |  3 +++
 drivers/pci/p2pdma.c           |  1 +
 fs/kernfs/file.c               | 15 ++++++++++++++-
 fs/sysfs/file.c                | 25 +++++++++++++++++++------
 include/linux/kernfs.h         |  7 +++++++
 include/linux/sysfs.h          |  1 +
 mm/gup.c                       |  9 ++++-----
 7 files changed, 49 insertions(+), 12 deletions(-)


base-commit: c3f38fa61af77b49866b006939479069cd451173
-- 
2.34.1



             reply	other threads:[~2024-06-05 19:30 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-06-05 19:29 Martin Oliveira [this message]
2024-06-05 19:29 ` [PATCH 1/6] kernfs: create vm_operations_struct without page_mkwrite() Martin Oliveira
2024-06-05 21:43   ` Bjorn Helgaas
2024-06-06 20:54   ` Greg Kroah-Hartman
2024-06-06 21:32     ` Logan Gunthorpe
2024-06-07  5:03     ` Christoph Hellwig
2024-06-07 16:16       ` Logan Gunthorpe
2024-06-07 19:18         ` Greg Kroah-Hartman
2024-06-05 19:29 ` [PATCH 2/6] sysfs: add mmap_allocates parameter to struct bin_attribute Martin Oliveira
2024-06-05 19:29 ` [PATCH 3/6] PCI/P2PDMA: create VMA without page_mkwrite() operator Martin Oliveira
2024-06-05 21:45   ` Bjorn Helgaas
2024-06-05 19:29 ` [PATCH 4/6] mm/gup: handle ZONE_DEVICE pages in folio_fast_pin_allowed() Martin Oliveira
2024-06-05 19:29 ` [PATCH 5/6] mm/gup: allow FOLL_LONGTERM & FOLL_PCI_P2PDMA Martin Oliveira
2024-06-05 19:29 ` [PATCH 6/6] RDMA/umem: add support for P2P RDMA Martin Oliveira
2024-06-10 12:11   ` Jason Gunthorpe
2024-06-06  8:53 ` [PATCH 0/6] Enable P2PDMA in Userspace RDMA Zhu Yanjun
2024-06-06 21:32   ` Martin Oliveira

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240605192934.742369-1-martin.oliveira@eideticom.com \
    --to=martin.oliveira@eideticom.com \
    --cc=akpm@linux-foundation.org \
    --cc=ardb@kernel.org \
    --cc=bhelgaas@google.com \
    --cc=dan.j.williams@intel.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=jgg@ziepe.ca \
    --cc=leon@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=logang@deltatee.com \
    --cc=lukas@wunner.de \
    --cc=michaelgur@nvidia.com \
    --cc=mike.marciniszyn@intel.com \
    --cc=rafael@kernel.org \
    --cc=tj@kernel.org \
    --cc=valesini@yandex-team.ru \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox