From: Keith Busch <kbusch@kernel.org>
To: Christoph Hellwig <hch@lst.de>
Cc: Caleb Sander Mateos <csander@purestorage.com>,
Jens Axboe <axboe@kernel.dk>, Sagi Grimberg <sagi@grimberg.me>,
Andrew Morton <akpm@linux-foundation.org>,
Kanchan Joshi <joshi.k@samsung.com>,
linux-nvme@lists.infradead.org, linux-mm@kvack.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v5 3/3] nvme/pci: make PRP list DMA pools per-NUMA-node
Date: Fri, 25 Apr 2025 12:02:55 -0600 [thread overview]
Message-ID: <aAvOT8meWyuBpUpS@kbusch-mbp.dhcp.thefacebook.com> (raw)
In-Reply-To: <20250425132111.GA5797@lst.de>
On Fri, Apr 25, 2025 at 03:21:11PM +0200, Christoph Hellwig wrote:
> On Thu, Apr 24, 2025 at 09:40:18AM -0600, Keith Busch wrote:
> > The dmapool allocates dma coherent memory, and it's mapped for the
> > remainder of lifetime of the pool. Allocating slab memory and dma
> > mapping per-io would be pretty costly in comparison, I think.
>
> True. Although we don't even need dma coherent memory, a single
> cache writeback after writing the PRPs/SGLs would probably be more
> efficient on not cache coherent platforms. But no one really cares
> about performance on those anyway..
Sure, but it's not just about non-coherent platform performance
concerns. Allocations out of the dma pool are iommu mapped if necessary
too. We frequently allocate and free these lists, and the dmapool makes
it quick and easy to reuse previously mapped memory.
next prev parent reply other threads:[~2025-04-25 18:03 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-04-22 22:09 [PATCH v5 0/3] nvme/pci: PRP list DMA pool partitioning Caleb Sander Mateos
2025-04-22 22:09 ` [PATCH v5 1/3] dmapool: add NUMA affinity support Caleb Sander Mateos
2025-04-25 21:44 ` Sagi Grimberg
2025-04-22 22:09 ` [PATCH v5 2/3] nvme/pci: factor out nvme_init_hctx() helper Caleb Sander Mateos
2025-04-22 22:09 ` [PATCH v5 3/3] nvme/pci: make PRP list DMA pools per-NUMA-node Caleb Sander Mateos
2025-04-24 14:12 ` Christoph Hellwig
2025-04-24 15:40 ` Keith Busch
2025-04-24 15:46 ` Caleb Sander Mateos
2025-04-25 13:21 ` Christoph Hellwig
2025-04-25 18:02 ` Keith Busch [this message]
2025-04-23 13:21 ` [PATCH v5 0/3] nvme/pci: PRP list DMA pool partitioning Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aAvOT8meWyuBpUpS@kbusch-mbp.dhcp.thefacebook.com \
--to=kbusch@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=axboe@kernel.dk \
--cc=csander@purestorage.com \
--cc=hch@lst.de \
--cc=joshi.k@samsung.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nvme@lists.infradead.org \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox