* Re: uncached page allocator [not found] ` <20070820094125.209e0811@the-village.bc.nu> @ 2007-08-21 6:05 ` Dave Airlie 2007-08-21 14:56 ` Peter Zijlstra 0 siblings, 1 reply; 4+ messages in thread From: Dave Airlie @ 2007-08-21 6:05 UTC (permalink / raw) To: Alan Cox; +Cc: dri-devel, Linux Kernel Mailing List, Linux Memory Management > Blame intel ;) > > > Any other ideas and suggestions? > > Without knowing exactly what you are doing: > > - Copies to uncached memory are very expensive on an x86 processor > (so it might be faster not to write and flush) > - Its not clear from your description how intelligent your transfer > system is. It is still possible to change the transfer system, but it should be intelligent enough or possible to make it more intelligent.. I also realise I need PAT + write combining but I believe this problem is othogonal... > > I'd expect for example that the process was something like > > Parse pending commands until either > 1. Queue empties > 2. A time target passes > > For each command we need to shove a pixmap over add it > to the buffer to transfer > > Do a single CLFLUSH and maybe IPI > > Fire up the command queue > > Keep the buffers hanging around until there is memory pressure > if we may reuse that pixmap > > Can you clarify that ? So at the moment a pixmap maps directly to a kernel buffer object which is a bunch of pages that get faulted in on the CPU or allocated when the buffer is to be used by the GPU. So when a pixmap is created a buffer object is created, when a pixmap is destroyed a buffer object is destroyed. Perhaps I can cache a bunch of buffer objects in userspace for re-use as pixmaps but I'm not really sure that will scale too well. When X wishes the GPU to access a buffer (pixmap), it calls into the kernel with a single ioctl with a list of all buffers the GPU is going to access along with a buffer containing the command to do the access, now at the moment, when each of those buffers is bound into the GART for the first time the system does a change_page_attr for each page and calls the global flush[1]. Now if a buffer is bound into the GART and gets accessed from the CPU later again (software fallback) we have the choice of taking it back out of the GART and letting the nopfn call fault back in the pages uncached or we can flush the tlb and bring them back in cached. We are hoping to avoid software fallbacks on the hardware platforms we want to work on as much as possible. Finally when a buffer is destroyed, the pages are released back to the system, so of course the pages are set back to cached and we need another tlb/cache flush per pixmap buffer destructor. So you can see why some sort of uncached+writecombined page cache would be useful, I could just allocate a bunch of pages at startup as uncached+writecombined, and allocate pixmaps from them and when I bind/free the pixmap I don't need the flush at all, now I'd really like this to be part of the VM so that under memory pressure it can just take the pages I've got in my cache back and after flushing turn them back into cached pages, the other option is for the DRM to do this on its own and penalise the whole system. [1]. (this is one inefficiency in that if multiple buffers are being bound in for the first time it'll flush for each of them, I'm trying to get rid of this inefficiency but I may need to tweak the order of things as at the moment, it crashes hard if I tried to leave the cache/tlb flush until later.) Dave. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: uncached page allocator 2007-08-21 6:05 ` uncached page allocator Dave Airlie @ 2007-08-21 14:56 ` Peter Zijlstra 2007-08-27 20:00 ` Christoph Lameter 2007-08-30 7:23 ` Nick Piggin 0 siblings, 2 replies; 4+ messages in thread From: Peter Zijlstra @ 2007-08-21 14:56 UTC (permalink / raw) To: Dave Airlie Cc: Alan Cox, dri-devel, Linux Kernel Mailing List, Linux Memory Management On Tue, 2007-08-21 at 16:05 +1000, Dave Airlie wrote: > So you can see why some sort of uncached+writecombined page cache > would be useful, I could just allocate a bunch of pages at startup as > uncached+writecombined, and allocate pixmaps from them and when I > bind/free the pixmap I don't need the flush at all, now I'd really > like this to be part of the VM so that under memory pressure it can > just take the pages I've got in my cache back and after flushing turn > them back into cached pages, the other option is for the DRM to do > this on its own and penalise the whole system. Can't you make these pages part of the regular VM by sticking them all into an address_space. And for this reclaim behaviour you'd only need to set PG_private and have a_ops->releasepage() dtrt. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: uncached page allocator 2007-08-21 14:56 ` Peter Zijlstra @ 2007-08-27 20:00 ` Christoph Lameter 2007-08-30 7:23 ` Nick Piggin 1 sibling, 0 replies; 4+ messages in thread From: Christoph Lameter @ 2007-08-27 20:00 UTC (permalink / raw) To: Dave Airlie Cc: Peter Zijlstra, Alan Cox, dri-devel, Linux Kernel Mailing List, Linux Memory Management, jes There is an uncached allocator in IA64 arch code (linux/arch/ia64/kernel/uncached.c). Maybe having a look at that will help? Jes wrote it. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: uncached page allocator 2007-08-21 14:56 ` Peter Zijlstra 2007-08-27 20:00 ` Christoph Lameter @ 2007-08-30 7:23 ` Nick Piggin 1 sibling, 0 replies; 4+ messages in thread From: Nick Piggin @ 2007-08-30 7:23 UTC (permalink / raw) To: Peter Zijlstra Cc: Dave Airlie, Alan Cox, dri-devel, Linux Kernel Mailing List, Linux Memory Management Peter Zijlstra wrote: > On Tue, 2007-08-21 at 16:05 +1000, Dave Airlie wrote: > > >>So you can see why some sort of uncached+writecombined page cache >>would be useful, I could just allocate a bunch of pages at startup as >>uncached+writecombined, and allocate pixmaps from them and when I >>bind/free the pixmap I don't need the flush at all, now I'd really >>like this to be part of the VM so that under memory pressure it can >>just take the pages I've got in my cache back and after flushing turn >>them back into cached pages, the other option is for the DRM to do >>this on its own and penalise the whole system. > > > Can't you make these pages part of the regular VM by sticking them all > into an address_space. > > And for this reclaim behaviour you'd only need to set PG_private and > have a_ops->releasepage() dtrt. I'd just suggest Dave just registers a shrinker to start with. You really want to be able to batch TLB flushes as well, which ->releasepage may not be so good at (you could add more machinery behind the releasepage to build batches and so on, but anyway, a shrinker might be the quickest way to get something working). -- SUSE Labs, Novell Inc. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2007-08-30 7:23 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <21d7e9970708191745h3b579f3bp72f138e089c624da@mail.gmail.com>
[not found] ` <20070820094125.209e0811@the-village.bc.nu>
2007-08-21 6:05 ` uncached page allocator Dave Airlie
2007-08-21 14:56 ` Peter Zijlstra
2007-08-27 20:00 ` Christoph Lameter
2007-08-30 7:23 ` Nick Piggin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox