RE: [RFC 0/2] New MAP_PMEM_AWARE mmap flag

["Rudoff, Andy" <andy.rudoff@intel.com>]

>> This is surprising to me, and goes completely against the proposed 
>> programming model.  In fact, this is a very basic tenet of the 
>> operation of the nvml libraries on pmem.io.
>
>It's simply impossible to provide.  But then again pmem.io seems to be much more about hype than reality anyway.

Well that comment woke me up :-)

I think several things are getting mixed together in this discussion:

First, one primary reason DAX exists is so that applications can access persistence directly.  Once mappings are set up, latency-sensitive apps get load/store access and can flush stores themselves using instructions rather than kernel calls.

Second, programming to load/store persistence is tricky, but the usual API for programming to memory-mapped files will "just work" and we built on that to avoid needlessly creating new permission & naming models.  If you want to use msync() or fsync(), it will work, but may not perform as well as using the instructions.  The instructions give you very fine-grain flushing control, but the downside is that the app must track what it changes at that fine granularity.  Both models work, but there's a trade-off.

So what can be done to make persistent memory easier to use?  I think this is where the debate really is.  Using memory-mapped files and the instructions directly is difficult.  The libraries available on pmem.io are meant to make it easier (providing transactions, memory allocation, etc) but it is still difficult.  But what about just taking applications that use mmap() and giving them DAX without their knowledge?  Is that a way to leverage pmem more easily, without forcing an application to change?  I think this is analogous to forcing O_DIRECT on applications without their knowledge.  There may be cases where it works, but there will always be better leverage of the technology if the application is architected to use it.

There are applications already modified to use DAX for pmem and to flush stores themselves (using NVDIMMs for testing, but planning for the higher-capacity pmem to become available).  Some are using the libraries on pmem.io, some are not.  Those are pmem-aware applications and I haven't seen any incorrect expectations on what happens with copy-on-write or page faults that fill in holes in a file.  Maybe there's a case to be made for applications getting DAX transparently, but I think that's not the only usage and the model we've been pushing where an application is pmem aware seems to be getting traction.

-andy

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>