Re: How to use huge pages in drivers?

[Matthew Wilcox <willy@infradead.org>]

On Thu, Sep 05, 2019 at 05:44:00PM +0200, Dominique Martinet wrote:
> > You shouldn't be calling vmf_insert_pfn_pmd() from a regular ->fault
> > handler, as by then the fault handler has already inserted a PMD.
> > The ->huge_fault handler is the place to call it from.
> > 
> > You may need to force PMD-alignment for your call to mmap().
> 
> I was missing setting the VM_HUGE_FAULT vm_flags2 bit in the vma - the
> huge_fault handler is now called, and I no longer have the pre-existing
> pmd problem; that's a much better solution than manually fiddling with
> flags :)
> 
> Question though - is it ok to insert small pages if the huge_fault
> handler is called with PE_SIZE_PMD ?
> (I think the pte insertion will automatically create the pmd, but would
> be good to confirm)

No, you need to return VM_FAULT_FALLBACK, at which point the generic code
will create a PMD for you and then call your ->fault handler which can
insert PTEs.

It works the same way from PUDs to PMDs by the way, in case you ever
have a 1GB mapping ;-)

> Now I've got this I'm back to where I stood with my kludge though,
> programs work until they exit, and the zap_huge_pmd() function tries to
> withdraw the pagetable from some magic field that was never set in my
> case... I realize this is old code no longer upstream, but my new
> workaround for this (looking at the zap_huge_pmd function) was to
> pretend my file is dax.
> Now that I've set it as dax I think it actually makes sense as in
> "there's memory here that points to something linux no longer manages
> directly, just let it be" and we might benefit from the other exceptions
> dax have, I'll need to look at what this implies in more details...

I think that should be fine, but I don't really know RHEL 7.3 all that
well ;-)

> > Hope these pointers are slightly more useful than a rubber duck ;-)
> 
> Much appreciated, thank you for taking the time! :)

No problem ... these APIs are relatively new and not necessarily all
that intuitive.