How to use huge pages in drivers?

[Dominique Martinet <asmadeus@codewreck.org>]

Hi ; not quite sure where to ask so will start here...


Some context first. I'm inquiring in the context of mckernel[1], a
lightweight kernel that works next to linux (basically offlines a
few/most cores, reserve some memory and have boot a second OS on that to
run HPC applications).
Being brutally honest here, this is mostly research and anyone here
looking into it will probably scream, but I might as well try not to add
too many more reasons to do so....

One of the mecanisms here is that sometimes we want to access the
mckernel memory from linux (either from the process that spawned the
mckernel side process or from a driver in linux), and to do that we have
mapped the mckernel side virtual memory range to that process so it can
page fault.
The (horrible) function doing that can be found here[2], rus_vm_fault -
sends a message to the other side to identify the physical address
corresponding from what we had reserved earlier and map it quite
manually.

We could know at this point if it had been a huge page (very likely) or
not; I'm observing a huge difference of performance with some
interconnect if I add a huge kludge emulating huge pages here (directly
manipulating the process' page table) so I'd very much like to use huge
pages when we know a huge page has been mapped on the other side.



What I'd like to know is:
 - we know (assuming the other side isn't too bugged, but if it is we're
fucked up anyway) exactly what huge-page-sized physical memory range has
been mapped on the other side, is there a way to manually gather the
pages corresponding and merge them into a huge page?

 - from what I understand that does not seem possible/recommended, the
way to go being to have a userland process get huge pages and pass these
to a device (ioctl or something); but I assume that means said process
needs to keep on running all the time that memory is required?
If the page fault needs to split the page (because the other side handed
a "small" page so we can only map a regular page here), can it be merged
back into a huge page for the next time this physical region is used?


[1] https://github.com/RIKEN-SysSoft/mckernel
[2] https://github.com/RIKEN-SysSoft/mckernel/blob/development/executer/kernel/mcctrl/syscall.c#L538

Any input will be appreciated,
-- 
Dominique Martinet