Re: HugeTLB mapping for drivers (sample driver)

[Alexey Korolev <akorolex@gmail.com>]

Hi,
>
> Did the OOM killer really trigger and select a process for killing or
> did the process itself just get killed with an out-of-memory message? I
> would have expected the latter.
>
OMM killer triggered in case of private mapping on attempt to access a
page under private mapping. It was because code did not check the pages
availability at mmap time. Will be fixed.

>> In fact there should be quite few cases when private mapping makes
>> sense for drivers and mapping DMA buffers. I thought about possible
>> solutions. The question is what to choose.
>>
>> 1. Forbid private mappings for drivers in case of hugetlb. (But this
>> limits functionality - it is not so good)
>
> For a long time, this was the "solution" for hugetlbfs.
>
>> 2. Allow private mapping. Use hugetlbfs hstates. (But it forces user
>> to know how much hugetlb memory it is necessary to reserve for
>> drivers)
>
> You can defer working out the reservations until mmap() time,
> particularly if you are using dynamic hugepage pool resizing instead of
> static allocation.
>
>> 3. Allow private mapping. Use special hstate for driver and driver
>> should tell how much memory needs to be reserved for it. (Not clear
>> yet how to behave if we are out of reserved space)
>>
>> Could you please suggest what is the best solution? May be some other options?
>>
>
> The only solution that springs to mind is the same one used by hugetlbfs
> and that is that reservations are taken at mmap() time for the size of the
> mapping. In your case, you prefault but either way, the hugepages exist.
>
Yes, that looks sane. I'll follow this way. In a particular case if
driver do not
need a private mapping mmap will return error. Thanks for the advice.
I'm about
to modify the patches. I'll try to involve  hugetlb reservation
functions as much  as
possible and track reservations by special hstate for drivers.

> What then happens for hugetlbfs is that only the process that called mmap()
> is guaranteed their faults will succeed. If a child process incurs a COW
> and the hugepages are not available, the child process gets killed. If
> the parent process performs COW and the huge pages are not available, it
> unmaps the pages from the child process so that COW becomes unnecessary. If
> the child process then faults, it gets killed.  This is implemented in
> mm/hugetlb.c#unmap_ref_private().

So on out of memory COW hugetlb code prefer applications to be killed by
SIGSEGV (SIGBUS?) instead of OOM. Okk.

Thanks,
Alexey

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>