hugetlb_cow and tlb invalidation on x86

hugetlb_cow and tlb invalidation on x86

[Anthony I]

[-- Attachment #1: Type: text/plain, Size: 1459 bytes --]

Hi all,

Currently huge_ptep_clear_flush on x86 is a nop. How is dtlb invalidation
handled, for the copy-on-write cases on huge pages ?

I have actually verified that dtlb consistency is maintained with an
example code, under the following scenario:

- parent allocates a 2M hugepage (via mmap/MAP_HUGETLB), fills page with
some random data
- forks a child process
- clones a thread
(all parent, child and thread run on separate cores, pinned).

The child process as well as the thread, read the contents of the hugepage
(thus there is a dtlb entry at the core that the thread runs)

At a later point, parent writes into the page, thus inducing a CoW fault.
Since this is a hugepage, there is no tlb page flushing taking place (no
tlb-flushing IPIs). My assumption is that the thread would be now reading
from the physical page that belongs to the child after CoW, since the
parent pgtable pte is now pointing to a newly allocated page, but the core
executing the thread has not received any tlb invalidation interrupts (thus
it would be following the "old" tlb entry). Oddly enough, this does not
hold true (the thread can see the updated page).

Going through the intel devel manuals, I do not see how this would happen.
It does not seem that large pages are treated differently from 4K pages as
far as tlb invalidation goes.

Any ideas ? Does the kernel somehow manage this in a different way, or is
it an x86 thing that is non-obvious ?

Regards,
Anthony

[-- Attachment #2: Type: text/html, Size: 1569 bytes --]