Re: Resurrecting the VM_PINNED discussion

[Eric B Munson <emunson@akamai.com>]

[-- Attachment #1: Type: text/plain, Size: 2789 bytes --]

On Tue, 03 Mar 2015, Vlastimil Babka wrote:

> On 03/03/2015 06:41 PM, Eric B Munson wrote:> All,
> >
> > After LSF/MM last year Peter revived a patch set that would create
> > infrastructure for pinning pages as opposed to simply locking them.
> > AFAICT, there was no objection to the set, it just needed some help
> > from the IB folks.
> >
> > Am I missing something about why it was never merged?  I ask because
> > Akamai has bumped into the disconnect between the mlock manpage,
> > Documentation/vm/unevictable-lru.txt, and reality WRT compaction and
> > locking.  A group working in userspace read those sources and wrote a
> > tool that mmaps many files read only and locked, munmapping them when
> > they are no longer needed.  Locking is used because they cannot afford a
> > major fault, but they are fine with minor faults.  This tends to
> > fragment memory badly so when they started looking into using hugetlbfs
> > (or anything requiring order > 0 allocations) they found they were not
> > able to allocate the memory.  They were confused based on the referenced
> > documentation as to why compaction would continually fail to yield
> > appropriately sized contiguous areas when there was more than enough
> > free memory.
> 
> So you are saying that mlocking (VM_LOCKED) prevents migration and thus
> compaction to do its job? If that's true, I think it's a bug as it is AFAIK
> supposed to work just fine.

Agreed.  But as has been discussed in the threads around the VM_PINNED
work, there are people that are relying on the fact that VM_LOCKED
promises no minor faults.  Which is why the behavoir has remained.

> 
> > I would like to see the situation with VM_LOCKED cleared up, ideally the
> > documentation would remain and reality adjusted to match and I think
> > Peter's VM_PINNED set goes in the right direction for this goal.  What
> > is missing and how can I help?
> 
> I don't think VM_PINNED would help you. In fact it is VM_PINNED that improves
> accounting for the kind of locking (pinning) that *does* prevent page migration
> (unlike mlocking)... quoting the patchset cover letter:

VM_PINNED itself doesn't help us, but it would allow us to make
VM_LOCKED use only the weaker 'no major fault' semantics while still
providing a way for anyone that needs the stronger 'no minor fault'
promise to get the semantics they need.

> 
> "These patches introduce VM_PINNED infrastructure, vma tracking of persistent
> 'pinned' page ranges. Pinned is anything that has a fixed phys address (as
> required for say IO DMA engines) and thus cannot use the weaker VM_LOCKED. One
> popular way to pin pages is through get_user_pages() but that not nessecarily
> the only way."
> 
> > Thanks,
> > Eric
> >
> 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]