Re: [DISCUSSION] anon_vma root lock contention and per anon_vma lock

[Matthew Wilcox <willy@infradead.org>]

On Mon, Sep 15, 2025 at 08:23:38AM +0800, Barry Song wrote:
> > I wonder if we could fix this by adding a new syscall:
> >
> >         mremap(addr, size, size, MREMAP_COW_NOW);
> >
> > That would create a new VMA that contains the COWed pages from the
> > old VMA, but crucially no longer attached to the anon_vma root of
> > the zygote.  You wouldn't want to call this for every VMA, of course.
> > Just the ones which are likely to be fully COWed.
> >
> > Maybe this isn't practical, but I thought it worth suggesting.
> 
> Lorenzo suggested possibly unlinking the child anon_vma from the root once all
> folios have been CoW-ed:
> 
> "Right now, even if you entirely CoW everything in a VMA, we are still
> attached to parents with all the overhead. That's something I can look at.
> "
> 
> My concern is that it’s difficult to determine whether a VMA has been completely
> CoW-ed, and a single shared folio would prevent the unlink.
> So I’m not sure this approach would work.

I'm concerned that tracking how many folios remain shared may be
inefficient.  Also that information needs to be gathered in both parent
and child.

> You seem to be proposing a forced CoW as a way to safely unlink from the root.
> 
> A side effect is the potential for sudden, heavy memory allocation,
> whereas CoW lets asynchronous tasks such as kswap work concurrently.

Perhaps you could help us out with some stats on that -- how much
anonymous memory starts out shared between the zygote and a newly
spawned process?

> Another issue is the extra memory use from folios that could have been
> shared but aren’t—likely minor on Android, since only a small portion
> of memory is actually shared, based on our observations.
> 
> Calling mremap for each VMA might be difficult. Something applied to the
> whole process could be more practical—similar to exec, but only
> performing CoW and unlinking the anon_vma root.

That seems like it would be worse for memory consumption than doing it
on the VMAs in question.

Another possibility would be for the zygote to set a flag on the VMA,
say EAGER_COW which forces a COW of all pages as soon as the first one
is COWed.  But then we're paying at fault time rather than in a syscall
that we can predict.

Another point in favour of COW_NOW or EAGER_COW is that we can choose to
allocate folios of the appropriate size at that time.  Unless something's
changed, I think we always COW individual pages rather than multiple
pages at once.