[PATCH v2] mm: skip folio_activate() for mlocked folios

[PATCH v2] mm: skip folio_activate() for mlocked folios

[Dmitry Ilvokhin]

__mlock_folio() does not move folio to unevicable LRU, when
folio_activate() removes folio from LRU.

To prevent this case also check for folio_test_mlocked() in
folio_mark_accessed(). If folio is not yet marked as unevictable, but
already marked as mlocked, then skip folio_activate() call to allow
__mlock_folio() to make all necessary updates. It should be safe to skip
folio_activate() here, because mlocked folio should end up in
unevictable LRU eventually anyway.

To observe the problem mmap() and mlock() big file and check Unevictable
and Mlocked values from /proc/meminfo. On freshly booted system without
any other mlocked memory we expect them to match or be quite close.

See below for more detailed reproduction steps. Source code of stat.c is
available at [1].

  $ head -c 8G < /dev/urandom > /tmp/random.bin

  $ cc -pedantic -Wall -std=c99 stat.c -O3 -o /tmp/stat
  $ /tmp/stat
  Unevictable:     8389668 kB
  Mlocked:         8389700 kB

  Need to run binary twice. Problem does not reproduce on the first run,
  but always reproduces on the second run.

  $ /tmp/stat
  Unevictable:     5374676 kB
  Mlocked:         8389332 kB

[1]: https://gist.github.com/ilvokhin/e50c3d2ff5d9f70dcbb378c6695386dd

Co-developed-by: Kiryl Shutsemau <kas@kernel.org>
Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
Acked-by: Usama Arif <usamaarif642@gmail.com>
---
Changes in v2:
  - Rephrase commit message: frame it in terms of unevicable LRU, not stat
    accounting.

 mm/swap.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/mm/swap.c b/mm/swap.c
index 2260dcd2775e..f682f070160b 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -469,6 +469,16 @@ void folio_mark_accessed(struct folio *folio)
 		 * this list is never rotated or maintained, so marking an
 		 * unevictable page accessed has no effect.
 		 */
+	} else if (folio_test_mlocked(folio)) {
+		/*
+		 * Pages that are mlocked, but not yet on unevictable LRU.
+		 * They might be still in mlock_fbatch waiting to be processed
+		 * and activating it here might interfere with
+		 * mlock_folio_batch(). __mlock_folio() will fail
+		 * folio_test_clear_lru() check and give up. It happens because
+		 * __folio_batch_add_and_move() clears LRU flag, when adding
+		 * folio to activate batch.
+		 */
 	} else if (!folio_test_active(folio)) {
 		/*
 		 * If the folio is on the LRU, queue it for activation via
-- 
2.47.3

Re: [PATCH v2] mm: skip folio_activate() for mlocked folios

[Nhat Pham]

On Mon, Oct 6, 2025 at 6:25 AM Dmitry Ilvokhin <d@ilvokhin.com> wrote:
>
> __mlock_folio() does not move folio to unevicable LRU, when
> folio_activate() removes folio from LRU.
>
> To prevent this case also check for folio_test_mlocked() in
> folio_mark_accessed(). If folio is not yet marked as unevictable, but
> already marked as mlocked, then skip folio_activate() call to allow
> __mlock_folio() to make all necessary updates. It should be safe to skip
> folio_activate() here, because mlocked folio should end up in
> unevictable LRU eventually anyway.
>
> To observe the problem mmap() and mlock() big file and check Unevictable
> and Mlocked values from /proc/meminfo. On freshly booted system without
> any other mlocked memory we expect them to match or be quite close.
>
> See below for more detailed reproduction steps. Source code of stat.c is
> available at [1].
>
>   $ head -c 8G < /dev/urandom > /tmp/random.bin
>
>   $ cc -pedantic -Wall -std=c99 stat.c -O3 -o /tmp/stat
>   $ /tmp/stat
>   Unevictable:     8389668 kB
>   Mlocked:         8389700 kB
>
>   Need to run binary twice. Problem does not reproduce on the first run,
>   but always reproduces on the second run.
>
>   $ /tmp/stat
>   Unevictable:     5374676 kB
>   Mlocked:         8389332 kB
>
> [1]: https://gist.github.com/ilvokhin/e50c3d2ff5d9f70dcbb378c6695386dd
>
> Co-developed-by: Kiryl Shutsemau <kas@kernel.org>
> Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
> Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
> Acked-by: Usama Arif <usamaarif642@gmail.com>

Acked-by: Nhat Pham <nphamcs@gmail.com>

Re: [PATCH v2] mm: skip folio_activate() for mlocked folios

[SeongJae Park]

On Mon, 6 Oct 2025 13:25:26 +0000 Dmitry Ilvokhin <d@ilvokhin.com> wrote:

> __mlock_folio() does not move folio to unevicable LRU, when
> folio_activate() removes folio from LRU.

A trivial opinion.  So the user-visible issue is the incorrect meminfo, right?

I read your changelog below saying you changed this message from v1 to frame on
unevictable LRU rather than stat accounting, and I think that's nice to
understand the detail.  But I think further describing the resulting
user-visible issue can be helpful at better understanding the motivation of
this nice patch.

> 
> To prevent this case also check for folio_test_mlocked() in
> folio_mark_accessed(). If folio is not yet marked as unevictable, but
> already marked as mlocked, then skip folio_activate() call to allow
> __mlock_folio() to make all necessary updates. It should be safe to skip
> folio_activate() here, because mlocked folio should end up in
> unevictable LRU eventually anyway.
> 
> To observe the problem mmap() and mlock() big file and check Unevictable
> and Mlocked values from /proc/meminfo. On freshly booted system without
> any other mlocked memory we expect them to match or be quite close.
> 
> See below for more detailed reproduction steps. Source code of stat.c is
> available at [1].
> 
>   $ head -c 8G < /dev/urandom > /tmp/random.bin
> 
>   $ cc -pedantic -Wall -std=c99 stat.c -O3 -o /tmp/stat
>   $ /tmp/stat
>   Unevictable:     8389668 kB
>   Mlocked:         8389700 kB
> 
>   Need to run binary twice. Problem does not reproduce on the first run,
>   but always reproduces on the second run.
> 
>   $ /tmp/stat
>   Unevictable:     5374676 kB
>   Mlocked:         8389332 kB
> 
> [1]: https://gist.github.com/ilvokhin/e50c3d2ff5d9f70dcbb378c6695386dd
> 
> Co-developed-by: Kiryl Shutsemau <kas@kernel.org>
> Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
> Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
> Acked-by: Usama Arif <usamaarif642@gmail.com>

Because this is a fix of a user-visible issue, I'm wondering if this deserves
Fixes: and Cc: stable@.

Anyway my comments are only trivial ones, and I think the change is good.

Reviewed-by: SeongJae Park <sj@kernel.org>

> ---
> Changes in v2:
>   - Rephrase commit message: frame it in terms of unevicable LRU, not stat
>     accounting.

Yet another trivial and personal opinion.  Adding a link to the previous
version could be helpful for reviewers like me.

Thanks,
SJ

[...]

Re: [PATCH v2] mm: skip folio_activate() for mlocked folios

[Kiryl Shutsemau]

On Tue, Oct 07, 2025 at 12:53:13PM -0700, SeongJae Park wrote:
> On Mon, 6 Oct 2025 13:25:26 +0000 Dmitry Ilvokhin <d@ilvokhin.com> wrote:
> 
> > __mlock_folio() does not move folio to unevicable LRU, when
> > folio_activate() removes folio from LRU.
> 
> A trivial opinion.  So the user-visible issue is the incorrect meminfo, right?

The user-visible effect is that we unnecessary postpone moving pages to
unevictable LRU that lead to unexpected stats: Mlocked > Unevictable.

> > ---
> > Changes in v2:
> >   - Rephrase commit message: frame it in terms of unevicable LRU, not stat
> >     accounting.
> 
> Yet another trivial and personal opinion.  Adding a link to the previous
> version could be helpful for reviewers like me.

You probably missed recent Linus rant on Link: tags :P

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCH v2] mm: skip folio_activate() for mlocked folios

[SeongJae Park]

On Wed, 8 Oct 2025 11:33:01 +0100 Kiryl Shutsemau <kas@kernel.org> wrote:

> On Tue, Oct 07, 2025 at 12:53:13PM -0700, SeongJae Park wrote:
> > On Mon, 6 Oct 2025 13:25:26 +0000 Dmitry Ilvokhin <d@ilvokhin.com> wrote:
> > 
> > > __mlock_folio() does not move folio to unevicable LRU, when
> > > folio_activate() removes folio from LRU.
> > 
> > A trivial opinion.  So the user-visible issue is the incorrect meminfo, right?
> 
> The user-visible effect is that we unnecessary postpone moving pages to
> unevictable LRU that lead to unexpected stats: Mlocked > Unevictable.

Thank you for clarifying!

> 
> > > ---
> > > Changes in v2:
> > >   - Rephrase commit message: frame it in terms of unevicable LRU, not stat
> > >     accounting.
> > 
> > Yet another trivial and personal opinion.  Adding a link to the previous
> > version could be helpful for reviewers like me.
> 
> You probably missed recent Linus rant on Link: tags :P

I didn't miss it [1] :)  And I pretty sure Linus wouldn't be angry for this,
since what he dislikes is meaningless Link: tags on commit messages.  We are
discussing something about changelog after the '---' line, which will anyway
not added to the commit message.  Also I'm saying a link in a general form, not
specifically Link: tag.

[1] https://lwn.net/Articles/1037069/


Thanks,
SJ

[...]

Re: [PATCH v2] mm: skip folio_activate() for mlocked folios

[Shakeel Butt]

[Somehow I messed up the subject, so resending]

Cc Hugh, yangge, David

On Mon, Oct 06, 2025 at 01:25:26PM +0000, Dmitry Ilvokhin wrote:
> __mlock_folio() does not move folio to unevicable LRU, when
> folio_activate() removes folio from LRU.
> 
> To prevent this case also check for folio_test_mlocked() in
> folio_mark_accessed(). If folio is not yet marked as unevictable, but
> already marked as mlocked, then skip folio_activate() call to allow
> __mlock_folio() to make all necessary updates. It should be safe to skip
> folio_activate() here, because mlocked folio should end up in
> unevictable LRU eventually anyway.
> 
> To observe the problem mmap() and mlock() big file and check Unevictable
> and Mlocked values from /proc/meminfo. On freshly booted system without
> any other mlocked memory we expect them to match or be quite close.
> 
> See below for more detailed reproduction steps. Source code of stat.c is
> available at [1].
> 
>   $ head -c 8G < /dev/urandom > /tmp/random.bin
> 
>   $ cc -pedantic -Wall -std=c99 stat.c -O3 -o /tmp/stat
>   $ /tmp/stat
>   Unevictable:     8389668 kB
>   Mlocked:         8389700 kB
> 
>   Need to run binary twice. Problem does not reproduce on the first run,
>   but always reproduces on the second run.
> 
>   $ /tmp/stat
>   Unevictable:     5374676 kB
>   Mlocked:         8389332 kB
> 
> [1]: https://gist.github.com/ilvokhin/e50c3d2ff5d9f70dcbb378c6695386dd
> 
> Co-developed-by: Kiryl Shutsemau <kas@kernel.org>
> Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
> Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
> Acked-by: Usama Arif <usamaarif642@gmail.com>
> ---
> Changes in v2:
>   - Rephrase commit message: frame it in terms of unevicable LRU, not stat
>     accounting.
> 
>  mm/swap.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/mm/swap.c b/mm/swap.c
> index 2260dcd2775e..f682f070160b 100644
> --- a/mm/swap.c
> +++ b/mm/swap.c
> @@ -469,6 +469,16 @@ void folio_mark_accessed(struct folio *folio)
>  		 * this list is never rotated or maintained, so marking an
>  		 * unevictable page accessed has no effect.
>  		 */
> +	} else if (folio_test_mlocked(folio)) {
> +		/*
> +		 * Pages that are mlocked, but not yet on unevictable LRU.
> +		 * They might be still in mlock_fbatch waiting to be processed
> +		 * and activating it here might interfere with
> +		 * mlock_folio_batch(). __mlock_folio() will fail
> +		 * folio_test_clear_lru() check and give up. It happens because
> +		 * __folio_batch_add_and_move() clears LRU flag, when adding
> +		 * folio to activate batch.
> +		 */

This makes sense as activating an mlocked folio should be a noop but I
am wondering why we are seeing this now. By this, I mean mlock()ed
memory being delayed to get to unevictable LRU. Also I remember Hugh
recently [1] removed the difference betwen mlock percpu cache and other
percpu caches of clearing LRU bit on entry. Does you repro work even
with Hugh's changes or without it?

[1] https://lore.kernel.org/all/05905d7b-ed14-68b1-79d8-bdec30367eba@google.com/

Re: [PATCH v2] mm: skip folio_activate() for mlocked folios

[Dmitry Ilvokhin]

On Wed, Oct 08, 2025 at 09:17:49AM -0700, Shakeel Butt wrote:
> [Somehow I messed up the subject, so resending]
> 
> Cc Hugh, yangge, David
> 
> On Mon, Oct 06, 2025 at 01:25:26PM +0000, Dmitry Ilvokhin wrote:
> > __mlock_folio() does not move folio to unevicable LRU, when
> > folio_activate() removes folio from LRU.
> > 
> > To prevent this case also check for folio_test_mlocked() in
> > folio_mark_accessed(). If folio is not yet marked as unevictable, but
> > already marked as mlocked, then skip folio_activate() call to allow
> > __mlock_folio() to make all necessary updates. It should be safe to skip
> > folio_activate() here, because mlocked folio should end up in
> > unevictable LRU eventually anyway.
> > 
> > To observe the problem mmap() and mlock() big file and check Unevictable
> > and Mlocked values from /proc/meminfo. On freshly booted system without
> > any other mlocked memory we expect them to match or be quite close.
> > 
> > See below for more detailed reproduction steps. Source code of stat.c is
> > available at [1].
> > 
> >   $ head -c 8G < /dev/urandom > /tmp/random.bin
> > 
> >   $ cc -pedantic -Wall -std=c99 stat.c -O3 -o /tmp/stat
> >   $ /tmp/stat
> >   Unevictable:     8389668 kB
> >   Mlocked:         8389700 kB
> > 
> >   Need to run binary twice. Problem does not reproduce on the first run,
> >   but always reproduces on the second run.
> > 
> >   $ /tmp/stat
> >   Unevictable:     5374676 kB
> >   Mlocked:         8389332 kB
> > 
> > [1]: https://gist.github.com/ilvokhin/e50c3d2ff5d9f70dcbb378c6695386dd
> > 
> > Co-developed-by: Kiryl Shutsemau <kas@kernel.org>
> > Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
> > Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
> > Acked-by: Usama Arif <usamaarif642@gmail.com>
> > ---
> > Changes in v2:
> >   - Rephrase commit message: frame it in terms of unevicable LRU, not stat
> >     accounting.
> > 
> >  mm/swap.c | 10 ++++++++++
> >  1 file changed, 10 insertions(+)
> > 
> > diff --git a/mm/swap.c b/mm/swap.c
> > index 2260dcd2775e..f682f070160b 100644
> > --- a/mm/swap.c
> > +++ b/mm/swap.c
> > @@ -469,6 +469,16 @@ void folio_mark_accessed(struct folio *folio)
> >  		 * this list is never rotated or maintained, so marking an
> >  		 * unevictable page accessed has no effect.
> >  		 */
> > +	} else if (folio_test_mlocked(folio)) {
> > +		/*
> > +		 * Pages that are mlocked, but not yet on unevictable LRU.
> > +		 * They might be still in mlock_fbatch waiting to be processed
> > +		 * and activating it here might interfere with
> > +		 * mlock_folio_batch(). __mlock_folio() will fail
> > +		 * folio_test_clear_lru() check and give up. It happens because
> > +		 * __folio_batch_add_and_move() clears LRU flag, when adding
> > +		 * folio to activate batch.
> > +		 */
> 
> This makes sense as activating an mlocked folio should be a noop but I
> am wondering why we are seeing this now. By this, I mean mlock()ed
> memory being delayed to get to unevictable LRU. Also I remember Hugh
> recently [1] removed the difference betwen mlock percpu cache and other
> percpu caches of clearing LRU bit on entry. Does you repro work even
> with Hugh's changes or without it?
>

Thanks Shakeel for mentioning Hugh's patch, I was not aware of it.
Indeed, I could not reproduce problem on top of Hugh's patch anymore,
which totally make sense, because folio_test_clear_lru() is gone from
__folio_batch_add_and_move().

Now I wonder does folio_test_mlocked() check still make sense in the
current codebase?

> [1] https://lore.kernel.org/all/05905d7b-ed14-68b1-79d8-bdec30367eba@google.com/

Re: [PATCH v2] mm: skip folio_activate() for mlocked folios

[Andrew Morton]

On Wed, 8 Oct 2025 18:06:07 +0000 Dmitry Ilvokhin <d@ilvokhin.com> wrote:

> > > +		 * They might be still in mlock_fbatch waiting to be processed
> > > +		 * and activating it here might interfere with
> > > +		 * mlock_folio_batch(). __mlock_folio() will fail
> > > +		 * folio_test_clear_lru() check and give up. It happens because
> > > +		 * __folio_batch_add_and_move() clears LRU flag, when adding
> > > +		 * folio to activate batch.
> > > +		 */
> > 
> > This makes sense as activating an mlocked folio should be a noop but I
> > am wondering why we are seeing this now. By this, I mean mlock()ed
> > memory being delayed to get to unevictable LRU. Also I remember Hugh
> > recently [1] removed the difference betwen mlock percpu cache and other
> > percpu caches of clearing LRU bit on entry. Does you repro work even
> > with Hugh's changes or without it?
> >
> 
> Thanks Shakeel for mentioning Hugh's patch, I was not aware of it.
> Indeed, I could not reproduce problem on top of Hugh's patch anymore,
> which totally make sense, because folio_test_clear_lru() is gone from
> __folio_batch_add_and_move().
> 
> Now I wonder does folio_test_mlocked() check still make sense in the
> current codebase?
> 
> > [1] https://lore.kernel.org/all/05905d7b-ed14-68b1-79d8-bdec30367eba@google.com/

So I take it that this patch ("mm: skip folio_activate() for mlocked
folios") is no longer needed?

Re: [PATCH v2] mm: skip folio_activate() for mlocked folios

[Dmitry Ilvokhin]

On Wed, Oct 15, 2025 at 12:59:11PM -0700, Andrew Morton wrote:
> On Wed, 8 Oct 2025 18:06:07 +0000 Dmitry Ilvokhin <d@ilvokhin.com> wrote:
> 
> > > > +		 * They might be still in mlock_fbatch waiting to be processed
> > > > +		 * and activating it here might interfere with
> > > > +		 * mlock_folio_batch(). __mlock_folio() will fail
> > > > +		 * folio_test_clear_lru() check and give up. It happens because
> > > > +		 * __folio_batch_add_and_move() clears LRU flag, when adding
> > > > +		 * folio to activate batch.
> > > > +		 */
> > > 
> > > This makes sense as activating an mlocked folio should be a noop but I
> > > am wondering why we are seeing this now. By this, I mean mlock()ed
> > > memory being delayed to get to unevictable LRU. Also I remember Hugh
> > > recently [1] removed the difference betwen mlock percpu cache and other
> > > percpu caches of clearing LRU bit on entry. Does you repro work even
> > > with Hugh's changes or without it?
> > >
> > 
> > Thanks Shakeel for mentioning Hugh's patch, I was not aware of it.
> > Indeed, I could not reproduce problem on top of Hugh's patch anymore,
> > which totally make sense, because folio_test_clear_lru() is gone from
> > __folio_batch_add_and_move().
> > 
> > Now I wonder does folio_test_mlocked() check still make sense in the
> > current codebase?
> > 
> > > [1] https://lore.kernel.org/all/05905d7b-ed14-68b1-79d8-bdec30367eba@google.com/
> 
> So I take it that this patch ("mm: skip folio_activate() for mlocked
> folios") is no longer needed?

Yes, this is my understanding as well. Hugh's patch addressed initial
problem and this patch is no longer needed.