Re: [RFC PATCH v0 2/2] mm: sched: Batch-migrate misplaced pages

[Gregory Price <gourry@gourry.net>]

On Wed, May 21, 2025 at 01:32:38PM +0530, Bharata B Rao wrote:
>  
> +static void task_check_pending_migrations(struct task_struct *curr)
> +{
> +	struct callback_head *work = &curr->numa_mig_work;
> +
> +	if (work->next != work)
> +		return;
> +
> +	if (time_after(jiffies, curr->numa_mig_interval) ||
> +	    (curr->migrate_count > NUMAB_BATCH_MIGRATION_THRESHOLD)) {
> +		curr->numa_mig_interval = jiffies + HZ;
> +		task_work_add(curr, work, TWA_RESUME);
> +	}
> +}
> +
>  /*
>   * Drive the periodic memory faults..
>   */
> @@ -3610,6 +3672,8 @@ static void task_tick_numa(struct rq *rq, struct task_struct *curr)
>  	if (!curr->mm || (curr->flags & (PF_EXITING | PF_KTHREAD)) || work->next != work)
>  		return;
>  
> +	task_check_pending_migrations(curr);
> +

So I know this was discussed in the cover leter a bit and alluded to in
the patch, but I want to add my 2cents from work on the unmapped page
cache set.

In that set, I chose to always schedule the task work on the next return
to user-space, rather than defer to a tick like the current numa-balance
code.  This was for two concerns:

1) I didn't want to leave a potentially large number of isolated folios
   on a list that may not be reaped for an unknown period of time.

   I don't know the real limitations on the number of isolated folios,
   but given what we have here I think we can represent a mathematical
   worst case on the nubmer of stranded folios.

   If (N=1,000,000, and M=511) then we could have ~1.8TB of pages
   stranded on these lists - never to be migrated because it never hits
   the threshhold.  In practice this won't happen to that extreme, but
   in practice it absolutely will happen for some chunk of tasks.

   So I chose to never leave kernel space with isolated folios on the
   task numa_mig_list.

   This discussion changes if the numa_mig_list is not on the
   task_struct and instead some per-cpu list routinely reaped by a
   kthread (kpromoted or whatever).
 

2) I was not confident I could measure the performance implications of
   the migrations directly when it was deferred.  When would I even know
   it happened?  The actual goal is to *not* know it happened, right?

   But now it might happen during a page fault, or any random syscall.

   This concerned me - so i just didn't defer.  That was largely out of
   lack of confidence in my own understanding of the task_work system.


So i think this, as presented, is a half-measure - and I don't think
it's a good half-measure.  I think we might need to go all the way to a
set of per-cpu migration lists that a kernel work can pluck the head of
on some interval.  That would bound the number of isolated folios to the
number of CPUs rather than the number of tasks.

~Gregory