Re: [PATCH v2 8/8] mm: defer second attempt at merge on mmap()

[Vlastimil Babka <vbabka@suse.cz>]

On 10/23/24 22:38, Lorenzo Stoakes wrote:
> Rather than trying to merge again when ostensibly allocating a new VMA,
> instead defer until the VMA is added and attempt to merge the existing
> range.
> 
> This way we have no complicated unwinding logic midway through the process
> of mapping the VMA.
> 
> In addition this removes limitations on the VMA not being able to be the
> first in the virtual memory address space which was previously implicitly
> required.
> 
> It also performs this merge after the final flag adjustments are performed,
> something that was not done previously and thus might have prevented
> possibly valid merges in the past.
> 
> In theory, for this very same reason, we should unconditionally attempt
> merge here, however this is likely to have a performance impact so it is
> better to avoid this given the unlikely outcome of a merge.

Maybe just expand the cases where we set map->retry_merge, i.e. in case the
final flag adjustments do anything?

> The vmg state will already have been reset by the first attempt at a merge
> so we only need to reset the iterator, set the vma and flags and try again.
> 
> Signed-off-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>

Reviewed-by: Vlastimil Babka <vbabka@suse.cz>

A nit:

> ---
>  mm/vma.c | 75 ++++++++++++++++++++------------------------------------
>  1 file changed, 26 insertions(+), 49 deletions(-)
> 
> diff --git a/mm/vma.c b/mm/vma.c
> index 065f5e1f65be..c493ecebf394 100644
> --- a/mm/vma.c
> +++ b/mm/vma.c
> @@ -19,6 +19,7 @@ struct mmap_state {
>  	struct file *file;
>  
>  	unsigned long charged;
> +	bool retry_merge;
>  
>  	struct vm_area_struct *prev;
>  	struct vm_area_struct *next;
> @@ -2280,9 +2281,9 @@ static int __mmap_prepare(struct mmap_state *map, struct vma_merge_struct *vmg,
>  	return 0;
>  }
>  
> +
>  static int __mmap_new_file_vma(struct mmap_state *map,
> -			       struct vma_merge_struct *vmg,
> -			       struct vm_area_struct **vmap, bool *mergedp)
> +			       struct vm_area_struct **vmap)
>  {
>  	struct vma_iterator *vmi = map->vmi;
>  	struct vm_area_struct *vma = *vmap;
> @@ -2311,37 +2312,11 @@ static int __mmap_new_file_vma(struct mmap_state *map,
>  			!(map->flags & VM_MAYWRITE) &&
>  			(vma->vm_flags & VM_MAYWRITE));
>  
> -	vma_iter_config(vmi, map->addr, map->end);
> -	/*
> -	 * If flags changed after mmap_file(), we should try merge
> -	 * vma again as we may succeed this time.
> -	 */
> -	if (unlikely(map->flags != vma->vm_flags && map->prev)) {
> -		struct vm_area_struct *merge;
> -
> -		vmg->flags = vma->vm_flags;
> -		/* If this fails, state is reset ready for a reattempt. */
> -		merge = vma_merge_new_range(vmg);
> -
> -		if (merge) {
> -			/*
> -			 * ->mmap() can change vma->vm_file and fput
> -			 * the original file. So fput the vma->vm_file
> -			 * here or we would add an extra fput for file
> -			 * and cause general protection fault
> -			 * ultimately.
> -			 */
> -			fput(vma->vm_file);
> -			vm_area_free(vma);
> -			vma = merge;
> -			*mergedp = true;
> -		} else {
> -			vma_iter_config(vmi, map->addr, map->end);
> -		}
> -	}
> +	/* If the flags change (and are mergeable), let's retry later. */
> +	map->retry_merge = vma->vm_flags != map->flags && !(vma->vm_flags & VM_SPECIAL);
>  
> +	vma_iter_config(vmi, map->addr, map->end);

Do we need this still? __mmap_new_vma() did that and nothing changed since
in the non-error case, AFAICS?

>  	map->flags = vma->vm_flags;
> -	*vmap = vma;
>  	return 0;
>  }
>