Re: [PATCH] mm: Add RWH_RMAP_EXCLUDE flag to exclude files from rmap sharing

[Pedro Falcato <pfalcato@suse.de>]

I'm not sure how you're using get_maintainer.pl, but PLEASE CC people
correctly.

On Tue, Apr 21, 2026 at 10:09:32AM +0800, Yibin Liu wrote:
> UnixBench execl/shellscript (dynamically linked binaries) at 64+ cores are
> bottlenecked on the i_mmap_rwsem semaphore due to heavy vma insert/remove
> operations on the i_mmap tree, where libc.so.6 is the most frequent,
> followed by ld-linux-x86-64.so.2 and the test executable itself.
> 
> This patch marks such files to skip rmap operations, avoiding frequent
> interval tree insert/remove that cause i_mmap_rwsem lock contention.
> The downside is these files can no longer be reclaimed (along with compact

This does not work.

> and ksm), but since they are small and resident anyway, it's acceptable.

Which ones? ELF executables commonly have a lot of padding. How do you know
what to mark as !no_rmap? Who has permissions for that? This patch allows
any user to create a full DoS of the system by simply marking a really large
file as no-reclaim.

> When all mapping processes exit, files can still be reclaimed normally.
> 
> Performance testing shows ~80% improvement in UnixBench execl/shellscript
> scores on Hygon 7490, AMD zen4 9754 and Intel emerald rapids platform.
> 
> Signed-off-by: Yibin Liu <liuyibin@hygon.cn>
> ---
>  fs/fcntl.c                 | 1 +
>  fs/open.c                  | 6 ++++++
>  include/linux/fs.h         | 3 +++
>  include/uapi/linux/fcntl.h | 1 +
>  mm/mmap.c                  | 3 ++-
>  mm/vma.c                   | 8 +++++---
>  6 files changed, 18 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/fcntl.c b/fs/fcntl.c
> index beab8080badf..9b7cc1544735 100644
> --- a/fs/fcntl.c
> +++ b/fs/fcntl.c
> @@ -349,6 +349,7 @@ static bool rw_hint_valid(u64 hint)
>  	case RWH_WRITE_LIFE_MEDIUM:
>  	case RWH_WRITE_LIFE_LONG:
>  	case RWH_WRITE_LIFE_EXTREME:
> +	case RWH_RMAP_EXCLUDE:
>  		return true;
>  	default:
>  		return false;
> diff --git a/fs/open.c b/fs/open.c
> index 681d405bc61e..643ab7c6b461 100644
> --- a/fs/open.c
> +++ b/fs/open.c
> @@ -46,6 +46,10 @@ int do_truncate(struct mnt_idmap *idmap, struct dentry *dentry,
>  	if (length < 0)
>  		return -EINVAL;
>  
> +	/* Prevent truncate on files marked as RMAP_EXCLUDE (e.g., libc, ld.so) */
> +	if (filp && (filp->f_mode & FMODE_RMAP_EXCLUDE))
> +		return -EPERM;
> +
>  	newattrs.ia_size = length;
>  	newattrs.ia_valid = ATTR_SIZE | time_attrs;
>  	if (filp) {
> @@ -892,6 +896,8 @@ static int do_dentry_open(struct file *f,
>  	path_get(&f->f_path);
>  	f->f_inode = inode;
>  	f->f_mapping = inode->i_mapping;
> +	if (inode->i_write_hint == RWH_RMAP_EXCLUDE)
> +		f->f_mode |= FMODE_RMAP_EXCLUDE;
>  	f->f_wb_err = filemap_sample_wb_err(f->f_mapping);
>  	f->f_sb_err = file_sample_sb_err(f);
>  
> diff --git a/include/linux/fs.h b/include/linux/fs.h
> index 11559c513dfb..d5c9e5a4c2b9 100644
> --- a/include/linux/fs.h
> +++ b/include/linux/fs.h
> @@ -189,6 +189,9 @@ typedef int (dio_iodone_t)(struct kiocb *iocb, loff_t offset,
>  /* File does not contribute to nr_files count */
>  #define FMODE_NOACCOUNT		((__force fmode_t)(1 << 29))
>  
> +/* File should exclude vma from rmap interval tree */
> +#define FMODE_RMAP_EXCLUDE	((__force fmode_t)(1 << 30))
> +
>  /*
>   * The two FMODE_NONOTIFY* define which fsnotify events should not be generated
>   * for an open file. These are the possible values of
> diff --git a/include/uapi/linux/fcntl.h b/include/uapi/linux/fcntl.h
> index aadfbf6e0cb3..4969b4762071 100644
> --- a/include/uapi/linux/fcntl.h
> +++ b/include/uapi/linux/fcntl.h
> @@ -72,6 +72,7 @@
>  #define RWH_WRITE_LIFE_MEDIUM	3
>  #define RWH_WRITE_LIFE_LONG	4
>  #define RWH_WRITE_LIFE_EXTREME	5
> +#define RWH_RMAP_EXCLUDE	6

Userspace does not know what "rmap" is, nor does it need to know. Even if you
found a workable solution for the permission side, this is not a good solution,
and it's not a good interface. It also might solve it for you, but definitely
does not solve the whole issue with having a lock there (which you are, for
the record, still acquiring).

-- 
Pedro