Re: [PATCH -next] mm: usercopy: add a debugfs interface to bypass the vmalloc check.

[Uladzislau Rezki <urezki@gmail.com>]

On Tue, Dec 03, 2024 at 09:30:09PM +0800, Kefeng Wang wrote:
> 
> 
> On 2024/12/3 21:10, zuoze wrote:
> > 
> > 
> > 在 2024/12/3 20:39, Uladzislau Rezki 写道:
> > > On Tue, Dec 03, 2024 at 07:23:44PM +0800, zuoze wrote:
> > > > We have implemented host-guest communication based on the TUN device
> > > > using XSK[1]. The hardware is a Kunpeng 920 machine (ARM architecture),
> > > > and the operating system is based on the 6.6 LTS version with kernel
> > > > version 6.6. The specific stack for hotspot collection is as follows:
> > > > 
> > > > -  100.00%     0.00%  vhost-12384  [unknown]      [k] 0000000000000000
> > > >     - ret_from_fork
> > > >        - 99.99% vhost_task_fn
> > > >           - 99.98% 0xffffdc59f619876c
> > > >              - 98.99% handle_rx_kick
> > > >                 - 98.94% handle_rx
> > > >                    - 94.92% tun_recvmsg
> > > >                       - 94.76% tun_do_read
> > > >                          - 94.62% tun_put_user_xdp_zc
> > > >                             - 63.53% __check_object_size
> > > >                                - 63.49% __check_object_size.part.0
> > > >                                     find_vmap_area
> > > >                             - 30.02% _copy_to_iter
> > > >                                  __arch_copy_to_user
> > > >                    - 2.27% get_rx_bufs
> > > >                       - 2.12% vhost_get_vq_desc
> > > >                            1.49% __arch_copy_from_user
> > > >                    - 0.89% peek_head_len
> > > >                         0.54% xsk_tx_peek_desc
> > > >                    - 0.68% vhost_add_used_and_signal_n
> > > >                       - 0.53% eventfd_signal
> > > >                            eventfd_signal_mask
> > > >              - 0.94% handle_tx_kick
> > > >                 - 0.94% handle_tx
> > > >                    - handle_tx_copy
> > > >                       - 0.59% vhost_tx_batch.constprop.0
> > > >                            0.52% tun_sendmsg
> > > > 
> > > > It can be observed that most of the overhead is concentrated in the
> > > > find_vmap_area function.
> > > > 
> > > I see. Yes, it is pretty contented, since you run the v6.6 kernel. There
> > > was a work that tends to improve it to mitigate a vmap lock contention.
> > > See it here: https://lwn.net/Articles/956590/
> > > 
> > > The work was taken in the v6.9 kernel:
> > > 
> > > <snip>
> > > commit 38f6b9af04c4b79f81b3c2a0f76d1de94b78d7bc
> > > Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
> > > Date:   Tue Jan 2 19:46:23 2024 +0100
> > > 
> > >      mm: vmalloc: add va_alloc() helper
> > > 
> > >      Patch series "Mitigate a vmap lock contention", v3.
> > > 
> > >      1. Motivation
> > > ...
> > > <snip>
> > > 
> > > Could you please try the v6.9 kernel on your setup?
> > > 
> > > How to solve it, probably, it can be back-ported to the v6.6 kernel.
> > 
> > All the vmalloc-related optimizations have already been merged into 6.6,
> > including the set of optimization patches you suggested. Thank you very
> > much for your input.
> > 
> 
> It is unclear, we have backported the vmalloc optimization into our 6.6
> kernel before, so the above stack already with those patches and even
> with those optimization, the find_vmap_area() is still the hotpots.
> 
> 
Could you please check that all below patches are in your v6.6 kernel?


<snip>
commit 8be4d46e12af32342569840d958272dbb3be3f4c
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Wed Jan 24 19:09:20 2024 +0100

    mm: vmalloc: refactor vmalloc_dump_obj() function

commit 15e02a39fb6b43f37100563c6a246252d5d1e6da
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Wed Jan 24 19:09:19 2024 +0100

    mm: vmalloc: improve description of vmap node layer

commit 7679ba6b36dbb300b757b672d6a32a606499e14b
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Tue Jan 2 19:46:33 2024 +0100

    mm: vmalloc: add a shrinker to drain vmap pools

commit 8f33a2ff307248c3e55a7696f60b3658b28edb57
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Tue Jan 2 19:46:32 2024 +0100

    mm: vmalloc: set nr_nodes based on CPUs in a system

commit 8e1d743f2c2671aa54f6f91a2b33823f92512870
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Tue Jan 2 19:46:31 2024 +0100

    mm: vmalloc: support multiple nodes in vmallocinfo

commit 53becf32aec1c8049b854f0c31a11df5ed75df6f
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Tue Jan 2 19:46:30 2024 +0100

    mm: vmalloc: support multiple nodes in vread_iter

commit 96aa8437d169b8e030a98e2b74fd9a8ee9d3be7e
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Fri Feb 2 20:06:28 2024 +0100

    mm: vmalloc: add a scan area of VA only once

commit 72210662c5a2b6005f6daea7fe293a0dc573e1a5
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Tue Jan 2 19:46:29 2024 +0100

    mm: vmalloc: offload free_vmap_area_lock lock

commit 282631cb2447318e2a55b41a665dbe8571c46d70
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Tue Jan 2 19:46:28 2024 +0100

    mm: vmalloc: remove global purge_vmap_area_root rb-tree

commit 55c49fee57af99f3c663e69dedc5b85e691bbe50
Author: Baoquan He <bhe@redhat.com>
Date:   Tue Jan 2 19:46:27 2024 +0100

    mm/vmalloc: remove vmap_area_list

commit d093602919ad5908532142a048539800fa94a0d1
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Tue Jan 2 19:46:26 2024 +0100

    mm: vmalloc: remove global vmap_area_root rb-tree

commit 7fa8cee003166ef6db0bba70d610dbf173543811
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Tue Jan 2 19:46:25 2024 +0100

    mm: vmalloc: move vmap_init_free_space() down in vmalloc.c


commit 5b75b8e1b9040b34f43809b1948eaa4e83e39112
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Tue Jan 2 19:46:24 2024 +0100

    mm: vmalloc: rename adjust_va_to_fit_type() function


commit 38f6b9af04c4b79f81b3c2a0f76d1de94b78d7bc
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Tue Jan 2 19:46:23 2024 +0100

    mm: vmalloc: add va_alloc() helper
<snip>

Thank you!

--
Uladzislau Rezki