Re: [PATCH 6.4 000/292] 6.4.5-rc1 review

[Naresh Kamboju <naresh.kamboju@linaro.org>]

On Mon, 24 Jul 2023 at 15:50, Alexander Potapenko <glider@google.com> wrote:
>
> On Sat, Jul 22, 2023 at 6:37 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > [ Removed the stable reviewers, bringing in the kfence people ]
> >
> > See
> >
> >   https://lore.kernel.org/lkml/CA+G9fYvgy22wiY=c3wLOrCM6o33636abhtEynXhJkqxJh4ca0A@mail.gmail.com/
> >
> > for the original report. The warning was introduced in 8f0b36497303
> > ("mm: kfence: fix objcgs vector allocation"), and Google doesn't find
> > any other cases of this.
> >
> > Anybody?
> >
> >                     Linus
> >
>
>
> > > NOTE:
> > > The following kernel warning was noticed while booting qemu-arm64
> > > with these configs enabled on stable rc 6.4.5-rc1.
> > >
> > >  CONFIG_ARM64_64K_PAGES=y
> > >  CONFIG_KFENCE=y
>
> Is there a full config somewhere?

Please find build details
 - https://storage.tuxsuite.com/public/linaro/lkft/builds/2StEPFnEfoD076PRu8fIxjexhnM/
 - https://storage.tuxsuite.com/public/linaro/lkft/builds/2StEPFnEfoD076PRu8fIxjexhnM/config
 - https://storage.tuxsuite.com/public/linaro/lkft/builds/2StEPFnEfoD076PRu8fIxjexhnM/vmlinux.xz
 - https://storage.tuxsuite.com/public/linaro/lkft/builds/2StEPFnEfoD076PRu8fIxjexhnM/System.map
 - https://storage.tuxsuite.com/public/linaro/lkft/builds/2StEPFnEfoD076PRu8fIxjexhnM/Image.gz

>
> > > This crash is not easily reproducible.
>
> CONFIG_KFENCE_SAMPLE_INTERVAL=10
> CONFIG_KFENCE_NUM_OBJECTS=2048
>
> might improve reproducibility.

The above test have following Kconfigs enabled.

CONFIG_HAVE_ARCH_KASAN=y
CONFIG_HAVE_ARCH_KASAN_SW_TAGS=y
CONFIG_HAVE_ARCH_KASAN_HW_TAGS=y
CONFIG_HAVE_ARCH_KASAN_VMALLOC=y
CONFIG_CC_HAS_KASAN_GENERIC=y
CONFIG_CC_HAS_KASAN_SW_TAGS=y
CONFIG_CC_HAS_WORKING_NOSANITIZE_ADDRESS=y
# CONFIG_KASAN is not set
CONFIG_HAVE_ARCH_KFENCE=y
CONFIG_KFENCE=y
CONFIG_KFENCE_SAMPLE_INTERVAL=100
CONFIG_KFENCE_NUM_OBJECTS=255
# CONFIG_KFENCE_DEFERRABLE is not set
CONFIG_KFENCE_STRESS_TEST_FAULTS=0

>
> > >
> > > boot logs:
> > > --------
> > > [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x000f0510]
> > > [    0.000000] Linux version 6.4.5-rc1 (tuxmake@tuxmake)
> > > (aarch64-linux-gnu-gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils
> > > for Debian) 2.40) #1 SMP PREEMPT @1689957802
> > > [    0.000000] random: crng init done
> > > [    0.000000] Machine model: linux,dummy-virt
> > > ...
> > > [    0.006821] kfence: initialized - using 33554432 bytes for 255
> > > objects at 0x(____ptrval____)-0x(____ptrval____)
> > > ...
> > > [    7.726994] ------------[ cut here ]------------
> > > [    7.727704] WARNING: CPU: 1 PID: 1 at mm/kfence/core.c:1097
> > > __kfence_free+0x84/0xc8
> > ...
> > > [    7.746478] Call trace:
> > > [    7.746776]  __kfence_free+0x84/0xc8
> > > [    7.747134]  __slab_free+0x490/0x508
> > > [    7.748063]  __kmem_cache_free+0x2b4/0x2d0
> > > [    7.748377]  kfree+0x78/0x140
> > > [    7.748638]  single_release+0x40/0x60
> > > [    7.750664]  __fput+0x78/0x260
> > > [    7.751065]  ____fput+0x18/0x30
> > > [    7.752086]  task_work_run+0x80/0xe0
> > > [    7.753122]  do_notify_resume+0x200/0x1398
> > > [    7.754292]  el0_svc+0xec/0x100
> > > [    7.754573]  el0t_64_sync_handler+0xf4/0x120
> > > [    7.755559]  el0t_64_sync+0x190/0x198
>
> It would be interesting to see the contents of
> /sys/kernel/debug/kfence/objects together with the object address.
> Would it be possible to boot the kernel with no_hash_pointers and add
> a line printing the object address:
>
> diff --git a/mm/kfence/core.c b/mm/kfence/core.c
> index dad3c0eb70a01..23f27f6cb18cf 100644
> --- a/mm/kfence/core.c
> +++ b/mm/kfence/core.c
> @@ -1094,7 +1094,10 @@ void __kfence_free(void *addr)
>         struct kfence_metadata *meta = addr_to_metadata((unsigned long)addr);
>
>  #ifdef CONFIG_MEMCG
> -       KFENCE_WARN_ON(meta->objcg);
> +       if (meta->objcg) {
> +               pr_err("ADDR: %px\n", addr);
> +               KFENCE_WARN_ON(1);
> +       }
>  #endif
>         /*
>          * If the objects of the cache are SLAB_TYPESAFE_BY_RCU, defer freeing
>
> , and then dump /sys/kernel/debug/kfence/objects?
>

This testing is running on CI loops and however, I will try to reproduce
this locally.

> Knowing the kfence pool location (the line starting with "kfence:
> initialized") and the object address, we can probably understand from
> the allocation stack in sysfs, whether the object is supposed to be
> deleted.

- Naresh