BUG: vdso changes expose elf mapping issue

[Ryan Roberts <ryan.roberts@arm.com>]

Hi,

I'm hitting a nasty bug which is preventing VSCode from connecting to my arm64 
VM running v6.15-rc3. Bisection fingers Commit 0b3bc3354eb9 ("arm64: vdso: 
Switch to generic storage implementation") as the point where this started 
failing.

Debugging this, the root cause is due to ldconfig crashing with a segmentation 
fault (I have no idea why VSCode thinks it needs to run this...). The segfault 
happens because ldconfig's attempt to expand the program break fails because 
vvar/vdso are in the way. The above change expands vvar by 2 pages and this 
causes the problem.

But I don't think we can really blame this commit...

ldconfig is a statically linked, PIE executable. The kernel treats this as an 
interpreter and therefore does not map it into low memory but instead maps it 
into high memory using mmap() (mmap is top-down on arm64). Once it's mapped, 
vvar/vdso gets mapped and fills the hole right at the top that is left due to 
ldconfig's alignment requirements. Before the above change, there were 2 pages 
free between the end of the data segment and vvar; this was enough for ldconfig 
to get it's required memory with brk(). But after the change there is no space:

Before:
fffff7f20000-fffff7fde000 r-xp 00000000 fe:02 8110426                    /home/ubuntu/glibc-2.35/build/elf/ldconfig
fffff7fee000-fffff7ff5000 rw-p 000be000 fe:02 8110426                    /home/ubuntu/glibc-2.35/build/elf/ldconfig
fffff7ff5000-fffff7ffa000 rw-p 00000000 00:00 0 
fffff7ffc000-fffff7ffe000 r--p 00000000 00:00 0                          [vvar]
fffff7ffe000-fffff8000000 r-xp 00000000 00:00 0                          [vdso]
fffffffdf000-1000000000000 rw-p 00000000 00:00 0                         [stack]

After:
fffff7f20000-fffff7fde000 r-xp 00000000 fe:02 8110426                    /home/ubuntu/glibc-2.35/build/elf/ldconfig
fffff7fee000-fffff7ff5000 rw-p 000be000 fe:02 8110426                    /home/ubuntu/glibc-2.35/build/elf/ldconfig
fffff7ff5000-fffff7ffa000 rw-p 00000000 00:00 0 
fffff7ffa000-fffff7ffe000 r--p 00000000 00:00 0                          [vvar]
fffff7ffe000-fffff8000000 r-xp 00000000 00:00 0                          [vdso]
fffffffdf000-1000000000000 rw-p 00000000 00:00 0                         [stack]

Note that this issue only occurs with ASLR disabled. When ASLR is enabled, the 
brk region is setup in the low memory region that would normally be used by 
primary executable.

So the issue is that when ASLR is disabled, these statically linked, PIE 
programs are mapped with insufficient space to expand the break.

I think in an ideal world, the kernel would notice that this is not an 
interpreter and map it to low memory. But I guess we can't know that for the 
case where the interpreter is invoked directly (as apposed to being referenced 
in the .interp section of the invoked binary)?

Another option would be to always relocate the break to low memory (but without 
the random offset for the ASLR=off case). But it looks like there could be some 
compat issues there? I see CONFIG_COMPAT_BRK...

Or we could just ensure we enforce some dead space after the end of the program 
that nothing else is (initially) mapped into. I think this could be done by 
overallocating the initial MAP_FIXED_NOREPLACE mmap, then munmapping the hole 
after ARCH_SETUP_ADDITIONAL_PAGES(). But it's not really clear what the correct 
reservation size would be, and any mmaps the program does will start to fill 
that space.

I'm hoping someone has some suggestions...

Thanks,
Ryan