Re: [PATCH v2] KASAN: prohibit KASAN+STRUCTLEAK combination

[Dennis Zhou <dennisszhou@gmail.com>]

Hi Kees,

On Mon, Apr 30, 2018 at 04:41:24PM -0700, Kees Cook wrote:
> I prefer this change over moving the plugin earlier since that ends up
> creating redundant initializers...

To be clear, what I was proposing was to move the plugin to execute
later rather than earlier. It currently runs before the
early_optimizations pass, while *all_optimizations is after inlining.
Apologizes for this being a half baked idea due to my limited
understanding.

I am hoping someone could chime in and help me understand how gcc
handles inlining. My assumption is that at the beginning, inlined
defined functions will be processed by the pass as any other function.
If the function can be inlined, it is inlined and no longer needs to be
kept around. If it cannot be inlined, it is kept around. An assumption
that I'm not sure is correct is that a function is either always inlined
or not inlined in a translation unit.

The current plugin puts an initializer in both the inlined function and
the locations that it will be inlined as both functions are around,
hence duplicate initializers. Below is a snippet of pass output from
earlier reproducing code of the issue.

My understanding is initializer 1 is created due to inlining moving
variable declarations to the encompassing functions scope. Then the
structleak_plugin performs the pass not finding an initializer and
creates one. Initializer 2 is created for the inlined function and is
propagated. So I guess this problem is also order dependent in which the
functions are processed.

An important difference in running in a later pass, which may be a deal
breaker, is that objects will only be initialized once. So if a function
gets inlined inside a for loop, the initializer will only be a part of
the encompassing function rather than also in each iteration. In the
example below, initializer 2 would not be there as the inlined function
wouldn't be around and processed by the structleak_plugin.

Thanks for taking the time to humor me, this is the extent of my
understanding of the problem and gcc.

Thanks,
Dennis

------

union
{
struct list_head * __val;
char __c[1];
} __u;

<bb 2> [0.00%]:
__u = {};    <---- initializer 1
p_8 = malloc (160);
i_9 = 0;
goto <bb 10>; [0.00%]

<bb 3> [0.00%]:
_1 = (long unsigned int) i_4;
_2 = _1 * 16;
_3 = p_8 + _2;
list_14 = _3;
__u = {};    <---- initializer 2
ASAN_MARK (UNPOISON, &__u, 8);