linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Ivan Babrou <ivan@cloudflare.com>
To: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Steven Rostedt <rostedt@goodmis.org>,
	kernel-team <kernel-team@cloudflare.com>,
	 Ignat Korchagin <ignat@cloudflare.com>,
	Hailong liu <liu.hailong6@zte.com.cn>,
	 Andrey Ryabinin <aryabinin@virtuozzo.com>,
	Alexander Potapenko <glider@google.com>,
	 Dmitry Vyukov <dvyukov@google.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	 Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	x86@kernel.org,  "H. Peter Anvin" <hpa@zytor.com>,
	Miroslav Benes <mbenes@suse.cz>,
	Julien Thierry <jthierry@redhat.com>,
	 Jiri Slaby <jirislaby@kernel.org>,
	kasan-dev@googlegroups.com, linux-mm@kvack.org,
	 linux-kernel <linux-kernel@vger.kernel.org>,
	Alasdair Kergon <agk@redhat.com>,
	 Mike Snitzer <snitzer@redhat.com>,
	dm-devel@redhat.com,  Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Martin KaFai Lau <kafai@fb.com>,
	 Song Liu <songliubraving@fb.com>, Yonghong Song <yhs@fb.com>,
	Andrii Nakryiko <andriin@fb.com>,
	 John Fastabend <john.fastabend@gmail.com>,
	KP Singh <kpsingh@chromium.org>,
	 Robert Richter <rric@kernel.org>,
	"Joel Fernandes (Google)" <joel@joelfernandes.org>,
	 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	 Linux Kernel Network Developers <netdev@vger.kernel.org>,
	bpf@vger.kernel.org
Subject: Re: BUG: KASAN: stack-out-of-bounds in unwind_next_frame+0x1df5/0x2650
Date: Thu, 4 Feb 2021 10:41:18 -0800	[thread overview]
Message-ID: <CABWYdi15x=-2qenWSdX_ONSha_Pz7GFJrx8axN6CJS5cWxTTSg@mail.gmail.com> (raw)
In-Reply-To: <20210204030948.dmsmwyw6fu5kzgey@treble>

On Wed, Feb 3, 2021 at 7:10 PM Josh Poimboeuf <jpoimboe@redhat.com> wrote:

> This line gives a big clue:
>
>   [160676.608966][    C4] RIP: 0010:0xffffffffc17d814c
>
> That address, without a function name, most likely means that it was
> running in some generated code (mostly likely BPF) when it got
> interrupted.

We do have eBPF/XDP in our environment.

> Right now, the ORC unwinder tries to fall back to frame pointers when it
> encounters generated code:
>
>         orc = orc_find(state->signal ? state->ip : state->ip - 1);
>         if (!orc)
>                 /*
>                  * As a fallback, try to assume this code uses a frame pointer.
>                  * This is useful for generated code, like BPF, which ORC
>                  * doesn't know about.  This is just a guess, so the rest of
>                  * the unwind is no longer considered reliable.
>                  */
>                 orc = &orc_fp_entry;
>                 state->error = true;
>         }
>
> Because the ORC unwinder is guessing from that point onward, it's
> possible for it to read the KASAN stack redzone, if the generated code
> hasn't set up frame pointers.  So the best fix may be for the unwinder
> to just always bypass KASAN when reading the stack.
>
> The unwinder has a mechanism for detecting and warning about
> out-of-bounds, and KASAN is short-circuiting that.
>
> This should hopefully get rid of *all* the KASAN unwinder warnings, both
> crypto and networking.

It definitely worked on my dm-crypt case, and I've tried it without
your previous AVX related patch. I will apply it to our tree and
deploy to the staging KASAN environment to see how it fares with
respect to networking stacks. Feel free to ping me if I don't get back
to you with the results on Monday.

Thanks for looking into this!


      reply	other threads:[~2021-02-04 18:41 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-29  3:35 Ivan Babrou
2021-02-03  3:09 ` Ivan Babrou
2021-02-03 16:46   ` Peter Zijlstra
2021-02-03 17:46     ` Ivan Babrou
2021-02-03 19:05       ` Josh Poimboeuf
2021-02-03 22:41         ` Ivan Babrou
2021-02-03 23:27           ` Josh Poimboeuf
2021-02-03 23:30             ` Ivan Babrou
2021-02-04  0:17               ` Josh Poimboeuf
2021-02-04  0:52                 ` Ivan Babrou
2021-02-04  2:37                   ` Josh Poimboeuf
2021-02-04 19:51                 ` Ivan Babrou
2021-02-04 20:22                   ` Josh Poimboeuf
2021-02-04  9:22       ` Peter Zijlstra
2021-02-04  2:44   ` Steven Rostedt
2021-02-04  3:09     ` Josh Poimboeuf
2021-02-04 18:41       ` Ivan Babrou [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CABWYdi15x=-2qenWSdX_ONSha_Pz7GFJrx8axN6CJS5cWxTTSg@mail.gmail.com' \
    --to=ivan@cloudflare.com \
    --cc=agk@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=andriin@fb.com \
    --cc=aryabinin@virtuozzo.com \
    --cc=ast@kernel.org \
    --cc=bp@alien8.de \
    --cc=bpf@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=dm-devel@redhat.com \
    --cc=dvyukov@google.com \
    --cc=glider@google.com \
    --cc=hpa@zytor.com \
    --cc=ignat@cloudflare.com \
    --cc=jirislaby@kernel.org \
    --cc=joel@joelfernandes.org \
    --cc=john.fastabend@gmail.com \
    --cc=jpoimboe@redhat.com \
    --cc=jthierry@redhat.com \
    --cc=kafai@fb.com \
    --cc=kasan-dev@googlegroups.com \
    --cc=kernel-team@cloudflare.com \
    --cc=kpsingh@chromium.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=liu.hailong6@zte.com.cn \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mbenes@suse.cz \
    --cc=mingo@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=rric@kernel.org \
    --cc=snitzer@redhat.com \
    --cc=songliubraving@fb.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    --cc=yhs@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox