From: Jens Axboe <axboe@kernel.dk>
To: Qian Cai <cai@lca.pw>
Cc: viro@zeniv.linux.org.uk, hare@suse.com, bcrl@kvack.org,
linux-aio@kvack.org, Linux-MM <linux-mm@kvack.org>
Subject: Re: io_submit with slab free object overwritten
Date: Fri, 22 Feb 2019 15:40:00 -0700 [thread overview]
Message-ID: <cd3bac6a-02b2-351e-3f81-322c2e0ca03e@kernel.dk> (raw)
In-Reply-To: <4a56fc9f-27f7-5cb5-feed-a4e33f05a5d1@lca.pw>
On 2/21/19 10:40 PM, Qian Cai wrote:
> This is only reproducible on linux-next (20190221), as v5.0-rc7 is fine. Running
> two LTP tests and then reboot will trigger this on ppc64le (CONFIG_IO_URING=n
> and CONFIG_SHUFFLE_PAGE_ALLOCATOR=y).
>
> # fgetxattr02
> # io_submit01
> # systemctl reboot
>
> There is a 32-bit (with all ones) overwritten of free slab objects (poisoned).
>
> [23424.121182] BUG aio_kiocb (Tainted: G B W L ): Poison overwritten
> [23424.121189]
> -----------------------------------------------------------------------------
> [23424.121189]
> [23424.121197] INFO: 0x000000009f1f5145-0x00000000841e301b. First byte 0xff
> instead of 0x6b
> [23424.121205] INFO: Allocated in io_submit_one+0x9c/0xb20 age=0 cpu=7 pid=12174
> [23424.121212] __slab_alloc+0x34/0x60
> [23424.121217] kmem_cache_alloc+0x504/0x5c0
> [23424.121221] io_submit_one+0x9c/0xb20
> [23424.121224] sys_io_submit+0xe0/0x350
> [23424.121227] system_call+0x5c/0x70
> [23424.121231] INFO: Freed in aio_complete+0x31c/0x410 age=0 cpu=7 pid=12174
> [23424.121234] kmem_cache_free+0x4bc/0x540
> [23424.121237] aio_complete+0x31c/0x410
> [23424.121240] blkdev_bio_end_io+0x238/0x3e0
> [23424.121243] bio_endio.part.3+0x214/0x330
> [23424.121247] brd_make_request+0x2d8/0x314 [brd]
> [23424.121250] generic_make_request+0x220/0x510
> [23424.121254] submit_bio+0xc8/0x1f0
> [23424.121256] blkdev_direct_IO+0x36c/0x610
> [23424.121260] generic_file_read_iter+0xbc/0x230
> [23424.121263] blkdev_read_iter+0x50/0x80
> [23424.121266] aio_read+0x138/0x200
> [23424.121269] io_submit_one+0x7c4/0xb20
> [23424.121272] sys_io_submit+0xe0/0x350
> [23424.121275] system_call+0x5c/0x70
> [23424.121278] INFO: Slab 0x00000000841158ec objects=85 used=85 fp=0x
> (null) flags=0x13fffc000000200
> [23424.121282] INFO: Object 0x000000007e677ed8 @offset=5504 fp=0x00000000e42bdf6f
> [23424.121282]
> [23424.121287] Redzone 000000005483b8fc: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb ................
> [23424.121291] Redzone 00000000b842fe53: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb ................
> [23424.121295] Redzone 00000000deb0d052: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb ................
> [23424.121299] Redzone 0000000014045233: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb ................
> [23424.121302] Redzone 00000000dd5d6c16: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb ................
> [23424.121306] Redzone 00000000538b5478: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb ................
> [23424.121310] Redzone 000000001f7fb704: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb ................
> [23424.121314] Redzone 0000000000e0484d: bb bb bb bb bb bb bb bb bb bb bb bb bb
> bb bb bb ................
> [23424.121318] Object 000000007e677ed8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b kkkkkkkkkkkkkkkk
> [23424.121322] Object 00000000e207f30b: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b kkkkkkkkkkkkkkkk
> [23424.121326] Object 00000000a7a45634: 6b 6b 6b 6b 6b 6b 6b 6b ff ff ff ff 6b
> 6b 6b 6b kkkkkkkk....kkkk
> [23424.121330] Object 00000000c85d951d: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b kkkkkkkkkkkkkkkk
> [23424.121334] Object 000000003104522f: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b kkkkkkkkkkkkkkkk
> [23424.121338] Object 00000000cfcdd820: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b kkkkkkkkkkkkkkkk
> [23424.121342] Object 00000000dded4924: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b kkkkkkkkkkkkkkkk
> [23424.121346] Object 00000000ff6687a4: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b kkkkkkkkkkkkkkkk
> [23424.121350] Object 00000000df3d67f6: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b kkkkkkkkkkkkkkkk
> [23424.121354] Object 00000000ddc188d1: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b kkkkkkkkkkkkkkkk
> [23424.121358] Object 000000002cee751a: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b kkkkkkkkkkkkkkkk
> [23424.121362] Object 00000000a994f007: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b a5 kkkkkkkkkkkkkkk.
> [23424.121366] Redzone 000000009f3d62e2: bb bb bb bb bb bb bb bb
> ........
> [23424.121370] Padding 00000000e5ccead8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a ZZZZZZZZZZZZZZZZ
> [23424.121374] Padding 000000002b0c1778: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a ZZZZZZZZZZZZZZZZ
> [23424.121378] Padding 00000000c67656c7: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a ZZZZZZZZZZZZZZZZ
> [23424.121382] Padding 0000000078348c5a: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a ZZZZZZZZZZZZZZZZ
> [23424.121386] Padding 00000000f3297820: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a ZZZZZZZZZZZZZZZZ
> [23424.121390] Padding 00000000e55789f4: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a ZZZZZZZZZZZZZZZZ
> [23424.121394] Padding 00000000d0fbb94c: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a ZZZZZZZZZZZZZZZZ
> [23424.121397] Padding 00000000bcb27a87: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a
> 5a 5a 5a ZZZZZZZZZZZZZZZZ
> [23424.121743] CPU: 7 PID: 12174 Comm: vgs Tainted: G B W L
> 5.0.0-rc7-next-20190221+ #7
> [23424.121758] Call Trace:
> [23424.121762] [c0000004ce5bf7b0] [c0000000007deb8c] dump_stack+0xb0/0xf4
> (unreliable)
> [23424.121770] [c0000004ce5bf7f0] [c00000000037d310] print_trailer+0x250/0x278
> [23424.121775] [c0000004ce5bf880] [c00000000036d578]
> check_bytes_and_report+0x138/0x160
> [23424.121779] [c0000004ce5bf920] [c00000000036fac8] check_object+0x348/0x3e0
> [23424.121784] [c0000004ce5bf990] [c00000000036fd18]
> alloc_debug_processing+0x1b8/0x2c0
> [23424.121788] [c0000004ce5bfa30] [c000000000372d14] ___slab_alloc+0xbb4/0xfa0
> [23424.121792] [c0000004ce5bfb60] [c000000000373134] __slab_alloc+0x34/0x60
> [23424.121802] [c0000004ce5bfb90] [c000000000373664] kmem_cache_alloc+0x504/0x5c0
> [23424.121812] [c0000004ce5bfc20] [c000000000476a9c] io_submit_one+0x9c/0xb20
> [23424.121824] [c0000004ce5bfd50] [c000000000477f10] sys_io_submit+0xe0/0x350
> [23424.121832] [c0000004ce5bfe20] [c00000000000b000] system_call+0x5c/0x70
> [23424.121836] FIX aio_kiocb: Restoring 0x000000009f1f5145-0x00000000841e301b=0x6b
> [23424.121836]
> [23424.121840] FIX aio_kiocb: Marking all objects used
Can you try this one? We only need to write it for polled, and for polled
the caller is the one that will reap the iocb. Hence it's safe to write
it after submission if we are marked polled.
diff --git a/fs/block_dev.c b/fs/block_dev.c
index 0e3155e817cc..f78fc7bf2225 100644
--- a/fs/block_dev.c
+++ b/fs/block_dev.c
@@ -419,11 +419,17 @@ __blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter, int nr_pages)
nr_pages = iov_iter_npages(iter, BIO_MAX_PAGES);
if (!nr_pages) {
- if (iocb->ki_flags & IOCB_HIPRI)
+ bool polled = false;
+
+ if (iocb->ki_flags & IOCB_HIPRI) {
bio_set_polled(bio, iocb);
+ polled = true;
+ }
qc = submit_bio(bio);
- WRITE_ONCE(iocb->ki_cookie, qc);
+
+ if (polled)
+ WRITE_ONCE(iocb->ki_cookie, qc);
break;
}
--
Jens Axboe
next prev parent reply other threads:[~2019-02-22 22:40 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-22 5:40 Qian Cai
2019-02-22 15:36 ` Christopher Lameter
2019-02-22 21:07 ` Qian Cai
2019-02-22 21:42 ` Eric Sandeen
2019-02-22 21:48 ` Qian Cai
2019-02-22 21:58 ` Eric Sandeen
2019-02-22 22:06 ` Qian Cai
2019-02-22 22:25 ` Jeff Moyer
2019-02-22 22:40 ` Jens Axboe [this message]
2019-02-23 1:58 ` Qian Cai
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cd3bac6a-02b2-351e-3f81-322c2e0ca03e@kernel.dk \
--to=axboe@kernel.dk \
--cc=bcrl@kvack.org \
--cc=cai@lca.pw \
--cc=hare@suse.com \
--cc=linux-aio@kvack.org \
--cc=linux-mm@kvack.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox