From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: ** X-Spam-Status: No, score=2.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BA9BC54FC9 for ; Tue, 21 Apr 2020 15:30:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D8E54206A2 for ; Tue, 21 Apr 2020 15:30:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=konsulko.com header.i=@konsulko.com header.b="iS9TeotT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D8E54206A2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=konsulko.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 899DF8E0006; Tue, 21 Apr 2020 11:30:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 84A508E0003; Tue, 21 Apr 2020 11:30:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 760028E0006; Tue, 21 Apr 2020 11:30:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0029.hostedemail.com [216.40.44.29]) by kanga.kvack.org (Postfix) with ESMTP id 57D308E0003 for ; Tue, 21 Apr 2020 11:30:28 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 136758245571 for ; Tue, 21 Apr 2020 15:30:28 +0000 (UTC) X-FDA: 76732248936.23.rate37_3ffe89740c5e X-HE-Tag: rate37_3ffe89740c5e X-Filterd-Recvd-Size: 18967 Received: from mail-lf1-f68.google.com (mail-lf1-f68.google.com [209.85.167.68]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 15:30:27 +0000 (UTC) Received: by mail-lf1-f68.google.com with SMTP id j14so11443874lfg.9 for ; Tue, 21 Apr 2020 08:30:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sH4bC4wEbDEHdPGGTIGHfIxneF2uFB6UdQbFt1FvhSQ=; b=iS9TeotTH8PoKWjB7kw8Gi5dnqW8HOF2vkT3lD30OP8IHQkcety4ikoga3fwOegxTq cbXWLvGdIqLsX7biN5wH+5U+d79UBsOWqLOJUBUsQ6jn0pLP1yU6Yc9Ypyhd0YLr8yph YS/jidaOUU8np8wN4M/d7EOTqXcUwx8Sz7cXw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sH4bC4wEbDEHdPGGTIGHfIxneF2uFB6UdQbFt1FvhSQ=; b=fnIhj9SCrQ0JgPyiiSh1qV3bwzPHUKIccdA8TA0nsp3Zrz6tTkXo+unmsZY5yxJkDA 0m+MsfGdaaCtzG/ZFnY6hlIAPemrwNFNH9O7W+/4UKjrmv42rG4QRARWsoWboOzOGq5S AOohqXI4oUCcWjxWiu9EQXP1dPkIbUvpAO4MFXihSaNohHsKARxnusuBLgeai0IU9pw3 BrlIWy7QGL7dLhaV11adqTd+xOgLgKTXQ/2Y0OzMU2WUVYT1Z9zwIpkiikrbSKbXWpC5 ej34PwVKaKsgI5Cmu+XpsPvHwUx6EFLEO6TyStDzJp0WbOSQWlrty6V4B1wTxJyGMsnn i3og== X-Gm-Message-State: AGi0PuYkhcPKWE/xg4c8BMEWyv7YYKBZSGaR+aIPS1VZUF80rGQYkfCj HdumA8DX/HEvf7uR2Qc3ja/58OZiYYNAedME+Gm/Rw== X-Google-Smtp-Source: APiQypKI1XZhOc6ibg0SX0V7/Pkat6rDLplDWhHdChcoHw3PB/SLp1LzpCNIDSwj3Ib/j6k7a2rvNchVcpBWF6tCFOI= X-Received: by 2002:a19:c14e:: with SMTP id r75mr14186578lff.62.1587483025700; Tue, 21 Apr 2020 08:30:25 -0700 (PDT) MIME-Version: 1.0 References: <8cae4626-90db-ff4e-a42b-75c11900d22a@suse.cz> In-Reply-To: <8cae4626-90db-ff4e-a42b-75c11900d22a@suse.cz> From: Vitaly Wool Date: Tue, 21 Apr 2020 17:30:11 +0200 Message-ID: Subject: Re: BUG in mm/zswap To: Vlastimil Babka Cc: Raymond Jennings , Linux-MM , Seth Jennings , Dan Streetman Content-Type: multipart/alternative; boundary="000000000000d5971705a3ceb13f" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --000000000000d5971705a3ceb13f Content-Type: text/plain; charset="UTF-8" On Tue, Apr 21, 2020, 5:19 PM Vlastimil Babka wrote: > On 4/20/20 1:15 PM, Raymond Jennings wrote: > > I got a bug check and the guys in #kernelnewbies in oftc told me to > email you > > guys about it, not sure what to do about it > > +CC zswap maintainers > Thanks Vlastimil, I might have a fix for this. I'm going to post a couple of patches this week and I'll make sure you are all CC'd. ~Vitaly > > > > 2036206:Apr 20 03:22:51 metalhead kernel: [103376.518888] kernel BUG at > > mm/zswap.c:1184! > > Hmm that's this: > > ret = crypto_comp_decompress(tfm, src, entry->length, dst, &dlen); > put_cpu_ptr(entry->pool->tfm); > kunmap_atomic(dst); > zpool_unmap_handle(entry->pool->zpool, entry->handle); > BUG_ON(ret); > > Looks like decompression failed? Are there any messages prior to the BUG > that > would indicate the failed decompression? > > From loaded modules it seems like z3fold and lz4_decompress could be in use > here. What's the output of: > grep . /sys/module/zswap/parameters/* > > And is this reproducible? Or happened just once? Is it a regression after > kernel > update? > > > 2036207-Apr 20 03:22:51 metalhead kernel: [103376.518893] invalid > opcode: 0000 > > [#1] PREEMPT SMP PTI > > 2036208-Apr 20 03:22:51 metalhead kernel: [103376.518895] CPU: 5 PID: > 2008 Comm: > > swapoff Not tainted 5.6.5-gentoo-x86_64 #1 > > 2036209-Apr 20 03:22:51 metalhead kernel: [103376.518896] Hardware name: > Dell > > Inc. OptiPlex 7020/02YYK5, BIOS A15 02/02/2018 > > 2036210-Apr 20 03:22:51 metalhead kernel: [103376.518900] RIP: > > 0010:zswap_frontswap_load+0x238/0x250 > > 2036211-Apr 20 03:22:51 metalhead kernel: [103376.518901] Code: 00 00 e8 > bb 04 > > e5 ff 65 8b 05 3c d3 dc 71 85 c0 0f 85 61 ff ff ff e8 3b 74 db ff e9 57 > ff ff ff > > e8 31 74 db ff e9 35 ff ff ff <0f> 0b e8 25 74 db ff e9 00 ff ff ff e8 > 37 13 e2 > > ff 0f 1f 80 00 00 > > 2036212-Apr 20 03:22:51 metalhead kernel: [103376.518902] RSP: > > 0018:ffffa7ed41f6fb20 EFLAGS: 00010282 > > 2036213-Apr 20 03:22:51 metalhead kernel: [103376.518903] RAX: > 0000000080000000 > > RBX: 00000000ffffffea RCX: 0000000000000000 > > 2036214-Apr 20 03:22:51 metalhead kernel: [103376.518904] RDX: > 0000000000000001 > > RSI: 0000000000000000 RDI: 00000000ffffffff > > 2036215-Apr 20 03:22:51 metalhead kernel: [103376.518905] RBP: > ffff8f37e9eab2a0 > > R08: ffff8f3a308de780 R09: 0000000000000000 > > 2036216-Apr 20 03:22:51 metalhead kernel: [103376.518905] R10: > 0000000000000000 > > R11: ffffa7ed41f6fb00 R12: ffff8f37bf4e4000 > > 2036217-Apr 20 03:22:51 metalhead kernel: [103376.518906] R13: > ffff8f3bf6908d28 > > R14: ffff8f3bf6908d20 R15: ffff8f3bc7cc5ec8 > > 2036218-Apr 20 03:22:51 metalhead kernel: [103376.518907] FS: > > 00007fc9c4cf0780(0000) GS:ffff8f3cfda00000(0000) knlGS:0000000000000000 > > 2036219-Apr 20 03:22:51 metalhead kernel: [103376.518908] CS: 0010 DS: > 0000 ES: > > 0000 CR0: 0000000080050033 > > 2036220-Apr 20 03:22:51 metalhead kernel: [103376.518909] CR2: > 00007f9e8a1d471c > > CR3: 0000000300754005 CR4: 00000000001606e0 > > 2036221-Apr 20 03:22:51 metalhead kernel: [103376.518909] Call Trace: > > 2036222-Apr 20 03:22:51 metalhead kernel: [103376.518916] > > __frontswap_load+0x9c/0xf0 > > 2036223-Apr 20 03:22:51 metalhead kernel: [103376.518918] > swap_readpage+0xfb/0x330 > > 2036224-Apr 20 03:22:51 metalhead kernel: [103376.518920] > > swap_cluster_readahead+0x1da/0x300 > > 2036225-Apr 20 03:22:51 metalhead kernel: [103376.518922] ? > 0xffffffff8e000000 > > 2036226-Apr 20 03:22:51 metalhead kernel: [103376.518924] > > swapin_readahead+0x2e4/0x4a0 > > 2036227-Apr 20 03:22:51 metalhead kernel: [103376.518926] ? > > put_swap_page+0x106/0x310 > > 2036228-Apr 20 03:22:51 metalhead kernel: [103376.518928] > > unuse_pte_range+0x167/0x760 > > 2036229-Apr 20 03:22:51 metalhead kernel: [103376.518930] > try_to_unuse+0x5a1/0x730 > > 2036230-Apr 20 03:22:51 metalhead kernel: [103376.518932] > > __do_sys_swapoff+0x1df/0x6d0 > > 2036231-Apr 20 03:22:51 metalhead kernel: [103376.518935] ? > > exit_to_usermode_loop+0x97/0xf0 > > 2036232-Apr 20 03:22:51 metalhead kernel: [103376.518937] > do_syscall_64+0x55/0x1b0 > > 2036233-Apr 20 03:22:51 metalhead kernel: [103376.518942] > > entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > 2036234-Apr 20 03:22:51 metalhead kernel: [103376.518944] RIP: > 0033:0x7fc9c4e25657 > > 2036235-Apr 20 03:22:51 metalhead kernel: [103376.518945] Code: 73 01 c3 > 48 8b > > 0d 39 b8 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 > 00 0f 1f > > 44 00 00 b8 a8 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 09 > b8 0c 00 > > f7 d8 64 89 01 48 > > 2036236-Apr 20 03:22:51 metalhead kernel: [103376.518946] RSP: > > 002b:00007ffc715f01e8 EFLAGS: 00000206 ORIG_RAX: 00000000000000a8 > > 2036237-Apr 20 03:22:51 metalhead kernel: [103376.518947] RAX: > ffffffffffffffda > > RBX: 0000000000000000 RCX: 00007fc9c4e25657 > > 2036238-Apr 20 03:22:51 metalhead kernel: [103376.518948] RDX: > 0000000000000001 > > RSI: 0000000000000003 RDI: 0000557e405007b0 > > 2036239-Apr 20 03:22:51 metalhead kernel: [103376.518948] RBP: > 00007ffc715f1442 > > R08: 0000557e404fe580 R09: 0000000000000001 > > 2036240-Apr 20 03:22:51 metalhead kernel: [103376.518949] R10: > 00007fc9c50018e0 > > R11: 0000000000000206 R12: 0000000000000000 > > 2036241-Apr 20 03:22:51 metalhead kernel: [103376.518949] R13: > 0000557e405007b0 > > R14: 0000000000000000 R15: 0000000000000000 > > 2036242-Apr 20 03:22:51 metalhead kernel: [103376.518951] Modules linked > in: > > z3fold bfq ipt_REJECT nf_reject_ipv4 xt_multiport iptable_filter > ip_tables > > af_packet snd_hda_codec_hdmi i915 i2c_algo_bit drm_kms_helper > intel_rapl_msr > > intel_rapl_common cec uvcvideo x86_pkg_temp_thermal intel_powerclamp > > snd_hda_codec_generic drm dell_wmi ledtrig_audio videobuf2_vmalloc > sparse_keymap > > iTCO_wdt kvm_intel wmi_bmof videobuf2_memops dell_smbios > dell_wmi_descriptor > > iTCO_vendor_support snd_hda_intel snd_usb_audio > drm_panel_orientation_quirks > > dcdbas snd_usbmidi_lib snd_rawmidi mousedev videobuf2_v4l2 kvm > videobuf2_common > > videodev intel_gtt agpgart snd_seq_device irqbypass input_leds joydev > > syscopyarea snd_intel_dspcfg sysfillrect sysimgblt fb_sys_fops > binfmt_misc > > crct10dif_pclmul i2c_i801 ghash_clmulni_intel snd_hda_codec i2c_core > > intel_cstate video wmi snd_hwdep intel_uncore snd_hda_core snd_pcm e1000e > > intel_rapl_perf snd_timer snd backlight evbug lpc_ich evdev pcspkr > soundcore > > mfd_core coretemp hwmon aesni_intel crypto_simd cryptd glue > > 2036243-Apr 20 03:22:51 metalhead kernel: helper > > 2036244-Apr 20 03:22:51 metalhead kernel: [103376.518983] algif_rng > algif_aead > > algif_hash algif_skcipher af_alg crc32c_intel crc32_pclmul crc32_generic > > configfs overlay squashfs lz4_decompress loop btrfs xor ext4 mbcache jbd2 > > raid6_pq libcrc32c dm_snapshot dm_mirror dm_region_hash dm_log_userspace > dm_log > > dm_bufio dm_mod firewire_core crc_itu_t hid_generic usbhid ohci_hcd > usb_storage > > hid xhci_plat_hcd xhci_pci xhci_hcd ehci_pci ehci_hcd usbcore usb_common > > scsi_transport_fc sr_mod cdrom sg sd_mod t10_pi ahci libahci libata > scsi_mod > > 2036245-Apr 20 03:22:51 metalhead kernel: [103376.519004] ---[ end trace > > 5959740853c6dbd4 ]--- > > --000000000000d5971705a3ceb13f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Tue, Apr 21, 2020, 5:19 PM Vlastimil Babka <vbabka@suse.cz> wrote:
On 4/20/20 1:15 PM, Raymond Jennings wrote:
> I got a bug check and the guys in #kernelnewbies in oftc told me to em= ail you
> guys about it, not sure what to do about it

+CC zswap maintainers

Thanks Vlastimil, I might have a fix for this. I'm= going to post a couple of patches this week and I'll make sure you are= all CC'd.

~Vitaly


> 2036206:Apr 20 03:22:51 metalhead kernel: [103376.518888] kernel BUG a= t
> mm/zswap.c:1184!

Hmm that's this:

ret =3D crypto_comp_decompress(tfm, src, entry->length, dst, &dlen);=
put_cpu_ptr(entry->pool->tfm);
kunmap_atomic(dst);
zpool_unmap_handle(entry->pool->zpool, entry->handle);
BUG_ON(ret);

Looks like decompression failed? Are there any messages prior to the BUG th= at
would indicate the failed decompression?

>From loaded modules it seems like z3fold and lz4_decompress could be in use=
here. What's the output of:
grep . /sys/module/zswap/parameters/*

And is this reproducible? Or happened just once? Is it a regression after k= ernel
update?

> 2036207-Apr 20 03:22:51 metalhead kernel: [103376.518893] invalid opco= de: 0000
> [#1] PREEMPT SMP PTI
> 2036208-Apr 20 03:22:51 metalhead kernel: [103376.518895] CPU: 5 PID: = 2008 Comm:
> swapoff Not tainted 5.6.5-gentoo-x86_64 #1
> 2036209-Apr 20 03:22:51 metalhead kernel: [103376.518896] Hardware nam= e: Dell
> Inc. OptiPlex 7020/02YYK5, BIOS A15 02/02/2018
> 2036210-Apr 20 03:22:51 metalhead kernel: [103376.518900] RIP:
> 0010:zswap_frontswap_load+0x238/0x250
> 2036211-Apr 20 03:22:51 metalhead kernel: [103376.518901] Code: 00 00 = e8 bb 04
> e5 ff 65 8b 05 3c d3 dc 71 85 c0 0f 85 61 ff ff ff e8 3b 74 db ff e9 5= 7 ff ff ff
> e8 31 74 db ff e9 35 ff ff ff <0f> 0b e8 25 74 db ff e9 00 ff ff= ff e8 37 13 e2
> ff 0f 1f 80 00 00
> 2036212-Apr 20 03:22:51 metalhead kernel: [103376.518902] RSP:
> 0018:ffffa7ed41f6fb20 EFLAGS: 00010282
> 2036213-Apr 20 03:22:51 metalhead kernel: [103376.518903] RAX: 0000000= 080000000
> RBX: 00000000ffffffea RCX: 0000000000000000
> 2036214-Apr 20 03:22:51 metalhead kernel: [103376.518904] RDX: 0000000= 000000001
> RSI: 0000000000000000 RDI: 00000000ffffffff
> 2036215-Apr 20 03:22:51 metalhead kernel: [103376.518905] RBP: ffff8f3= 7e9eab2a0
> R08: ffff8f3a308de780 R09: 0000000000000000
> 2036216-Apr 20 03:22:51 metalhead kernel: [103376.518905] R10: 0000000= 000000000
> R11: ffffa7ed41f6fb00 R12: ffff8f37bf4e4000
> 2036217-Apr 20 03:22:51 metalhead kernel: [103376.518906] R13: ffff8f3= bf6908d28
> R14: ffff8f3bf6908d20 R15: ffff8f3bc7cc5ec8
> 2036218-Apr 20 03:22:51 metalhead kernel: [103376.518907] FS:
> =C2=A000007fc9c4cf0780(0000) GS:ffff8f3cfda00000(0000) knlGS:000000000= 0000000
> 2036219-Apr 20 03:22:51 metalhead kernel: [103376.518908] CS: =C2=A000= 10 DS: 0000 ES:
> 0000 CR0: 0000000080050033
> 2036220-Apr 20 03:22:51 metalhead kernel: [103376.518909] CR2: 00007f9= e8a1d471c
> CR3: 0000000300754005 CR4: 00000000001606e0
> 2036221-Apr 20 03:22:51 metalhead kernel: [103376.518909] Call Trace:<= br> > 2036222-Apr 20 03:22:51 metalhead kernel: [103376.518916]
> =C2=A0__frontswap_load+0x9c/0xf0
> 2036223-Apr 20 03:22:51 metalhead kernel: [103376.518918] =C2=A0swap_r= eadpage+0xfb/0x330
> 2036224-Apr 20 03:22:51 metalhead kernel: [103376.518920]
> =C2=A0swap_cluster_readahead+0x1da/0x300
> 2036225-Apr 20 03:22:51 metalhead kernel: [103376.518922] =C2=A0? 0xff= ffffff8e000000
> 2036226-Apr 20 03:22:51 metalhead kernel: [103376.518924]
> =C2=A0swapin_readahead+0x2e4/0x4a0
> 2036227-Apr 20 03:22:51 metalhead kernel: [103376.518926] =C2=A0?
> put_swap_page+0x106/0x310
> 2036228-Apr 20 03:22:51 metalhead kernel: [103376.518928]
> =C2=A0unuse_pte_range+0x167/0x760
> 2036229-Apr 20 03:22:51 metalhead kernel: [103376.518930] =C2=A0try_to= _unuse+0x5a1/0x730
> 2036230-Apr 20 03:22:51 metalhead kernel: [103376.518932]
> =C2=A0__do_sys_swapoff+0x1df/0x6d0
> 2036231-Apr 20 03:22:51 metalhead kernel: [103376.518935] =C2=A0?
> exit_to_usermode_loop+0x97/0xf0
> 2036232-Apr 20 03:22:51 metalhead kernel: [103376.518937] =C2=A0do_sys= call_64+0x55/0x1b0
> 2036233-Apr 20 03:22:51 metalhead kernel: [103376.518942]
> =C2=A0entry_SYSCALL_64_after_hwframe+0x44/0xa9
> 2036234-Apr 20 03:22:51 metalhead kernel: [103376.518944] RIP: 0033:0x= 7fc9c4e25657
> 2036235-Apr 20 03:22:51 metalhead kernel: [103376.518945] Code: 73 01 = c3 48 8b
> 0d 39 b8 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 0= 0 00 0f 1f
> 44 00 00 b8 a8 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b= 0d 09 b8 0c 00
> f7 d8 64 89 01 48
> 2036236-Apr 20 03:22:51 metalhead kernel: [103376.518946] RSP:
> 002b:00007ffc715f01e8 EFLAGS: 00000206 ORIG_RAX: 00000000000000a8
> 2036237-Apr 20 03:22:51 metalhead kernel: [103376.518947] RAX: fffffff= fffffffda
> RBX: 0000000000000000 RCX: 00007fc9c4e25657
> 2036238-Apr 20 03:22:51 metalhead kernel: [103376.518948] RDX: 0000000= 000000001
> RSI: 0000000000000003 RDI: 0000557e405007b0
> 2036239-Apr 20 03:22:51 metalhead kernel: [103376.518948] RBP: 00007ff= c715f1442
> R08: 0000557e404fe580 R09: 0000000000000001
> 2036240-Apr 20 03:22:51 metalhead kernel: [103376.518949] R10: 00007fc= 9c50018e0
> R11: 0000000000000206 R12: 0000000000000000
> 2036241-Apr 20 03:22:51 metalhead kernel: [103376.518949] R13: 0000557= e405007b0
> R14: 0000000000000000 R15: 0000000000000000
> 2036242-Apr 20 03:22:51 metalhead kernel: [103376.518951] Modules link= ed in:
> z3fold bfq ipt_REJECT nf_reject_ipv4 xt_multiport iptable_filter ip_ta= bles
> af_packet snd_hda_codec_hdmi i915 i2c_algo_bit drm_kms_helper intel_ra= pl_msr
> intel_rapl_common cec uvcvideo x86_pkg_temp_thermal intel_powerclamp > snd_hda_codec_generic drm dell_wmi ledtrig_audio videobuf2_vmalloc spa= rse_keymap
> iTCO_wdt kvm_intel wmi_bmof videobuf2_memops dell_smbios dell_wmi_desc= riptor
> iTCO_vendor_support snd_hda_intel snd_usb_audio drm_panel_orientation_= quirks
> dcdbas snd_usbmidi_lib snd_rawmidi mousedev videobuf2_v4l2 kvm videobu= f2_common
> videodev intel_gtt agpgart snd_seq_device irqbypass input_leds joydev<= br> > syscopyarea snd_intel_dspcfg sysfillrect sysimgblt fb_sys_fops binfmt_= misc
> crct10dif_pclmul i2c_i801 ghash_clmulni_intel snd_hda_codec i2c_core > intel_cstate video wmi snd_hwdep intel_uncore snd_hda_core snd_pcm e10= 00e
> intel_rapl_perf snd_timer snd backlight evbug lpc_ich evdev pcspkr sou= ndcore
> mfd_core coretemp hwmon aesni_intel crypto_simd cryptd glue
> 2036243-Apr 20 03:22:51 metalhead kernel: helper
> 2036244-Apr 20 03:22:51 metalhead kernel: [103376.518983] =C2=A0algif_= rng algif_aead
> algif_hash algif_skcipher af_alg crc32c_intel crc32_pclmul crc32_gener= ic
> configfs overlay squashfs lz4_decompress loop btrfs xor ext4 mbcache j= bd2
> raid6_pq libcrc32c dm_snapshot dm_mirror dm_region_hash dm_log_userspa= ce dm_log
> dm_bufio dm_mod firewire_core crc_itu_t hid_generic usbhid ohci_hcd us= b_storage
> hid xhci_plat_hcd xhci_pci xhci_hcd ehci_pci ehci_hcd usbcore usb_comm= on
> scsi_transport_fc sr_mod cdrom sg sd_mod t10_pi ahci libahci libata sc= si_mod
> 2036245-Apr 20 03:22:51 metalhead kernel: [103376.519004] ---[ end tra= ce
> 5959740853c6dbd4 ]---

--000000000000d5971705a3ceb13f--