From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCD2EC282C5 for ; Mon, 3 Mar 2025 07:48:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3BD3E280007; Mon, 3 Mar 2025 02:48:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 36CB5280003; Mon, 3 Mar 2025 02:48:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 25B6E280007; Mon, 3 Mar 2025 02:48:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 04C97280003 for ; Mon, 3 Mar 2025 02:48:14 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 98D0047017 for ; Mon, 3 Mar 2025 07:48:14 +0000 (UTC) X-FDA: 83179461708.26.B1F6D0F Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) by imf27.hostedemail.com (Postfix) with ESMTP id 740D040008 for ; Mon, 3 Mar 2025 07:48:12 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=exxDC9fo; spf=pass (imf27.hostedemail.com: domain of hare@suse.com designates 209.85.218.41 as permitted sender) smtp.mailfrom=hare@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740988092; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LbsbyNEYb54uhiNHNi0Sebz3O0VlglRg1DC7jpquZQY=; b=nAoyhVuMU0Ithus/8bHHjqCkTVzYYPg+kqfwICSYWJ8LoVBWrVNMtJIEJRUitCf2vKhzDP TLuTRkRFNVPa/DbwXil61yTiwnpETAfNsEcImKXcUmX/vag/Gpv1Ntu0QesvDmre+LQOdv 4vm+aXf/LXCUiG/GEAkVoBZ0v8yMo+E= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740988092; a=rsa-sha256; cv=none; b=yr9VDQuVmY8j3oxtb73F0f64WRYLd8JWxXzQbNfCsohNJndLjhumfEUWec9JWFEaooqi+m YNid7Fp6YkJGTlZkJAHR4UPbI7qw8ohgYEksfxRxWiGsRK58+9aKWCUA+8RrDsQPyFJHcx IME8/DKF6Ny/rEbCdIw8zLzctsQUuZA= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=exxDC9fo; spf=pass (imf27.hostedemail.com: domain of hare@suse.com designates 209.85.218.41 as permitted sender) smtp.mailfrom=hare@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-abf42913e95so330512766b.2 for ; Sun, 02 Mar 2025 23:48:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1740988091; x=1741592891; darn=kvack.org; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id:from:to :cc:subject:date:message-id:reply-to; bh=LbsbyNEYb54uhiNHNi0Sebz3O0VlglRg1DC7jpquZQY=; b=exxDC9foqaY3vgHa7MsgTvb7eulRqrHAA93NoW2mKpwfh6X8PebAjJobhEAoywDV7w Ukht4ajnreN6yaqsMiPGuG4BtI08M6ka5Mq/DA/72u7sGFdgZuIOGObAZ2SRHq40srul SMfnjiKZ6eaCoyQXQ6MgaH1P7KKBXGiUkfETm4XrUEsUbDqfftAS4IREhmzQbXHYBw6W 2aLOikkNQhIrcjjsquM0tWNTaCqHzF1zCmE/7mgBsv7J4YgV8Ol4pa69EFa5vYmDI2Ap r7hJBEn5mFckUPRvjWuH/X9IZ2WQ76yQ2M/mw0dsKD3+YVw2nEKRBdQVjwf4N0yoMrZb eNkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740988091; x=1741592891; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=LbsbyNEYb54uhiNHNi0Sebz3O0VlglRg1DC7jpquZQY=; b=jTmMn9oKDO/EutvbCJ2QtS9sQybS0uds2jLrMXY/zkQeR8ForCxm03IvRCDMV6hmUO FUUC572QUk1Iv7lOpPtIGFwnD+tOqn21PAgLgU00BIAym3L+PwkihS7QEyP7lstVl8A/ kMrxAUhj9TUZ0s6JtbH7j0XCTIXVKekobmLzOimuNKXfD2dsy/sDLO0vca4K3wbnkh5b BLJDQU4VbsgMkqMh5hwUlPbGigKWbQVze//ei4DSuMhK9jc1y45ru5U335u42TdLIu4G ghs1jw0O+23RcAizhFkcsINeaQ6HdZIFEBKVTKn4nst8F321uvpdHbB9lpqkkeQeOvfY Y5/w== X-Forwarded-Encrypted: i=1; AJvYcCULZyoAsPgYHAcvq2kVoAcp+nGxW7Hhrz40P9Un++/cm2bZjROoBfTPxq7kj7rg7CkASPbZh54fUg==@kvack.org X-Gm-Message-State: AOJu0Yzzmx5auy9eK/YFqUP9C2S/51mKXh1BQbNMfW5LpNAg2zInYZr8 mxiBiVlN6Ms4Rqrc2ATj2Ps0xs3KQxdFXPiH6vn61suqoubw6NgwAFvmxKmqeQs= X-Gm-Gg: ASbGncsytByluYNhoIqaQgoHqQifBuFQpub2BGGxQFdOuSsYPsUccF5yBWReWUfkaEL CXlRvRRYIwN9pvixuPESZePS6PXZwtRDshk0tDxoy86Rk5brF3x3lxVuiLGVE2HWqvhzs68mZ08 KuAZcMe2ykHnFGK7jEi0zA8wCrND34u9oJw550iwv6wbYLaU+hd1Og2BSN4nlmzo6fOHe7d7mYm C4NpydbrKDbPLLEAd4c6oN/tm0PeEOExQUeuGXUyQf7GR7tGFAqk15rz471hcy5VYlXdZQ8eWHh Pl4gFwqBaRqnn+fHTnbGqbGdEcjjdQmYY7HcxH4nMsEA5ySk6383bg+6IyX4bAwn8MDjRIuX8KI 51MA8XNwegQ== X-Google-Smtp-Source: AGHT+IEN/KOQRG0jauTgtDMQPughhXuUV7Aw/Cetq+l5sD6X4ZpwdEPLjPATdWfnaufKMmGcXq91gg== X-Received: by 2002:a17:907:3fa6:b0:abf:65c8:70e5 with SMTP id a640c23a62f3a-abf65c8922cmr655138066b.25.1740988090599; Sun, 02 Mar 2025 23:48:10 -0800 (PST) Received: from [192.168.178.47] (aftr-82-135-83-43.dynamic.mnet-online.de. [82.135.83.43]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac1e6a5c13bsm5906866b.132.2025.03.02.23.48.09 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 02 Mar 2025 23:48:10 -0800 (PST) Message-ID: <509dd4d3-85e9-40b2-a967-8c937909a1bf@suse.com> Date: Mon, 3 Mar 2025 08:48:09 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Kernel oops with 6.14 when enabling TLS From: Hannes Reinecke To: Sagi Grimberg , Matthew Wilcox Cc: "linux-nvme@lists.infradead.org" , "linux-block@vger.kernel.org" , linux-nvme@lists.infradead.org, linux-mm@kvack.org References: <08c29e4b-2f71-4b6d-8046-27e407214d8c@suse.com> Content-Language: en-US In-Reply-To: <08c29e4b-2f71-4b6d-8046-27e407214d8c@suse.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam02 X-Stat-Signature: d3earo8wk43rr7arujynciw9f9ad19bd X-Rspamd-Queue-Id: 740D040008 X-Rspam-User: X-HE-Tag: 1740988092-427231 X-HE-Meta: U2FsdGVkX18A9s9crqMzrBqaT5dANjxs1faujKfaBiJ+RebvxAH63nBAJY29Se0M5EjkC7tUbhb7nfztYYgAMqDSCtTAp4wzQoae6dQXBJHrNcGOjTbxZbF7mhOE8JstpIsG4TQgPX0ZDfVsHS+Wr0seML5JHMfPy/bod6l73nfYk2OUNMNPe2CUydFb9P/WvmUuChLU495DhbnXvdLrj3zit39J5CPCOj/f35KhG8hG0DSeHZnZzgj2ZE3LbkQ3O70VTFhiZ2u3H7AjD4jOv5/S9CJIEJZoytTSNbyLgb2CA69m9F2sGtCIiNkGLcJhmBpygeW55h2EF8sswaS0Nj52vTMWHusU+2N4mJ/X1VGi8kFW/+oPwBJwICAVLNk9h6kZU5MqInV6K986AxXoSugD6ic6d45Ebh5mTtH7abMPLFuvBdCZg7e3tMtta1PvAiSnR/i4T0ZCtVsvXN9ajmGGQloZRv9m+qDXxzZmjr/lZ+ydHiMiE70n05tB2MisNuDEbC8Drmyrijl3yeCSKrJERqG6+M9XPnv6pecu58BTfjx4OSx+ExIkZsdximemlf4jeiqlOEMIBDIHOSBWgfo4lXKO69dXo+HCszrrbUu98I9T2He9To5RAp7IjlG+8JDEBzzjc68ebEeB2kSgmvc4Ko7rZdyp59HFjHaDifJ5HiVK2bFnGZVW+TPJ1my5LEpjgjXLzFflySmK8vraXMk1p9tmbki7/GY8sisOIc/UvVse98xZGV3AsWUeWySSBaIR1Yh4KlH6SA+jUKKa8KEhVuLBvzRhXTDi5Honso48PtEOeBZd4TzlYCyf4ffC29ySezO/HVUEPXKW/i4LTYAZNE+Ff0lvtk2FTbPyp9u8GtcJZ//xfI4pZ4Ja/xIWzDBNHyyoWU9ZVBNJWyhi9rL+QlRkf1eXx2j2/lWTkXwM+s7BoxQpSRhDOp3l0R4CYsSXX6ZUkziKTwA58eX Xwp2J3CZ bqX1DlAkpVtr3WP8B2QxeOcM3vw0JdWmV6sr/XJcIHpoMBvgjNfaOufJGfirjk/RX+dtkmg/PBoTFFXkEan6pL6Mlx1u8Ydq0sCAMx+Omh04jg0f8lC2HQ4qgVfbsVPBnp9jlvrtgPsoqaubuOaWWyZXuRpoc9DqogKR0z9YhBPv4KA0KOXu4XWHBKqWbfcCM+MWhMILUboFgYK9/70W1e1xjPbo5MOR0ba/c0PSykasOM3ZxGthccI2dyDBiIbfWA1fCTRJjv1yQ76YN3sOrLo8D4SnH1oLIg2P7z/T3H4e9wu0TJ5Q/0JL0FdUWnlC6jduxciPAJtIfrTMvS1O66xnTzjv9yAKDJskNUrznotzOgezkpTZWvM5ok0bj0cN9Fp4gXQkt8ZcrYVysAoWFPeauv8cuNFOkXvw7EzwxEVMGbtswtWJWSTK7rIoNabHWrAPJHaphdDlVlaQINo/byuozVma6FG8gGM2b X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2/28/25 11:47, Hannes Reinecke wrote: > Hi Sagi, > > enabling TLS on latest linus tree reliably crashes my system: > > [  487.018058] ------------[ cut here ]------------ > [  487.024046] WARNING: CPU: 9 PID: 6159 at mm/slub.c:4719 > free_large_kmalloc+0x15/0xa0 > [  487.033549] Modules linked in: tls(E) nvme_tcp(E) af_packet(E) > iscsi_ibft(E) iscsi_boot_sysfs(E) amd_atl(E) intel_rapl_msr(E) > intel_rapl_common(E) amd64_edac(E) edac_mce_amd(E) nls_iso8859_1(E) > nls_cp437(E) dax_hmem(E) vfat(E) cxl_acpi(E) fat(E) kvm_amd(E) > ipmi_ssif(E) cxl_port(E) xfs(E) tg3(E) cxl_core(E) ipmi_si(E) i40e(E) > kvm(E) einj(E) wmi_bmof(E) acpi_cpufreq(E) ipmi_devintf(E) libphy(E) > k10temp(E) libie(E) i2c_piix4(E) i2c_smbus(E) ipmi_msghandler(E) > i2c_designware_platform(E) i2c_designware_core(E) button(E) > nvme_fabrics(E) nvme_keyring(E) fuse(E) efi_pstore(E) configfs(E) > dmi_sysfs(E) ip_tables(E) x_tables(E) ahci(E) libahci(E) > ghash_clmulni_intel(E) libata(E) sha512_ssse3(E) sd_mod(E) > sha256_ssse3(E) ast(E) sha1_ssse3(E) drm_client_lib(E) scsi_dh_emc(E) > i2c_algo_bit(E) aesni_intel(E) drm_shmem_helper(E) scsi_dh_rdac(E) > crypto_simd(E) cryptd(E) xhci_pci(E) drm_kms_helper(E) scsi_dh_alua(E) > nvme(E) sg(E) xhci_hcd(E) nvme_core(E) scsi_mod(E) drm(E) nvme_auth(E) > scsi_common(E) usbcore(E) sp5100_tco(E) ccp(E) > [  487.033696]  wmi(E) btrfs(E) blake2b_generic(E) xor(E) raid6_pq(E) > efivarfs(E) > [  487.033707] CPU: 9 UID: 0 PID: 6159 Comm: nvme Kdump: loaded Tainted: > G        W   E      6.14.0-rc4-default+ #292 > f1e35f01b401c038558e67f3c2d644747de50dbd > [  487.033713] Tainted: [W]=WARN, [E]=UNSIGNED_MODULE > [  487.033715] Hardware name: Lenovo ThinkSystem SR655V3/SB27B09914, > BIOS KAE111E-2.10 04/11/2023 > [  487.033717] RIP: 0010:free_large_kmalloc+0x15/0xa0 > [  487.033722] Code: 80 00 00 00 00 90 90 90 90 90 90 90 90 90 90 90 90 > 90 90 90 90 0f 1f 44 00 00 55 53 48 89 fb 48 83 ec 08 48 8b 07 a8 40 75 > 4b <0f> 0b 80 3d 24 4e 8e 01 00 ba 00 f0 ff ff 74 5d 9c 58 0f 1f 40 00 > [  487.205280] RSP: 0018:ff4de44e432a7b70 EFLAGS: 00010246 > [  487.205284] RAX: 000fffffc0000000 RBX: ffd659b280210b80 RCX: > ff42118489e0cd80 > [  487.205286] RDX: 0000000000000000 RSI: ff4211848842e000 RDI: > ffd659b280210b80 > [  487.205288] RBP: ff4de44e432a7be0 R08: 0000000000000001 R09: > 0000000000000002 > [  487.205289] R10: ff4de44e432a7c00 R11: 0000000000000104 R12: > ffd659b280210b80 > [  487.205291] R13: ff4211848842e000 R14: ff421186d0696520 R15: > ff421186e19c4000 > [  487.205292] FS:  00007f66b8ffd800(0000) GS:ff4211874d980000(0000) > knlGS:0000000000000000 > [  487.205294] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [  487.205296] CR2: 00007f66b9150d7e CR3: 0000000281a5e005 CR4: > 0000000000771ef0 > [  487.205298] PKRU: 55555554 > [  487.205306] Call Trace: > [  487.205309]  > [  487.205314]  ? __warn+0x85/0x130 > [  487.296763]  ? free_large_kmalloc+0x15/0xa0 > [  487.296772]  ? report_bug+0xf8/0x1e0 > [  487.296779]  ? handle_bug+0x50/0xa0 > [  487.296783]  ? exc_invalid_op+0x13/0x60 > [  487.296786]  ? asm_exc_invalid_op+0x16/0x20 > [  487.296796]  ? free_large_kmalloc+0x15/0xa0 > [  487.296801]  kfree+0x234/0x320 > [  487.332065]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  487.332075]  ? nvmf_connect_admin_queue+0x105/0x1a0 [nvme_fabrics > 34d997d53c805aa2fae8e8baee6a736e8da38358] > [  487.332081]  ? nvmf_connect_admin_queue+0xa1/0x1a0 [nvme_fabrics > 34d997d53c805aa2fae8e8baee6a736e8da38358] > [  487.332084]  nvmf_connect_admin_queue+0x105/0x1a0 [nvme_fabrics > 34d997d53c805aa2fae8e8baee6a736e8da38358] > [  487.332093]  nvme_tcp_start_queue+0x18f/0x310 [nvme_tcp > 68f6be106f52ac467179f8a0922f02aeb6fa1f1c] > [  487.332102]  nvme_tcp_setup_ctrl+0xf8/0x700 [nvme_tcp > 68f6be106f52ac467179f8a0922f02aeb6fa1f1c] > [  487.394479]  ? nvme_change_ctrl_state+0x99/0x1b0 [nvme_core > 22f0ce18ead628230226a9b87ebf48eb576bf299] > [  487.394495]  nvme_tcp_create_ctrl+0x2e3/0x4d0 [nvme_tcp > 68f6be106f52ac467179f8a0922f02aeb6fa1f1c] > [  487.394503]  nvmf_dev_write+0x323/0x3d0 [nvme_fabrics > 34d997d53c805aa2fae8e8baee6a736e8da38358] > [  487.394508]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  487.394514]  vfs_write+0xd9/0x430 > [  487.394521]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  487.394523]  ? __handle_mm_fault+0x7da/0xca0 > [  487.394531]  ksys_write+0x68/0xe0 > [  487.394536]  do_syscall_64+0x74/0x160 > [  487.394543]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  487.394545]  ? __count_memcg_events+0x98/0x130 > [  487.394550]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  487.476947]  ? count_memcg_events.constprop.163+0x1a/0x30 > [  487.476956]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  487.476960]  ? handle_mm_fault+0xa1/0x290 > [  487.476966]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  487.476968]  ? do_user_addr_fault+0x56b/0x830 > [  487.476975]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  487.476977]  ? exc_page_fault+0x68/0x150 > [  487.476983]  entry_SYSCALL_64_after_hwframe+0x76/0x7e > [  487.476989] RIP: 0033:0x7f66b91216f0 > [  487.476994] Code: 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90 90 > 90 90 90 90 90 90 90 90 90 80 3d 19 c3 0e 00 00 74 17 b8 01 00 00 00 0f > 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 48 83 ec 28 48 89 > [  487.551619] RSP: 002b:00007ffef5393ed8 EFLAGS: 00000202 ORIG_RAX: > 0000000000000001 > [  487.551623] RAX: ffffffffffffffda RBX: 000055e61c2113b0 RCX: > 00007f66b91216f0 > [  487.551625] RDX: 00000000000000ed RSI: 000055e61c2113b0 RDI: > 0000000000000003 > [  487.551627] RBP: 0000000000000003 R08: 00000000000000ed R09: > 000055e61c2113b0 > [  487.551628] R10: 0000000000000000 R11: 0000000000000202 R12: > 00000000000000ed > [  487.551630] R13: 00007f66b9380008 R14: 000055e61c20a980 R15: > 000055e61c20b100 > [  487.551637]  > [  487.551639] ---[ end trace 0000000000000000 ]--- > [  487.551642] object pointer: 0x00000000346cb6fc > [  487.554112] nvme nvme1: creating 32 I/O queues. > [  489.396262] nvme nvme1: mapped 32/0/0 default/read/poll queues. > [  489.405197] Oops: general protection fault, probably for non- > canonical address 0xdead000000000100: 0000 [#1] PREEMPT SMP NOPTI > [  489.418790] CPU: 9 UID: 0 PID: 6159 Comm: nvme Kdump: loaded Tainted: > G        W   E      6.14.0-rc4-default+ #292 > f1e35f01b401c038558e67f3c2d644747de50dbd > [  489.435212] Tainted: [W]=WARN, [E]=UNSIGNED_MODULE > [  489.441381] Hardware name: Lenovo ThinkSystem SR655V3/SB27B09914, > BIOS KAE111E-2.10 04/11/2023 > [  489.451841] RIP: 0010:__rmqueue_pcplist+0xe1/0xc80 > [  489.458016] Code: 06 48 83 e8 08 48 89 44 24 70 48 8b 45 00 48 39 c5 > 0f 84 72 01 00 00 48 8b 55 00 48 8b 32 48 8b 4a 08 48 8d 42 f8 48 89 4e > 08 <48> 89 31 48 b9 00 01 00 00 00 00 ad de 48 89 0a 48 8b 4c 24 20 48 > [  489.479905] RSP: 0018:ff4de44e432a7688 EFLAGS: 00010293 > [  489.486567] RAX: ffd659b280210a00 RBX: ff4211874d9bf400 RCX: > dead000000000100 > [  489.495370] RDX: ffd659b280210a08 RSI: ffd659b281ed7e08 RDI: > ff421184fffd60c0 > [  489.504174] RBP: ff4211874d9bf4b0 R08: ff4211874d9bf400 R09: > ff4211874d9bf4b0 > [  489.512976] R10: 0000000000002acd R11: 0000000000000002 R12: > 0000000000000003 > [  489.521779] R13: 0000000000000003 R14: 0000000000252800 R15: > ff421184fffd60c0 > [  489.530584] FS:  00007f66b8ffd800(0000) GS:ff4211874d980000(0000) > knlGS:0000000000000000 > [  489.540460] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [  489.547700] CR2: 00007f66b9150d7e CR3: 0000000281a5e005 CR4: > 0000000000771ef0 > [  489.556502] PKRU: 55555554 > [  489.560337] Call Trace: > [  489.563876]  > [  489.567022]  ? __die_body+0x1a/0x60 > [  489.571728]  ? die_addr+0x38/0x60 > [  489.576241]  ? exc_general_protection+0x19e/0x430 > [  489.582325]  ? asm_exc_general_protection+0x22/0x30 > [  489.588603]  ? __rmqueue_pcplist+0xe1/0xc80 > [  489.594092]  ? __rmqueue_pcplist+0x51b/0xc80 > [  489.599687]  get_page_from_freelist+0xe10/0x1680 > [  489.605675]  __alloc_frozen_pages_noprof+0x171/0x340 > [  489.612048]  new_slab+0x90/0x4d0 > [  489.616466]  ___slab_alloc+0x6f3/0xb20 > [  489.621469]  ? sbitmap_init_node+0x77/0x1a0 > [  489.626961]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  489.633134]  ? _get_random_bytes.part.18+0x90/0x120 > [  489.639409]  ? __slab_alloc.isra.98+0x22/0x40 > [  489.645101]  __slab_alloc.isra.98+0x22/0x40 > [  489.650597]  __kmalloc_node_noprof+0x218/0x510 > [  489.656380]  ? sbitmap_init_node+0x77/0x1a0 > [  489.661874]  ? sbitmap_init_node+0x77/0x1a0 > [  489.667362]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  489.673534]  sbitmap_init_node+0x77/0x1a0 > [  489.678830]  sbitmap_queue_init_node+0x24/0x150 > [  489.684709]  blk_mq_init_tags+0x7e/0x110 > [  489.689915]  blk_mq_alloc_map_and_rqs+0x44/0x320 > [  489.695898]  __blk_mq_alloc_map_and_rqs+0x3b/0x60 > [  489.701973]  blk_mq_alloc_tag_set+0x1f1/0x380 > [  489.707662]  nvme_alloc_io_tag_set+0xc2/0x1a0 [nvme_core > 22f0ce18ead628230226a9b87ebf48eb576bf299] > [  489.718534]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  489.724707]  ? nvme_tcp_alloc_queue+0x293/0x7b0 [nvme_tcp > 68f6be106f52ac467179f8a0922f02aeb6fa1f1c] > [  489.735676]  ? __pfx_nvme_tcp_tls_done+0x10/0x10 [nvme_tcp > 68f6be106f52ac467179f8a0922f02aeb6fa1f1c] > [  489.746737]  nvme_tcp_setup_ctrl+0x3ee/0x700 [nvme_tcp > 68f6be106f52ac467179f8a0922f02aeb6fa1f1c] > [  489.757401]  nvme_tcp_create_ctrl+0x2e3/0x4d0 [nvme_tcp > 68f6be106f52ac467179f8a0922f02aeb6fa1f1c] > [  489.768165]  nvmf_dev_write+0x323/0x3d0 [nvme_fabrics > 34d997d53c805aa2fae8e8baee6a736e8da38358] > [  489.778732]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  489.784906]  vfs_write+0xd9/0x430 > [  489.789422]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  489.795591]  ? __handle_mm_fault+0x7da/0xca0 > [  489.801183]  ksys_write+0x68/0xe0 > [  489.805700]  do_syscall_64+0x74/0x160 > [  489.810616]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  489.816794]  ? __count_memcg_events+0x98/0x130 > [  489.822580]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  489.828750]  ? count_memcg_events.constprop.163+0x1a/0x30 > [  489.835606]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  489.841777]  ? handle_mm_fault+0xa1/0x290 > [  489.847070]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  489.853241]  ? do_user_addr_fault+0x56b/0x830 > [  489.858928]  ? srso_alias_return_thunk+0x5/0xfbef5 > [  489.865101]  ? exc_page_fault+0x68/0x150 > [  489.870302]  entry_SYSCALL_64_after_hwframe+0x76/0x7e > [  489.876769] RIP: 0033:0x7f66b91216f0 > [  489.881573] Code: 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90 90 > 90 90 90 90 90 90 90 90 90 80 3d 19 c3 0e 00 00 74 17 b8 01 00 00 00 0f > 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 48 83 ec 28 48 89 > [  489.903462] RSP: 002b:00007ffef5393ed8 EFLAGS: 00000202 ORIG_RAX: > 0000000000000001 > [  489.912761] RAX: ffffffffffffffda RBX: 000055e61c2113b0 RCX: > 00007f66b91216f0 > [  489.921564] RDX: 00000000000000ed RSI: 000055e61c2113b0 RDI: > 0000000000000003 > [  489.930366] RBP: 0000000000000003 R08: 00000000000000ed R09: > 000055e61c2113b0 > [  489.939169] R10: 0000000000000000 R11: 0000000000000202 R12: > 00000000000000ed > [  489.947970] R13: 00007f66b9380008 R14: 000055e61c20a980 R15: > 000055e61c20b100 > [  489.956783]  > [  489.960024] Modules linked in: tls(E) nvme_tcp(E) af_packet(E) > iscsi_ibft(E) iscsi_boot_sysfs(E) amd_atl(E) intel_rapl_msr(E) > intel_rapl_common(E) amd64_edac(E) edac_mce_amd(E) nls_iso8859_1(E) > nls_cp437(E) dax_hmem(E) vfat(E) cxl_acpi(E) fat(E) kvm_amd(E) > ipmi_ssif(E) cxl_port(E) xfs(E) tg3(E) cxl_core(E) ipmi_si(E) i40e(E) > kvm(E) einj(E) wmi_bmof(E) acpi_cpufreq(E) ipmi_devintf(E) libphy(E) > k10temp(E) libie(E) i2c_piix4(E) i2c_smbus(E) ipmi_msghandler(E) > i2c_designware_platform(E) i2c_designware_core(E) button(E) > nvme_fabrics(E) nvme_keyring(E) fuse(E) efi_pstore(E) configfs(E) > dmi_sysfs(E) ip_tables(E) x_tables(E) ahci(E) libahci(E) > ghash_clmulni_intel(E) libata(E) sha512_ssse3(E) sd_mod(E) > sha256_ssse3(E) ast(E) sha1_ssse3(E) drm_client_lib(E) scsi_dh_emc(E) > i2c_algo_bit(E) aesni_intel(E) drm_shmem_helper(E) scsi_dh_rdac(E) > crypto_simd(E) cryptd(E) xhci_pci(E) drm_kms_helper(E) scsi_dh_alua(E) > nvme(E) sg(E) xhci_hcd(E) nvme_core(E) scsi_mod(E) drm(E) nvme_auth(E) > scsi_common(E) usbcore(E) sp5100_tco(E) ccp(E) > [  489.960207]  wmi(E) btrfs(E) blake2b_generic(E) xor(E) raid6_pq(E) > efivarfs(E) > > Haven't found a culprit for that one for now, started bisecting. > Just wanted to report that as a heads-up, maybe you have some idea. > bisect is pointing to 9aec2fb0fd5e ("slab: allocate frozen pages") and, indeed, reverting this patch on top of linus current resolves the issue. Sorry Matthew. Cheers, Hannes -- Dr. Hannes Reinecke Kernel Storage Architect hare@suse.com +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich