From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 910DCC19F32 for ; Wed, 5 Mar 2025 18:59:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 09031280018; Wed, 5 Mar 2025 13:59:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 021DF28000B; Wed, 5 Mar 2025 13:59:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD7F5280018; Wed, 5 Mar 2025 13:59:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B5B6628000B for ; Wed, 5 Mar 2025 13:59:02 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 31C581A0161 for ; Wed, 5 Mar 2025 08:21:00 +0000 (UTC) X-FDA: 83186801880.26.DE0409D Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by imf17.hostedemail.com (Postfix) with ESMTP id 07F124000D for ; Wed, 5 Mar 2025 08:20:57 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=NroaPe+i; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf17.hostedemail.com: domain of hare@suse.com designates 209.85.221.41 as permitted sender) smtp.mailfrom=hare@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741162858; a=rsa-sha256; cv=none; b=RCwL/M8WLkInVShKWTrFMjASeMeBGTPlKgYKU7Ipdg+pu4PdBhmUOxs4+RvyCfVN/SbMCU QqqwHS5V720FrdTdbwYORccVHFdrwpgq+3PBolt3KXcUolPQwEgXnHqMwmuTaMuMBCtsqW jgbTh4qLnfSgWU0iRUkbKe2+cRik4oc= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=NroaPe+i; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf17.hostedemail.com: domain of hare@suse.com designates 209.85.221.41 as permitted sender) smtp.mailfrom=hare@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741162858; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WstflJwOKcHhDHAGRoeZWYR9qd28A8X/ITaM3cnKcLU=; b=pmET3Zxm8yv7en8jIq1r5Bp7VFeqazI+5cJXRjfVY0nvzJXcydgLz6IJqwDWgUsP5rnb78 eum4y953JUw0z/UV4JbZS5UtK0eEEZlfoao3s5UKipZF8C1oclz1Nx7Q4fIYC65VMlrqko cqkNsnNS7Mty9L4zF5k63kqhcuKRyys= Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-388cae9eb9fso3405070f8f.3 for ; Wed, 05 Mar 2025 00:20:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1741162856; x=1741767656; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=WstflJwOKcHhDHAGRoeZWYR9qd28A8X/ITaM3cnKcLU=; b=NroaPe+iPX1MKRy3b+6a/ZenVTERJb7m0CmLWFt8h4TuIhSPXv/pallpv0KjzE2IXZ +YNFQ5vBheXxMEPcH1RG2irKTm/+Zdbbke5gwYu2sFlw0NL776Qbc5yBVsyxBcoKSwwE UwwNgQrUuL4niXmGLigLCGIoUBBnUzmFxmcn4Jnk28T6K5MNXejnLsy2uhWD/ooJR4LB TQyUdCKGa6p9VpwDSAjvVTDVH/f48e/PEbzO9zCkQ/5qGR9iHNlJKGh9GZTspQPXqg1V DH4i3AV63zpGgONZvABoQOauyBCFXXG+QRmyz2te9hq+u9AL6m+Sv8O7QkDgNoQOmSG7 2uGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741162856; x=1741767656; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WstflJwOKcHhDHAGRoeZWYR9qd28A8X/ITaM3cnKcLU=; b=WIlzoujDKQoK9JPxcTJLNFhEwV4QlEwEYNlwIm1734nD5lNVAoRQrw8fdnL0sxKOY8 s0x3hvSTW0LvMO/QZ66TvrOj/lnR5ZYawQo/wsl8A6qRJ/nymgkWBK57Q+I0/mW00rsn MMa1FSnNQYC6YgWZvcxK1dp2Tg+7hmFOFWxO67aR8bmmpiOD7Jd+69fyYDL2CkCgSr7+ UqAcEdXPzVeBedOn9ymQWtHsL11ELu3+AYfjWMforxZeAnrpttgFqDpCNr2WbxhGDa8R EvOVNu3deIQT//YBV0D54eiFrMVZm5/V33pDMa50HHPr4lO2NWA/egKCRhZX3gMusYYi ePFQ== X-Forwarded-Encrypted: i=1; AJvYcCXdaj5imQ9AB7DgoWFwNMByRYE1OG8jAEDmMvQ++7Xj4kHfsiDhh7maVIR85R9lky453vhUKoW5Rg==@kvack.org X-Gm-Message-State: AOJu0YxWZMaTNMw+7NstfvXulIt0a95kJH3KGM1LJUUVcm7Jq8ruWkJC 7sFhqxzV/U8Zf4yVF7v9XdUQYaTXsU0HdOhnPjWsirhtYN1N7bdkg+FjyoUVyac= X-Gm-Gg: ASbGncvdx2iIxfX45wazdAQZEKDaGTYvAjRrWfTnLD+FE36P1/hUVF6r5FzL+PYXzrB 2VNSjz/T0oMp1SCEbRheM6UWXs7xq7sTa8rPgmsAxokz/Kkb0dr4fDWPTcXPbrVnaSpZnwbVnKS gfCAnW6+Lk8aulbgf0sEfgbxkHPks0pPKuDid2TlH2iIW/H3VUAiPDWnzJLlv4LznDVSYNutEqx 1I+8OKBH3cuh37j8xayqaya/AF9W3VlgvJU3+Xm0eW3O6pkF+WSaHAJdjThGa+J7NcDnpXvxhBs WIM5wg1e+oJkxuTPlXvUCRcH1E4UUDhWGqMKt8a/r4smveB5Q4GuK7net+tOuZaWzJOAB5nukxl ZfnkHbH3EJg== X-Google-Smtp-Source: AGHT+IEAkj4nFKh5XmT6XTDGea5rdD5iNWGSoZ6uQmLsfSiyS4yVckbQm32RaOrZ0zLDfWxwkT/AOQ== X-Received: by 2002:a5d:6d06:0:b0:391:122c:8ab with SMTP id ffacd0b85a97d-3911f756724mr1275068f8f.22.1741162856237; Wed, 05 Mar 2025 00:20:56 -0800 (PST) Received: from [192.168.178.47] (aftr-62-216-202-7.dynamic.mnet-online.de. [62.216.202.7]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-390e4844adfsm20346370f8f.62.2025.03.05.00.20.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 05 Mar 2025 00:20:55 -0800 (PST) Message-ID: Date: Wed, 5 Mar 2025 09:20:54 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Kernel oops with 6.14 when enabling TLS To: Vlastimil Babka , Hannes Reinecke , Matthew Wilcox Cc: Boris Pismenny , John Fastabend , Jakub Kicinski , Sagi Grimberg , "linux-nvme@lists.infradead.org" , "linux-block@vger.kernel.org" , linux-mm@kvack.org, Harry Yoo , "netdev@vger.kernel.org" References: <95b0b93b-3b27-4482-8965-01963cc8beb8@suse.cz> <6877dfb1-9f44-4023-bb6d-e7530d03e33c@suse.com> <27111897-0b36-4d8c-8be9-4f8bdbae88b7@suse.cz> Content-Language: en-US From: Hannes Reinecke In-Reply-To: <27111897-0b36-4d8c-8be9-4f8bdbae88b7@suse.cz> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 07F124000D X-Stat-Signature: 4rbxb3szahkrfhbxifhx66pszsgskwxt X-HE-Tag: 1741162857-393896 X-HE-Meta: U2FsdGVkX1/8kpYVXzyfuoINI1Xs+OHGj/OQxAG9ED3y6guaIrpuNHlI1CKkKFEBBvbd7H6duZFn6lA5jmRDJMO79SZEXRz6gpFVHR70g5adI5Vj6jriTuqYXI3UR4WdFQel+PKo91Ct+TbI/TO38r/Dxok5rGvltpotqLi0T0U3ColpNDEUwIqgh5eArmI/xhPiRdfDULvDifFNRbWmuMH4f+QUsUKnEaSWoWCaXkn4x/8U8sq+UrC0LtsH+LVFVled/nxxH5ekOfA35jAzJ/eOm6HwFt6GlC7y3qOsn3vacS5UHMxFwOHmnF4YB40Ow9FC5ca+RiKwYSzFfXbPg2cmDYESTNghLJo050A3BRiMZ2ZH7djcFSzJtrNESTlboiRdCT1Zw8VR+B1D/EGuVl1qdwaOr8CtKeejKpr8gq8Eln5W0jwk8pkdg69iDLjSXUKXvlzlL3tCO44FhAtFBxIjDN/ren/u+tcG+qzUX+V/pG2Vh7+/9U7bdi9HAPsetRM5y5mU9yZRw2Yr/1RT7LLOENJu1Egq6hydGNDUMxw9hombmTldtFDlsCOS4JhRjP+/1np56LMcCg7qdNVgdeyI4J5jQY4mLHfDnUuQtpBCFUJwuqnZDzGRBB491NDMODw2b6qgtKkX8+STbmnpgkRl6wNxCHnmqnBDlO+pb1Q46tyfnkecYx/b070gha16PEFD31iMNYzzGSyRSaSO6Py0exyn5ehZdUqqsRBGO0Njwcz4+b0UMzjhJqhnTZEFoF3Fdfv7iDIFXMc+aBsyDuerTuEVavs6S6jWfbDFacvJ4c2I5/JlmgVxnyjw9geppRQxHvBxlpduifZBFm+GDGr8M9u84KIZzgj5BPxrO4PMFPjB6GfxY7IF4kqvRuygDcynUc/VH7rcQDzgIHO3mFOFoGWQRLTv8nLZlvjn4KrOd10LPm4w8FnE0i30BJPhBRzOkXwn+tezDwJ9hcq +5WWSTQh klWQFbltuGJxYbQVnchtMh3PEuy0P0WSughCoiC+MDJb/F6T+EsKEQ4vzNqyrBEjo+RWvDa22oAFOyxPGtXiEBG9idT3p5jpMUZIxEAmvP1s2BgEWkAr6fNpHTP0xWDmvr/WnyYa+bGxFa6x4uzV0PoRzqOUnkDBLZPUU1UbBmX6Ettf69sas2Y6mbUVr4aFOB8m6DSmKPEDOy2mshXoappRpLLixNCKWtnOt6z/tbAOI3+CyFRO5c/e5oSySSeM3JAU2g00Q1JVLL8067mOScvUSxyjfsRcl8yxQATYoNpujL6Zq+Ji/lN+as1Pjlx6bUu6boR37pdlmkbDN7hGjoe6yeq9nZA61Xd3fSfyfCK2jAMs6WmqK8t3/x8nBIDUOBsYdXhdC/amHYI39pEp7KGVjOkAzjI6aaqBr1rUZPC9qW7d9d6Ow9KY2GaTKrOFid/g3rKBsFhmI818FccgyxLgId/KSbGKIAGHAoQSqWokDNXT5ySolnQ55IzTTJG1Mxutn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 3/4/25 20:44, Vlastimil Babka wrote: > On 3/4/25 20:39, Hannes Reinecke wrote: [ .. ] >> >> Good news and bad news ... >> Good news: TLS works again! >> Bad news: no errors. > > Wait, did you add a WARN_ON_ONCE() to the put_page() as I suggested? If yes > and there was no error, it would have to be leaking the page. Or the path > uses folio_put() and we'd need to put the warning there. > That triggers: [ 42.364339] page dumped because: VM_WARN_ON_FOLIO(folio_test_slab(folio)) [ 42.364379] ------------[ cut here ]------------ [ 42.375500] WARNING: CPU: 0 PID: 236 at ./include/linux/mm.h:1564 sk_msg_free_elem+0x157/0x180 [ 42.375642] Modules linked in: tls(E) nvme_tcp(E) af_packet(E) iscsi_ibft(E) iscsi_boot_sysfs(E) xfs(E) nls_iso8859_1(E) nls_cp437(E) vfat(E) fat(E) iTCO_wdt(E) intel_pmc_bxt(E) intel_rapl_msr(E) iTCO_vendor_support(E) intel_rapl_common(E) i2c_i801(E) bnxt_en(E) i2c_mux(E) lpc_ich(E) mfd_core(E) i2c_smbus(E) virtio_balloon(E) joydev(E) button(E) nvme_fabrics(E) nvme_keyring(E) nvme_core(E) fuse(E) nvme_auth(E) efi_pstore(E) configfs(E) dmi_sysfs(E) ip_tables(E) x_tables(E) hid_generic(E) usbhid(E) ahci(E) libahci(E) libata(E) virtio_scsi(E) sd_mod(E) scsi_dh_emc(E) scsi_dh_rdac(E) scsi_dh_alua(E) qxl(E) sg(E) ghash_clmulni_intel(E) xhci_pci(E) drm_client_lib(E) drm_exec(E) drm_ttm_helper(E) sha512_ssse3(E) xhci_hcd(E) ttm(E) sha256_ssse3(E) drm_kms_helper(E) scsi_mod(E) sha1_ssse3(E) usbcore(E) scsi_common(E) drm(E) serio_raw(E) btrfs(E) blake2b_generic(E) xor(E) raid6_pq(E) efivarfs(E) qemu_fw_cfg(E) virtio_rng(E) aesni_intel(E) crypto_simd(E) cryptd(E) [ 42.393292] CPU: 0 UID: 0 PID: 236 Comm: kworker/0:1H Kdump: loaded Tainted: G E 6.14.0-rc4-default+ #316 cadaa81909a6170d00e1f47f3fc0db03c6a03650 [ 42.393303] Tainted: [E]=UNSIGNED_MODULE [ 42.393305] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015 [ 42.393310] Workqueue: nvme_tcp_wq nvme_tcp_io_work [nvme_tcp] [ 42.393323] RIP: 0010:sk_msg_free_elem+0x157/0x180 [ 42.393331] Code: ff 48 c7 c6 d0 42 4f 82 48 89 ef e8 b3 63 8a ff 0f 0b 48 8d 6a ff e9 6c ff ff ff 48 c7 c6 a0 42 4f 82 48 89 ef e8 99 63 8a ff <0f> 0b e9 c7 fe ff ff 2b 87 78 01 00 00 8b 97 c0 00 00 00 29 d0 ba [ 42.393336] RSP: 0018:ffffc9000040b798 EFLAGS: 00010282 [ 42.393341] RAX: 000000000000003d RBX: ffff888110ab0858 RCX: 0000000000000027 [ 42.393344] RDX: 0000000000000000 RSI: 0000000000000002 RDI: ffff88817f423748 [ 42.393347] RBP: ffffea0004295e00 R08: 0000000000000000 R09: 0000000000000001 [ 42.393350] R10: ffffc9000040b780 R11: ffffc9000040b4e0 R12: 0000000000000400 [ 42.393353] R13: ffff888110ab0818 R14: 0000000000000002 R15: ffff88810fa669d8 [ 42.393361] FS: 0000000000000000(0000) GS:ffff88817f400000(0000) knlGS:0000000000000000 [ 42.393365] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 42.393369] CR2: 00007f56a6ea6da4 CR3: 000000011bfc0000 CR4: 0000000000350ef0 [ 42.416071] Call Trace: [ 42.416078] [ 42.416084] ? __warn+0x85/0x130 [ 42.416095] ? sk_msg_free_elem+0x157/0x180 [ 42.418893] ? report_bug+0xf8/0x1e0 [ 42.418904] ? handle_bug+0x50/0xa0 [ 42.418910] ? exc_invalid_op+0x13/0x60 [ 42.418916] ? asm_exc_invalid_op+0x16/0x20 [ 42.418935] ? sk_msg_free_elem+0x157/0x180 [ 42.423206] ? sk_msg_free_elem+0x157/0x180 [ 42.423215] __sk_msg_free+0x4f/0x100 [ 42.423224] tls_tx_records+0x118/0x190 [tls 80cce2d02933ba636eb5845a829121ac309b44ed] [ 42.426506] bpf_exec_tx_verdict+0x249/0x5e0 [tls 80cce2d02933ba636eb5845a829121ac309b44ed] [ 42.426519] ? srso_return_thunk+0x5/0x5f [ 42.426526] ? __pfx_stack_trace_consume_entry+0x10/0x10 [ 42.426572] tls_sw_sendmsg+0x72f/0x9f0 [tls 80cce2d02933ba636eb5845a829121ac309b44ed] [ 42.432016] __sock_sendmsg+0x98/0xc0 [ 42.432025] sock_sendmsg+0x5c/0xa0 [ 42.432030] ? srso_return_thunk+0x5/0x5f [ 42.432034] ? __sock_sendmsg+0x98/0xc0 [ 42.432040] ? srso_return_thunk+0x5/0x5f [ 42.436134] ? sock_sendmsg+0x5c/0xa0 [ 42.436146] nvme_tcp_try_send_data+0x13f/0x410 [nvme_tcp 9f4f1c84141d3edfcd3e478eb7c2fb638b4a92b3] [ 42.436159] ? srso_return_thunk+0x5/0x5f [ 42.439452] ? sched_balance_newidle+0x2f6/0x400 [ 42.439468] nvme_tcp_try_send+0x299/0x330 [nvme_tcp 9f4f1c84141d3edfcd3e478eb7c2fb638b4a92b3] [ 42.439479] nvme_tcp_io_work+0x37/0xb0 [nvme_tcp 9f4f1c84141d3edfcd3e478eb7c2fb638b4a92b3] [ 42.443603] process_scheduled_works+0x97/0x400 [ 42.443614] ? __pfx_worker_thread+0x10/0x10 [ 42.443619] worker_thread+0x105/0x240 [ 42.443625] ? __pfx_worker_thread+0x10/0x10 [ 42.443630] kthread+0xec/0x200 [ 42.443639] ? __pfx_kthread+0x10/0x10 [ 42.443646] ret_from_fork+0x30/0x50 [ 42.443652] ? __pfx_kthread+0x10/0x10 [ 42.443658] ret_from_fork_asm+0x1a/0x30 [ 42.451127] [ 42.451131] ---[ end trace 0000000000000000 ]--- Not surprisingly, though, as the original code did a get_page(), so there had to be a corresponding put_page() somewhere. Cheers, Hannes -- Dr. Hannes Reinecke Kernel Storage Architect hare@suse.com +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich