From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15832C52D7C for ; Fri, 23 Aug 2024 18:43:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79497800C0; Fri, 23 Aug 2024 14:43:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7450C800B4; Fri, 23 Aug 2024 14:43:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E518800C0; Fri, 23 Aug 2024 14:43:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3E898800B4 for ; Fri, 23 Aug 2024 14:43:01 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BE6491A131E for ; Fri, 23 Aug 2024 18:43:00 +0000 (UTC) X-FDA: 82484382120.21.FC088AF Received: from mail-yb1-f172.google.com (mail-yb1-f172.google.com [209.85.219.172]) by imf13.hostedemail.com (Postfix) with ESMTP id 076CC2000D for ; Fri, 23 Aug 2024 18:42:58 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=R0RkjenI; spf=pass (imf13.hostedemail.com: domain of flintglass@gmail.com designates 209.85.219.172 as permitted sender) smtp.mailfrom=flintglass@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724438514; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vDPsqzAoxenIzuwcubdWFxYwhXNsFjnpCK8a2v37uUA=; b=pCt9qTbflxGrxTY33BzrPSHoX4grUj+seIM+4g7+jojqRAAkmxhITVl5zr7nZXMzlrNkVO MEb6welMTmdK/BnI8tDoV93YVoho44DMSIDzLrHKFEikVBblqIk8/fYlZZNxZRREjxJxmF j24yyQVTHbIRdKC/Ug6t0FEo5xPHI1M= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=R0RkjenI; spf=pass (imf13.hostedemail.com: domain of flintglass@gmail.com designates 209.85.219.172 as permitted sender) smtp.mailfrom=flintglass@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724438514; a=rsa-sha256; cv=none; b=NHv2E0Yjvh7A+HCKI76N1PzcrmPDWsmLQ/KErlk27ap9hHw0lb+VNCG0ZrMqiwCNNaAQDG NFBvUBoo8CU/vG4S2zzfEk/g9wohbdmGWu4LPdxCNHXdcA4GuWUFnGhgFRpKgI2GeT/olF VqdJSELzjWnk5CIODwzpE8cmoHIE0pI= Received: by mail-yb1-f172.google.com with SMTP id 3f1490d57ef6-dff1ccdc17bso2387336276.0 for ; Fri, 23 Aug 2024 11:42:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724438578; x=1725043378; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=vDPsqzAoxenIzuwcubdWFxYwhXNsFjnpCK8a2v37uUA=; b=R0RkjenIhi7InUREx0hvTf127amOkAIi/UoEinE+DAQnJLVxtKit7H9w3B5deDSYJp YQrGCPGSenAnAwZGmcCgJfTfgCfaQgrHxMtOYiKMVt6EBZfoOIjQ2c7xDOhkt1zH4uSO gY4R12MiHp/0CUKqFqVj9hqblzLMkwdMsLuIhb7cL164wt3pDCANphlD0Qqa1qshXyHp YSctFmRT8lWP66Kd7dEA7jw7LEpwsyCyCFHeIPJuCBRUHG/CdnECfXnq9+xW802OtfOY qZ5kvapLWeOdpx6VDCRx1esFfvPn/G6qTNFwgEilI4hwcez6PMeOeIGEOqENGS2Cccxa 7Wug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724438578; x=1725043378; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vDPsqzAoxenIzuwcubdWFxYwhXNsFjnpCK8a2v37uUA=; b=bhyxde8AOz3LXvz8upPWUUHG/ud096PRNQSBORulFcepcfiSobxrzE4frsoLOcQve/ lUCalx16aOPKoUNjlB76K2a/mmKnxfzyMSgDaZ/bifYWK3Zijrti/JG2waBzXVLaoneu a/2ydFAXwOTAnbkWj2oRayZh02fnqwnVkY+mw4aOu17VNObfIAaKrOMmxoQrwfzP0j41 gyuApn2FKaKUni2sYMPN6K56IfmpB4TEgjQsXGpnkGJtMA6iQYuolrGnE5wbcKgOWcIZ evX0pRlGkgmXzWnxaOhJYwDm19ymxuzm9vJe1iqMzRhzv4dqFBj63KKEtmkdKR+x0tGU 3/HA== X-Forwarded-Encrypted: i=1; AJvYcCVe1ZD/0j37ztLx9iQyMtzWHfFKROwzoIiX2PoOjIeHzMHfPUcXXVDuw35l+guMyZDgxAp5E7VlYA==@kvack.org X-Gm-Message-State: AOJu0YwrVAxRdWLbFE/QgPlG2rYgDIXyWHorIhKm36d8ELaKmIajOcrr H20vl71lDwzXZNqlo3n4RaMZF7AA4ecq7RyxnhMbPbjnVL5Y/ljuYUoEheORuMKUemi6pYdoFOf dtFfzYyT+Hb9vvP+t3t/8xMklfm4= X-Google-Smtp-Source: AGHT+IGP5XOWs22Ob+Od8cQep7S+t0IxbSv28R6vyQfhdmlVTie8ZZ9QOHt1cMUuQWZzMD4Z85T69CYO082JM6cj92s= X-Received: by 2002:a05:6902:1501:b0:e13:c854:2a2 with SMTP id 3f1490d57ef6-e17a83b28d3mr4049002276.11.1724438577951; Fri, 23 Aug 2024 11:42:57 -0700 (PDT) MIME-Version: 1.0 References: <6f65e3a6-5f1a-4fda-b406-17598f4a72d5@leemhuis.info> <02D2DA66-4A91-4033-8B98-ED25FC2E0CD6@gmail.com> In-Reply-To: <02D2DA66-4A91-4033-8B98-ED25FC2E0CD6@gmail.com> From: Takero Funaki Date: Sat, 24 Aug 2024 03:42:47 +0900 Message-ID: Subject: Re: [regression] oops on heavy compilations ("kernel BUG at mm/zswap.c:1005!" and "Oops: invalid opcode: 0000") To: Piotr Oniszczuk Cc: Matthew Wilcox , Linux regressions mailing list , LKML , Johannes Weiner , Yosry Ahmed , Nhat Pham , Linux-MM Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: twx8zwwzak4jap9qdbq6mwgyyy5ghg9u X-Rspam-User: X-Rspamd-Queue-Id: 076CC2000D X-Rspamd-Server: rspam02 X-HE-Tag: 1724438578-951402 X-HE-Meta: U2FsdGVkX185sAdeGmsJybye+bPWhhjNMg6KJ+yY3RwpjQ6c/nJscLN+DyDWKZW6WjnW5ry6sDZPQTgwNCLZHAo3erULDOKza2/SrMuHNs10rqotKaIMDvHRsqmHauybNBLlfh2Qso1Kg3WzIQLPmdoBh/g04K74OIEuDNtJb8lja9MMdVPPT6666JLQ59uxyd7kqwaT1x2hzUiSqYCxwbIcKBHmm0hFaOD+COJWlQbkvlaxMEyTeUpCNtoxzwAc4/7uKWOrvCIRqPuHAEurTOURNdksMpCujnmJNkIJYd2ap9YHZO1yH993x++Er5N+hpKV4ypvaP/DFk+T8aFr3BetAeAnhUGSRO1S1AeFYYYpl6cNHr2K/5f4f+fqmMyR0+18pat/JcESUNSUSUP1oRNgLMP0Mnl6ppmvdVg4/WoOM9qWa9rc7dCotjJ5Bd5O8iPSXpe84E4GhFzXS/nMC0tmOCDFwdYYzH/n8XRHTzdDIhfg0Fob4fXRKlivMkp5RQWaXfjS7GLjkAmQzCGnVcm1ntpYjWY4sgZxByD3puP1kadQnEgR769v5hhV6uv5guUxQTESRbVFiiQOOTMsRGoxb3yRNdPJGOYDNefPveDZIMI5iIzBGj4NHdbD83FHY99A1y/X10m3e9b6eDW6Xr1doHCTdLGX5JTAUl18YK4OAO2Lgl6erUbocfBkrbOTvoWggFUyNJF3qq0uWn7IcpMQHHo8yFCPrc4cT1+0JyKBB1uFIFuIeQEWMkuyciACWoqV9eyuvCShhXczSDOy74YP5dmMdnVn+PYjQGyezntfXyaB3xqN7I7oqwHKm/VVUd+t2QCFVfJ9TR2lojH7jo65GEPKQO3/vkxjIRTjGNDVgvVxQKJld23CffbmtTpyNLviwMFzFzqucD1KpEeIOmsWawjahB+4FXSaRCI8Oag2yOCUOwUrLgE6kxuSdtCIRlM2glg7FZn7EM8awGv hm15UJOT 1IF/AFXv/XOj3krEmHqgA83nY85u5s1dRashAMveMWpvpMBo1x0FmE8nZO1g71g08AtzI5ZadRlChzdLlBCxrw3QrR3zx7U+XMXU9l9QmRqGr/M+a2bTUCJ0SYOC116nnhB6sunHKABEM4lyFZsmzf/t8GY09BdUl0ySzQUZnMW4Nu6CdZ05NdNFt0zHGaMhhPNVIX0bR/Se9ttiuRnVMboZ1JCOs1SifHzhqush5jnKSU8Ckx0FFOiDS+72NQHVfersSJRyZ3mCF0T/6w5BnLPDWaBakwgqhf5IlS3CAPujFuQD38gyqzxeVypAFIPfHwujZTrFRlReinebr9DHLLzHho4Kz3FBYfF8jj+uxE8fYLqOYCRwRkwkTr9OQGwdx1bf4pT/DomedlU4u/EqYvbL/UiVRsE2BQlDi6ZRdUUTPvDdiKTZRATgm0L5zdujsmuchJy9bZfZaiT/uAOXNJWN33jCD7Yv+LZmvWcuYo+rHLeX1YCgPXFoxFr/xxS3c8nsz92DcJxWo/ENX64iHCuV/RaszIMEp8gr8hBGFZUqsRLV3HXi+fv2tJprRgEUOSzyu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: 2024=E5=B9=B48=E6=9C=8824=E6=97=A5(=E5=9C=9F) 0:07 Piotr Oniszczuk : > > > > > Wiadomo=C5=9B=C4=87 napisana przez Matthew Wilcox = w dniu 23.08.2024, o godz. 15:13: > > > > I wouldn't be surprised if this were dodgy ram. > > > Well - that was my initial hypothesis. > > in fact i had few of them. Ranked (and ordered) like this: > 1. downstream kernel patches > 2. hw (ram) issue > 3. kernel bug > > So full history was: > -build myself archlinux 6.10.2 kernel; upgrade builder OS (only kernel; n= othing else) > -run normal devel process and (to my surprise) discover interrupted CI/CD= builds by kernel oops > -downgrade to 6.8.2 and done 4 full builds (full takes 8..9h of constant = 12c/24/t compile). all good. > -prepare vanilla 6.10.6 (to exclude potential downstream (ArchLinux) root= causes) > -run normal devel process and still discover oops > -make sure hw is ok by week of test with 6.8.2 (recompiling for 3 archite= ctures on 4 OS (3 in kvm). This was almost 5 full days of 12c/24 compiling.= All good > -because last steep was all good - decide to go to you :-) > > sure - this is possible that 6.8.2 had luck with my ram and 6.10.6 had no= luck=E2=80=A6.but i personally don=E2=80=99t believe this is a case=E2=80= =A6. > > btw: we can go with elimination strategy. > So what i need to change/disable to be closer to finding root cause? > swap? > now it is swapfile on system nvme > > Hello, I=E2=80=99m encountering a similar crash and trace in the issue I posted on= Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=3D219154 If this is the same issue caused by virtio_net corrupting memory, you should be able to reproduce the crash by sending data to the VM over virtio interface while it is actively allocating memory (e.g., using iperf3 -s on the VM and running iperf -c from another host). In my case, as Thorsten suggested, reverting bisected commit f9dac92ba908 (virtio_ring: enable premapped mode regardless of use_dma_api) along with two related commits in this series resolved the issue: https://lore.kernel.org/all/7774ac707743ad8ce3afeacbd4bee63ac96dd927.172361= 7902.git.mst@redhat.com/