From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60391C4167D for ; Tue, 31 Oct 2023 15:42:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C95748D0018; Tue, 31 Oct 2023 11:42:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C45078D0012; Tue, 31 Oct 2023 11:42:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0C8A8D0018; Tue, 31 Oct 2023 11:42:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A191D8D0012 for ; Tue, 31 Oct 2023 11:42:28 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 42DA4B5482 for ; Tue, 31 Oct 2023 15:42:28 +0000 (UTC) X-FDA: 81406173576.29.2D53F2C Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by imf02.hostedemail.com (Postfix) with ESMTP id 068868000B for ; Tue, 31 Oct 2023 15:42:25 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=invisiblethingslab.com header.s=fm3 header.b=BeyyQ7Ba; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=nYOyPpfj; spf=none (imf02.hostedemail.com: domain of marmarek@invisiblethingslab.com has no SPF policy when checking 64.147.123.25) smtp.mailfrom=marmarek@invisiblethingslab.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698766946; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mb0gYnV0H7yJxOK1w9f/YAAaMqkDuUMTlzoUd82abGI=; b=sYaVhhGPxpAZxaKJI5zY0T5Jz438AY7ClgqTWYKKedLqBffXFD6tqBZSGvXh2l1h1rFLgM gtlTxahIsgvP3rq7EoR3Hs9Phv4Pxd8CwfUFBFQ0zOtcrKMMvRhxbtjOzdPhLF8rFhW88T L/AmgmhsHtwZf3uxiCur3nQbVU5Payo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698766946; a=rsa-sha256; cv=none; b=XQa2eHrRbz2JehYgInnQBKAWYRx+4SwX/dvMQiJtK+LrPBtF9ewh4KKzlqZGi8fl/sm76q uCOw1hKU1Tx0WM/rbwPaqx1LrylIs65AwG8LG0gXlGNKNbY2vrTramZPzX0qt8QJW2rY7b boCFLHZk9IwnZf0c2xsGMzNHjjIuxJc= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=invisiblethingslab.com header.s=fm3 header.b=BeyyQ7Ba; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=nYOyPpfj; spf=none (imf02.hostedemail.com: domain of marmarek@invisiblethingslab.com has no SPF policy when checking 64.147.123.25) smtp.mailfrom=marmarek@invisiblethingslab.com; dmarc=none Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 5976A32009DC; Tue, 31 Oct 2023 11:42:23 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Tue, 31 Oct 2023 11:42:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1698766942; x=1698853342; bh=mb0gYnV0H7yJxOK1w9f/YAAaMqkDuUMTlzo Ud82abGI=; b=BeyyQ7BaR/RTfnF3twpmruJUylTd2KwhIKx23X70HU3YCqspbzF X6EO2K57TKbc7Mn25vORN7XhERC2lI7nOmpJKMeEwz5n0wjbPBMLg3s4PnwuS4oE GNX43jlbcIMsVCwylSZtStBr05IYD1KDNN/cBYAszZF+iu3wYfKg/MS56zhIo3Wr Euf10bzZMJDP0WiPo3/pj8+EfLhb5WZV+5xwHz990+Fa3hqO/KHp/v+K1vPRIB3d PUIdXssOculZP1Xy52e6gRm775dRaeVuY0ID9JkfePdqqEp4Cd7W80TXWX3J4/45 sY2g6fjxWQ0dgzqObJfz1KNUT7Ip0KBM/9w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1698766942; x=1698853342; bh=mb0gYnV0H7yJx OK1w9f/YAAaMqkDuUMTlzoUd82abGI=; b=nYOyPpfjVRmq9pCxHWrqyTLGx9rpA GnCHemUTpulOxIjaEqeBtLvLo2+PPa50Bc+14ETaISQyFnIvTHmtVb3J7Ac2kfnq +y4WewQfvMPiob7Bg1j1UTc6fO/DyO1wdA8NFZ10J8omUVproUWww+gHZDgSjEFI YnqRJ8NF8uIWG+r15Lmq4wBaTnIIn9QLpJbfJIG+J6s3QNPoSnbkR0SKWVXze0xG 57itohLuUUuo6/nadLbx5t8gn57YUf+ODQbzpCoKCgPiT8lzMcUNHAAS3hYISYxU VrZC3ut/SmXRrwB+ZRTDBqtpVeULJ+nk2KrVjIOLi3Ts4gXuoSSpj/7YQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedruddtvddgjeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepgfdu leetfeevhfefheeiteeliefhjefhleduveetteekveettddvgeeuteefjedunecuvehluh hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghk sehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh X-ME-Proxy: Feedback-ID: i1568416f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 31 Oct 2023 11:42:19 -0400 (EDT) Date: Tue, 31 Oct 2023 16:42:15 +0100 From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= To: Jan Kara Cc: Mikulas Patocka , Vlastimil Babka , Andrew Morton , Matthew Wilcox , Michal Hocko , stable@vger.kernel.org, regressions@lists.linux.dev, Alasdair Kergon , Mike Snitzer , dm-devel@lists.linux.dev, linux-mm@kvack.org Subject: Re: Intermittent storage (dm-crypt?) freeze - regression 6.4->6.5 Message-ID: References: <3514c87f-c87f-f91f-ca90-1616428f6317@redhat.com> <1a47fa28-3968-51df-5b0b-a19c675cc289@suse.cz> <20231030122513.6gds75hxd65gu747@quack3> <20231030155603.k3kejytq2e4vnp7z@quack3> <98aefaa9-1ac-a0e4-fb9a-89ded456750@redhat.com> <20231031140136.25bio5wajc5pmdtl@quack3> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="T+T3sop+0bQ1u8dt" Content-Disposition: inline In-Reply-To: <20231031140136.25bio5wajc5pmdtl@quack3> X-Stat-Signature: zq16n7j9dtod36cf8sntoc595aofuz3p X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 068868000B X-Rspam-User: X-HE-Tag: 1698766945-274631 X-HE-Meta: U2FsdGVkX1+kOVaj/SK9D7cfH3KRwVsA6I35J77RQ2FxINYbgeaycHnPoaiZS1I/e2QR3rwp26giDWCVUMlwqO523ks0bv1Nw4t2bVM+MhH4Rqsao+4v78Iveu9VpDWcfbPHtzISqfmgrDEUimoMSKMHXNy/nKGV5OZIkjhsVYFY6ZdFsnHbHUbMAsGg+cXlTchImgyhm9XF/4upaPUVKCEpicoB7xTPwdBPJ5TCnL7rNi4UgMl1hb4g9PymRmMr8AThAu1v/j/RUsROZXacmOemXBfQ87hqQhciRLGGZA35SybBIMJBCYWqL1e6+nZWSiVYqqRnmHbsEyy9vgFKSVogEDravfRDeT1gdJ5rLyjUtpeAhv/hNbv65YUwEIDGd6ksH3MEcrRuCiwPhyCEmLCW5psoiGHsrJ7VMmZjoVqj0AxtkMc5gPcXczFROQDFH8znyHuXCgbMbSOhPWyUX3LyyA6HdTy2S7J4+4lhbuuBBB/6gPtPqJ3Si0cKyiVeRjawqOkaS4VRpa4wjzAaRdlqsEFya/ZkfpsjNJgu7AX6PIfpsuONhbAWd5WO2u+IX55zGcyogvN6yDD/XkQpzwhZ3NpZsty67WpWaODbbUaCVKhZmkEzIT0wrElDPWf22EqRKu3fYxvKIzHKXvkMWSIO0l7PgT2a+Ii/UwSS/5xdmVGlH9DB2WrdIuNus45q/NL5kAWeixBuKDmINkqUnP5pAyI5UcWMS2ZW1XrnF6wPzMtltTADkzjIJE6wkfisuOPhdWhcHygNSNsFhM+rPm0JAGHw6ay7VzNJKuzGLYJUQIRqonP8a0VQxUnOzmW09OiuFL4IJudH84dj0XM/U9/IgxINJ0Et192FeFdt6E5x8n3KvKoQD/ik6vUf+Zr4hXubPdzF+YzzvoLlwQdQFBHiTgicoq0yGPa1x/JqNm5IAu+oxQ/Sj3QaWef1MOTaTak2KDE17FbtqxSCiVj 3EN4hvfH iAVB/YO7nRURZf3IlKwSJfGZlNPIlEc3fPSmNvv3whNrYP+QfiEyE0V1xNNCtQX7YCP419ZrVlD9Ro76Fv9xRMWVySszLBmhRSivrrUMGa3CyXtl0K1+Esw7pTnhPNP4bpsT2NsPfUjK3MEQVwLSNAXB3KwHaX/sDcXPxe+2YsAYHR4JMlS6JaOw3GnvDrNPTLMWkzbhxsLrOKxkEdRrNAcdK85QZN8CSJg6NsI8J/VDgU0v7BSN8XwNfQiw1uVgoidapcKTwNlTBLKEkJ4+DJHzceAMTdmLuPJnBuGTUM0/DZHg09BOnTIxWSM16pf6L5Rl0cIAUAnEcBxaWwOYIeWx49ay04r2OxFLCLltJLmdP81/4tXKxCKa3RrH/w17A7fUfZPGjqQwOxjEpqeg8vEygw+b1EaeyI0X425vAkLEhugPatgAiBoJ5QSZD7fcBwjGi/Gh7rvHA0FM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: --T+T3sop+0bQ1u8dt Content-Type: text/plain; protected-headers=v1; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Date: Tue, 31 Oct 2023 16:42:15 +0100 From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= To: Jan Kara Cc: Mikulas Patocka , Vlastimil Babka , Andrew Morton , Matthew Wilcox , Michal Hocko , stable@vger.kernel.org, regressions@lists.linux.dev, Alasdair Kergon , Mike Snitzer , dm-devel@lists.linux.dev, linux-mm@kvack.org Subject: Re: Intermittent storage (dm-crypt?) freeze - regression 6.4->6.5 On Tue, Oct 31, 2023 at 03:01:36PM +0100, Jan Kara wrote: > On Tue 31-10-23 04:48:44, Marek Marczykowski-G=C3=B3recki wrote: > > Then tried: > > - PAGE_ALLOC_COSTLY_ORDER=3D4, order=3D4 - cannot reproduce, > > - PAGE_ALLOC_COSTLY_ORDER=3D4, order=3D5 - cannot reproduce, > > - PAGE_ALLOC_COSTLY_ORDER=3D4, order=3D6 - freeze rather quickly > >=20 > > I've retried the PAGE_ALLOC_COSTLY_ORDER=3D4,order=3D5 case several tim= es > > and I can't reproduce the issue there. I'm confused... >=20 > And this kind of confirms that allocations > PAGE_ALLOC_COSTLY_ORDER > causing hangs is most likely just a coincidence. Rather something either = in > the block layer or in the storage driver has problems with handling bios > with sufficiently high order pages attached. This is going to be a bit > painful to debug I'm afraid. How long does it take for you trigger the > hang? I'm asking to get rough estimate how heavy tracing we can afford so > that we don't overwhelm the system... Sometimes it freezes just after logging in, but in worst case it takes me about 10min of more or less `tar xz` + `dd`. --=20 Best Regards, Marek Marczykowski-G=C3=B3recki Invisible Things Lab --T+T3sop+0bQ1u8dt Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmVBIFgACgkQ24/THMrX 1yxXiwf/STqQWBIoStONLIYTVENKHHERIECrbOJ3yP9Go8RfO3rJ23Ex6CAj0jvm eoItbb+4wfzvg6Cko+yvRa1jlK1Bau9YT17OhaJydq3WIbfUbxlMq8Rj8q43jKak PMzvgY2aHJ+P33UbnAweZTH9zVE2hOBi1WDs4tLpHXSqaiRxtRTLxkhEOR3+ESDz HJNb3lNtD6/8H6fIDdElrizvvpDmbbLoA2ulPJGzBHTz9gt55taD22b9GChRE9Bv pYSyINW8cN1jQzYG/eLtcJGrBpeI0F8LrlCfBnK3A9/QiV8NLBJmT2h+WEjv7Azp bUEnlyfGpodVuj6V5NGcWtAAkl6R2Q== =krBC -----END PGP SIGNATURE----- --T+T3sop+0bQ1u8dt--