From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED1F9C4167D for ; Mon, 30 Oct 2023 08:37:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6868D6B0190; Mon, 30 Oct 2023 04:37:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 636AA6B0192; Mon, 30 Oct 2023 04:37:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4FE446B0193; Mon, 30 Oct 2023 04:37:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 407D26B0190 for ; Mon, 30 Oct 2023 04:37:16 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0D1B88058A for ; Mon, 30 Oct 2023 08:37:16 +0000 (UTC) X-FDA: 81401473272.03.F1D9DE7 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf01.hostedemail.com (Postfix) with ESMTP id A1E8A40012 for ; Mon, 30 Oct 2023 08:37:13 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="24qfnBZ/"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="rLkfd/tX"; dmarc=none; spf=pass (imf01.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698655034; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OT3gH6W3eVii6TzxQ2SSrakkNp82aIS3RiPQ/z/s9nk=; b=5AkoJQeoP/rZ/lFKM2O72at2Er/HwArCCJCVpn+MSmiAX/fibC2tYubJei7OFgIpHiDS9G SBOQUybq2bLAt70XwQrSJdTRpHsD6JeKTgCIrDa2NFXmDVGeSCxX5NI82eM8A3IE9gduzl KyWoel04sVJnFbrrAbEisKHDhuSsA5w= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="24qfnBZ/"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="rLkfd/tX"; dmarc=none; spf=pass (imf01.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698655034; a=rsa-sha256; cv=none; b=jFoCdmfNyujY9u+qEN6iDMf4wBqbOEKz1tbY/hG6xi8tR1p+YQnJwbCS7Hje9L57nyONQH MyCGKyYCDI2gFnZWMslRK4E8o+rhxSgBQWdLo7yY+vHNQo60wLRkVOD8za/cZHJcYAFqtw stglEePaoEJV6wJT1xHRE3AAxj0LATE= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 4F81621992; Mon, 30 Oct 2023 08:37:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1698655031; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OT3gH6W3eVii6TzxQ2SSrakkNp82aIS3RiPQ/z/s9nk=; b=24qfnBZ/ngwGoEVrHhkIOaZDKFpbYpOmf8t/yCcsFyZa+/v9gE/ClAV4rZvDOhxPaN8ros gsam/yFyrhJAs5tSDCdBBvTo5TRBwXf+BkZhTDQfUx4fduhNYRvwuWYjX9ymkSHcx4xrmi KRnScYNc+iWKHQPUKx7WktfpOAN6/XE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1698655031; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OT3gH6W3eVii6TzxQ2SSrakkNp82aIS3RiPQ/z/s9nk=; b=rLkfd/tXI4lLSxLJFuDtXWYfqb3NogmTxO2wxKiFYibHKn/nPUhId+nKEdOauiRL09Cm9C hdz9hnWo4o93nqDw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2C2F6138EF; Mon, 30 Oct 2023 08:37:11 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id g8kTCjdrP2XlbAAAMHmgww (envelope-from ); Mon, 30 Oct 2023 08:37:11 +0000 Message-ID: <18a38935-3031-1f35-bc36-40406e2e6fd2@suse.cz> Date: Mon, 30 Oct 2023 09:37:10 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 Subject: Re: Intermittent storage (dm-crypt?) freeze - regression 6.4->6.5 To: Mikulas Patocka Cc: =?UTF-8?Q?Marek_Marczykowski-G=c3=b3recki?= , Andrew Morton , Matthew Wilcox , Michal Hocko , stable@vger.kernel.org, regressions@lists.linux.dev, Alasdair Kergon , Mike Snitzer , dm-devel@lists.linux.dev, linux-mm@kvack.org, Jan Kara References: <89320668-67a2-2a41-e577-a2f561e3dfdd@suse.cz> <818a23f2-c242-1c51-232d-d479c3bcbb6@redhat.com> Content-Language: en-US From: Vlastimil Babka In-Reply-To: <818a23f2-c242-1c51-232d-d479c3bcbb6@redhat.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: A1E8A40012 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: b6b114xwyas3di8kyqqfeyroi1u1f5eo X-HE-Tag: 1698655033-490516 X-HE-Meta: U2FsdGVkX19MSrxvuUNW5xpB4qf0FdZ0XkqxnI6qpTMf+9Rrcj0C4+wUubi2h3kJC33XtXOy9Uwk+IlnpbntzlOOQd0KYVulkpyI3Ecaorwn5HunbPpXiCcFx3y7IZYHK9n3XLrvHSOhJYhcbzVG15zob40RirTSkWLCPl9bxtYtC+JXkMtEs46QZAf6LJvlmrRfnSiAELq2VDHS1x+FuHgFBUv+KrdYcoyZBuNCWqLzSP8pVxKJ5sK6iuHOKqXbj54kNGcX8j5PMK3kn5mrn5cYx8iD7c/OI0aqJ1X7oJoRj/EMLEyrN8V43U6ntfenY1Mq7JcYGGDhP6XVW57mBCJMgkhrNbFCwY9+WFCztjFJT4apojrQn0IVyqY4ry0b//mWUvwU+zjlO2IxkQwC5e9tp9ZZAlRHRQRApQNMZCLb8UJB5acOd+Qin7uQbYRESu0jfWSiZY2Y3NCM+2C2IW8/9eSAR8otvr+JYOrW5/I4uWJLLwny5j62VOYVYNVr2l3Oood/DDnWK2Yv1iP70IGj3rRnrGdULjmpHGjo9+Qlcnmo7NN3+AfxzqmoLNea0pq9cHHL8B9UW2+oVHKmOewj2aC2Xq5gLb9tH/uODiWnxYG37kO633dpEZb/cWWTPsRaYtEsy/JHZ4h5F/z8CyY5nMY6vUdOnZo37UOKa0TTwd4sO3eLZ8zfDad48UeX5yH+PY7KTgMUwD0FPs5nCkNt8uC/EUyq9La2RTw6XdpC/jmC6QHyOmqwZCZWAvuocs4wdGNFbajvYIAcQxjHG1OL2y0/ZIahzuD2JY1wkw3mqficUbAf+B3ooD4JcMi3dl/iI79ZYfPbAM+aXtPaUFqETdsfBJoEq9hsF+Zs5aZBj10DwPnR4QLc/S6AQ3g37D4HvdPfZEXGzcnlNWXf1O8i+qdfFaKae/mi2i+QTWRXKeWeAW5APNgn7h6CGj/4wy1/8xFDBrKuScHmVED pEsLrkuj wvCAiOosnGdat15D/Zyj+llGhVqgnXOkgxBJFcIOsWrIJBMXxCVRqXSq1ajtEwyU/mFmsvyVi7mDRYkV8qVFB+7W2nUgg1FvyLlQy0ltYbgZUjpPQItjEShGLVVLEZUO6s04NOpGBHHFhNOD/3uy6W45rHf44JiTKjvOH1KRDWc90fZURzem0lM/FO30+Wpq23uccqpAu6KtWaiweFP157oLWdJgzkHYGQptACkVt4f2y83Iultd98/8qktrHKE3STvHpPP6qCznVCwQqxC///lvNj2IlpZ1HIGSsRevL1Q9yaZVPTTQ19XspaBPzB/G2C1e6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 10/30/23 08:37, Mikulas Patocka wrote: > > > On Sun, 29 Oct 2023, Vlastimil Babka wrote: > >> Haven't found any. However I'd like to point out some things I noticed in >> crypt_alloc_buffer(), although they are probably not related. >> >> > static struct bio *crypt_alloc_buffer(struct dm_crypt_io *io, unsigned int size) >> > { >> > struct crypt_config *cc = io->cc; >> > struct bio *clone; >> > unsigned int nr_iovecs = (size + PAGE_SIZE - 1) >> PAGE_SHIFT; >> > gfp_t gfp_mask = GFP_NOWAIT | __GFP_HIGHMEM; >> > unsigned int remaining_size; >> > unsigned int order = MAX_ORDER - 1; >> > >> > retry: >> > if (unlikely(gfp_mask & __GFP_DIRECT_RECLAIM)) >> > mutex_lock(&cc->bio_alloc_lock); >> >> What if we end up in "goto retry" more than once? I don't see a matching > > It is impossible. Before we jump to the retry label, we set > __GFP_DIRECT_RECLAIM. mempool_alloc can't ever fail if > __GFP_DIRECT_RECLAIM is present (it will just wait until some other task > frees some objects into the mempool). Ah, missed that. And the traces don't show that we would be waiting for that. I'm starting to think the allocation itself is really not the issue here. Also I don't think it deprives something else of large order pages, as per the sysrq listing they still existed. What I rather suspect is what happens next to the allocated bio such that it works well with order-0 or up to costly_order pages, but there's some problem causing a deadlock if the bio contains larger pages than that? Cc Honza. The thread starts here: https://lore.kernel.org/all/ZTNH0qtmint%2FzLJZ@mail-itl/ The linked qubes reports has a number of blocked task listings that can be expanded: https://github.com/QubesOS/qubes-issues/issues/8575