From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61346C48BC3 for ; Mon, 19 Feb 2024 11:40:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DBD6D6B007B; Mon, 19 Feb 2024 06:40:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D46B96B007D; Mon, 19 Feb 2024 06:40:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC0BB6B007E; Mon, 19 Feb 2024 06:40:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A4E6A6B007B for ; Mon, 19 Feb 2024 06:40:08 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7CCAD1C02B5 for ; Mon, 19 Feb 2024 11:40:06 +0000 (UTC) X-FDA: 81808359612.30.BA21B78 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf07.hostedemail.com (Postfix) with ESMTP id 2F2834001B for ; Mon, 19 Feb 2024 11:40:03 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=qyxDsVl2; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=nTDEHiH1; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=qyxDsVl2; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=nTDEHiH1; dmarc=none; spf=pass (imf07.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708342804; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VpNd7pScMzcCua32coUiV3InuIJm0wwq2wM8HmpeqA4=; b=TsH09ksu3Jg+tHDPAS3OMMOisHo/+GSgBcqRU2ZwyB3Pf59yPAbCGFMWfZ+g+A3XCROnYG dxv0+401hjgvTUrrVuXLNODZQ7HvEbqYWtGQhulHzHki+AxPE0dUyD2qApigxEfssOnACF bwqQ7zx4bGCu7GlvhCOtvc/d+UzxsSY= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=qyxDsVl2; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=nTDEHiH1; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=qyxDsVl2; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=nTDEHiH1; dmarc=none; spf=pass (imf07.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708342804; a=rsa-sha256; cv=none; b=lK5aG6bNzjBXQqyDh49thR2HCWCKkBUQoLp7b3Pym+zqsVR6TRGDZWMv4RtSYYcYcYK1NB JT3UFmnpM9ARskQMQxBSumcF4FvnP7bx0cMBMd32+OAElbsD52xHdmctVdHZINejeaQIne J0gMa1ff5bkxcXjPUWlHjrwDPU+fcsU= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 3741E1FD12; Mon, 19 Feb 2024 11:40:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1708342802; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VpNd7pScMzcCua32coUiV3InuIJm0wwq2wM8HmpeqA4=; b=qyxDsVl2nUf/elcWaG06ru0iwhkV3yDVeyszL4wtreYDPp2GliqZBCGCJhtrrsqr8j4cEU 9FUpM5nyW3uDaGDQjC/F6gz9D5YSQxsqvFmC+YQYo1LfuB+MLTRJY38d3wm5txQIO3+R+H QjokKUPFJ/HyFhVFStL/1SucI/1Wt4g= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1708342802; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VpNd7pScMzcCua32coUiV3InuIJm0wwq2wM8HmpeqA4=; b=nTDEHiH188LXGOCghi9RFOmdhpIEfrthQIl/Ne7U4pmnEm8boyJrpa2vN4ckwPYguaGbcV 6V1jV3BEUoIdpbCw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1708342802; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VpNd7pScMzcCua32coUiV3InuIJm0wwq2wM8HmpeqA4=; b=qyxDsVl2nUf/elcWaG06ru0iwhkV3yDVeyszL4wtreYDPp2GliqZBCGCJhtrrsqr8j4cEU 9FUpM5nyW3uDaGDQjC/F6gz9D5YSQxsqvFmC+YQYo1LfuB+MLTRJY38d3wm5txQIO3+R+H QjokKUPFJ/HyFhVFStL/1SucI/1Wt4g= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1708342802; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VpNd7pScMzcCua32coUiV3InuIJm0wwq2wM8HmpeqA4=; b=nTDEHiH188LXGOCghi9RFOmdhpIEfrthQIl/Ne7U4pmnEm8boyJrpa2vN4ckwPYguaGbcV 6V1jV3BEUoIdpbCw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1EF9D139D0; Mon, 19 Feb 2024 11:40:02 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id bBw4BxI+02W+NgAAD6G6ig (envelope-from ); Mon, 19 Feb 2024 11:40:02 +0000 Message-ID: <235ab5aa-90a4-4dd7-b2c6-70469605bcfb@suse.cz> Date: Mon, 19 Feb 2024 12:40:01 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Stall at page allocations with __GFP_RETRY_MAYFAIL (Re: [PATCH v1] ALSA: memalloc: Fix indefinite hang in non-iommu case) Content-Language: en-US To: Takashi Iwai , Karthikeyan Ramasubramanian Cc: LKML , Sven van Ashbrook , Brian Geffon , stable@vger.kernel.org, Curtis Malainey , Jaroslav Kysela , Takashi Iwai , linux-sound@vger.kernel.org, linux-mm@kvack.org References: <20240214170720.v1.1.Ic3de2566a7fd3de8501b2f18afa9f94eadb2df0a@changeid> <87jzn0ofdb.wl-tiwai@suse.de> From: Vlastimil Babka In-Reply-To: <87jzn0ofdb.wl-tiwai@suse.de> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 2F2834001B X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: oqyirzi18m6roht4u4aerexdnz11ioak X-HE-Tag: 1708342803-537062 X-HE-Meta: U2FsdGVkX190mMpomb4RhlQNha8a1xNbqLvQR99bttgPEp+kDHA6LDp6l5vjXgKv3hk0SQKtB47pzMmMtsdBk+h0VP3GQHhe0lbnwe/S1tk74IL+p9iKZ2Q+lkhWALZ3WcX8wND8jknau0FSuIFfiEdIyAZ5mx3A81Td6YkRdngAZUrhrBaCwhNfJODhei0egXJkk2yjk3oXuo9jhjsLCHm8Zpu/MCY+5tIqdKmosz27mw2oNm9qLkwvHXF3d198fU5/Zf1WNxSdmZdyJFOagGjHPXbMDJVsMFPi1sPHYUJMC3lmXmVRLdLABakTDTvtzi5LifBlNXUY+UNSyrAOcpvHZ6yR/OpOlxZHCGncsM1NypjCG6imbNmfRy2oWprZBYUda1iIOo5yutV3j4pci/z7oXEL+kVYVYd6pJToDjVsv0CeFWcRQa+ujr9XlDUk6cXWV7Q/h7hGg6ABs2nK+vTNPH8JuqMOuxpWiQi+5GiQrKrdMB5vso7x2wkgnXfJMrElz3NQhHPdVohrgQlEqXd4xsEafmt30M5geeH8tVTNcViYlKPSR4PSTtLlTDOnky9ddUMl9IK6hwT9j+HdlmJuAwnlE0Pvc5MmYHhNOR+ceWuT6AaSjzMo/NHTWJBwVJTM7uCj9mi0hmwuvu83n/TlKh0KzvvBqKmrLC8tE7zOIobr0sewLppXGdp+fn/hRurq6t+FKGPhDBockKW5TQo9GXWpz0VCldhYE7AqQ1NWuCRJsMdm1x/bGoRHjqw0EIsh8MImCdqmSGuibU8Df7/uKY7FuoSQTKJfqGJ4USZHH3//YRsR33fVIlGXFFPqFoJOwZlUfH8lzLyUpYgviCzqLvllL/W7Rc8lpWmnq/u7EluSNeo4RqRCDGHTyGiaYj8aADD4jDMgC1cD824Gywz/K+FisbNxkOMWyHNQYqJSaBXQHdzLxFti6H3dTav+ylOTTMPrynLatWyBfbX /SBKXDpf HKFyiikU1yeng0UtoO6ZheCpYJv2k0AeK/MeBNf9kEHDT7x5CdBq1IgBncjimO6ormX9Nkb54DPjwm/gr0EXwf02ugIZgevKIRXgsuFr9HCD13qxhdXSRhGK/0F71BHD5/l2yGOhwZBLl7oHv1SrsniM9OT9R+yYzwf6gQ8OROtuRUl5wRPpC74kzPey3DC+Yl4ITTlu4dQXnQpv+tXT1hzG7lS38wLwqgxQDf6ItDkXqOSrBzAJBH7wLkYuEgvwCfmpa2UT1Y4TvglXf+4HjQqzrpX8Z+9zbPALmjC53l2nL4g1EqGM7hnhLGX9hxgC8k0wchcGhgbrHD8QVyXsmM8wHL0wboRvZHcX4tyGVd4fFkJ3Si0Wx7UkoVpKv6Yq7a+0eMMG6KDFYTLaJf2knkzHTfw1nnQzuZZtFNBhBUwQIV3I5Q+FaCPEED9atGKFzplRTfrTI5k9TkAs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2/19/24 12:36, Takashi Iwai wrote: > On Thu, 15 Feb 2024 01:07:25 +0100, > Karthikeyan Ramasubramanian wrote: >> >> Before 9d8e536 ("ALSA: memalloc: Try dma_alloc_noncontiguous() at first") >> the alsa non-contiguous allocator always called the alsa fallback >> allocator in the non-iommu case. This allocated non-contig memory >> consisting of progressively smaller contiguous chunks. Allocation was >> fast due to the OR-ing in of __GFP_NORETRY. >> >> After 9d8e536 ("ALSA: memalloc: Try dma_alloc_noncontiguous() at first") >> the code tries the dma non-contig allocator first, then falls back to >> the alsa fallback allocator. In the non-iommu case, the former supports >> only a single contiguous chunk. >> >> We have observed experimentally that under heavy memory fragmentation, >> allocating a large-ish contiguous chunk with __GFP_RETRY_MAYFAIL >> triggers an indefinite hang in the dma non-contig allocator. This has >> high-impact, as an occurrence will trigger a device reboot, resulting in >> loss of user state. >> >> Fix the non-iommu path by letting dma_alloc_noncontiguous() fail quickly >> so it does not get stuck looking for that elusive large contiguous chunk, >> in which case we will fall back to the alsa fallback allocator. >> >> Note that the iommu dma non-contiguous allocator is not affected. While >> assembling an array of pages, it tries consecutively smaller contiguous >> allocations, and lets higher-order chunk allocations fail quickly. >> >> Suggested-by: Sven van Ashbrook >> Suggested-by: Brian Geffon >> Fixes: 9d8e536d36e7 ("ALSA: memalloc: Try dma_alloc_noncontiguous() at first") >> Cc: stable@vger.kernel.org >> Cc: Sven van Ashbrook >> Cc: Brian Geffon >> Cc: Curtis Malainey >> Signed-off-by: Karthikeyan Ramasubramanian > > After chatting with Vlastimil, he recommended to get linux-mm people > involved, as basically such a problem shouldn't happen in the page > allocator side. So let's widen the audience. > > To recap the thread: the problem is that dma_alloc_contiguous() call > with high order pages and __GFP_FAIL_MAYRETRY leads to indefinite > stall. (It was __GFP_NORETRY beforehand.) This looks like the code > path with the direct page allocation where no IOMMU is involved. > > Karthikeyan, Sven, and co: could you guys show the stack trace at the > stall? This may give us more clear light. Yeah, if the inifinite loop with __GFP_RETRY_MAYFAIL happens in a call to __alloc_pages and not in some retry loop around it in an upper layer (I tried to check the dma functions but got lost quickly so the exact call stack would be useful), we definitely want to know the details. It shouldn't happen for costly orders (>3) because the retries are hard limited for those despite apparent progress or reclaim or compaction. > Also, Vlastimil suggested that tracepoints would be helpful if that's > really in the page allocator, too. > > > Thanks! > > Takashi > >> --- >> >> sound/core/memalloc.c | 7 ++++++- >> 1 file changed, 6 insertions(+), 1 deletion(-) >> >> diff --git a/sound/core/memalloc.c b/sound/core/memalloc.c >> index f901504b5afc1..5f6526a0d731c 100644 >> --- a/sound/core/memalloc.c >> +++ b/sound/core/memalloc.c >> @@ -540,13 +540,18 @@ static void *snd_dma_noncontig_alloc(struct snd_dma_buffer *dmab, size_t size) >> { >> struct sg_table *sgt; >> void *p; >> + gfp_t gfp_flags = DEFAULT_GFP; >> >> #ifdef CONFIG_SND_DMA_SGBUF >> if (cpu_feature_enabled(X86_FEATURE_XENPV)) >> return snd_dma_sg_fallback_alloc(dmab, size); >> + >> + /* Non-IOMMU case: prevent allocator from searching forever */ >> + if (!get_dma_ops(dmab->dev.dev)) >> + gfp_flags |= __GFP_NORETRY; >> #endif >> sgt = dma_alloc_noncontiguous(dmab->dev.dev, size, dmab->dev.dir, >> - DEFAULT_GFP, 0); >> + gfp_flags, 0); >> #ifdef CONFIG_SND_DMA_SGBUF >> if (!sgt && !get_dma_ops(dmab->dev.dev)) >> return snd_dma_sg_fallback_alloc(dmab, size); >> -- >> 2.43.0.687.g38aa6559b0-goog >>