From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 581B5C83F27 for ; Sat, 19 Jul 2025 10:34:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 942F56B0089; Sat, 19 Jul 2025 06:34:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F3D36B008C; Sat, 19 Jul 2025 06:34:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 809636B0092; Sat, 19 Jul 2025 06:34:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 70CC56B0089 for ; Sat, 19 Jul 2025 06:34:45 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 088891A02D2 for ; Sat, 19 Jul 2025 10:34:45 +0000 (UTC) X-FDA: 83680655730.06.7FDDAC4 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by imf23.hostedemail.com (Postfix) with ESMTP id 16006140006 for ; Sat, 19 Jul 2025 10:34:42 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=NiFWjnGK; spf=pass (imf23.hostedemail.com: domain of ydfan@suse.com designates 209.85.128.45 as permitted sender) smtp.mailfrom=ydfan@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752921283; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=YLPSgROEAhCnKRwI31RE1TO5M8HQINGoz4GugGu3XKg=; b=jf/R6OIQznH1E7by0DVMZWVezmFOwR94ZRz3wEs8IoHUbzp9VjHcN7NE/D12Vjg0VGsVaJ 35X7esNuRvf86qXIktcTetNXECRhvUd7XbTv+hJ3TkSwflt9wLSSf01Py74xlrTDN85WnT rz8cqEnxdL/RQe2RcQhpCkWgwjSh1pQ= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=NiFWjnGK; spf=pass (imf23.hostedemail.com: domain of ydfan@suse.com designates 209.85.128.45 as permitted sender) smtp.mailfrom=ydfan@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752921283; a=rsa-sha256; cv=none; b=r/bQlsl05SFM/qNU3eB7pi2+adELdWb+vpasuZE3Uzb8i3qVVzcRTpZ+e2jYfmvJ3zqmUo AvWPFGOkK7ekrLGByMOhNlnJ792HnOcaKq0FSH2Y86shtHXHYutWdhBNthfNqqweMWBhwd aoVEPrdMLdyOH0qJ4Ga+Ru4+32o9axQ= Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-4561ed868b5so20628805e9.0 for ; Sat, 19 Jul 2025 03:34:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1752921281; x=1753526081; darn=kvack.org; h=content-transfer-encoding:subject:from:cc:to:content-language :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=YLPSgROEAhCnKRwI31RE1TO5M8HQINGoz4GugGu3XKg=; b=NiFWjnGKSfLp/y53wDp0U9OQCuYyepdE9oMlpeEyVyB9K/s0Crun2IKiC46fLw/IMo grWN+lzLLE+gwmVWSYM5enRIP35+jJ8kBjnY0dd/ztnC93Xm+EDqqOOIRtH4MnVMsTjM FSEktQ1/Pjg4jt4X/3JsnICn2RnpwfG5xO3P89knMrTB6hFaKiDcr44gPdbrCMyCcK+t +k73z09At8+jymsuc6o/gm+pggAxKdqjupbo5RLNkRfJQlmOeLjIjoeQDiR+XMSCmADU jD7TnTFMSqk9eFW4G5uQ5Z7302njkscdhnDAKUw9e/lwPcEHmC2Uuz4Gkhc9PfwK9Kmr Vj6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752921281; x=1753526081; h=content-transfer-encoding:subject:from:cc:to:content-language :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=YLPSgROEAhCnKRwI31RE1TO5M8HQINGoz4GugGu3XKg=; b=Dc5m1+l/wbbnqxnTBJUgfFG3x/Moimnk8l2Pb94bhuvY0wASHf9Lbb877oBZB+WC1v tWnJ39Ifp0bO9b30QCuA8y2hzm5Igub26ik+itYcxOQKt1AwuGtiO4+ME7UuOJsWXD6s ebEdPDBWnn01wrM6NtAi0kakkPpkMi8uY7Qme59bFxjsp9oJlYdW/l6VPeDqW00l2HYV 098EkkFzHfnYHL0cC72+bEtu7SdNYyW1c3a4NUYXRVcyXFYsJFjKby/QdCc/raHBBYRD 7Ilixnrm491nEOxkt20PAdEaVBjB90RybTVtaQGt8LCIHqKEzoWVGFCj3sENXW5wI43b 8ZtA== X-Gm-Message-State: AOJu0YxH4P89u+ws9ltH69VScSQK1WzPSWW/xp+Cs0LxNHUYF5vx4iiI 48ki5H7VVPjLstmuPxLirs+gGHNpqgRQclnluuB/dwW36h6LrUhfw70L2SIFLcIrhik= X-Gm-Gg: ASbGncuvDX/yhNy9yRDwK3oH8sKS8vJCQb++2AzP2FaHU73t5Uhx1BXAwUa7Hesa9a0 ndiyPLDw5UTC9qMjq6xwIA6vW+6s36UHsPHjxTXKC7qITvVUCn6sYwy23MNUhucen3p29RNZdef qNhEiHPuzzOGVgIA4uHQtedtgQViFc4DfLn0ecd1plM4j2gFxpdUq90GMxabefsmmuOFLZIAlsl 4cYEnXsefx3l9sscb2zrDegy2JlMhiADriRx1S2WGWGNw9OlfTo3LaFGqZN7S7tXnwjPyAxX+K/ 1iS3AzgjgaC7qHKppVVAdPEFp3Zaq4+Yo3wOSY8LPo2GWvWHPPhfPTjOzh1/HuroSMZkvv7pO5K oyJtsKjRl+JcDjrd3ipE= X-Google-Smtp-Source: AGHT+IGjz3rNU3dd+9li05K6Tq18JANydrGITOOvXwZniSAWqrMVuX0mS/Ho0NHEf170VZRNEMycHA== X-Received: by 2002:a05:6000:26c7:b0:3b5:e29c:190 with SMTP id ffacd0b85a97d-3b60e5134e6mr10245430f8f.29.1752921281247; Sat, 19 Jul 2025 03:34:41 -0700 (PDT) Received: from [10.211.55.30] ([103.172.41.197]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-23e3b5e9efbsm27098825ad.5.2025.07.19.03.34.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 19 Jul 2025 03:34:40 -0700 (PDT) Message-ID: Date: Sat, 19 Jul 2025 06:06:51 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org From: Yadan Fan Subject: [PATCH v2] mm: mempool: fix wake-up edge case bug for zero-minimum pools Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Stat-Signature: zeuyzrirt4m47q4ajqcnuhbh9rs7i6m9 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 16006140006 X-Rspam-User: X-HE-Tag: 1752921282-697008 X-HE-Meta: U2FsdGVkX18nydc3zWijcI5jrEXJnolFyIuiNC4/JQObr4yvku0TwiMisu0YNx6pdba9WZeDuuQLnM4RDdpzqy+oyIwpB58tKnyye9KSx7QiZv4wW3MT3kzVlF1CN74sA9XOlO9UVhL0NDwJXA41M7miYHyP4T7sywgqzd133qcgree3eHtfhwwnCl/CigWHlfAIZgOjE6MdMtJ2hXGY8WshH2yU4qyFWT73cTPYflTVbSaLQHGRuBDbiZAJfC6HCt7yIt3Cwvl4DrahZzJse7hLf/O2ycGyCQTVGt+eBWNcx3W1iHSbZfWDTQ7JD61kpltazFfJHKDdAFPN2flhmcVlf6m+Fi/alc0r1iV18wtvj80gEog9ofM/lWFUgkOy4f4HyRW7H0A/0LeAn3y8X29iOhAEcqgEVBM4ubrEk5qcav+VdDyetK959+OScjHZT+vsdCA7IDoGEEwNrtgqd0/EsXpT3PBA9gDGVvAQ6BaVRCLY2OB/+pUQQWUXfXistXm7l5XSZXFzIFL9RXbTrS+QZYs88t+Ci6lyc8i6mV+3wgxDLlDy5PWwOW6mvETd8d6RsL1J8ZWdlFksdidT5xYF1xmY1QA3W60+FtGY4i9xdvo3oRd2t8vaVkwcPxMd42T8Rr0Yk4wQi1DjqizvaicFrz9m/u1VELEYPsPDuSFJUp77jgovcGGZ6/0T7+XGGa8j4FnTQyxsnCYbarBPHfsWhy2eFSS1SOo7zjGTA496UDEfqtGb1TiNUNUskdYaq8sHIuL4R1wlhRbWEZLjPUdop8lF3G6RK5qKu9rXH0t/fDg0qAXP6uFzW4vMKSVlV+VIAEa0TDth6OaHEk53pf+VWTL7cu+62qaxwsg5vSO7tJ+1+YChYUctPxZiuL9XrkAa9Iwm8Y0f0WceS7KJaUno6O58daUrMRW9vRfuG5AmOBso0C+V5ckBb8N4TXPbF1niiCrQBxP2HY6yK3m fJQwml8D 2QAipLSuzMwyfSsJhc4ldcEL0Qte4/C2dHUnOVpFzJXCQdAxW66LRJ1K1OEa4c2o02AuHMIyLNcUyVBiRHsZPPVNo8/vsCqmgQJizFUB82WFz9s2Zu1CAhauBl+ob+A5xeGL+JRkx8JDUoDf2osgqLMbbgJL1VytlUHsxOW7FGpLv68JWUdVVMIl9QX00K5cU/WExTIZPmNz55H5ZRpKrF7e4QVsaw9C+43x/ZRZS6qX5q4lchQxJPvtkRH9XFouUT/TiM3pt8qa7r4jSDGAWy7rTSfH9nUf7PBM25tHwLN5Tv58N95FmB+eB3Sg/YOS0T5hg+1HQXzrAA74sIBP4bKOvMs0VmszQJePHr0aPkZD8Jj5jYudN8jYT3g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The mempool wake-up path has a edge case bug that affects pools created with min_nr=0. When a thread blocks waiting for memory from an empty pool (curr_nr == 0), subsequent mempool_free() calls fail to wake the waiting thread because the condition "curr_nr < min_nr" evaluates to "0 < 0" which is false, this can cause threads to sleep indefinitely according to the code logic. There is at least 2 places where the mempool created with min_nr=0: 1. lib/btree.c:191: mempool_create(0, btree_alloc, btree_free, NULL) 2. drivers/md/dm-verity-fec.c:791: mempool_init_slab_pool(&f->extra_pool, 0, f->cache) Add an explicit check in mempool_free() to handle the min_nr=0 case: when the pool has zero minimum reserves, is currently empty, and has active waiters, allocate the element then wake up the sleeper. Changes in v2: - Inline the same logic from previous test for zero min_nr pool to return an element to the pool - Inline the necessary parts of add_element() to avoid the BUG_ON which refuses the scenario of curr_nr == min_nr - Use wq_has_sleeper() instead of unconditional wake_up() calls for better performance when no waiters are present Signed-off-by: Yadan Fan --- mm/mempool.c | 34 +++++++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) diff --git a/mm/mempool.c b/mm/mempool.c index 3223337135d0..204a216b6418 100644 --- a/mm/mempool.c +++ b/mm/mempool.c @@ -540,11 +540,43 @@ void mempool_free(void *element, mempool_t *pool) if (likely(pool->curr_nr < pool->min_nr)) { add_element(pool, element); spin_unlock_irqrestore(&pool->lock, flags); - wake_up(&pool->wait); + if (wq_has_sleeper(&pool->wait)) + wake_up(&pool->wait); return; } spin_unlock_irqrestore(&pool->lock, flags); } + + /* + * Handle the min_nr = 0 edge case: + * + * For zero-minimum pools, curr_nr < min_nr (0 < 0) never succeeds, + * so waiters sleeping on pool->wait would never be woken by the + * wake-up path of previous test. This explicit check ensures the + * allocation of element when both min_nr and curr_nr are 0, and + * any active waiters are properly awakened. + * + * Inline the same logic as previous test, add_element() cannot be + * directly used here since it has BUG_ON to deny if min_nr equals + * curr_nr, so here picked rest of add_element() to use without + * BUG_ON check. + */ + if (unlikely(pool->min_nr == 0 && + READ_ONCE(pool->curr_nr) == 0)) { + spin_lock_irqsave(&pool->lock, flags); + if (likely(pool->curr_nr == 0)) { + /* Inline the logic of add_element() */ + poison_element(pool, element); + if (kasan_poison_element(pool, element)) + pool->elements[pool->curr_nr++] = element; + spin_unlock_irqrestore(&pool->lock, flags); + if (wq_has_sleeper(&pool->wait)) + wake_up(&pool->wait); + return; + } + spin_unlock_irqrestore(&pool->lock, flags); + } + pool->free(element, pool->pool_data); } EXPORT_SYMBOL(mempool_free); -- 2.50.1