From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5C24E77199 for ; Wed, 8 Jan 2025 05:00:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 246C06B0093; Wed, 8 Jan 2025 00:00:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1F7506B0095; Wed, 8 Jan 2025 00:00:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E5E06B0096; Wed, 8 Jan 2025 00:00:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DECD26B0093 for ; Wed, 8 Jan 2025 00:00:52 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5A87D1A0511 for ; Wed, 8 Jan 2025 05:00:52 +0000 (UTC) X-FDA: 82983084744.04.2448A9B Received: from out-177.mta0.migadu.com (out-177.mta0.migadu.com [91.218.175.177]) by imf08.hostedemail.com (Postfix) with ESMTP id 676D616001E for ; Wed, 8 Jan 2025 05:00:50 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=hVqksTQT; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf08.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.177 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736312450; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yFj1VXynprGXG6/ML3EG7dB5MksGwHIqxnVihq/iLvo=; b=jdgW0M2NgDv71DN4N3miWmPLzkAH9Pjdr3Ayx4BAJj5p7j5PAUTu46tRNeyAVbVg/K5RvX RkCbfam3cLGjlNAIW64Qw+roOVjUmv5ynaFPo60JtV4x+NPT+wg1TGwHlCCU78gPono3Al RNoMIyKX0KNF4+ARLlBQxMCxNMdcsxg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736312450; a=rsa-sha256; cv=none; b=hoXB3IFkxdTS8cb2rbIBAfuIIy6aKVJLNLpiDN+KWPPgY5joLcpEgfdY32ocMizwQkZh3S TptTm0F4Exbl9WbHm10pTH/A+Kk6srCptbARZOjyxQpuQMOk478/0KyvFQeIPHuhfmMFJx dckZJzxkJFTkoaujIdcsamRpov4YNJQ= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=hVqksTQT; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf08.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.177 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev Message-ID: <857acdc4-c4b7-44ea-a67d-2df83ca245ed@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1736312443; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yFj1VXynprGXG6/ML3EG7dB5MksGwHIqxnVihq/iLvo=; b=hVqksTQT3AsE39BS5OnW9/CYbNPt6qWDh5JgS9OPpmtlLBkG0FNnSkssMNsCGWeDLz+JIF QczpuhzKlEivMit7xj4lDVwGxwsaL+qIRJ9W/VBOo6RPYuq/2en6RYRb28yqXKICkTZU4t /EExKhkWqwb1jbxv3Aphv6kPv6NvpPw= Date: Wed, 8 Jan 2025 13:00:24 +0800 MIME-Version: 1.0 Subject: Re: [PATCH v2 2/2] mm: zswap: disable migration while using per-CPU acomp_ctx To: Nhat Pham , Yosry Ahmed Cc: Johannes Weiner , Andrew Morton , Vitaly Wool , Barry Song , Sam Sun , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , "stable@vger.kernel.org" , "Sridhar, Kanchana P" References: <20250107222236.2715883-1-yosryahmed@google.com> <20250107222236.2715883-2-yosryahmed@google.com> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Stat-Signature: q38jj5y4jrid7m6o5fyy1ghxq1p9hn1w X-Rspamd-Queue-Id: 676D616001E X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1736312450-397069 X-HE-Meta: U2FsdGVkX18/4D0wHHM6MFEAXqmeuD1CrkuLZCNnh/tRcciiC2Uf1oeHXK+tq3l3LmqTXCkTvF3Kip1mvqsqrtAMTR5SdkplO1AWALKBhDgnzQ4zpLtp9hxiWfB/IIUOmHdbfJyAt1HMP45cO5dO5OUJLbyGRpdQKmH9MroRLFURGQ9p+2Gs9bKFGvojT8/gTrwCuVXpHi5V8WRMov9loYfsXBJfarnWwfhGnFkcTh5mG3KvuKjv9zY1YiBMqB4bBkaTsESe/KiXVJvJzKjIZZF1ASH6POEpnbcjEopXDARZO1ViFRmgfSqdqkzyNP8HnguxdaOBcugYEjZvEQC0OPRWI9VsFTCCAGS7m/gOUPDjhlt8kCt49jX/J/Mm3zNuVz1mDaMMBSqnfjO1C1NKTdXakivY32FEkKdxzmz/qF8BU7qJQW2w6bbR4fOpfvT2CFKTcSjT7cM2dcS/S7hwSUtusmWi9kHSTMIaxt2jMCrvXX7GKmQ2i1bvJ5sxleNvQe1H/P1s9dJuQyOyPfsQ0J4JBcpY/5lfYbv9O9ujtOzTbMxA85lXaTyzNtm13cQ59B9cj43oCXI8we4tLCb1qEP8p6tn77/EH7gF8rpB+wQPEg0F35iBtU+d1ajNUQCWDTx4ImJBQ1W75mAeAF90xbv1cBukFEdzH0hy194RKNfqKAIwz+bvFQ725tNpl5yU2glK56W+AmWWNTgPwZR9nIk5QtgmEjXTPyL9MIYkW2UHgCZk3Lc0M7RAnEhs6FwX//ImUvk32LewRfpvSrdgSQeUuIc9r/4Ax+UJV5gK3QHf219FD9GP4ULyYMXfZQjsgKJERCTi43eOYQqJVsD++S5a28EJ2IPSAUgduzuQ9zktashwpyUr7sliwCVZBGehJiQEz7BHM82nUxy2CWOdFFWE8YxzIv8h7+Z2NG5a5Rks7XJY4RBJrMpf1gv/imLk0Pz0zNdCD9jgCnt14jd HjWDQ9lN 0aR43UQMgtkDgqlRvOkUjX8dopHc5eJbuNzMsZL6lB7cHUjmgPtYIqO2lX919SF/v3YLVkHohEekrUSaXjt/Sbp6NXbbxwyM6FZBAmTHdldUWIHDR79CfUSD5Gws7eAhILWx0HgA3fYIbriy1x+Wp14H6S/jD5W1GYcfJY4gz1R75Xe3/IhR0RbcaEu/bUcRAugUVJq7djze0XiQV2CHGWOuRPoEOIUnjwI55qDxTzZOPQr95PsxAkZ7p+INMZ9BygGo/lQIwwOeOVihE5RMGBMldCeviWDHpWvOo X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2025/1/8 12:46, Nhat Pham wrote: > On Wed, Jan 8, 2025 at 9:34 AM Yosry Ahmed wrote: >> >> >> Actually, using the mutex to protect against CPU hotunplug is not too >> complicated. The following diff is one way to do it (lightly tested). >> Johannes, Nhat, any preferences between this patch (disabling >> migration) and the following diff? > > I mean if this works, this over migration diasbling any day? :) > >> >> diff --git a/mm/zswap.c b/mm/zswap.c >> index f6316b66fb236..4d6817c679a54 100644 >> --- a/mm/zswap.c >> +++ b/mm/zswap.c >> @@ -869,17 +869,40 @@ static int zswap_cpu_comp_dead(unsigned int cpu, >> struct hlist_node *node) >> struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); >> struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); >> >> + mutex_lock(&acomp_ctx->mutex); >> if (!IS_ERR_OR_NULL(acomp_ctx)) { >> if (!IS_ERR_OR_NULL(acomp_ctx->req)) >> acomp_request_free(acomp_ctx->req); >> + acomp_ctx->req = NULL; >> if (!IS_ERR_OR_NULL(acomp_ctx->acomp)) >> crypto_free_acomp(acomp_ctx->acomp); >> kfree(acomp_ctx->buffer); >> } >> + mutex_unlock(&acomp_ctx->mutex); >> >> return 0; >> } >> >> +static struct crypto_acomp_ctx *acomp_ctx_get_cpu_locked( >> + struct crypto_acomp_ctx __percpu *acomp_ctx) >> +{ >> + struct crypto_acomp_ctx *ctx; >> + >> + for (;;) { >> + ctx = raw_cpu_ptr(acomp_ctx); >> + mutex_lock(&ctx->mutex); > > I'm a bit confused. IIUC, ctx is per-cpu right? What's protecting this > cpu-local data (including the mutex) from being invalidated under us > while we're sleeping and waiting for the mutex? Yeah, it's not safe, we can only use this_cpu_ptr(), which will disable preempt (so cpu offline can't kick in), and get refcount of ctx. Since we can't mutex_lock in the preempt disabled section. Thanks. > > If it is somehow protected, then yeah this seems quite elegant :) > >> + if (likely(ctx->req)) >> + return ctx; >> + /* Raced with zswap_cpu_comp_dead() on CPU hotunplug */ >> + mutex_unlock(&ctx->mutex); >> + } >> +} >> + >> +static void acomp_ctx_put_unlock(struct crypto_acomp_ctx *ctx) >> +{ >> + mutex_unlock(&ctx->mutex); >> +} >> + >> static bool zswap_compress(struct page *page, struct zswap_entry *entry, >> struct zswap_pool *pool) >> { >> @@ -893,10 +916,7 @@ static bool zswap_compress(struct page *page, >> struct zswap_entry *entry, >> gfp_t gfp; >> u8 *dst; >> >> - acomp_ctx = raw_cpu_ptr(pool->acomp_ctx); >> - >> - mutex_lock(&acomp_ctx->mutex); >> - >> + acomp_ctx = acomp_ctx_get_cpu_locked(pool->acomp_ctx); >> dst = acomp_ctx->buffer; >> sg_init_table(&input, 1); >> sg_set_page(&input, page, PAGE_SIZE, 0); >> @@ -949,7 +969,7 @@ static bool zswap_compress(struct page *page, >> struct zswap_entry *entry, >> else if (alloc_ret) >> zswap_reject_alloc_fail++; >> >> - mutex_unlock(&acomp_ctx->mutex); >> + acomp_ctx_put_unlock(acomp_ctx); >> return comp_ret == 0 && alloc_ret == 0; >> } >> >> @@ -960,9 +980,7 @@ static void zswap_decompress(struct zswap_entry >> *entry, struct folio *folio) >> struct crypto_acomp_ctx *acomp_ctx; >> u8 *src; >> >> - acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); >> - mutex_lock(&acomp_ctx->mutex); >> - >> + acomp_ctx = acomp_ctx_get_cpu_locked(entry->pool->acomp_ctx); >> src = zpool_map_handle(zpool, entry->handle, ZPOOL_MM_RO); >> /*