From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5339E77188 for ; Wed, 8 Jan 2025 05:56:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3DD956B0082; Wed, 8 Jan 2025 00:56:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 33F7D6B0083; Wed, 8 Jan 2025 00:56:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1929E6B0088; Wed, 8 Jan 2025 00:56:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EAB0A6B0082 for ; Wed, 8 Jan 2025 00:56:17 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 6F517A03FC for ; Wed, 8 Jan 2025 05:56:17 +0000 (UTC) X-FDA: 82983224394.08.03554B6 Received: from mail-qv1-f46.google.com (mail-qv1-f46.google.com [209.85.219.46]) by imf03.hostedemail.com (Postfix) with ESMTP id 93F9D20005 for ; Wed, 8 Jan 2025 05:56:15 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=SMDSWQ7W; spf=pass (imf03.hostedemail.com: domain of yosryahmed@google.com designates 209.85.219.46 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736315775; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zX/cnpkXYzxhGFAOz7+6qXIi6/SLArdxuHweSSCM17s=; b=HS+yG4d2HqopiP3YI2Xe73qkTDFSH7vDWoPrQ9/7iQuTORlS4DVehgBWbcArjL0UNP0VAc 6L1yRHEj6Pd7VlhJC1XdX5sjj7hguuKoslp0IBdeQalsdFdMBY7hSMknbL3zMPJTwadLLu neVuzeirqnQ2a5s1BPmT5MBSgUlhVyU= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=SMDSWQ7W; spf=pass (imf03.hostedemail.com: domain of yosryahmed@google.com designates 209.85.219.46 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736315775; a=rsa-sha256; cv=none; b=6+4L+RHc8EL2S2g1EEEzf+Ekg2aXUQy+Y520Mly6T4hCwBbaWpQcFbsINDqXyv73gEDh1U CB6kAgwLHAl1IYPN+VjaoSuhKh7fjwwcP6/GjHULP68EQ2KSsXFUC3dTRIdWQDS+qF8DG1 wvs1cgLNofdBtGl4Wdb1JnOJRHEF4mQ= Received: by mail-qv1-f46.google.com with SMTP id 6a1803df08f44-6d8ece4937fso108224026d6.2 for ; Tue, 07 Jan 2025 21:56:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736315775; x=1736920575; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=zX/cnpkXYzxhGFAOz7+6qXIi6/SLArdxuHweSSCM17s=; b=SMDSWQ7WMrZsaYeYvLtrjZPfHrmkKm0dFaTDDva7HheKydZqXSKpIU86plb+SJE4ZE Tmwqyew1wdWhXtb1kOnvDhiYqwiwrwr7JHnO6LIveYLThoD48W61a2YhiJLBxaRvW0/9 eJ9aqFf1s7GfN0kCM/bGmxAQ6/kR3gth+OTU2xoqgtEvDxpU3nK/B0XQ16gXGNtvjdj2 fc87ViJyMhECTVxahecUhXmO9bM79DWy6nkj5WWD3u2CXecu2otahurwgAAVeSjWqmEL STIkerE3JnJYYi7TOUUPTHCLQVTGuoFgr7QIyfxY8wn473HbH3ZVVncL46uZqNhoeK+w B6/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736315775; x=1736920575; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zX/cnpkXYzxhGFAOz7+6qXIi6/SLArdxuHweSSCM17s=; b=EcYr0t/E9hqQg3uVwZGfN2L3qQoV4CsJ+PbkHxnpKFRQW9uBGeQLo9DkBBOjnMkpJB oj5joHzNBf6ONRcyeHXk+qkDIwezVhusP4vQSqJc2TsbHMxAPCf11BYGaXpgxy/dqUc8 HJJctovpsfagyopeg1rwzLjXhHBXTauIiRt0e49DOxIAvcq5G/zxownHNHsqaaHUIC6I nyu9yIlV6pzLmNgvEkdN5Yy5iOEnXjN4FKeDpFe7WGPHnK0yiWueoYcDllKGtC6RBgsQ prb9MDIc9LLTqgfw9axm7W7Z/rq9Q1qPWal5bAy0cMC72+fk3Enam7SQJU7AklqnM2EM 80Uw== X-Forwarded-Encrypted: i=1; AJvYcCXgdiFWKTCtGIt2KlDE9kKEATzOtAMUepxfHhJt5fENmlciE+QydGI6OKrqAhXNlS1PD7+gRagllg==@kvack.org X-Gm-Message-State: AOJu0YzCKl2ddOq6s11UfK5ZoJcP4bVQgOvxXvkmaf3suGw+BHXYdkIf R3Ehn7vpAvMc8tfPX6V7sOJbnddLk0UVXYnM8V3p9QykSsDnk5bYuDmMH4t6aNsb0CvAF7hZaAl yI8+TLDGRceFkcqq3hq0CogxiKyfuK29tdUZ+ X-Gm-Gg: ASbGncvdDIPfC5KQKvQCJE62m+c5fznp7+L01laO7UJU3CJX1sY95eM3Ce7MH0hMUN5 b5F4R8008UYSev83TcFF1EfSDPAFKTSvtzQc= X-Google-Smtp-Source: AGHT+IHOpR+lmzo2NW+3OlN9CS63bZTHb8o98pgQMbFH1CH9aK2p0ryiPkG+rmndPVeNjWJUUGSFeDoDt62tcK8AgvY= X-Received: by 2002:a05:6214:c25:b0:6df:99e9:39b3 with SMTP id 6a1803df08f44-6df9b1eebccmr29452826d6.15.1736315774580; Tue, 07 Jan 2025 21:56:14 -0800 (PST) MIME-Version: 1.0 References: <20250107222236.2715883-1-yosryahmed@google.com> <20250107222236.2715883-2-yosryahmed@google.com> <857acdc4-c4b7-44ea-a67d-2df83ca245ed@linux.dev> In-Reply-To: From: Yosry Ahmed Date: Tue, 7 Jan 2025 21:55:38 -0800 X-Gm-Features: AbW1kvZ1fMp-hR4rSSt6sFl8GDUeXhqcsXaUXMlCrVfqamXbhTq-mrU7nYcGw7M Message-ID: Subject: Re: [PATCH v2 2/2] mm: zswap: disable migration while using per-CPU acomp_ctx To: Chengming Zhou , Nhat Pham Cc: Johannes Weiner , Andrew Morton , Vitaly Wool , Barry Song , Sam Sun , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , "stable@vger.kernel.org" , "Sridhar, Kanchana P" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 93F9D20005 X-Rspam-User: X-Stat-Signature: s4sdj57kagoqrg1wo6ks7y9b63hk9pwt X-HE-Tag: 1736315775-481004 X-HE-Meta: U2FsdGVkX180vveUqlc0U8m1LDsdG/wXTmVNaru0s+SQG3PILmw6TdIT7jO5bclOOKB+yFvB5bpcLfOtfAxlRybhuFH+1SfMtQIGVU+Q2sFfvbW9Nlku3zAKTqNUx3hkJY/qT3DFghg0ILeTDx5RzroqexSTdgvM1HXSE+LLEBZZ8A0j9a5ZEXczJ3eqSCXs2p/MZYiRgbalUwXJMbgFmL2gggpejd3TTVF1NPGl687O89gmRothSWdvNGEGD3pTYvHy5WegJQs0L31TVTCiX6HKAckcRuW/FjDj6oTyf6yCJtM2wt7pR+xNaGe4Ua7TK/0TOQixl/IdC1/hWUvaNvSkSiULYodA+6i6z7A9CghwXGhOb4s1nJ8Bj0RcOJVHjUF1BfyNL0GySBWfPRY2Q+uQFAJ6ULApds/9tbei5vgo/mWIZ1Lm1CBuldDgr/Jkqkm0TM5OFvaJ3BYHGVpqOQ/Rr2f9/OULZ8+HkbwX4NmzX9KMr4n0QA8wLzFBS3HdTMqPSoe1I3nlrxQDLmX9xuwQ3M3ub79JTpIf2fi0v4Hj/G9O8TU3XagH7BwlN0pKTqbHakBvW1rG1XOz+Co2A715P3QlGGJA07ki/+PVWfIEWscYMvvdlj9QMksQTN4tplCpZlXmyJY/OoFn5M2hreSkI4AYWsOr2p8vNS+bZQPwnLx6tyYz7wEQa8p9x0lF4kTPFn81LueFr97nJ9YuNS4O4WWKrqZgIetBwrlQe0LeVw+mvQVKlMP5MPkpheS7PV98ZzTxFL1uHxcSt7ihU7SAGhmLE+vcZQaYXgU6Ed10WkoctSIxGitzEpwa++cWK/DSQyDrevH16Y7I1eWPNruLjyyZ0AJ/gYgSOJ9HT/U7L6+AaIyKqMZU9IeJtnKP9v5Jo+FxCxzmu5HFu/7YK2lyGtBYdiwnCB/8g0GecWjnqF2BKkCtrM2fxdr9fYdNbjKniHPNL+M7i6uAq5t 3RnXFxz6 v9H52u25U9sH5TXpV7uJf0OTZ4HNxXWCBqdFT495JQAnQwiKM7Y9hl/xezu0GlCmiOJu/R+STFOjluxjCdSRt+So63qYif47qk7LKBGWrTJSXUY7WpG63tClJXr+4DxhGaUEewF2CxVWozRKDH78szGu+JlRVn/B6nSqPIfO9kp/1PBsortvB8TCN1dY6iRGP038PVcr32e3TdFafspx8jQLrcTN0kppC+2JtYezP6OoGCeGltl9pxkOZkpt83lxaSEjIPrLlHlIioLsMYpixkvSvF+iGnsVGIZtjUO6APAaQxX75fjkuvrPTlg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jan 7, 2025 at 9:34=E2=80=AFPM Yosry Ahmed = wrote: > > On Tue, Jan 7, 2025 at 9:00=E2=80=AFPM Chengming Zhou wrote: > > > > On 2025/1/8 12:46, Nhat Pham wrote: > > > On Wed, Jan 8, 2025 at 9:34=E2=80=AFAM Yosry Ahmed wrote: > > >> > > >> > > >> Actually, using the mutex to protect against CPU hotunplug is not to= o > > >> complicated. The following diff is one way to do it (lightly tested)= . > > >> Johannes, Nhat, any preferences between this patch (disabling > > >> migration) and the following diff? > > > > > > I mean if this works, this over migration diasbling any day? :) > > > > > >> > > >> diff --git a/mm/zswap.c b/mm/zswap.c > > >> index f6316b66fb236..4d6817c679a54 100644 > > >> --- a/mm/zswap.c > > >> +++ b/mm/zswap.c > > >> @@ -869,17 +869,40 @@ static int zswap_cpu_comp_dead(unsigned int cp= u, > > >> struct hlist_node *node) > > >> struct zswap_pool *pool =3D hlist_entry(node, struct zswap_= pool, node); > > >> struct crypto_acomp_ctx *acomp_ctx =3D per_cpu_ptr(pool->ac= omp_ctx, cpu); > > >> > > >> + mutex_lock(&acomp_ctx->mutex); > > >> if (!IS_ERR_OR_NULL(acomp_ctx)) { > > >> if (!IS_ERR_OR_NULL(acomp_ctx->req)) > > >> acomp_request_free(acomp_ctx->req); > > >> + acomp_ctx->req =3D NULL; > > >> if (!IS_ERR_OR_NULL(acomp_ctx->acomp)) > > >> crypto_free_acomp(acomp_ctx->acomp); > > >> kfree(acomp_ctx->buffer); > > >> } > > >> + mutex_unlock(&acomp_ctx->mutex); > > >> > > >> return 0; > > >> } > > >> > > >> +static struct crypto_acomp_ctx *acomp_ctx_get_cpu_locked( > > >> + struct crypto_acomp_ctx __percpu *acomp_ctx) > > >> +{ > > >> + struct crypto_acomp_ctx *ctx; > > >> + > > >> + for (;;) { > > >> + ctx =3D raw_cpu_ptr(acomp_ctx); > > >> + mutex_lock(&ctx->mutex); > > > > > > I'm a bit confused. IIUC, ctx is per-cpu right? What's protecting thi= s > > > cpu-local data (including the mutex) from being invalidated under us > > > while we're sleeping and waiting for the mutex? > > Please correct me if I am wrong, but my understanding is that memory > allocated with alloc_percpu() is allocated for each *possible* CPU, > and does not go away when CPUs are offlined. We allocate the per-CPU > crypto_acomp_ctx structs with alloc_percpu() (including the mutex), so > they should not go away with CPU offlining. > > OTOH, we allocate the crypto_acomp_ctx.acompx, crypto_acomp_ctx.req, > and crypto_acomp_ctx.buffer only for online CPUs through the CPU > hotplug notifiers (i.e. zswap_cpu_comp_prepare() and > zswap_cpu_comp_dead()). These are the resources that can go away with > CPU offlining, and what we need to protect about. ..and now that I explain all of this I am wondering if the complexity is warranted in the first place. It goes back all the way to the first zswap commit, so I can't tell the justification for it. I am not sure if they are setups that have significantly different numbers of online and possible CPUs. Maybe we should just bite the bullet and just allocate everything with alloc_percpu() and rip out the hotplug callbacks completely.