From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F1F4C0218A for ; Sat, 1 Feb 2025 09:21:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 11F536B007B; Sat, 1 Feb 2025 04:21:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A93F6B0082; Sat, 1 Feb 2025 04:21:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E8A7C6B0083; Sat, 1 Feb 2025 04:21:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id C1EE96B007B for ; Sat, 1 Feb 2025 04:21:56 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2AB1B121627 for ; Sat, 1 Feb 2025 09:21:56 +0000 (UTC) X-FDA: 83070833832.09.6AFE45D Received: from mail-lj1-f178.google.com (mail-lj1-f178.google.com [209.85.208.178]) by imf26.hostedemail.com (Postfix) with ESMTP id 13A01140007 for ; Sat, 1 Feb 2025 09:21:53 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=U0YkDCVM; spf=pass (imf26.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.208.178 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738401714; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Zfx3S2dBIyfRumerX8ffl1DnCCuzv9ZN168RxmRN/LY=; b=y0nwAwBFiDuy9c9kJ1AB4XHkiHY1bqjqm2DQuOTEmyYnKFng9fRxRd4ingra+b/Y0Y543u dJ4eH5Wwk0d+9fH4aPxULhbKZBi+FhfdfYgXax2wUp6CIqnRC8bLiGohrhYd6OPdnj7F5+ S8ynJoPH8MM/zBwZJqzw/TzTGGeghCQ= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=U0YkDCVM; spf=pass (imf26.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.208.178 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738401714; a=rsa-sha256; cv=none; b=cjaMYYobU5Bk4jd8lQLaXXYNVVYf3vgFWoFqf6o4goOmjFUaDx+IwCoHR31dMv+E8zUW6N i3X9kFQ5LszsiYJCSVsTuOCdtBrwiMY7OdJ6DAVulqx1aHB4zX3Q7S5iN4d4eEV47RqzYO mj/1z6gvBvq/fIZHRqDudpkO6SD3uPw= Received: by mail-lj1-f178.google.com with SMTP id 38308e7fff4ca-306007227d3so27646271fa.0 for ; Sat, 01 Feb 2025 01:21:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738401712; x=1739006512; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Zfx3S2dBIyfRumerX8ffl1DnCCuzv9ZN168RxmRN/LY=; b=U0YkDCVMRassDdfbcjnu+7EQCn0StsA1z/fubOUgYOjX6gu5IBJ4IMH5RlHEZmrqgT 2cP4BPhFYWeSxID1qYfc3PEnDGpsAiffD/iW8Qs4k+autvGhrOXii4uY2vM7B5ZR4e0B hIHD6RzR5P9p7DqiA3xSzcmgjb5hhFkYw5ZDnetuksGIy1Ew9jPUNfczaytVs+8cLaSH z3+BJUakt7Rr5ORe2iSFGjda3Djq4L6Yl+/kEYF4APtJRZsUD/nWE53puBzpmuHBzuSc bX5oJjJDkMDLs5T5ggPP5zvSOJ3blSVaqXpsoBTzvFV3RCQiuN1VLQ7jLbitlD91cs1U i4wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738401712; x=1739006512; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Zfx3S2dBIyfRumerX8ffl1DnCCuzv9ZN168RxmRN/LY=; b=RaNEs24H77SLqVgCu6Ttn7ycCiBFqO35RfwMb8VoBpwLdtQPu64+Zbwc51RAdMnCWA OuK6Ryvv2SEJh7d+sMtZGz7+eJD+RpSwglaJP7TeizNM1D/bKHtngf4YGvX90RQLXgQG kydBD5v0yZS6EZKaT9rDSqXbsKRsayPm7Qdsnq9zhXs5aCWlvKW6bmNyPuGv6aMdL/s1 EDKSIUFOMzPVDOq8A7A6pygics97XFeDNHGClQNMJim62E0x54Iis1lgVWZ0vvip5KpK Gv0FloaFQfLpLuAlku6G9yOOOshxhmbsIOcC2oHbQaHcchLPio8hUjlwmrfVaj16RKmb 1r0g== X-Forwarded-Encrypted: i=1; AJvYcCX/bD51cUiovpApdjhZJdzuno39zpHeqemRISn8+qyiMwT9B4sXHzM8mcszyyFueOWDOIUDVRwdnA==@kvack.org X-Gm-Message-State: AOJu0YykpMbeet6IznAlO2ETu4bfQeqe7xiBUkLsNswYmMcvLK+UCgQz 4ZrxdfMGdnWceUwGoG5MfT/GkwquRltdT9yVm2mYaIbhh0b22OO5dB4s4Y1LhrZd5u9WPt3VhJb 35jaVCOk4tYKsjF9Re+8/LRSdhIUDjlG5yWLuvSpU X-Gm-Gg: ASbGncuY0D4TEFlJGlWqa7Z/9jb/AoGXlEI+NDxkoe7TuQHSTPsdJJImVXylc9dPsCn 0za+bkFftQaG/Zr1HAmHuN+Bb3WuBXAWkerfd6Lnhpve9A3/aXrms1vSpZ2HI5/lZrNQA+q3xcg == X-Google-Smtp-Source: AGHT+IHhTHe02EWhJXsULCEd+AhMIELapgSMV13y4pUoGEJ0c4m6oV2uwfl6MNORV5arGrEf6FQ3xPEYhjkueljJdwI= X-Received: by 2002:a2e:b8cb:0:b0:300:3de4:ff72 with SMTP id 38308e7fff4ca-307968ec551mr61124941fa.27.1738401711787; Sat, 01 Feb 2025 01:21:51 -0800 (PST) MIME-Version: 1.0 References: <20250131090658.3386285-1-senozhatsky@chromium.org> <20250131090658.3386285-3-senozhatsky@chromium.org> In-Reply-To: <20250131090658.3386285-3-senozhatsky@chromium.org> From: Kairui Song Date: Sat, 1 Feb 2025 17:21:35 +0800 X-Gm-Features: AWEUYZllhQpdsad0F-KjtRxX3og_LaVLQGsbxr6aKjR7KceagBBKBSsX-k-wnEk Message-ID: Subject: Re: [PATCHv4 02/17] zram: do not use per-CPU compression streams To: Sergey Senozhatsky Cc: Andrew Morton , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 13A01140007 X-Stat-Signature: 7tht3egz6mms7i1arjsqmdzaz4j8eyia X-Rspam-User: X-HE-Tag: 1738401713-893761 X-HE-Meta: U2FsdGVkX18gu2K6ql+rqa8f1thx98NcJYwyCVDF5gq3VfMtrZSerqB0MWMgQkhxmczSAeUuASxg25WZ0dhctPBmKMtI1ZFjoWU60Vh32T5jJHxvakBrSQ4ifYRZTBQyydmAFHK9JqAew58hMPkXER7x8tIgP0G8TkwlZAVEN5BlnnahzK7DYFmRDMsp52NpiNxK8Bsgo92q+LBfG+ybX1d2SDKwmst4LcnnldUaDxdRuCvMss8CHTuHpGZIUiUwoqWAKQR0U35FOAT+aerwtlNPoxnT8PsAx9pdBw049z1PJiNvsCh/WBq6PVJP02qZvMxva9FWW2ImJfqPhkqvcLlDZEq8VuGY0wKH9Z2szOOVPMTjjYJXtgJSuZ4oTYoGU/9aL1mywLyGYjc0CkhoMf1VpSrqZrkEti0BEp+31ODe50L9U2zpEkwWYjbGnBLITNHac7E7YaYCjk563U/w2txTpps3h53q+8zxTvA8H+1qU8u0ywfz7730Hi/Z25SdvcA0eRRXd8N2EeJwX1J9T/HPttNvlIGpQ+6Q+zSzy6zgTXuPbNO4H/wkKJhfCdctShzPafGyDBuk+gipKojg8uiaPDDAIomDuBaBk8dJoZnANCJAZ8TJouN9tBg1FXDyDB7ck2c7vwwaTw+Q2O3orA+PZy4DR0Ck6K5zChLdCZ0iM+hPKxuERXP/gPwgqu4teLzZdbXhpGkWa7k1BgZc0B30C3jDYrNSWrR/CxZvG7TYem/JUYWqcXR4lUut4geOdRVQcaOzWvpX03EkW5DZzR5u3CSIlw9uBF3nPuiuVYPsDmTJYXBQ3wG4y4lzDjZ0axLYdbXGGoZoMZ6/i5rgj3UBRDmtUwURHox+uLv7qDz43fvlGPWm38QLk37+H9Sm9U3RS4uHFA1zjlQ0k3HjqyVpcf69WY+lJxp1joB8DR1YZq88nW4sf/fZCOoqKqRwNrYM4Yf28xdMtGjxS/T GWWVn0VP UbYhulek0lOEkVaBCF5N8/xLtWEi8UJc93NR5B0wQz4M+cr8FHHLiyWlNQIX15iPJG3NjUbt3pMQ9Um3t8PKbUUKUcYH/fG+JOCk6kI03xtGdnm3MXDReAKeLl9fEcRnVbyTpxX0wsIHXber2MzosPcZwtx6zvleBMvGPEaMUnf4Q35w0xGYZcCPi8iim2D+UPCi9oGAtX8kpmNSs0QlI47nlJ7BM4Fy71pNDViy2jtf96SjuVXSzRuoCYqSgtxh1ZFpdWjrUuUnDQDSYL5tMuYbGa2e2k/Shaf8ADgJBeRgZ/gTfMIvW1oyc1fCilfC03i422ySLG+geaa9+wMM5Ez7kOw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Sergey, On Fri, Jan 31, 2025 at 5:07=E2=80=AFPM Sergey Senozhatsky wrote: > > Similarly to per-entry spin-lock per-CPU compression streams > also have a number of shortcoming. > > First, per-CPU stream access has to be done from a non-preemptible > (atomic) section, which imposes the same atomicity requirements on > compression backends as entry spin-lock do and makes it impossible > to use algorithms that can schedule/wait/sleep during compression > and decompression. > > Second, per-CPU streams noticeably increase memory usage (actually > more like wastage) of secondary compression streams. The problem > is that secondary compression streams are allocated per-CPU, just > like the primary streams are. Yet we never use more that one > secondary stream at a time, because recompression is a single > threaded action. Which means that remaining num_online_cpu() - 1 > streams are allocated for nothing, and this is per-priority list > (we can have several secondary compression algorithms). Depending > on the algorithm this may lead to a significant memory wastage, in > addition each stream also carries a workmem buffer (2 physical > pages). > > Instead of per-CPU streams, maintain a list of idle compression > streams and allocate new streams on-demand (something that we > used to do many years ago). So that zram read() and write() become > non-atomic and ease requirements on the compression algorithm > implementation. This also means that we now should have only one > secondary stream per-priority list. > > Signed-off-by: Sergey Senozhatsky > --- > drivers/block/zram/zcomp.c | 164 +++++++++++++++++++--------------- > drivers/block/zram/zcomp.h | 17 ++-- > drivers/block/zram/zram_drv.c | 29 +++--- > include/linux/cpuhotplug.h | 1 - > 4 files changed, 109 insertions(+), 102 deletions(-) > > diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c > index bb514403e305..982c769d5831 100644 > --- a/drivers/block/zram/zcomp.c > +++ b/drivers/block/zram/zcomp.c > @@ -6,7 +6,7 @@ > #include > #include > #include > -#include > +#include > #include > #include > > @@ -43,31 +43,40 @@ static const struct zcomp_ops *backends[] =3D { > NULL > }; > > -static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *zstrm= ) > +static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *strm) > { > - comp->ops->destroy_ctx(&zstrm->ctx); > - vfree(zstrm->buffer); > - zstrm->buffer =3D NULL; > + comp->ops->destroy_ctx(&strm->ctx); > + vfree(strm->buffer); > + kfree(strm); > } > > -static int zcomp_strm_init(struct zcomp *comp, struct zcomp_strm *zstrm) > +static struct zcomp_strm *zcomp_strm_alloc(struct zcomp *comp) > { > + struct zcomp_strm *strm; > int ret; > > - ret =3D comp->ops->create_ctx(comp->params, &zstrm->ctx); > - if (ret) > - return ret; > + strm =3D kzalloc(sizeof(*strm), GFP_KERNEL); > + if (!strm) > + return NULL; > + > + INIT_LIST_HEAD(&strm->entry); > + > + ret =3D comp->ops->create_ctx(comp->params, &strm->ctx); > + if (ret) { > + kfree(strm); > + return NULL; > + } > > /* > - * allocate 2 pages. 1 for compressed data, plus 1 extra for the > - * case when compressed size is larger than the original one > + * allocate 2 pages. 1 for compressed data, plus 1 extra in case = if > + * compressed data is larger than the original one. > */ > - zstrm->buffer =3D vzalloc(2 * PAGE_SIZE); > - if (!zstrm->buffer) { > - zcomp_strm_free(comp, zstrm); > - return -ENOMEM; > + strm->buffer =3D vzalloc(2 * PAGE_SIZE); > + if (!strm->buffer) { > + zcomp_strm_free(comp, strm); > + return NULL; > } > - return 0; > + return strm; > } > > static const struct zcomp_ops *lookup_backend_ops(const char *comp) > @@ -109,13 +118,59 @@ ssize_t zcomp_available_show(const char *comp, char= *buf) > > struct zcomp_strm *zcomp_stream_get(struct zcomp *comp) > { > - local_lock(&comp->stream->lock); > - return this_cpu_ptr(comp->stream); > + struct zcomp_strm *strm; > + > + might_sleep(); > + > + while (1) { > + spin_lock(&comp->strm_lock); > + if (!list_empty(&comp->idle_strm)) { > + strm =3D list_first_entry(&comp->idle_strm, > + struct zcomp_strm, > + entry); > + list_del(&strm->entry); > + spin_unlock(&comp->strm_lock); > + return strm; > + } > + > + /* cannot allocate new stream, wait for an idle one */ > + if (comp->avail_strm >=3D num_online_cpus()) { > + spin_unlock(&comp->strm_lock); > + wait_event(comp->strm_wait, > + !list_empty(&comp->idle_strm)); > + continue; > + } > + > + /* allocate new stream */ > + comp->avail_strm++; > + spin_unlock(&comp->strm_lock); > + > + strm =3D zcomp_strm_alloc(comp); > + if (strm) > + break; > + > + spin_lock(&comp->strm_lock); > + comp->avail_strm--; > + spin_unlock(&comp->strm_lock); > + wait_event(comp->strm_wait, !list_empty(&comp->idle_strm)= ); > + } > + > + return strm; > } > > -void zcomp_stream_put(struct zcomp *comp) > +void zcomp_stream_put(struct zcomp *comp, struct zcomp_strm *strm) > { > - local_unlock(&comp->stream->lock); > + spin_lock(&comp->strm_lock); > + if (comp->avail_strm <=3D num_online_cpus()) { > + list_add(&strm->entry, &comp->idle_strm); > + spin_unlock(&comp->strm_lock); > + wake_up(&comp->strm_wait); > + return; > + } > + > + comp->avail_strm--; > + spin_unlock(&comp->strm_lock); > + zcomp_strm_free(comp, strm); > } > > int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, > @@ -148,61 +203,19 @@ int zcomp_decompress(struct zcomp *comp, struct zco= mp_strm *zstrm, > return comp->ops->decompress(comp->params, &zstrm->ctx, &req); > } > > -int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node) > -{ > - struct zcomp *comp =3D hlist_entry(node, struct zcomp, node); > - struct zcomp_strm *zstrm; > - int ret; > - > - zstrm =3D per_cpu_ptr(comp->stream, cpu); > - local_lock_init(&zstrm->lock); > - > - ret =3D zcomp_strm_init(comp, zstrm); > - if (ret) > - pr_err("Can't allocate a compression stream\n"); > - return ret; > -} > - > -int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *node) > -{ > - struct zcomp *comp =3D hlist_entry(node, struct zcomp, node); > - struct zcomp_strm *zstrm; > - > - zstrm =3D per_cpu_ptr(comp->stream, cpu); > - zcomp_strm_free(comp, zstrm); > - return 0; > -} > - > -static int zcomp_init(struct zcomp *comp, struct zcomp_params *params) > -{ > - int ret; > - > - comp->stream =3D alloc_percpu(struct zcomp_strm); > - if (!comp->stream) > - return -ENOMEM; > - > - comp->params =3D params; > - ret =3D comp->ops->setup_params(comp->params); > - if (ret) > - goto cleanup; > - > - ret =3D cpuhp_state_add_instance(CPUHP_ZCOMP_PREPARE, &comp->node= ); > - if (ret < 0) > - goto cleanup; > - > - return 0; > - > -cleanup: > - comp->ops->release_params(comp->params); > - free_percpu(comp->stream); > - return ret; > -} > - > void zcomp_destroy(struct zcomp *comp) > { > - cpuhp_state_remove_instance(CPUHP_ZCOMP_PREPARE, &comp->node); > + struct zcomp_strm *strm; > + > + while (!list_empty(&comp->idle_strm)) { > + strm =3D list_first_entry(&comp->idle_strm, > + struct zcomp_strm, > + entry); > + list_del(&strm->entry); > + zcomp_strm_free(comp, strm); > + } > + > comp->ops->release_params(comp->params); > - free_percpu(comp->stream); > kfree(comp); > } > > @@ -229,7 +242,12 @@ struct zcomp *zcomp_create(const char *alg, struct z= comp_params *params) > return ERR_PTR(-EINVAL); > } > > - error =3D zcomp_init(comp, params); > + INIT_LIST_HEAD(&comp->idle_strm); > + init_waitqueue_head(&comp->strm_wait); > + spin_lock_init(&comp->strm_lock); > + > + comp->params =3D params; > + error =3D comp->ops->setup_params(comp->params); > if (error) { > kfree(comp); > return ERR_PTR(error); > diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h > index ad5762813842..62330829db3f 100644 > --- a/drivers/block/zram/zcomp.h > +++ b/drivers/block/zram/zcomp.h > @@ -3,10 +3,10 @@ > #ifndef _ZCOMP_H_ > #define _ZCOMP_H_ > > -#include > - > #define ZCOMP_PARAM_NO_LEVEL INT_MIN > > +#include > + > /* > * Immutable driver (backend) parameters. The driver may attach private > * data to it (e.g. driver representation of the dictionary, etc.). > @@ -31,7 +31,7 @@ struct zcomp_ctx { > }; > > struct zcomp_strm { > - local_lock_t lock; > + struct list_head entry; > /* compression buffer */ > void *buffer; > struct zcomp_ctx ctx; > @@ -60,16 +60,15 @@ struct zcomp_ops { > const char *name; > }; > > -/* dynamic per-device compression frontend */ > struct zcomp { > - struct zcomp_strm __percpu *stream; > + struct list_head idle_strm; > + spinlock_t strm_lock; > + u32 avail_strm; > + wait_queue_head_t strm_wait; > const struct zcomp_ops *ops; > struct zcomp_params *params; > - struct hlist_node node; > }; > > -int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node); > -int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *node); > ssize_t zcomp_available_show(const char *comp, char *buf); > bool zcomp_available_algorithm(const char *comp); > > @@ -77,7 +76,7 @@ struct zcomp *zcomp_create(const char *alg, struct zcom= p_params *params); > void zcomp_destroy(struct zcomp *comp); > > struct zcomp_strm *zcomp_stream_get(struct zcomp *comp); > -void zcomp_stream_put(struct zcomp *comp); > +void zcomp_stream_put(struct zcomp *comp, struct zcomp_strm *strm); > > int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, > const void *src, unsigned int *dst_len); > diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.= c > index 1c2df2341704..8d5974ea8ff8 100644 > --- a/drivers/block/zram/zram_drv.c > +++ b/drivers/block/zram/zram_drv.c > @@ -31,7 +31,6 @@ > #include > #include > #include > -#include > #include > #include > > @@ -1601,7 +1600,7 @@ static int read_compressed_page(struct zram *zram, = struct page *page, u32 index) > ret =3D zcomp_decompress(zram->comps[prio], zstrm, src, size, dst= ); > kunmap_local(dst); > zs_unmap_object(zram->mem_pool, handle); > - zcomp_stream_put(zram->comps[prio]); > + zcomp_stream_put(zram->comps[prio], zstrm); > > return ret; > } > @@ -1762,14 +1761,14 @@ static int zram_write_page(struct zram *zram, str= uct page *page, u32 index) > kunmap_local(mem); > > if (unlikely(ret)) { > - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); > + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); > pr_err("Compression failed! err=3D%d\n", ret); > zs_free(zram->mem_pool, handle); > return ret; > } > > if (comp_len >=3D huge_class_size) { > - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); > + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); > return write_incompressible_page(zram, page, index); > } > > @@ -1793,7 +1792,7 @@ static int zram_write_page(struct zram *zram, struc= t page *page, u32 index) > __GFP_HIGHMEM | > __GFP_MOVABLE); > if (IS_ERR_VALUE(handle)) { > - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); > + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); > atomic64_inc(&zram->stats.writestall); > handle =3D zs_malloc(zram->mem_pool, comp_len, > GFP_NOIO | __GFP_HIGHMEM | > @@ -1805,7 +1804,7 @@ static int zram_write_page(struct zram *zram, struc= t page *page, u32 index) > } > > if (!zram_can_store_page(zram)) { > - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); > + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); > zs_free(zram->mem_pool, handle); > return -ENOMEM; > } > @@ -1813,7 +1812,7 @@ static int zram_write_page(struct zram *zram, struc= t page *page, u32 index) > dst =3D zs_map_object(zram->mem_pool, handle, ZS_MM_WO); > > memcpy(dst, zstrm->buffer, comp_len); > - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); > + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); > zs_unmap_object(zram->mem_pool, handle); > > zram_slot_write_lock(zram, index); > @@ -1972,7 +1971,7 @@ static int recompress_slot(struct zram *zram, u32 i= ndex, struct page *page, > kunmap_local(src); > > if (ret) { > - zcomp_stream_put(zram->comps[prio]); > + zcomp_stream_put(zram->comps[prio], zstrm); > return ret; > } > > @@ -1982,7 +1981,7 @@ static int recompress_slot(struct zram *zram, u32 i= ndex, struct page *page, > /* Continue until we make progress */ > if (class_index_new >=3D class_index_old || > (threshold && comp_len_new >=3D threshold)) { > - zcomp_stream_put(zram->comps[prio]); > + zcomp_stream_put(zram->comps[prio], zstrm); > continue; > } > > @@ -2040,13 +2039,13 @@ static int recompress_slot(struct zram *zram, u32= index, struct page *page, > __GFP_HIGHMEM | > __GFP_MOVABLE); > if (IS_ERR_VALUE(handle_new)) { > - zcomp_stream_put(zram->comps[prio]); > + zcomp_stream_put(zram->comps[prio], zstrm); > return PTR_ERR((void *)handle_new); > } > > dst =3D zs_map_object(zram->mem_pool, handle_new, ZS_MM_WO); > memcpy(dst, zstrm->buffer, comp_len_new); > - zcomp_stream_put(zram->comps[prio]); > + zcomp_stream_put(zram->comps[prio], zstrm); > > zs_unmap_object(zram->mem_pool, handle_new); > > @@ -2794,7 +2793,6 @@ static void destroy_devices(void) > zram_debugfs_destroy(); > idr_destroy(&zram_index_idr); > unregister_blkdev(zram_major, "zram"); > - cpuhp_remove_multi_state(CPUHP_ZCOMP_PREPARE); > } > > static int __init zram_init(void) > @@ -2804,15 +2802,9 @@ static int __init zram_init(void) > > BUILD_BUG_ON(__NR_ZRAM_PAGEFLAGS > sizeof(zram_te.flags) * 8); > > - ret =3D cpuhp_setup_state_multi(CPUHP_ZCOMP_PREPARE, "block/zram:= prepare", > - zcomp_cpu_up_prepare, zcomp_cpu_dea= d); > - if (ret < 0) > - return ret; > - > ret =3D class_register(&zram_control_class); > if (ret) { > pr_err("Unable to register zram-control class\n"); > - cpuhp_remove_multi_state(CPUHP_ZCOMP_PREPARE); > return ret; > } > > @@ -2821,7 +2813,6 @@ static int __init zram_init(void) > if (zram_major <=3D 0) { > pr_err("Unable to get major number\n"); > class_unregister(&zram_control_class); > - cpuhp_remove_multi_state(CPUHP_ZCOMP_PREPARE); > return -EBUSY; > } > > diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h > index 6cc5e484547c..092ace7db8ee 100644 > --- a/include/linux/cpuhotplug.h > +++ b/include/linux/cpuhotplug.h > @@ -119,7 +119,6 @@ enum cpuhp_state { > CPUHP_MM_ZS_PREPARE, > CPUHP_MM_ZSWP_POOL_PREPARE, > CPUHP_KVM_PPC_BOOK3S_PREPARE, > - CPUHP_ZCOMP_PREPARE, > CPUHP_TIMERS_PREPARE, > CPUHP_TMIGR_PREPARE, > CPUHP_MIPS_SOC_PREPARE, > -- > 2.48.1.362.g079036d154-goog This seems will cause a huge regression of performance on multi core systems, this is especially significant as the number of concurrent tasks increases: Test build linux kernel using ZRAM as SWAP (1G memcg): Before: + /usr/bin/time make -s -j48 2495.77user 2604.77system 2:12.95elapsed 3836%CPU (0avgtext+0avgdata 863304maxresident)k After: + /usr/bin/time make -s -j48 2403.60user 6676.09system 3:38.22elapsed 4160%CPU (0avgtext+0avgdata 863276maxresident)k `perf lock contention -ab sleep 3` also indicates the big spin lock in zcomp_stream_get/put is having significant contention: contended total wait max wait avg wait type caller 793357 28.71 s 2.66 ms 36.19 us spinlock zcomp_stream_get+0x37 793170 28.60 s 2.65 ms 36.06 us spinlock zcomp_stream_put+0x1f 444007 15.26 s 2.58 ms 34.37 us spinlock zcomp_stream_put+0x1f 443960 15.21 s 2.68 ms 34.25 us spinlock zcomp_stream_get+0x37 5516 152.50 ms 3.30 ms 27.65 us spinlock evict_folios+0x7e 4523 137.47 ms 3.66 ms 30.39 us spinlock folio_lruvec_lock_irqsave+0xc3 4253 108.93 ms 2.92 ms 25.61 us spinlock folio_lruvec_lock_irqsave+0xc3 49294 71.73 ms 15.87 us 1.46 us spinlock list_lru_del+0x7c 2327 51.35 ms 3.48 ms 22.07 us spinlock evict_folios+0x5c0