From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C0AEC433E1 for ; Tue, 19 May 2020 20:20:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4AF1F2070A for ; Tue, 19 May 2020 20:20:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4AF1F2070A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linutronix.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9E66A80015; Tue, 19 May 2020 16:20:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9963A900003; Tue, 19 May 2020 16:20:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D40080015; Tue, 19 May 2020 16:20:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0071.hostedemail.com [216.40.44.71]) by kanga.kvack.org (Postfix) with ESMTP id 76573900003 for ; Tue, 19 May 2020 16:20:49 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 16B5C2A87 for ; Tue, 19 May 2020 20:20:49 +0000 (UTC) X-FDA: 76834587018.22.bit38_5e62c81c9ce62 X-HE-Tag: bit38_5e62c81c9ce62 X-Filterd-Recvd-Size: 5471 Received: from Galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by imf21.hostedemail.com (Postfix) with ESMTP for ; Tue, 19 May 2020 20:20:48 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=flow.W.breakpoint.cc) by Galois.linutronix.de with esmtp (Exim 4.80) (envelope-from ) id 1jb8ii-00012c-4u; Tue, 19 May 2020 22:20:12 +0200 From: Sebastian Andrzej Siewior To: linux-kernel@vger.kernel.org Cc: Peter Zijlstra , Ingo Molnar , Steven Rostedt , Will Deacon , Thomas Gleixner , "Paul E . McKenney" , Linus Torvalds , "Luis Claudio R. Goncalves" , Seth Jennings , Dan Streetman , Vitaly Wool , Andrew Morton , linux-mm@kvack.org, Sebastian Andrzej Siewior Subject: [PATCH 8/8] mm/zswap: Use local lock to protect per-CPU data Date: Tue, 19 May 2020 22:19:12 +0200 Message-Id: <20200519201912.1564477-9-bigeasy@linutronix.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200519201912.1564477-1-bigeasy@linutronix.de> References: <20200519201912.1564477-1-bigeasy@linutronix.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Luis Claudio R. Goncalves" zwap uses per-CPU compression. The per-CPU data pointer is acquired with get_cpu_ptr() which implicitly disables preemption. It allocates memory inside the preempt disabled region which conflicts with the PREEMPT_RT semantics. Replace the implicit preemption control with an explicit local lock. This allows RT kernels to substitute it with a real per CPU lock, which serializes the access but keeps the code section preemptible. On non RT kernels this maps to preempt_disable() as before, i.e. no functional change. [bigeasy: Use local_lock(), additional hunks, patch description] Cc: Seth Jennings Cc: Dan Streetman Cc: Vitaly Wool Cc: Andrew Morton Cc: linux-mm@kvack.org Signed-off-by: Luis Claudio R. Goncalves Signed-off-by: Sebastian Andrzej Siewior --- mm/zswap.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index fbb782924ccc5..1db2ad941e501 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -388,6 +389,8 @@ static struct zswap_entry *zswap_entry_find_get(struct = rb_root *root, * per-cpu code **********************************/ static DEFINE_PER_CPU(u8 *, zswap_dstmem); +/* Used for zswap_dstmem and tfm */ +static DEFINE_LOCAL_LOCK(zswap_cpu_lock); =20 static int zswap_dstmem_prepare(unsigned int cpu) { @@ -919,10 +922,11 @@ static int zswap_writeback_entry(struct zpool *pool, = unsigned long handle) dlen =3D PAGE_SIZE; src =3D (u8 *)zhdr + sizeof(struct zswap_header); dst =3D kmap_atomic(page); - tfm =3D *get_cpu_ptr(entry->pool->tfm); + local_lock(zswap_cpu_lock); + tfm =3D *this_cpu_ptr(entry->pool->tfm); ret =3D crypto_comp_decompress(tfm, src, entry->length, dst, &dlen); - put_cpu_ptr(entry->pool->tfm); + local_unlock(zswap_cpu_lock); kunmap_atomic(dst); BUG_ON(ret); BUG_ON(dlen !=3D PAGE_SIZE); @@ -1074,12 +1078,12 @@ static int zswap_frontswap_store(unsigned type, pgo= ff_t offset, } =20 /* compress */ - dst =3D get_cpu_var(zswap_dstmem); - tfm =3D *get_cpu_ptr(entry->pool->tfm); + local_lock(zswap_cpu_lock); + dst =3D *this_cpu_ptr(&zswap_dstmem); + tfm =3D *this_cpu_ptr(entry->pool->tfm); src =3D kmap_atomic(page); ret =3D crypto_comp_compress(tfm, src, PAGE_SIZE, dst, &dlen); kunmap_atomic(src); - put_cpu_ptr(entry->pool->tfm); if (ret) { ret =3D -EINVAL; goto put_dstmem; @@ -1103,7 +1107,7 @@ static int zswap_frontswap_store(unsigned type, pgoff= _t offset, memcpy(buf, &zhdr, hlen); memcpy(buf + hlen, dst, dlen); zpool_unmap_handle(entry->pool->zpool, handle); - put_cpu_var(zswap_dstmem); + local_unlock(zswap_cpu_lock); =20 /* populate entry */ entry->offset =3D offset; @@ -1131,7 +1135,7 @@ static int zswap_frontswap_store(unsigned type, pgoff= _t offset, return 0; =20 put_dstmem: - put_cpu_var(zswap_dstmem); + local_unlock(zswap_cpu_lock); zswap_pool_put(entry->pool); freepage: zswap_entry_cache_free(entry); @@ -1176,9 +1180,10 @@ static int zswap_frontswap_load(unsigned type, pgoff= _t offset, if (zpool_evictable(entry->pool->zpool)) src +=3D sizeof(struct zswap_header); dst =3D kmap_atomic(page); - tfm =3D *get_cpu_ptr(entry->pool->tfm); + local_lock(zswap_cpu_lock); + tfm =3D *this_cpu_ptr(entry->pool->tfm); ret =3D crypto_comp_decompress(tfm, src, entry->length, dst, &dlen); - put_cpu_ptr(entry->pool->tfm); + local_unlock(zswap_cpu_lock); kunmap_atomic(dst); zpool_unmap_handle(entry->pool->zpool, entry->handle); BUG_ON(ret); --=20 2.26.2