From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41A87C433E6 for ; Wed, 10 Mar 2021 12:01:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C08FF64FEE for ; Wed, 10 Mar 2021 12:01:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C08FF64FEE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 50F008D01A0; Wed, 10 Mar 2021 07:01:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E6398D0195; Wed, 10 Mar 2021 07:01:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3604A8D01A0; Wed, 10 Mar 2021 07:01:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0149.hostedemail.com [216.40.44.149]) by kanga.kvack.org (Postfix) with ESMTP id 1BC3F8D0195 for ; Wed, 10 Mar 2021 07:01:46 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C624C181AF5EA for ; Wed, 10 Mar 2021 12:01:45 +0000 (UTC) X-FDA: 77903825370.25.FDAE26E Received: from raptor.unsafe.ru (raptor.unsafe.ru [5.9.43.93]) by imf13.hostedemail.com (Postfix) with ESMTP id D31D3E007A6A for ; Wed, 10 Mar 2021 12:01:42 +0000 (UTC) Received: from comp-core-i7-2640m-0182e6.redhat.com (ip-94-113-225-162.net.upcbroadband.cz [94.113.225.162]) by raptor.unsafe.ru (Postfix) with ESMTPSA id A860540EF6; Wed, 10 Mar 2021 12:01:43 +0000 (UTC) From: Alexey Gladkov To: LKML , io-uring@vger.kernel.org, Kernel Hardening , Linux Containers , linux-mm@kvack.org Cc: Alexey Gladkov , Andrew Morton , Christian Brauner , "Eric W . Biederman" , Jann Horn , Jens Axboe , Kees Cook , Linus Torvalds , Oleg Nesterov Subject: [PATCH v8 3/8] Use atomic_t for ucounts reference counting Date: Wed, 10 Mar 2021 13:01:28 +0100 Message-Id: <59ee3289194cd97d70085cce701bc494bfcb4fd2.1615372955.git.gladkov.alexey@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: References: MIME-Version: 1.0 X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.4 (raptor.unsafe.ru [0.0.0.0]); Wed, 10 Mar 2021 12:01:44 +0000 (UTC) X-Stat-Signature: 6sf8rjam1j7ffdprpwu9zdzuy8u1yku4 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: D31D3E007A6A Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=raptor.unsafe.ru; client-ip=5.9.43.93 X-HE-DKIM-Result: none/none X-HE-Tag: 1615377702-53636 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The current implementation of the ucounts reference counter requires the use of spin_lock. We're going to use get_ucounts() in more performance critical areas like a handling of RLIMIT_SIGPENDING. Now we need to use spin_lock only if we want to change the hashtable. Signed-off-by: Alexey Gladkov --- include/linux/user_namespace.h | 4 +-- kernel/ucount.c | 60 +++++++++++++++------------------- 2 files changed, 28 insertions(+), 36 deletions(-) diff --git a/include/linux/user_namespace.h b/include/linux/user_namespac= e.h index f71b5a4a3e74..d84cc2c0b443 100644 --- a/include/linux/user_namespace.h +++ b/include/linux/user_namespace.h @@ -92,7 +92,7 @@ struct ucounts { struct hlist_node node; struct user_namespace *ns; kuid_t uid; - int count; + atomic_t count; atomic_long_t ucount[UCOUNT_COUNTS]; }; =20 @@ -104,7 +104,7 @@ void retire_userns_sysctls(struct user_namespace *ns)= ; struct ucounts *inc_ucount(struct user_namespace *ns, kuid_t uid, enum u= count_type type); void dec_ucount(struct ucounts *ucounts, enum ucount_type type); struct ucounts *alloc_ucounts(struct user_namespace *ns, kuid_t uid); -struct ucounts *get_ucounts(struct ucounts *ucounts); +struct ucounts * __must_check get_ucounts(struct ucounts *ucounts); void put_ucounts(struct ucounts *ucounts); =20 #ifdef CONFIG_USER_NS diff --git a/kernel/ucount.c b/kernel/ucount.c index 50cc1dfb7d28..bb3203039b5e 100644 --- a/kernel/ucount.c +++ b/kernel/ucount.c @@ -11,7 +11,7 @@ struct ucounts init_ucounts =3D { .ns =3D &init_user_ns, .uid =3D GLOBAL_ROOT_UID, - .count =3D 1, + .count =3D ATOMIC_INIT(1), }; =20 #define UCOUNTS_HASHTABLE_BITS 10 @@ -139,6 +139,22 @@ static void hlist_add_ucounts(struct ucounts *ucount= s) spin_unlock_irq(&ucounts_lock); } =20 +/* 127: arbitrary random number, small enough to assemble well */ +#define refcount_zero_or_close_to_overflow(ucounts) \ + ((unsigned int) atomic_read(&ucounts->count) + 127u <=3D 127u) + +struct ucounts *get_ucounts(struct ucounts *ucounts) +{ + if (ucounts) { + if (refcount_zero_or_close_to_overflow(ucounts)) { + WARN_ONCE(1, "ucounts: counter has reached its maximum value"); + return NULL; + } + atomic_inc(&ucounts->count); + } + return ucounts; +} + struct ucounts *alloc_ucounts(struct user_namespace *ns, kuid_t uid) { struct hlist_head *hashent =3D ucounts_hashentry(ns, uid); @@ -155,7 +171,7 @@ struct ucounts *alloc_ucounts(struct user_namespace *= ns, kuid_t uid) =20 new->ns =3D ns; new->uid =3D uid; - new->count =3D 0; + atomic_set(&new->count, 1); =20 spin_lock_irq(&ucounts_lock); ucounts =3D find_ucounts(ns, uid, hashent); @@ -163,33 +179,12 @@ struct ucounts *alloc_ucounts(struct user_namespace= *ns, kuid_t uid) kfree(new); } else { hlist_add_head(&new->node, hashent); - ucounts =3D new; + spin_unlock_irq(&ucounts_lock); + return new; } } - if (ucounts->count =3D=3D INT_MAX) - ucounts =3D NULL; - else - ucounts->count +=3D 1; spin_unlock_irq(&ucounts_lock); - return ucounts; -} - -struct ucounts *get_ucounts(struct ucounts *ucounts) -{ - unsigned long flags; - - if (!ucounts) - return NULL; - - spin_lock_irqsave(&ucounts_lock, flags); - if (ucounts->count =3D=3D INT_MAX) { - WARN_ONCE(1, "ucounts: counter has reached its maximum value"); - ucounts =3D NULL; - } else { - ucounts->count +=3D 1; - } - spin_unlock_irqrestore(&ucounts_lock, flags); - + ucounts =3D get_ucounts(ucounts); return ucounts; } =20 @@ -197,15 +192,12 @@ void put_ucounts(struct ucounts *ucounts) { unsigned long flags; =20 - spin_lock_irqsave(&ucounts_lock, flags); - ucounts->count -=3D 1; - if (!ucounts->count) + if (atomic_dec_and_test(&ucounts->count)) { + spin_lock_irqsave(&ucounts_lock, flags); hlist_del_init(&ucounts->node); - else - ucounts =3D NULL; - spin_unlock_irqrestore(&ucounts_lock, flags); - - kfree(ucounts); + spin_unlock_irqrestore(&ucounts_lock, flags); + kfree(ucounts); + } } =20 static inline bool atomic_long_inc_below(atomic_long_t *v, int u) --=20 2.29.2