From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91B45C433FE for ; Fri, 4 Nov 2022 23:15:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1126B6B0078; Fri, 4 Nov 2022 19:15:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C2FF6B007B; Fri, 4 Nov 2022 19:15:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECCE28E0001; Fri, 4 Nov 2022 19:15:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id DAAE66B0078 for ; Fri, 4 Nov 2022 19:15:53 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id AF2DB141722 for ; Fri, 4 Nov 2022 23:15:53 +0000 (UTC) X-FDA: 80097319386.14.7E338F2 Received: from mail-yb1-f180.google.com (mail-yb1-f180.google.com [209.85.219.180]) by imf24.hostedemail.com (Postfix) with ESMTP id 58CE3180006 for ; Fri, 4 Nov 2022 23:15:53 +0000 (UTC) Received: by mail-yb1-f180.google.com with SMTP id g127so7469538ybg.8 for ; Fri, 04 Nov 2022 16:15:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=gfQYYmCbfCdmjxGtiSGnXE7+iSfuVxiEGd6gtYgqdUg=; b=W7QfOvf2wl5F1GgempFO5HuRpFHpBoZFuLire4tTDUmcIFG0V0llXW6eY60TN4eKwj mgqQY4kV0HG1PwRq4ETPLqO2Ir6Ngk2DRQ0nbQK3MnjpiijFSmgbwsBKaDlPDYxRmDue l+lSlmAT+regGBQ1XuTGfERpdMReMZtE5VEMTcnASO9eKaV1LzeZvhPnPhCfsi5BtQMb k86hY1TpDd51WohK4ComlXZnZduj0C4dJzYLSckz7B1HD9FCjGnk1tKnvay2eldhq5d9 YK7JFTIt5KZYTxbDvRBJxHQ4rLRG/enxlZD8C2uaFRkV2V6pgmBbOPOsb7r3AHJQKJ+s vj/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=gfQYYmCbfCdmjxGtiSGnXE7+iSfuVxiEGd6gtYgqdUg=; b=wwza/MoUEHfiNI7plGYBBVJG83bXy7LPXkTFIzdKyL9J5gkCSahnYk7ycMNh2OPRhO CK03oGtsRzru6fI9hV3sQbTmZhC38cLRxVQd2Ie4O1w73sTlfF0AcuE1TnKtHn7xYfh7 jILFU1AQK++umGMvc8oUNtKmc7VSULrDU7iCMSmc9PCUuetwLDSZrH2WqTw2JyCOJG8F JMSAbvHv0CvUDD8vHxdlJGQC/adyXV+F8Cz3zdVgDgws4UE1EJd1wdqWmV0AEbQEOpZ7 2UW3/ui+EDTzZFh6aP5YTswdndU7vkKm0j1Qqw8I94plvvwzFgqD/zbPm6tjHDfywaEz cyyA== X-Gm-Message-State: ACrzQf2dm33qYlGz50njHlyYGp/gZ90MAVYeF/hPBsyWinTOcPMnzWHB uBbEZ4Q8/dX4LPYY89HiiGIdhbKFCSsFr7cFOzt3TA== X-Google-Smtp-Source: AMsMyM6Y9C9l1opVH7d9hQqg25kh4n4jR2xkbBg4vrbQWw1H6AOqjy6XSalLTDDqueYCQwiSumzX7y/MS6laK3Q5rhI= X-Received: by 2002:a25:c00a:0:b0:6cf:dda2:552e with SMTP id c10-20020a25c00a000000b006cfdda2552emr221195ybf.363.1667603752365; Fri, 04 Nov 2022 16:15:52 -0700 (PDT) MIME-Version: 1.0 References: <20221024052841.3291983-1-shakeelb@google.com> <20221103171407.ydubp43x7tzahriq@google.com> <20221104160552.c249397512c5c7f8b293869f@linux-foundation.org> In-Reply-To: <20221104160552.c249397512c5c7f8b293869f@linux-foundation.org> From: Shakeel Butt Date: Fri, 4 Nov 2022 16:15:41 -0700 Message-ID: Subject: Re: [PATCH] mm: convert mm's rss stats into percpu_counter To: Andrew Morton Cc: Marek Szyprowski , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667603753; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gfQYYmCbfCdmjxGtiSGnXE7+iSfuVxiEGd6gtYgqdUg=; b=Z4BDpz7qEKZbD+dqrIFBR0j/7UA29QQfl0kX3TPgByxBYNCqwblnLgtfNyRmhxDnLxtEZd 7z+e6Hw4Afz14gKX75vZiCHZwhYJMYNbes2jbNp4cL/6Qasm7bUeteBJ6PJFnkC5lZUP9G OIQwnW2xxXIa6nDWR9WA/9GHoYVoVmU= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=W7QfOvf2; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of shakeelb@google.com designates 209.85.219.180 as permitted sender) smtp.mailfrom=shakeelb@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667603753; a=rsa-sha256; cv=none; b=NPCuYaPsC0j2vAQPFVL4eT4jSMayTFy75CzJj2ZemVV+LS4g9PhPVHRc4ZxsmNYgWoPRdQ hC1XytmvOBDErUZa52qp6oAm8FoBijDnZZ1cbH1VUQRg1y+hsI6mcicLToy/7a6P/zE/0T g7gdj/IOjRvLun/BeWU+RBmAYpWBS0Q= Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=W7QfOvf2; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of shakeelb@google.com designates 209.85.219.180 as permitted sender) smtp.mailfrom=shakeelb@google.com X-Rspam-User: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 58CE3180006 X-Stat-Signature: ipofg7p8jq3abay9jc67s1z6gm8t7ycf X-HE-Tag: 1667603753-972289 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Nov 4, 2022 at 4:05 PM Andrew Morton wrote: > > On Thu, 3 Nov 2022 17:14:07 +0000 Shakeel Butt wrote: > > > > > ... > > > > Thanks for the report. It seems like there is a race between > > for_each_online_cpu() in __percpu_counter_sum() and > > percpu_counter_cpu_dead()/cpu-offlining. Normally this race is fine for > > percpu_counter users but for check_mm() is not happy with this race. Can > > you please try the following patch: > > percpu-counters supposedly avoid such races via the hotplup notifier. > So can you please fully describe the race and let's see if it can be > fixed at the percpu_counter level? > Yes, I am writing a more detailed commit message explaining the race and why it is not really an issue for current users. > > > > From: Shakeel Butt > > Date: Thu, 3 Nov 2022 06:05:13 +0000 > > Subject: [PATCH] mm: percpu_counter: use race free percpu_counter sum > > interface > > > > percpu_counter_sum can race with cpu offlining. Add a new interface > > which does not race with it and use that for check_mm(). > > I'll grab this version for now, as others might be seeing this issue. > Thanks. > > > --- > > include/linux/percpu_counter.h | 11 +++++++++++ > > kernel/fork.c | 2 +- > > lib/percpu_counter.c | 24 ++++++++++++++++++------ > > 3 files changed, 30 insertions(+), 7 deletions(-) > > > > diff --git a/include/linux/percpu_counter.h b/include/linux/percpu_counter.h > > index bde6c4c1f405..3070c1043acf 100644 > > --- a/include/linux/percpu_counter.h > > +++ b/include/linux/percpu_counter.h > > @@ -45,6 +45,7 @@ void percpu_counter_set(struct percpu_counter *fbc, s64 amount); > > void percpu_counter_add_batch(struct percpu_counter *fbc, s64 amount, > > s32 batch); > > s64 __percpu_counter_sum(struct percpu_counter *fbc); > > +s64 __percpu_counter_sum_all(struct percpu_counter *fbc); > > int __percpu_counter_compare(struct percpu_counter *fbc, s64 rhs, s32 batch); > > void percpu_counter_sync(struct percpu_counter *fbc); > > > > @@ -85,6 +86,11 @@ static inline s64 percpu_counter_sum(struct percpu_counter *fbc) > > return __percpu_counter_sum(fbc); > > } > > > > +static inline s64 percpu_counter_sum_all(struct percpu_counter *fbc) > > +{ > > + return __percpu_counter_sum_all(fbc); > > +} > > We haven't been good about documenting these interfaces. Can we please > start now? ;) > Yup will do. > > > > ... > > > > + > > +/* > > + * Add up all the per-cpu counts, return the result. This is a more accurate > > + * but much slower version of percpu_counter_read_positive() > > + */ > > +s64 __percpu_counter_sum(struct percpu_counter *fbc) > > +{ > > + return __percpu_counter_sum_mask(fbc, cpu_online_mask); > > +} > > EXPORT_SYMBOL(__percpu_counter_sum); > > > > +s64 __percpu_counter_sum_all(struct percpu_counter *fbc) > > +{ > > + return __percpu_counter_sum_mask(fbc, cpu_possible_mask); > > +} > > +EXPORT_SYMBOL(__percpu_counter_sum_all); > > Probably here is a good place to document it. > > Is there any point in having the > percpu_counter_sum_all()->__percpu_counter_sum_all() inlined wrapper? > Why not name this percpu_counter_sum_all() directly? > Ack. thanks, Shakeel