From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 252F7C4167B for ; Fri, 16 Dec 2022 22:47:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9EB98E0005; Fri, 16 Dec 2022 17:47:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B4F5F8E0001; Fri, 16 Dec 2022 17:47:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3E448E0005; Fri, 16 Dec 2022 17:47:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 95FCF8E0001 for ; Fri, 16 Dec 2022 17:47:26 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6F0161C5C22 for ; Fri, 16 Dec 2022 22:47:26 +0000 (UTC) X-FDA: 80249657292.12.51C44D2 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf23.hostedemail.com (Postfix) with ESMTP id EEDCF140007 for ; Fri, 16 Dec 2022 22:47:23 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jXs70tcB; spf=pass (imf23.hostedemail.com: domain of frederic@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671230844; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ib84dVpx1D/MjAdbZVUiiYllvINs95WACwm95MQzH3c=; b=Zc02t9jzKsNQ20NvDMRpqMcXW1/GRFmXCHCGPIxB/SAcBiu3jEMIjCAduiTBxCg9or5t/9 msniN2ZcQnX3ob/N5ZCvgkoW6Rbv0RGAxafUjPSEGk0ffRplOsItUvRJpTcEK3AiNyacPI 9kv80ccVaJFTRQTphXRz2s/NBNU8I9U= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jXs70tcB; spf=pass (imf23.hostedemail.com: domain of frederic@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671230844; a=rsa-sha256; cv=none; b=usgCIyOBdowUM1qs5Ez/yF05RYEGS2Ef+5R4ipn/x0k2lx6llJwecRDcFvhkscl8guRdqf NR4SRQx5LOx9mtGe8t2R4zUJxUdSkJdXbWjvdhW2YqqpVIvG1AASiej+sldZKeLiTHvj7R Dbf1iunACiV0IgvFk+/E9e2HS+FcIUI= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id F220562274; Fri, 16 Dec 2022 22:47:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87439C433F1; Fri, 16 Dec 2022 22:47:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671230841; bh=UDGxwodPxON5TS5sciX0FiTPylhbUIAKvOCV1p/a2JE=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=jXs70tcBCwjWPNomP7NvNBi3B1eykeZCZm5yr7lSDN7nkDKCJrWYXslWirlBunY5o IgLw4P+2fGLf+Mv24xuAeigK8vWf+Ya0M5aepznyvwnYltJOS4s0/njqsKiLrJZl91 jiz4h1HcKPvizmjs3ByM82OAUHLkNQbr+NiKD82wpCruhAiJxvpCxNMVA4pWiKABJs +W0VwIeBmZB41e5Y8OUQPsmu0a6rZjmM8bcs7U9Dnyke0bar38in7WDS1OXBZYTjwS OVvCu5EqIVrbSZPP8jksmPmPjqRzoAgq0PlFeS7Ub+JWafEqjm+wzazzeIROQa60Ul IfUfoQgEDyL1Q== Date: Fri, 16 Dec 2022 23:47:19 +0100 From: Frederic Weisbecker To: Marcelo Tosatti Cc: atomlin@redhat.com, cl@linux.com, tglx@linutronix.de, mingo@kernel.org, peterz@infradead.org, pauld@redhat.com, neelx@redhat.com, oleksandr@natalenko.name, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v9 3/5] mm/vmstat: manage per-CPU stats from CPU context when NOHZ full Message-ID: <20221216224719.GB1967978@lothringen> References: <20221206161826.698593151@redhat.com> <20221206162416.474800121@redhat.com> <20221214133302.GA1931356@lothringen> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: EEDCF140007 X-Rspam-User: X-Stat-Signature: csgedirrg5eur1qgyhrz17chrn1aaynm X-HE-Tag: 1671230843-373903 X-HE-Meta: U2FsdGVkX18p1oS9ZfCwdIbeaNz0IezWn/GMoNIc+CZffskQCNKCF40rX+IOB1kOMdz4wfES1yfSG+v8GrAaGbSwC44hM5b8Q4Qcgh8r7Wx7t5Gqj5zGElh6sQiRiOkAnDrc6a20MuOBfymADdKz4cWsfZLMlPLAvVKRrOCaYM3PEXea4zb+EWSkr6zJJvVTbMr9utQjuRWSWoY1Hgw3bPZh/TwAH+YN3L+mg8OPiVlev3TPbP9PfCYbtq0tzjxj6PRlc7V8wWAy14JbeqLyQOfkugR1sQh836oVkuw05EldrYbmaen2TuAcnPvQDnRLTTho9KnALZZMe+I+lBeN2LZVQR+4TocMPr3hNq9dQyjDmF0RyiG50ps0aUNyRnYfRRetZOO/c+iTLS6z9KA4HFqE+iGao+icJAvenh1KYBYS/HVTcK1Mi0Jjt22fc2MeSFEJvI85Tp78nabVYge8X3BG/Y22hLn+N3rpubehUNyoBJhKUqrqTbeLGFxvhnJaMQ1pPVGsOXQy4Cv/zcw1DwN5Wp4rjL2PK+xKTHsFgZ4kVJSz3mZrBPvb8tbHtIxpdWsA3vjVlk2r6i41vVpUHUxpn7gwo/b7EcYvXkfnCM/bz+Pi8l/5DxSNjAkfBfWZs18abPjdKjxdkZQf6for7Dv5VQo7XLhG6hngonyBieJU4DJaSkZim3wSTV5yDlz95w4z+DeGtU9RRpA5AWIvXgdV2dbXy1aiFf9gcqPPNB/K8Txe16tBzv8FH3JiD4lqaR0N5Xv3iDWhYhGSEq2jTNXMpDvw/yzBTA1oh6DEsZ8KIdxFmRgq+vCOycz6hrsn5sLY07xqyAoeWCiRRawJSUN4ujxTfdRbV2okRvOMl9GgUI7FajKHuJTz6bvfC+64hBFvdmo/g3Iw5kD7vybTA0DmpV4qUuqWr7AGhdERuM/SWUX821gpkigBMAyRVoF/xASSk6qAml6LVHU9mGF ES1B+REa eghw+l2HKlxrRcBvLvnFkfTa5b7dhVjy+lZ3f X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Dec 16, 2022 at 01:16:09PM -0300, Marcelo Tosatti wrote: > On Wed, Dec 14, 2022 at 02:33:02PM +0100, Frederic Weisbecker wrote: > > On Tue, Dec 06, 2022 at 01:18:29PM -0300, Marcelo Tosatti wrote: > > > static inline void vmstat_mark_dirty(void) > > > { > > > + int cpu = smp_processor_id(); > > > + > > > + if (tick_nohz_full_cpu(cpu) && !this_cpu_read(vmstat_dirty)) { > > > + struct delayed_work *dw; > > > + > > > + dw = &per_cpu(vmstat_work, cpu); > > > + if (!delayed_work_pending(dw)) { > > > + unsigned long delay; > > > + > > > + delay = round_jiffies_relative(sysctl_stat_interval); > > > + queue_delayed_work_on(cpu, mm_percpu_wq, dw, delay); > > > > Currently the vmstat_work is flushed on cpu_hotplug (CPUHP_AP_ONLINE_DYN). > > vmstat_shepherd makes sure to not rearm it afterward. But now it looks > > possible for the above to do that mistake? > > Don't think the mistake is an issue. In case of a > queue_delayed_work_on being called after cancel_delayed_work_sync, > either vmstat_update executes on the local CPU, or on a > different CPU (after the bound kworkers have been moved). But after the CPU goes offline, its workqueue pool becomes UNBOUND. Which means that the vmstat_update() from the offline CPU can then execute partly on CPU 0, then gets preempted and executes halfway on CPU 1, then gets preempted and... Having a quick look at refresh_cpu_vm_stats(), I doesn't look ready for that... Thanks. > > Each case is fine (see vmstat_update). > > > > + } > > > + } > > > this_cpu_write(vmstat_dirty, true); > > > } > > > @@ -2009,6 +2028,10 @@ static void vmstat_shepherd(struct work_ > > > for_each_online_cpu(cpu) { > > > struct delayed_work *dw = &per_cpu(vmstat_work, cpu); > > > > > > + /* NOHZ full CPUs manage their own vmstat flushing */ > > > + if (tick_nohz_full_cpu(smp_processor_id())) > > > > It should be the remote CPU instead of the current one. > > Fixed. >