From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D465C5472D for ; Wed, 28 Aug 2024 05:37:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7429E6B0082; Wed, 28 Aug 2024 01:37:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6CA8A6B0083; Wed, 28 Aug 2024 01:37:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56AC06B0085; Wed, 28 Aug 2024 01:37:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 28DCC6B0082 for ; Wed, 28 Aug 2024 01:37:41 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 9A536AAE8E for ; Wed, 28 Aug 2024 05:37:40 +0000 (UTC) X-FDA: 82500547080.15.5067E80 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by imf03.hostedemail.com (Postfix) with ESMTP id C6D7520021 for ; Wed, 28 Aug 2024 05:37:38 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux.microsoft.com header.s=default header.b=FXiaFZoE; spf=pass (imf03.hostedemail.com: domain of ssengar@linux.microsoft.com designates 13.77.154.182 as permitted sender) smtp.mailfrom=ssengar@linux.microsoft.com; dmarc=pass (policy=none) header.from=linux.microsoft.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724823389; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MvlhFl0YgUXxQqCx/QzeuztzvE6H4pK/dbDElYS3dL4=; b=E2kAJCR1BTQX5cI+3uthQvMP/h8vcC/ZLWYOZoxlHHKMfnRchRZLe4jQLBXaKZ0yZUH8tw CsF41VGCzioHZZ1uleqSTeNubDCtBRDnSsK6Y+rCgXTwy0leD7e/480lVs3eOQ9U1/QVzI 96bAFb/fPt93jyo8yYUgwcuIuphRYSE= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=linux.microsoft.com header.s=default header.b=FXiaFZoE; spf=pass (imf03.hostedemail.com: domain of ssengar@linux.microsoft.com designates 13.77.154.182 as permitted sender) smtp.mailfrom=ssengar@linux.microsoft.com; dmarc=pass (policy=none) header.from=linux.microsoft.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724823389; a=rsa-sha256; cv=none; b=v72vrjkmy1TNsjwoc7R3H/TdmXmn1LMZlhlnBH3yGMkV7bzUPHSFLUhthnjFkAY1YhKCxt yy4KPyevhUOKhx7WtDSlY8za25aycyBrWebx9x1VgxPBPzaD/cpK0lgVRyUv7v2AVbW6ym EQDVfx1LqLIB5jVXPVPbHb8UA3aGDec= Received: by linux.microsoft.com (Postfix, from userid 1127) id 4E71C20B7165; Tue, 27 Aug 2024 22:37:37 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 4E71C20B7165 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1724823457; bh=MvlhFl0YgUXxQqCx/QzeuztzvE6H4pK/dbDElYS3dL4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=FXiaFZoE/NqE2BnrcjX67ysJN+HN98y21fpCR9Y2Qan481LHEae4dor1wM2GNvMNC kuWpcuK3YkghqHGvlrMFIz1ZzwQQtnbB98Jxli/t7HDQ9vxZmG1Q2DERxvSIcCcNVD L1/JOh64UiMPwtL4XarqqITFYD062U3zuE09JnzM= Date: Tue, 27 Aug 2024 22:37:37 -0700 From: Saurabh Singh Sengar To: "Christoph Lameter (Ampere)" Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, ssengar@microsoft.com, wei.liu@kernel.org Subject: Re: [PATCH] mm/vmstat: Defer the refresh_zone_stat_thresholds after all CPUs bringup Message-ID: <20240828053737.GA22322@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> References: <1720169301-21002-1-git-send-email-ssengar@linux.microsoft.com> <20240705135911.4a6e38379ae95c3fc6bbe7e2@linux-foundation.org> <20240709045750.GA32083@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> <5a18105c-8e6f-39fa-370f-2d839d9ab843@gentwo.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5a18105c-8e6f-39fa-370f-2d839d9ab843@gentwo.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Stat-Signature: j3ab6j4kq54rxs8tdbqnj8epa59dowfm X-Rspam-User: X-Rspamd-Queue-Id: C6D7520021 X-Rspamd-Server: rspam02 X-HE-Tag: 1724823458-74341 X-HE-Meta: U2FsdGVkX19Z9rOAOgK0b99MhWk6w66dk6FA3WhieSnvZLRztU6rkRSDb8vx6ED8k4OTpJ1SwYLcAHdufWMGCUfp76lcXI05LHXawBZjENO40EL8W2+swmnlcVt/XUpTlicpNDvM6EYWma9pQN34AqtYxjJZJYbgqXZLOb7z6KWHniNBK8YhzGVfsEHiXjtL+TRwHWTK+p7b9LQile/GTx1hgkBVbQs0lrWEXWkelGg7vwKqSYzNkxBtxNJHv18u8dNOueCS3NNSxfqSlB1XS5aRGemiZSYMOO/iPVMEXywmf3O9sSjW5yymSOEiDPLAUVe2SHkd69Skqsm5JtupXU6kgVeSWKh8hkFh8AxTfR+qYwAaOjPYgvyiT+CBbbdpUNQspqosGQJIoBeLqv6OJ0COMKOks9DD1ctfe1eJxZWhdjYmUxIrgwRoxcmyMR6gXJ31cyKS1YQ05uF6M8VD6lczQ/KK63C4tT/6R1/RT4FaJARcvvCpYMXFeN5qhVgQwpnMakLy9kM9Msdphno6q+lgOWDiRcs0bwwgn2n9pqP0ZZvLGe8krOWQRFmwNDLgsbcNQnGeByda4nFPbGTjlJdwZM90dfFuaXaVgd7pDVNf7UqKjSRGsWIb95Lp9Ysdo7uWTksB408yz1J28x7OGvMY/w7c/TtBQtv43F464Ql8pyxIzbKDP53cTEEIiI6/jmr2z+4XIKY8e2vnTvSMqmhaLPqEFiEn93DXI1GF+K4Ns1l26NktOyp1El1RgfUsmZH+rZQ4gqXpNaa6JmL3YGZ/TbV3+reb1Tcoc1UsiH9Q2ew7gg6f1l2j+NeKkeggoTyL+/1sB7e1qkSi2kCcAniIAb8NPN2r3oatWgQqYOtsjOQRNcmWFJtH65EXRt0myt8XRQ+rkVvj2ddhOECBndGnurLO7hO/zscWs4AZQ3IR6z1qYWyLNUaIokVucCZW8plTJoBL1D0YL9XTN/j sS4azC1T ppLawRD6E9679hJOxxzq4fIYD4+03I4Ps1KWoUVcxX3bmrXuTuLZ78mbMAvl8FU3Ck0avD9tsqjUeH7/7EQFCyJ50prC1TNSnTLZgUVse4W8L9+OX+qJcx2X2hvGkaHDwDaos5zYiKUmI845NeXjTIO2EcvdmSsY5amw8XqhSF8ePnpU/RrtxjEXZyZAHkmn/r6+BbH9cLvVaysJJTqxVq6zFsnuurr6PdaqVYMQCSLVMFtntBWnNsW9axQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Aug 23, 2024 at 08:32:43AM -0700, Christoph Lameter (Ampere) wrote: > On Mon, 8 Jul 2024, Saurabh Singh Sengar wrote: > > > > > Calling this for each CPU is expensive when there are large number > > > > of CPUs along with multiple NUMAs. Fix this by deferring > > > > refresh_zone_stat_thresholds to be called later at once when all the > > > > secondary CPUs are up. Also, register the DYN hooks to keep the > > > > existing hotplug functionality intact. > > > > > > > > > > Seems risky - we'll now have online CPUs which have unintialized data, > > > yes? What assurance do we have that this data won't be accessed? > > > > I understand that this data is only accessed by userspace tools, and they can > > only access it post late_initcall. Please let me know if there are any other > > cases, I will look to address them. > > stat_threshold is used in all statistics functions that modify VM > counters. It is core to the functioning of the VM statistics. > > However, if the threshold is zero (not initialized) then the VM counter > handling will simply be less effective because it will not do the per cpu > counter diffs anymore. This may increase contention and eat up the benefit > you are getting from deferring the calculation of the threshholds. Christoph, Thank you for your review. I would like to gain a better understanding of how to measure contention. Could you please inform me if there is any recommended method for doing so? In my testing, this patch has resulted in a significant improvement in boot time. > > What may be more promising is to make it possible to calculate the > threshholds per cpu instead of recalculating the thresholds for every zone > again and again. I am happy to explore alternatives, can you please share more details around this approach. Are you referring to avoiding the repetition of the calculation below? mem = zone_managed_pages(zone) >> (27 - PAGE_SHIFT); - Saurabh