From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7A26C38A24 for ; Thu, 7 May 2020 12:17:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 87F3C208E4 for ; Thu, 7 May 2020 12:17:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 87F3C208E4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2B6DF900004; Thu, 7 May 2020 08:17:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 26616900002; Thu, 7 May 2020 08:17:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 17B47900004; Thu, 7 May 2020 08:17:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0190.hostedemail.com [216.40.44.190]) by kanga.kvack.org (Postfix) with ESMTP id F228C900002 for ; Thu, 7 May 2020 08:17:16 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B75A5824559C for ; Thu, 7 May 2020 12:17:16 +0000 (UTC) X-FDA: 76789822872.01.bait03_79b6f26d28c06 X-HE-Tag: bait03_79b6f26d28c06 X-Filterd-Recvd-Size: 6128 Received: from mail-wm1-f66.google.com (mail-wm1-f66.google.com [209.85.128.66]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Thu, 7 May 2020 12:17:16 +0000 (UTC) Received: by mail-wm1-f66.google.com with SMTP id u127so6562207wmg.1 for ; Thu, 07 May 2020 05:17:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=x8vf8a0RtXwF+U6/vuiecNbW1tvIfw6NFoZKkZipDlw=; b=DgeoYdzUE+52W4AoXBKRxo/RWC1XgkaRQjpLEZBZzx1jFJVLeT5LLmGqSwW9n6Yshx KPTMQZYwhMVhHykP2bSDjGkf8XTCZdZPjPnBa1hj4qCdlPOEKWJBAsH5SvaRCV8vhLaE IGaPq+qBqPJE3EXNRqIT4GskGaFuUG7Mes2CexWPB8EZEAHZfk9bVg4yQ8MflB+gXgsz sUvdC5kPCxVWPpr3XddM3pogUWp1iNYY2yqeJcLbBywg1I+af1UIdcWT+L8XGDsMwt5S bsIy0rj+QVVlFyCQU6MqWoB5HgKa3YgSd54pJgfJlgfFQsOchjBRyXQYepw9cL5EeEQu uMQA== X-Gm-Message-State: AGi0PuaruMvBfI1TljEzpTLd5DuUmOAjd+dicqsY2LmnYECtWkGFPMY+ mgLNpSCTv9WHwOml+aX73u6KIRtf X-Google-Smtp-Source: APiQypKAPCLf4pCrnsBJUWeKWU+a+SQ5cBDiIeIBdVGmTlKz6Y0Zs9Mh9ciJ98+3WJd/ZQnDMsMwsw== X-Received: by 2002:a7b:c1c4:: with SMTP id a4mr10274637wmj.86.1588853835178; Thu, 07 May 2020 05:17:15 -0700 (PDT) Received: from localhost (ip-37-188-183-9.eurotel.cz. [37.188.183.9]) by smtp.gmail.com with ESMTPSA id a13sm7854976wrv.67.2020.05.07.05.17.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2020 05:17:14 -0700 (PDT) Date: Thu, 7 May 2020 14:17:12 +0200 From: Michal Hocko To: Vlastimil Babka Cc: Jonathan Corbet , linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sandipan Das Subject: Re: [PATCH] Documentation: update numastat explanation Message-ID: <20200507121712.GJ6345@dhcp22.suse.cz> References: <20200507120217.12313-1-vbabka@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200507120217.12313-1-vbabka@suse.cz> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu 07-05-20 14:02:17, Vlastimil Babka wrote: > During recent patch discussion [1] it became apparent that the "other_node" > definition in the numastat documentation has always been different from actual > implementation. It was also noted that the stats can be innacurate on systems > with memoryless nodes. > > This patch corrects the other_node definition (with minor tweaks to two more > definitions), adds a note about memoryless nodes and also two introductory > paragraphs to the numastat documentation. > > [1] https://lore.kernel.org/linux-mm/20200504070304.127361-1-sandipan@linux.ibm.com/T/#u > > Signed-off-by: Vlastimil Babka Acked-by: Michal Hocko Thanks! > --- > Documentation/admin-guide/numastat.rst | 31 +++++++++++++++++++++++--- > 1 file changed, 28 insertions(+), 3 deletions(-) > > diff --git a/Documentation/admin-guide/numastat.rst b/Documentation/admin-guide/numastat.rst > index aaf1667489f8..08ec2c2bdce3 100644 > --- a/Documentation/admin-guide/numastat.rst > +++ b/Documentation/admin-guide/numastat.rst > @@ -6,6 +6,21 @@ Numa policy hit/miss statistics > > All units are pages. Hugepages have separate counters. > > +The numa_hit, numa_miss and numa_foreign counters reflect how well processes > +are able to allocate memory from nodes they prefer. If they succeed, numa_hit > +is incremented on the preferred node, otherwise numa_foreign is incremented on > +the preferred node and numa_miss on the node where allocation succeeded. > + > +Usually preferred node is the one local to the CPU where the process executes, > +but restrictions such as mempolicies can change that, so there are also two > +counters based on CPU local node. local_node is similar to numa_hit and is > +incremented on allocation from a node by CPU on the same node. other_node is > +similar to numa_miss and is incremented on the node where allocation succeeds > +from a CPU from a different node. Note there is no counter analogical to > +numa_foreign. > + > +In more detail: > + > =============== ============================================================ > numa_hit A process wanted to allocate memory from this node, > and succeeded. > @@ -14,11 +29,13 @@ numa_miss A process wanted to allocate memory from another node, > but ended up with memory from this node. > > numa_foreign A process wanted to allocate on this node, > - but ended up with memory from another one. > + but ended up with memory from another node. > > -local_node A process ran on this node and got memory from it. > +local_node A process ran on this node's CPU, > + and got memory from this node. > > -other_node A process ran on this node and got memory from another node. > +other_node A process ran on a different node's CPU > + and got memory from this node. > > interleave_hit Interleaving wanted to allocate from this node > and succeeded. > @@ -28,3 +45,11 @@ For easier reading you can use the numastat utility from the numactl package > (http://oss.sgi.com/projects/libnuma/). Note that it only works > well right now on machines with a small number of CPUs. > > +Note that on systems with memoryless nodes (where a node has CPUs but no > +memory) the numa_hit, numa_miss and numa_foreign statistics can be skewed > +heavily. In the current kernel implementation, if a process prefers a > +memoryless node (i.e. because it is running on one of its local CPU), the > +implementation actually treats one of the nearest nodes with memory as the > +preferred node. As a result, such allocation will not increase the numa_foreign > +counter on the memoryless node, and will skew the numa_hit, numa_miss and > +numa_foreign statistics of the nearest node. > -- > 2.26.2 -- Michal Hocko SUSE Labs