From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D95FDC4724C for ; Thu, 7 May 2020 12:02:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8F3A321473 for ; Thu, 7 May 2020 12:02:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8F3A321473 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 35D20900003; Thu, 7 May 2020 08:02:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E6D0900002; Thu, 7 May 2020 08:02:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D580900003; Thu, 7 May 2020 08:02:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0164.hostedemail.com [216.40.44.164]) by kanga.kvack.org (Postfix) with ESMTP id F366B900002 for ; Thu, 7 May 2020 08:02:35 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 31B5E180AD82F for ; Thu, 7 May 2020 12:02:34 +0000 (UTC) X-FDA: 76789785828.12.books30_8acb070f58a63 X-HE-Tag: books30_8acb070f58a63 X-Filterd-Recvd-Size: 4463 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Thu, 7 May 2020 12:02:33 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 7FD70ADC8; Thu, 7 May 2020 12:02:34 +0000 (UTC) From: Vlastimil Babka To: Jonathan Corbet Cc: linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Michal Hocko , Sandipan Das , Vlastimil Babka Subject: [PATCH] Documentation: update numastat explanation Date: Thu, 7 May 2020 14:02:17 +0200 Message-Id: <20200507120217.12313-1-vbabka@suse.cz> X-Mailer: git-send-email 2.26.2 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: During recent patch discussion [1] it became apparent that the "other_nod= e" definition in the numastat documentation has always been different from a= ctual implementation. It was also noted that the stats can be innacurate on sys= tems with memoryless nodes. This patch corrects the other_node definition (with minor tweaks to two m= ore definitions), adds a note about memoryless nodes and also two introductor= y paragraphs to the numastat documentation. [1] https://lore.kernel.org/linux-mm/20200504070304.127361-1-sandipan@lin= ux.ibm.com/T/#u Signed-off-by: Vlastimil Babka --- Documentation/admin-guide/numastat.rst | 31 +++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/Documentation/admin-guide/numastat.rst b/Documentation/admin= -guide/numastat.rst index aaf1667489f8..08ec2c2bdce3 100644 --- a/Documentation/admin-guide/numastat.rst +++ b/Documentation/admin-guide/numastat.rst @@ -6,6 +6,21 @@ Numa policy hit/miss statistics =20 All units are pages. Hugepages have separate counters. =20 +The numa_hit, numa_miss and numa_foreign counters reflect how well proce= sses +are able to allocate memory from nodes they prefer. If they succeed, num= a_hit +is incremented on the preferred node, otherwise numa_foreign is incremen= ted on +the preferred node and numa_miss on the node where allocation succeeded. + +Usually preferred node is the one local to the CPU where the process exe= cutes, +but restrictions such as mempolicies can change that, so there are also = two +counters based on CPU local node. local_node is similar to numa_hit and = is +incremented on allocation from a node by CPU on the same node. other_nod= e is +similar to numa_miss and is incremented on the node where allocation suc= ceeds +from a CPU from a different node. Note there is no counter analogical to +numa_foreign. + +In more detail: + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D numa_hit A process wanted to allocate memory from this node, and succeeded. @@ -14,11 +29,13 @@ numa_miss A process wanted to allocate memory from an= other node, but ended up with memory from this node. =20 numa_foreign A process wanted to allocate on this node, - but ended up with memory from another one. + but ended up with memory from another node. =20 -local_node A process ran on this node and got memory from it. +local_node A process ran on this node's CPU, + and got memory from this node. =20 -other_node A process ran on this node and got memory from another node. +other_node A process ran on a different node's CPU + and got memory from this node. =20 interleave_hit Interleaving wanted to allocate from this node and succeeded. @@ -28,3 +45,11 @@ For easier reading you can use the numastat utility fr= om the numactl package (http://oss.sgi.com/projects/libnuma/). Note that it only works well right now on machines with a small number of CPUs. =20 +Note that on systems with memoryless nodes (where a node has CPUs but no +memory) the numa_hit, numa_miss and numa_foreign statistics can be skewe= d +heavily. In the current kernel implementation, if a process prefers a +memoryless node (i.e. because it is running on one of its local CPU), t= he +implementation actually treats one of the nearest nodes with memory as t= he +preferred node. As a result, such allocation will not increase the numa_= foreign +counter on the memoryless node, and will skew the numa_hit, numa_miss an= d +numa_foreign statistics of the nearest node. --=20 2.26.2