From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56CDBCD1292 for ; Mon, 1 Apr 2024 14:46:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C22AC6B0089; Mon, 1 Apr 2024 10:46:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD2766B008A; Mon, 1 Apr 2024 10:46:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A9A206B008C; Mon, 1 Apr 2024 10:46:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8D6726B0089 for ; Mon, 1 Apr 2024 10:46:55 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 382C8C0744 for ; Mon, 1 Apr 2024 14:46:55 +0000 (UTC) X-FDA: 81961239990.09.C96EC65 Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) by imf23.hostedemail.com (Postfix) with ESMTP id 59538140024 for ; Mon, 1 Apr 2024 14:46:53 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CtitCy6r; spf=pass (imf23.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.50 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711982813; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=w0i1loy1PtBydQybSVTedNI2psegIDa6Olwwmuq/ycE=; b=Zc7NxBGh0F1KZR34ReU2yxDeKYEfqJPiFjrPI1YXnWtYTo0N0EouuYSHhJeOHk2RZjysQm GCrBncDqeL60Oli6z5Kyjq7S6UgPbvaoaThtiARELt3ce7V2vbEGZFj8iu+MaHRim87oQv Z9UQTdyfu6vzRbm8nX76qjHdT0XCP6w= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711982813; a=rsa-sha256; cv=none; b=NY+ntXqIxNqlnlU/DHTDeAooksyXnshiHcL3tcsCswX/1X2kdn9QouAz+S3DE4/4UWFWDO qELfUuIyT715r5ZnGRI8RDT1GW5R+FtdZo7iNgH2VL20dDMvlMhUrrjiO4QyXSUkODn0TG YqT4qYChkWP4bY0hJqW96xZJix6iGRY= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CtitCy6r; spf=pass (imf23.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.50 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-4154d38ce9dso188155e9.0 for ; Mon, 01 Apr 2024 07:46:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1711982812; x=1712587612; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=w0i1loy1PtBydQybSVTedNI2psegIDa6Olwwmuq/ycE=; b=CtitCy6rKDttGjP6kU5vxzyBYKCkqwXCMT9O5hlUXAcpVTlM/tVv5jhto9px6nKLjn B+0VjlawlIwPs+Nlw+Evrzy3NUAFCybXxlqZb5hoYiROY8x+WCikmmQBFfpKAfXnMiC2 EAavixu5HCuXpn/DJrnSjyHWDJLX98aMPm27GE6VuGIIP/dX6/eKEWKw7kwKYT5SBGGf BNx1HOnFmDXTcQOOJqsplBC5O8/wwderVCkNGn4iWYsXcVlU5IjCDhKHCVUCWsW0+QZB FNLboR9hwt8W1jqKRTx25n4eKuEa+FHwJl6RJBcBUYZRmLcs2nzU69OJunyk03ftJF78 LpeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711982812; x=1712587612; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=w0i1loy1PtBydQybSVTedNI2psegIDa6Olwwmuq/ycE=; b=beUi0zTEzIyLlzPrmPK7T3sIHIEigNzNjm7HkGSQn/2VZAQOytcBrNwQnHEcrOVnzO 6LUwc+UrIXsCxUrjODG85oMw92Jq8VGu3jeTZO8Xrwbb2Pl81FFBOUWE3cVE6sh9hA4B oXFL0QO0LpIa6UOlCDLP3jXsgdFRy0+c/LwDD+0YkAaiAI0ybmcpgRn8z7x8byIHCpe3 QKnIxFHniwIRGwJ35jD9kjlOvVjeCCXKKGo7fAnnTIX1w/wYr7SsOexDXRcM0hTt6Yxc rSbwKYzTZrzI5WVV7Af4aqNWDWyrr97z0ddxQgKVF0MebOoOC8jy2ha1oVjpix6RTmh+ dRMQ== X-Forwarded-Encrypted: i=1; AJvYcCXRDVwJYjDTOc3WfphUObJ+tL0UDoef7d/JcdlwaYFmVom9R8W6wsm7JMkFSFXDMklHAMd9vHUogc34wEwd1SlMiVU= X-Gm-Message-State: AOJu0YzkzSM7b0pARGq7ju6beLObahlzXdu6EstKjV9OVf26vNj16QiM uaR6cjgKr1PKt5Sa2hmaxODsi0f7w8ju0XWa4kJcYqV9FDBrutfc99+O8fZ+eonHziqPEU6cfum jOWMu1GyMbknht1nt2udRheoKC1qrkSscBVY/ X-Google-Smtp-Source: AGHT+IGkCelZ4Pg29XWrF958XnThlqbcR2HJ9EaF1ieZ/FqWEQFotriv1GI3CH/ePOQBIlHnUzPQ8juVJhVaJzMfSZc= X-Received: by 2002:a05:600c:3c93:b0:414:1400:a776 with SMTP id bg19-20020a05600c3c9300b004141400a776mr518047wmb.5.1711982811707; Mon, 01 Apr 2024 07:46:51 -0700 (PDT) MIME-Version: 1.0 References: <20240328095139.143374-1-21cnbao@gmail.com> In-Reply-To: <20240328095139.143374-1-21cnbao@gmail.com> From: Yu Zhao Date: Mon, 1 Apr 2024 10:46:12 -0400 Message-ID: Subject: Re: [PATCH v2] mm: add per-order mTHP alloc_success and alloc_fail counters To: Barry Song <21cnbao@gmail.com> Cc: akpm@linux-foundation.org, linux-mm@kvack.org, willy@infradead.org, david@redhat.com, ryan.roberts@arm.com, cerasuolodomenico@gmail.com, kasong@tencent.com, surenb@google.com, v-songbaohua@oppo.com, yosryahmed@google.com, chrisl@kernel.org, peterx@redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 59538140024 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 36g7g4s9ydmtcdfhpepzzh81wstm13mj X-HE-Tag: 1711982813-8982 X-HE-Meta: U2FsdGVkX1+S1BaPz84w+K5QgZ2fwIOSSUmhMbERnHsRHV3UGV+7opkL2w3LCOeal03LSBNZBTgBdWWX3eYxThfBmsK2tfXVkL/D//7b2n2N5cHE2l9nS/GsnWpps7Nfg/SQl7q1ytLI7VFYmYKLQ0RNcTEMDhA09TNuS8v9/mujK0hX8YrvFbX+Q7fgvS/jVEeFUJqpAcFZDub8hwwglyNsR7cjEu9vAkfRR9vKL0MgDLhALhfeS1Tf4RbPbvWGkQVlxbKR29sS+SDbj6ZxCkZv+6M0rYT1KT1w9PxaDAuTKDtiv/wObes5/ZT0QAKX/dPHsQqUK1FIjSvuX6Uo7mpKJvMiMqj5nvOcThPRnOWBamFDm+0djmjqzzdPsi1Y0XBQAb8UitdDXvki9LSQsT1759azKPxRZ17fsyfVdbUrbqLZJQZBLbexRoRMwSXgh/af8ubDEa6OMWhGL3oS2dxwf+a7wbjTJf4U1MOV2ZcS//3CKrF17DNe5Lxh8CfxZg3/DyOsOG29wos3y+w1EczVEi2HzCZRxLBwIpa3LFw+gS7vP8yR24bq6V6b/tBr9xslJt21SctSwytzUijviXHtpvKLZSk9chAhxlWQkCKP+oKnNRt5sYSjnU/ytxgGh+DORKoI9oD8HRegfYm9Aii1gQ7dTDkrfQxxxmZvEdXqUfu6zcsndfbz7xHWlWgO11KYhnwctDUw4IlawXes8Hcl0Cz9Adv2TnG3yZmtFlDMqAyz4+tu7Zz2yZe3iQOV7RpD48zS6NyEx05YHhhFYHcy/wj4tt4aNGpJF16POv3U274lOUJjXeBYG3RH07Tf7/bm6E4H19f8jmdhjBursRuFYc2olX0SJWcQWE1NHUbVzXIXCyqMY39PFgHN9K3HKQj2qJ6qlYBut8Jw0tt+9JipoqnZ4Jcbx0n3Ek/TwbdzWtWGFkILf/m35z2XGhKaTvJ+p/Btz4AWqSLeHqZ HNRQIDdF a9/J+7lsFMR8ZIVrtWll1BzqXPqqg/lIU+bf6gT6yEvR7u2o41CdS/1AjkyqPeXn6McVesRBtB/gRfx1bNRooM0/kNZWqCQGjYCUWMKEZwWcEzudHIGVl+gJkTu9mL5N5YgEvdvAVZTqF+4selrrQ9Z7GxNHCjGrIRsPJrcQkw2sCL/azOqPxIvVgCum/r3mscuR+sj1ON2N1LbzPXwNEfP3ocgn9putyk7XMQjmzeQ0H/KdpRcG5wccXnFOPw3s8LPQnoq0jKY/XvFj/nPaSjrOOBQPlOQ+OB7mIHhnz/fEwEogfd+zFCejLxnY6N6Qs8MtoCFpJSbyfvTvJcoFFBJBkDgQZ0aRhsJVXIC0ig4fCil/k5NyzLae/d3rzUA9w+y+k1lAw+nhXLCcyH77Qvg5i32dbK1z67v76 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Mar 28, 2024 at 5:51=E2=80=AFAM Barry Song <21cnbao@gmail.com> wrot= e: > > From: Barry Song > > Profiling a system blindly with mTHP has become challenging due > to the lack of visibility into its operations. Presenting the > success rate of mTHP allocations appears to be pressing need. > > Recently, I've been experiencing significant difficulty debugging > performance improvements and regressions without these figures. > It's crucial for us to understand the true effectiveness of > mTHP in real-world scenarios, especially in systems with > fragmented memory. > > This patch sets up the framework for per-order mTHP counters, > starting with the introduction of alloc_success and alloc_fail > counters. Incorporating additional counters should now be > straightforward as well. > > The initial two unsigned longs for each event are unused, given > that order-0 and order-1 are not mTHP. Nonetheless, this refinement > improves code clarity. > > Signed-off-by: Barry Song > --- > -v2: > * move to sysfs and provide per-order counters; David, Ryan, Willy > -v1: > https://lore.kernel.org/linux-mm/20240326030103.50678-1-21cnbao@gmail.co= m/ > > include/linux/huge_mm.h | 17 +++++++++++++ > mm/huge_memory.c | 54 +++++++++++++++++++++++++++++++++++++++++ > mm/memory.c | 3 +++ > 3 files changed, 74 insertions(+) > > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h > index e896ca4760f6..27fa26a22a8f 100644 > --- a/include/linux/huge_mm.h > +++ b/include/linux/huge_mm.h > @@ -264,6 +264,23 @@ unsigned long thp_vma_allowable_orders(struct vm_are= a_struct *vma, > enforce_sysfs, orders); > } > > +enum thp_event_item { > + THP_ALLOC_SUCCESS, > + THP_ALLOC_FAIL, > + NR_THP_EVENT_ITEMS > +}; > + > +struct thp_event_state { > + unsigned long event[PMD_ORDER + 1][NR_THP_EVENT_ITEMS]; > +}; > + > +DECLARE_PER_CPU(struct thp_event_state, thp_event_states); Do we have existing per-CPU counters that cover all possible THP orders? I.e., foo_counter[PMD_ORDER + 1][BAR_ITEMS]. I don't think we do but I want to double check. This might be fine if BAR_ITEMS is global, not per memcg. Otherwise on larger systems, e.g., 512 CPUs which is not uncommon, we'd have high per-CPU memory overhead. For Google's datacenters, per-CPU memory overhead has been a problem. I'm not against this patch since NR_THP_EVENT_ITEMS is not per memcg. Alternatively, we could make the per-CPU counters to track only one order and flush the local counter to a global atomic counter if the new order doesn't match the existing order stored in the local counter. WDYT?