From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2F4BC54E64 for ; Thu, 28 Mar 2024 09:51:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 588866B0087; Thu, 28 Mar 2024 05:51:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 537596B008A; Thu, 28 Mar 2024 05:51:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3FFE46B008C; Thu, 28 Mar 2024 05:51:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1F5696B0087 for ; Thu, 28 Mar 2024 05:51:54 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id D7E54120DE2 for ; Thu, 28 Mar 2024 09:51:53 +0000 (UTC) X-FDA: 81945981306.07.C8DB00B Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf28.hostedemail.com (Postfix) with ESMTP id 23DF1C0009 for ; Thu, 28 Mar 2024 09:51:51 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gABA8kKP; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf28.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711619512; a=rsa-sha256; cv=none; b=KY6MfGfE+LrVZdoe9gwWWjdFquFZ1jRlCJrOsaglv36V2q7oTKvaLZNdcd1GTT821GECbp +hjR5ODoAeekh6KNjb/q6wenCoVKfUIO1jOZvWLAVTiCLjt5Ga5WWx0Qgx1FlBDYZcf3Ak /3Plq7n9Np46QN6l84uyeU1pynLSBpU= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=gABA8kKP; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf28.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711619512; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=AbTuCU8sWuF995wX6nEUzkH5NjNFGQ3SD4YC2TRHNM8=; b=76RoqisL5zpTkCUexwJUbvnoiqIemvNHRvSeQSYIWPLyrl06Q9u1M909+PpXDByIvfAV2T FrHTwKWh5RSNryUXO6Db7ticwxXxannoXO63/+QnvbudS1Z906j/gZsKqBZb0lqcyjDVo3 eeM8FfWj1u6FARO3KDgS/uv7+4FlQIo= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-1df01161b39so6705165ad.3 for ; Thu, 28 Mar 2024 02:51:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1711619511; x=1712224311; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=AbTuCU8sWuF995wX6nEUzkH5NjNFGQ3SD4YC2TRHNM8=; b=gABA8kKPVliYEOHuGjNXkDznCpUPXHaSVRlzYww/jt3WM5kW7uzhqLHKYaaj4jcl+C yD91/NN1AkGWQAT++QslLObeKodhuti6zfKEdqE+hxzXzwUdGwEpXcrcv502u0pBRcGv zespkVr32RGZY0Tl3CkwXE0Ey+y7blzI6PaqrDHXhiRYN/AQEMcelhcSTtrZjYdnSxCJ 6dWVwoQo8RYvqui9wDm/SPnpft4tF7qV2Cy/8G6jpsPgSNQiQStEyCUOptjRZ5TC990r f1gjeV33rS7tfJAMaotzANOK8BNdNLL1CbKxWrgVMCrBCHGfwlRMeqPIDnxKLDzFuoky WCSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711619511; x=1712224311; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AbTuCU8sWuF995wX6nEUzkH5NjNFGQ3SD4YC2TRHNM8=; b=IejX5y4QHXhVRPoUHHG6dpj141lbqVn6gx2iIaw4qO4wvDBFFTNZmpsnJQekEEerwM BpvgGSaCMH/65KB9IQsUAQ0t9U5RbGnrbtUQc5RPdQVLRZe4Kz/JZt760pC6BDLOT44y ZWsQUHPGWrZfWUAjxnDvQyW6e6XurliHQOvdrYY1KWeKqE8nEfWAn5TcLU0Ka9V3qa67 u7yKy09IKGOaGS/DbbdxY3JBjtG1nvr2aIB6MfBhYF+sCf9VVGVyNMCqZ72RPvPuISE5 0X8nv0Wp0J5jWDdNGF0zA1juvSDBRwPEQI//swjc0cBtmcrvnnNes6e2MIZAYvM3yxeo ZsRQ== X-Forwarded-Encrypted: i=1; AJvYcCXMQ9i2gDZ12voxVR6az5X6U/acsRtR2MfF/+lQ1mA8zE0HOKNfSVVpgSvsMLiqSAERb1rGyxU6jaYOPPFtAIOt0EI= X-Gm-Message-State: AOJu0Yzyy/a0/8ZzqgvZdvGRu5X507e1Keja7i8QnTGwlnGsMB0y3isM 6pFEMI+vHuOQmQtzEhP9aRz2A6u1xMQdXZZSDzo67/bPduvlnUtR X-Google-Smtp-Source: AGHT+IFa0955lP2oGdrAJEscwF8aQVPANmnV6FuzOW+YDzndIJVq6BqQywPmQpWT3JTFEK8DSgLejQ== X-Received: by 2002:a17:903:98f:b0:1dd:7e30:4b15 with SMTP id mb15-20020a170903098f00b001dd7e304b15mr2779718plb.29.1711619510774; Thu, 28 Mar 2024 02:51:50 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id s2-20020a170902b18200b001ddb505d50asm1116432plr.244.2024.03.28.02.51.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Mar 2024 02:51:50 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: willy@infradead.org, david@redhat.com, ryan.roberts@arm.com, cerasuolodomenico@gmail.com, kasong@tencent.com, surenb@google.com, v-songbaohua@oppo.com, yosryahmed@google.com, yuzhao@google.com, chrisl@kernel.org, peterx@redhat.com Subject: [PATCH v2] mm: add per-order mTHP alloc_success and alloc_fail counters Date: Thu, 28 Mar 2024 22:51:39 +1300 Message-Id: <20240328095139.143374-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 23DF1C0009 X-Stat-Signature: kfetbydjhr454foa7gfmhi8h5rzt74mz X-HE-Tag: 1711619511-77953 X-HE-Meta: U2FsdGVkX19LK1/x1co0shzry8f2xJld91zcKquL4WvO63tljVySt58lvtm8NKOvD3JzE8k8hauKWdzcaqAINJGkoXiZvPTqRWqvhyKTZt8jTK0Ky2IstG7kwZkr7OwzJWmatiGAVXyIEdTvfnOF7niOfAV6r0ivsuyoOXaEqfuDqIdICa8t+gS1pjoxKgNHwI61hj2WeH/38o0MV3EwN2GBmWiLo2JT3dD3Gljw1uOVP/wrpQ4kloS9ON4xWXFiyheFADW8iyoTJxEhvVNYupHhHSRLOgVqicWwo3e3v+yA3Yf7orLXv2e/EM/kH6qApDm6FwhZNqWwqlaenY9K5OlXD069FF4tH2hClHw5FOQfK6IdcNdkfdfGz28U14pwyV59Y8EalG9e1ccqY5ovqS5aZ5W40k1yqvyB+ZPRDcF8/ijPkMfbOkQ+f7SCpS6UHoexOYyDXAfJYeWffH10YpPpZuGKYUadScQSUx5OWrP1KuTDUBeoPjKDIulEjzMYLnPyDg0Ywu7uRrxvCUSGSS5K3x4/aAMqcYUeyh3teYeadf2isGqKAwLxzFKj17mItfq8YOMq6qcs+4o/vNvIpYo/zc2Sd6UAbzWuKC7bIc7s8IOVB/zffRQlFLYwCmSngN0OTc2F+elNVEP+cBy9f4W0tWg0WqNyVT4kAHCddIxXzFcoABPL+dlv+STl7sLAkqkKWee7d33rL+KiLDK6Pnacrge3izBFOHH72tuZKtJtm4r2CQEPGaeDxWZATqg/nNGLCb/fgOhOF4i3N2/dVKBbiGop8p7uoD35kg5E5Uaudj35xO++O0rjZVphBRMb8KfORpDTUMODfTw68F38OL3JhjFFKa7ouJdksSF5NCfLH0xcn2r5XWqoh57e4NGWt4FjfM3fhhs077xbRrha1S++upWcFu01bd0T+CI27Sq7q4M9pnO/VRg8LcZHShrBln3Z0nsef0QkkQJaV3l cB89GZYO +XvbX8yzuWIuc4cb0GnVOiahOmav/ErpkHTL114+/JXYPg6ZdabU6WBiIOhXqZKKeVu0+FJUdMK4fBiSjEo9dD/JWNPsuQL5VxIXwfymP0v9FA+RxJfPtRYH9UQAw0fbxLKSmtHbZ8XFJYgqaxXY6PCn0EvgFy3XensAPIfckVqJwakrF2Q9Yr+ZsXOJg3FGab7RLt0NnYmqZTFnYsc5mahVVgChgRN9AQjEBC7jlQpv/poqy6KI7U+LV6sFmUaDihMAhvl6VfyOMxq8xnii5VvZktBxig9HlH6Oh4ErUYOMG9iEhHPdRVoU5WZ0TnhPQzAvMwI6hib65pvphMNnLP2WPayw7mUHzvVSIhaDS/3JHkuf5YgGI670AbreXXn3R6OyT9oYp1B4Z1QNvR84RqEoKn6r5hFyrfVCIUUCMfd2X54JCPIbxEp2LHPUoSNYagjOsU4vE3msRymQvT4at6Frinnj1/f5E21P/WZLDNVcyVL9jbB1g+uJexsOQy9ZJ1EuHT+Od21ef3iz3tVMr8hnlmW3wZHgjlb0Zqpx9OOyIPfv0FZSJZdc4EuHlKBWVWztP/rJwMSjYC1F/pbcGO8jFBFgLRh7a6QcOKe2JEQcZwyaWruOCbPcnDg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song Profiling a system blindly with mTHP has become challenging due to the lack of visibility into its operations. Presenting the success rate of mTHP allocations appears to be pressing need. Recently, I've been experiencing significant difficulty debugging performance improvements and regressions without these figures. It's crucial for us to understand the true effectiveness of mTHP in real-world scenarios, especially in systems with fragmented memory. This patch sets up the framework for per-order mTHP counters, starting with the introduction of alloc_success and alloc_fail counters. Incorporating additional counters should now be straightforward as well. The initial two unsigned longs for each event are unused, given that order-0 and order-1 are not mTHP. Nonetheless, this refinement improves code clarity. Signed-off-by: Barry Song --- -v2: * move to sysfs and provide per-order counters; David, Ryan, Willy -v1: https://lore.kernel.org/linux-mm/20240326030103.50678-1-21cnbao@gmail.com/ include/linux/huge_mm.h | 17 +++++++++++++ mm/huge_memory.c | 54 +++++++++++++++++++++++++++++++++++++++++ mm/memory.c | 3 +++ 3 files changed, 74 insertions(+) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index e896ca4760f6..27fa26a22a8f 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -264,6 +264,23 @@ unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma, enforce_sysfs, orders); } +enum thp_event_item { + THP_ALLOC_SUCCESS, + THP_ALLOC_FAIL, + NR_THP_EVENT_ITEMS +}; + +struct thp_event_state { + unsigned long event[PMD_ORDER + 1][NR_THP_EVENT_ITEMS]; +}; + +DECLARE_PER_CPU(struct thp_event_state, thp_event_states); + +static inline void count_thp_event(int order, enum thp_event_item item) +{ + this_cpu_inc(thp_event_states.event[order][item]); +} + #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ (1<event[order][item]; + } + + return sum; +} + +static ssize_t alloc_success_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + int order = to_thpsize(kobj)->order; + + return sysfs_emit(buf, "%lu\n", sum_thp_events(order, THP_ALLOC_SUCCESS)); +} + +static ssize_t alloc_fail_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + int order = to_thpsize(kobj)->order; + + return sysfs_emit(buf, "%lu\n", sum_thp_events(order, THP_ALLOC_FAIL)); +} + +static struct kobj_attribute alloc_success_attr = __ATTR_RO(alloc_success); +static struct kobj_attribute alloc_fail_attr = __ATTR_RO(alloc_fail); + +static struct attribute *stats_attrs[] = { + &alloc_success_attr.attr, + &alloc_fail_attr.attr, + NULL, +}; + +static struct attribute_group stats_attr_group = { + .name = "stats", + .attrs = stats_attrs, +}; + static struct thpsize *thpsize_create(int order, struct kobject *parent) { unsigned long size = (PAGE_SIZE << order) / SZ_1K; @@ -549,6 +595,12 @@ static struct thpsize *thpsize_create(int order, struct kobject *parent) return ERR_PTR(ret); } + ret = sysfs_create_group(&thpsize->kobj, &stats_attr_group); + if (ret) { + kobject_put(&thpsize->kobj); + return ERR_PTR(ret); + } + thpsize->order = order; return thpsize; } @@ -1050,8 +1102,10 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) folio = vma_alloc_folio(gfp, HPAGE_PMD_ORDER, vma, haddr, true); if (unlikely(!folio)) { count_vm_event(THP_FAULT_FALLBACK); + count_thp_event(HPAGE_PMD_ORDER, THP_ALLOC_FAIL); return VM_FAULT_FALLBACK; } + count_thp_event(HPAGE_PMD_ORDER, THP_ALLOC_SUCCESS); return __do_huge_pmd_anonymous_page(vmf, &folio->page, gfp); } diff --git a/mm/memory.c b/mm/memory.c index 984b138f85b4..c9c1031c2ecb 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4365,7 +4365,10 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf) } folio_throttle_swaprate(folio, gfp); clear_huge_page(&folio->page, vmf->address, 1 << order); + count_thp_event(order, THP_ALLOC_SUCCESS); return folio; + } else { + count_thp_event(order, THP_ALLOC_FAIL); } next: order = next_order(&orders, order); -- 2.34.1