From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF255C00144 for ; Mon, 1 Aug 2022 13:28:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3DB218E0001; Mon, 1 Aug 2022 09:28:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 38AC96B0072; Mon, 1 Aug 2022 09:28:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 252788E0001; Mon, 1 Aug 2022 09:28:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 14E336B0071 for ; Mon, 1 Aug 2022 09:28:21 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id DA765120ADB for ; Mon, 1 Aug 2022 13:28:20 +0000 (UTC) X-FDA: 79751102760.10.5F00039 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf03.hostedemail.com (Postfix) with ESMTP id 54C15200F8 for ; Mon, 1 Aug 2022 13:28:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1659360499; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9HeT4J2pwN84U0RWrWPAlfTJpr5y0EVb+TVvboAarr4=; b=hCamkqLDwQbEplQglLNvN1DCXzTN+LUTuS6g0m7/0c5RHQi1Yi3GAcKqBum2yik38SNJRO 1gaNaD20hS+tQ8dLZOfI0oJdIM7+oIgw0rcz8Q6clEQd07gw8AzFmIJjfVQI6cgCow4hOy oX1r3inix1Pf6GEi5Nf4AkxINYC3NPE= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-654-QPzCfNjLNdu15a9merzogw-1; Mon, 01 Aug 2022 09:28:18 -0400 X-MC-Unique: QPzCfNjLNdu15a9merzogw-1 Received: by mail-wr1-f72.google.com with SMTP id n7-20020adfc607000000b0021a37d8f93aso2593318wrg.21 for ; Mon, 01 Aug 2022 06:28:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent :content-language:to:cc:references:from:organization:subject :in-reply-to:content-transfer-encoding; bh=9HeT4J2pwN84U0RWrWPAlfTJpr5y0EVb+TVvboAarr4=; b=ykWXWHrH2vyfKdwsn+t4+eP215jpe8oOk9Gya90N2LhCZyYo8K7DpMmVQWGeZYgcjT mQh1aNnayRDyCPGpgNgDdVJNS80sIujY6WnFcOHmLmhIH7e0fBXdCVW/d6P+5o6IytCw cYXWeezyxrLQYqc1wgyu9Cl5ho8atMDpe/xt4wdyPznvl2sFa48iqXNE3yNInvh5WJhw mLlsgfvnH1ArHcONgHycrCetkOLo2qPkinZQcZV19ljOYXOmez57KRfnxeG6lhmuxPL5 uPsTFuzaeLMzukL8shTcUWxq8ex9za5bhtojteP58gxHXmLGE2GhSPWBynG9ZFelysI5 iS2w== X-Gm-Message-State: ACgBeo0VFvuKqIX111PSR/fBD1Z+or9dRFNF1lntaZKNQ50zYeEypCL5 SQfb1uSgApwt0tDMUiSJjml3WrRMIxjug87mfPh96fslDmlq1fs9djApDPze+lgjoYa9QweCfpL JZk04pf3+wPw= X-Received: by 2002:adf:e881:0:b0:21f:abb:fc0a with SMTP id d1-20020adfe881000000b0021f0abbfc0amr9692045wrm.103.1659360497232; Mon, 01 Aug 2022 06:28:17 -0700 (PDT) X-Google-Smtp-Source: AA6agR7UYK7ZDuMNgtMyeC5Mwc+2jyxv+KxRym0qux7n9+Y6OT4H9DgkvxuzUxC/BH0QrzhH/lRuFA== X-Received: by 2002:adf:e881:0:b0:21f:abb:fc0a with SMTP id d1-20020adfe881000000b0021f0abbfc0amr9692021wrm.103.1659360496952; Mon, 01 Aug 2022 06:28:16 -0700 (PDT) Received: from ?IPV6:2003:cb:c704:6900:6d08:8df1:dd2c:bf00? (p200300cbc70469006d088df1dd2cbf00.dip0.t-ipconnect.de. [2003:cb:c704:6900:6d08:8df1:dd2c:bf00]) by smtp.gmail.com with ESMTPSA id z12-20020a05600c03cc00b003a2eacc8179sm14598761wmd.27.2022.08.01.06.28.16 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 01 Aug 2022 06:28:16 -0700 (PDT) Message-ID: <0c1f9e76-9b1d-7069-bb09-c18e4f19f0c4@redhat.com> Date: Mon, 1 Aug 2022 15:28:15 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 To: Xin Hao , willy@infradead.org Cc: akpm@linux-foundation.org, adobriyan@gmail.com, keescook@chromium.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20220801121727.76186-1-xhao@linux.alibaba.com> <20220801121727.76186-2-xhao@linux.alibaba.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [RFC PATCH V4 1/1] mm: add last level page table numa info to /proc/pid/numa_pgtable In-Reply-To: <20220801121727.76186-2-xhao@linux.alibaba.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659360500; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9HeT4J2pwN84U0RWrWPAlfTJpr5y0EVb+TVvboAarr4=; b=f8t2MFqfZtZdVzVL3GBcwpjGLgsx7Z9aVOYo3M+1TbZbHQDMihy2iPGkfy2i0d9xhB1Me3 dKnb2VskjrlRE3MdOPfBxqxz+qHiJJBkT+RiFuRQ8YF4Vp1L5rhm63fNFth28YXnEIiskO GVh1d8+KoX9swzZdD/+LWTn4pEgkGnA= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hCamkqLD; spf=pass (imf03.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659360500; a=rsa-sha256; cv=none; b=UzyxSapnWvbGSyEMaQAtzzV/3HOXwuvNc8eqi0VrU5ruy8OHdnY13Ku4p/8t4QsmLL36Bk 9Wjsl5hlqDY+apsz96HKJmZo4g022+I1oKSVvS2gVCkUP1/lNg5wENuvlcQeW6VDYLO43z P2yM5RIvR67Pi/8k8ubl3gAKa5kpo+8= X-Rspam-User: X-Stat-Signature: njgr8jgk8utuoynif3x4s6i7h5r6tsq6 X-Rspamd-Queue-Id: 54C15200F8 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hCamkqLD; spf=pass (imf03.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam08 X-HE-Tag: 1659360500-373161 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 01.08.22 14:17, Xin Hao wrote: > In many data center servers, the shared memory architectures is > Non-Uniform Memory Access (NUMA), remote numa node data access > often brings a high latency problem, but what we are easy to ignore > is that the page table remote numa access, It can also leads to a > performance degradation. Let me try rewriting: " Many data center servers employ Non-Uniform Memory Access (NUMA) architectures. Remote numa memory access results in high latency. While memory placement is one issue, sub-optimal page table placement can also result in surprise performance degradation. " > > So there add a new interface in /proc, This will help developers to > get more info about performance issues if they are caused by cross-NUMA. Why do we only care about "last level page table", why not about the others? IMHO, we could emit something like "0, 1, 3, 0" instead for a given user space address, showing the NUMA node the page table belongs to from highest to lowest page table level. > > Reported-by: kernel test robot The kernel test robot reported that we need "/proc/pid/numa_pgtable" ?! :) Just drop that unless it's a follow-up fix. > Signed-off-by: Xin Hao > --- > fs/proc/base.c | 2 ++ > fs/proc/internal.h | 1 + > fs/proc/task_mmu.c | 87 ++++++++++++++++++++++++++++++++++++++++++++++ > 3 files changed, 90 insertions(+) > > diff --git a/fs/proc/base.c b/fs/proc/base.c > index 8dfa36a99c74..487e82dd3275 100644 > --- a/fs/proc/base.c > +++ b/fs/proc/base.c > @@ -3224,6 +3224,7 @@ static const struct pid_entry tgid_base_stuff[] = { > REG("maps", S_IRUGO, proc_pid_maps_operations), > #ifdef CONFIG_NUMA > REG("numa_maps", S_IRUGO, proc_pid_numa_maps_operations), > + REG("numa_pgtable", S_IRUGO, proc_pid_numa_pgtable_operations), > #endif > REG("mem", S_IRUSR|S_IWUSR, proc_mem_operations), > LNK("cwd", proc_cwd_link), > @@ -3571,6 +3572,7 @@ static const struct pid_entry tid_base_stuff[] = { > #endif > #ifdef CONFIG_NUMA > REG("numa_maps", S_IRUGO, proc_pid_numa_maps_operations), > + REG("numa_pgtable", S_IRUGO, proc_pid_numa_pgtable_operations), > #endif > REG("mem", S_IRUSR|S_IWUSR, proc_mem_operations), > LNK("cwd", proc_cwd_link), > diff --git a/fs/proc/internal.h b/fs/proc/internal.h > index 06a80f78433d..e7ed9ef097b6 100644 > --- a/fs/proc/internal.h > +++ b/fs/proc/internal.h > @@ -296,6 +296,7 @@ struct mm_struct *proc_mem_open(struct inode *inode, unsigned int mode); > > extern const struct file_operations proc_pid_maps_operations; > extern const struct file_operations proc_pid_numa_maps_operations; > +extern const struct file_operations proc_pid_numa_pgtable_operations; > extern const struct file_operations proc_pid_smaps_operations; > extern const struct file_operations proc_pid_smaps_rollup_operations; > extern const struct file_operations proc_clear_refs_operations; > diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c > index 2d04e3470d4c..77b7a49757f5 100644 > --- a/fs/proc/task_mmu.c > +++ b/fs/proc/task_mmu.c > @@ -1999,4 +1999,91 @@ const struct file_operations proc_pid_numa_maps_operations = { > .release = proc_map_release, > }; > > +struct pgtable_numa_private { > + struct proc_maps_private proc_maps; > + unsigned long node[MAX_NUMNODES]; > +}; > + > +static int gather_pgtable_numa_stats(pmd_t *pmd, unsigned long addr, > + unsigned long end, struct mm_walk *walk) > +{ > + struct pgtable_numa_private *priv = walk->private; > + struct page *page; > + int nid; > + > + if (pmd_huge(*pmd)) { > + page = virt_to_page(pmd); > + } else { > + page = pmd_page(*pmd); > + } > + > + nid = page_to_nid(page); > + priv->node[nid]++; > + > + return 0; > +} > + > +static const struct mm_walk_ops show_numa_pgtable_ops = { > + .pmd_entry = gather_pgtable_numa_stats, > +}; > + > +/* > + * Display the page talbe allocated per node via /proc. s/talbe/table/ but the comment somehow doesn't make sense. We don't display a page table. > + */ > +static int show_numa_pgtable(struct seq_file *m, void *v) > +{ > + struct pgtable_numa_private *numa_priv = m->private; > + struct vm_area_struct *vma = v; > + struct mm_struct *mm = vma->vm_mm; > + struct file *file = vma->vm_file; > + int nid; > + > + if (!mm) > + return 0; > + > + memset(numa_priv->node, 0, sizeof(numa_priv->node)); > + > + seq_printf(m, "%08lx ", vma->vm_start); > + > + if (file) { > + seq_puts(m, " file="); > + seq_file_path(m, file, "\n\t= "); > + } else if (vma->vm_start <= mm->brk && vma->vm_end >= mm->start_brk) { > + seq_puts(m, " heap"); > + } else if (is_stack(vma)) { > + seq_puts(m, " stack"); > + } > + > + /* mmap_lock is held by m_start */ > + walk_page_vma(vma, &show_numa_pgtable_ops, numa_priv); > + > + for_each_node_state(nid, N_MEMORY) { > + if (numa_priv->node[nid]) > + seq_printf(m, " N%d=%lu", nid, numa_priv->node[nid]); > + } > + seq_putc(m, '\n'); > + > + return 0; > +} > + > +static const struct seq_operations proc_pid_numa_pgtable_op = { > + .start = m_start, > + .next = m_next, > + .stop = m_stop, > + .show = show_numa_pgtable, > +}; > + > +static int pid_numa_pgtable_open(struct inode *inode, struct file *file) > +{ > + return proc_maps_open(inode, file, &proc_pid_numa_pgtable_op, > + sizeof(struct pgtable_numa_private)); > +} > + > +const struct file_operations proc_pid_numa_pgtable_operations = { > + .open = pid_numa_pgtable_open, > + .read = seq_read, > + .llseek = seq_lseek, > + .release = proc_map_release, > +}; > + > #endif /* CONFIG_NUMA */ > -- > 2.31.0 > -- Thanks, David / dhildenb