From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45331C46CD3 for ; Wed, 20 Dec 2023 15:42:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D8D8B8D0006; Wed, 20 Dec 2023 10:42:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D3D378D0001; Wed, 20 Dec 2023 10:42:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2B6E8D0006; Wed, 20 Dec 2023 10:42:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AF0928D0001 for ; Wed, 20 Dec 2023 10:42:20 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 7FE25160517 for ; Wed, 20 Dec 2023 15:42:20 +0000 (UTC) X-FDA: 81587613240.14.09EAF00 Received: from gentwo.org (gentwo.org [62.72.0.81]) by imf20.hostedemail.com (Postfix) with ESMTP id D103F1C0005 for ; Wed, 20 Dec 2023 15:42:18 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=linux.com (policy=none); spf=softfail (imf20.hostedemail.com: 62.72.0.81 is neither permitted nor denied by domain of cl@linux.com) smtp.mailfrom=cl@linux.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703086939; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AyBmT4GxwkZacKn1E9wvfLuZaHSUMVoSVz4QCc6anUs=; b=8XLpIQTpX40tPVr2aKTFJvO0Eil+wNoi4LqzpWq0QX6QC7qxCE1Y4lFTo1qVV5Vlp4801L 5es+FWQvAg4MVh3hsyeNT1D+tZ2G/Oc5SDByylwxaTdhnBYc+b0tbeHwrswWl1f/IAGjvC Vqiq7EJuE1Hci6XxT0vSZS60u2DR/Yo= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=linux.com (policy=none); spf=softfail (imf20.hostedemail.com: 62.72.0.81 is neither permitted nor denied by domain of cl@linux.com) smtp.mailfrom=cl@linux.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703086939; a=rsa-sha256; cv=none; b=WI080GaZfLnntoAvft8AWp+Q8IOJS0lUl2m6hA4zRSyYQ1n4e9qkdpP1wJE0Mx8c0wEqMF rD32YbdmlViom5d2sFmTuNhbEFt7BQy10o+icW47VqEwE5+LAYfGk2iWUtzUdlYhhdqFl+ HHRiwue2U3d1JFEU+8TM7fbSmoGMkcs= Received: by gentwo.org (Postfix, from userid 1003) id 61D2440A8A; Wed, 20 Dec 2023 07:42:17 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by gentwo.org (Postfix) with ESMTP id 5F15D40A89; Wed, 20 Dec 2023 07:42:17 -0800 (PST) Date: Wed, 20 Dec 2023 07:42:17 -0800 (PST) From: "Christoph Lameter (Ampere)" To: Yin Fengwei cc: Yang Shi , kernel test robot , Rik van Riel , oe-lkp@lists.linux.dev, lkp@intel.com, Linux Memory Management List , Andrew Morton , Matthew Wilcox , ying.huang@intel.com, feng.tang@intel.com Subject: Re: [linux-next:master] [mm] 1111d46b5c: stress-ng.pthread.ops_per_sec -84.3% regression In-Reply-To: Message-ID: References: <202312192310.56367035-oliver.sang@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: D103F1C0005 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: ahdzyyxza3h9uk4raeumpurfa4nnec8d X-HE-Tag: 1703086938-238815 X-HE-Meta: U2FsdGVkX1872lpVssBSOFquIxjeueHbzX6/0hkDwfIHieoqA9o7f+DhI92ERIHGAbUuAqNP6OT30FWjYvQxn0tinwHdL/m8jPXc9eO6+8BTVVKLzaDGeqdLnfOhPakjlIaTu7KnBhpxAeUjKNVFTViB/XnyqVp7nciMso36of5+zeJQk6xlGZm5QpE0+TFP4yvWq/CgOOuF7++DuFketvF/4CkLBr0rvGmCxXvFboao+6ivj7tSSabWfesoAdvcws7tBxUkESWet//4YNJf8a4fFQLq+OK2e/ODZ//+Km8roTylEXHgPeUnHn9Rx382pjb1/5MIltyUYogACTHDgkE18oHXW5ORAfLBmsEaMlopszi2o7ZzUAIhZxASdEaf4cGLV8RoviRtH67J7xV7YIip+IchgxrCTe5QKZFrX7F9wHssTGAJcTSpM7ft5vsMqS1MpEjiZf3x0p6HZMRCn5WAtOYpuG4X2+NKIPt0alCFDFiBwf49Ug0oAcS04U8oIicWrWe9LGLfPVqkS7Yg6PrlcpQJxeKcwuiDmrRW5mkpySI5kZvY38+yk6lqjOvAWbHpWn26N9Sq85owMohzLlaCwgWQ0nC+5cwBpd3GwaP5ea7CJOFBahGp630nbYgZa+80ZGQt7IWAWmKAeY8QHuawcYxWjcFrU7vwZlJuL1UN8fR2dCCo5WFM6E255TuDdjQ8HD8BDI+9QKleO3IR2lkndT1HND8WNIEVaXbVJH4d0iZdw468aDHqwOArFy5YN+QcX8iYaHvVxTIlLJCPY3x/jAdBJv6MR2ltwp9BYrRGjt8fONC7MvD+M7/1pGBntfqNAFiGrsxlUfPclMEt5DSGUeXzbv72As3QKJWizc/kcxhv7r7gCgM/4szXPmNL/dL929XPZRd1sEDHREi58Nc/4Meufznt2e39jiTK7xU5aXzyfMTlrQbKRjUOgq6bVC5+kumS5mFUZeHDFYa N4lKNVDi I9sGMwcKidJ22s8F9Np7dPF0OA6e3btlo8FC68KjunWnEy9CWBJWlQvooTNDeVT0HJYmA56gZJzZOvlkzh/uLiljWYjfGk/a6EBzjrFkUZ/eVFO831VGtFZxR6H2ipZyYLj0Lyk5hrXZdc1xk2Wu3Y879hCe0/w2p96Ej5cte/onc1EIdFjFaR27BhV0fn8QKHIn97YJvjN7742Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, 20 Dec 2023, Yin Fengwei wrote: >> Interesting, wasn't the same regression seen last time? And I'm a >> little bit confused about how pthread got regressed. I didn't see the >> pthread benchmark do any intensive memory alloc/free operations. Do >> the pthread APIs do any intensive memory operations? I saw the >> benchmark does allocate memory for thread stack, but it should be just >> 8K per thread, so it should not trigger what this patch does. With >> 1024 threads, the thread stacks may get merged into one single VMA (8M >> total), but it may do so even though the patch is not applied. > stress-ng.pthread test code is strange here: > > https://github.com/ColinIanKing/stress-ng/blob/master/stress-pthread.c#L573 > > Even it allocates its own stack, but that attr is not passed > to pthread_create. So it's still glibc to allocate stack for > pthread which is 8M size. This is why this patch can impact > the stress-ng.pthread testing. Hmmm... The use of calloc() for 8M triggers an mmap I guess. Why is that memory slower if we align the adress to a 2M boundary? Because THP can act faster and creates more overhead? > while this time, the hotspot is in (pmd_lock from do_madvise I suppose): > - 55.02% zap_pmd_range.isra.0 > - 53.42% __split_huge_pmd > - 51.74% _raw_spin_lock > - 51.73% native_queued_spin_lock_slowpath > + 3.03% asm_sysvec_call_function > - 1.67% __split_huge_pmd_locked > - 0.87% pmdp_invalidate > + 0.86% flush_tlb_mm_range > - 1.60% zap_pte_range > - 1.04% page_remove_rmap > 0.55% __mod_lruvec_page_state Ok so we have 2M mappings and they are split because of some action on 4K segments? Guess because of the guard pages? >> More time spent in madvise and munmap. but I'm not sure whether this >> is caused by tearing down the address space when exiting the test. If >> so it should not count in the regression. > It's not for the whole address space tearing down. It's for pthread > stack tearing down when pthread exit (can be treated as address space > tearing down? I suppose so). > > https://github.com/lattera/glibc/blob/master/nptl/allocatestack.c#L384 > https://github.com/lattera/glibc/blob/master/nptl/pthread_create.c#L576 > > Another thing is whether it's worthy to make stack use THP? It may be > useful for some apps which need large stack size? No can do since a calloc is used to allocate the stack. How can the kernel distinguish the allocation?