From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0C1FCCA47C for ; Fri, 15 Jul 2022 02:47:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B7F879401C5; Thu, 14 Jul 2022 22:47:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B568C9401A5; Thu, 14 Jul 2022 22:47:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A1E829401C5; Thu, 14 Jul 2022 22:47:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 8E25B9401A5 for ; Thu, 14 Jul 2022 22:47:29 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 53FED604A3 for ; Fri, 15 Jul 2022 02:47:29 +0000 (UTC) X-FDA: 79687798218.06.E67A7AD Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by imf18.hostedemail.com (Postfix) with ESMTP id DFEDC1C0075 for ; Fri, 15 Jul 2022 02:47:27 +0000 (UTC) Received: from canpemm500009.china.huawei.com (unknown [172.30.72.55]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4LkbN218m3zFpxV; Fri, 15 Jul 2022 10:46:26 +0800 (CST) Received: from [10.67.102.169] (10.67.102.169) by canpemm500009.china.huawei.com (7.192.105.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 15 Jul 2022 10:47:23 +0800 CC: , Andrew Morton , Linux-MM , LAK , x86 , Catalin Marinas , Will Deacon , Linux Doc Mailing List , Jonathan Corbet , Arnd Bergmann , LKML , Darren Hart , , =?UTF-8?B?5p2O5Z+56ZSLKHdpbmsp?= , =?UTF-8?B?5byg6K+X5piOKFNpbW9uIFpoYW5nKQ==?= , =?UTF-8?B?6YOt5YGl?= , real mz , , , , , , "tiantao (H)" Subject: Re: [PATCH v2 0/4] mm: arm64: bring up BATCHED_UNMAP_TLB_FLUSH To: Barry Song <21cnbao@gmail.com>, References: <20220711034615.482895-1-21cnbao@gmail.com> <24f5e25b-3946-b92a-975b-c34688005398@linux.alibaba.com> From: Yicong Yang Message-ID: <77e245a3-56b8-d624-187d-d8dacaf8d043@huawei.com> Date: Fri, 15 Jul 2022 10:47:22 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.67.102.169] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To canpemm500009.china.huawei.com (7.192.105.203) X-CFilter-Loop: Reflected ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657853249; a=rsa-sha256; cv=none; b=YNDbZhn75f+fxwLGjK/HqGDs8MGS8LANwEbyPsaIDiYIza4ocRbe1wV4CE4Ruaj33ymHq5 77iGGwaCwAq/Ro+nQjEm3X5hO9dTRpTSTFnOqG7swXXcD9XCSDmRckrx1XlwRjAV1ZjUWS +D42Wnuvv+rso4mlkCz7hwpSU604mOg= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf18.hostedemail.com: domain of yangyicong@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=yangyicong@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657853249; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QJD90NqMoxsCIUtRZcH26A1bbRwLGfOzJhG9DJFNH+M=; b=NVw1+vOuffqMu3mpqQ54PMx9+oI3oRoapVOltjBWchbptwD6SwvweRQKRFLbyFfC3Ln5xj V9hDg1Zx3/CAU9m42VQ23zB6KJgjM3R8M9AjzKcjWOTdTutwr6E0qQzDz5IaQoOkBPygOP T/IJONgryiBIl8EmrDyUHP6UmFEFL08= X-Rspam-User: Authentication-Results: imf18.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf18.hostedemail.com: domain of yangyicong@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=yangyicong@huawei.com X-Stat-Signature: yt51kt8bogtexoqkd5973a8e16u8y7z8 X-Rspamd-Queue-Id: DFEDC1C0075 X-Rspamd-Server: rspam08 X-HE-Tag: 1657853247-536600 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2022/7/14 12:51, Barry Song wrote: > On Thu, Jul 14, 2022 at 3:29 PM Xin Hao wrote: >> >> Hi barry. >> >> I do some test on Kunpeng arm64 machine use Unixbench. >> >> The test result as below. >> >> One core, we can see the performance improvement above +30%. > > I am really pleased to see the 30%+ improvement on unixbench on single core. > >> ./Run -c 1 -i 1 shell1 >> w/o >> System Benchmarks Partial Index BASELINE RESULT INDEX >> Shell Scripts (1 concurrent) 42.4 5481.0 1292.7 >> ======== >> System Benchmarks Index Score (Partial Only) 1292.7 >> >> w/ >> System Benchmarks Partial Index BASELINE RESULT INDEX >> Shell Scripts (1 concurrent) 42.4 6974.6 1645.0 >> ======== >> System Benchmarks Index Score (Partial Only) 1645.0 >> >> >> But with whole cores, there have little performance degradation above -5% > > That is sad as we might get more concurrency between mprotect(), madvise(), > mremap(), zap_pte_range() and the deferred tlbi. > >> >> ./Run -c 96 -i 1 shell1 >> w/o >> Shell Scripts (1 concurrent) 80765.5 lpm (60.0 s, 1 >> samples) >> System Benchmarks Partial Index BASELINE RESULT INDEX >> Shell Scripts (1 concurrent) 42.4 80765.5 19048.5 >> ======== >> System Benchmarks Index Score (Partial Only) 19048.5 >> >> w >> Shell Scripts (1 concurrent) 76333.6 lpm (60.0 s, 1 >> samples) >> System Benchmarks Partial Index BASELINE RESULT INDEX >> Shell Scripts (1 concurrent) 42.4 76333.6 18003.2 >> ======== >> System Benchmarks Index Score (Partial Only) 18003.2 >> >> ---------------------------------------------------------------------------------------------- >> >> >> After discuss with you, and do some changes in the patch. >> >> ndex a52381a680db..1ecba81f1277 100644 >> --- a/mm/rmap.c >> +++ b/mm/rmap.c >> @@ -727,7 +727,11 @@ void flush_tlb_batched_pending(struct mm_struct *mm) >> int flushed = batch >> TLB_FLUSH_BATCH_FLUSHED_SHIFT; >> >> if (pending != flushed) { >> +#ifdef CONFIG_ARCH_HAS_MM_CPUMASK >> flush_tlb_mm(mm); >> +#else >> + dsb(ish); >> +#endif >> > > i was guessing the problem might be flush_tlb_batched_pending() > so i asked you to change this to verify my guess. > > /* >> * If the new TLB flushing is pending during flushing, leave >> * mm->tlb_flush_batched as is, to avoid losing flushing. >> >> there have a performance improvement with whole cores, above +30% > > But I don't think it is a proper patch. There is no guarantee the cpu calling > flush_tlb_batched_pending is exactly the cpu sending the deferred > tlbi. so the solution is unsafe. But since this temporary code can bring the > 30%+ performance improvement back for high concurrency, we have huge > potential to finally make it. > > Unfortunately I don't have an arm64 server to debug on this. I only have > 8 cores which are unlikely to reproduce regression which happens in > high concurrency with 96 parallel tasks. > > So I'd ask if @yicong or someone else working on kunpeng or other > arm64 servers is able to actually debug and figure out a proper > patch for this, then add the patch as 5/5 into this series? > sure, Tiantao and I will look into this on Kunpeng 920. >> >> ./Run -c 96 -i 1 shell1 >> 96 CPUs in system; running 96 parallel copies of tests >> >> Shell Scripts (1 concurrent) 109229.0 lpm (60.0 s, 1 samples) >> System Benchmarks Partial Index BASELINE RESULT INDEX >> Shell Scripts (1 concurrent) 42.4 109229.0 25761.6 >> ======== >> System Benchmarks Index Score (Partial Only) 25761.6 >> >> >> Tested-by: Xin Hao > > Thanks for your testing! > >> >> Looking forward to your next version patch. >> >> On 7/11/22 11:46 AM, Barry Song wrote: >>> Though ARM64 has the hardware to do tlb shootdown, the hardware >>> broadcasting is not free. >>> A simplest micro benchmark shows even on snapdragon 888 with only >>> 8 cores, the overhead for ptep_clear_flush is huge even for paging >>> out one page mapped by only one process: >>> 5.36% a.out [kernel.kallsyms] [k] ptep_clear_flush >>> >>> While pages are mapped by multiple processes or HW has more CPUs, >>> the cost should become even higher due to the bad scalability of >>> tlb shootdown. >>> >>> The same benchmark can result in 16.99% CPU consumption on ARM64 >>> server with around 100 cores according to Yicong's test on patch >>> 4/4. >>> >>> This patchset leverages the existing BATCHED_UNMAP_TLB_FLUSH by >>> 1. only send tlbi instructions in the first stage - >>> arch_tlbbatch_add_mm() >>> 2. wait for the completion of tlbi by dsb while doing tlbbatch >>> sync in arch_tlbbatch_flush() >>> My testing on snapdragon shows the overhead of ptep_clear_flush >>> is removed by the patchset. The micro benchmark becomes 5% faster >>> even for one page mapped by single process on snapdragon 888. >>> >>> >>> -v2: >>> 1. Collected Yicong's test result on kunpeng920 ARM64 server; >>> 2. Removed the redundant vma parameter in arch_tlbbatch_add_mm() >>> according to the comments of Peter Zijlstra and Dave Hansen >>> 3. Added ARCH_HAS_MM_CPUMASK rather than checking if mm_cpumask >>> is empty according to the comments of Nadav Amit >>> >>> Thanks, Yicong, Peter, Dave and Nadav for your testing or reviewing >>> , and comments. >>> >>> -v1: >>> https://lore.kernel.org/lkml/20220707125242.425242-1-21cnbao@gmail.com/ >>> >>> Barry Song (4): >>> Revert "Documentation/features: mark BATCHED_UNMAP_TLB_FLUSH doesn't >>> apply to ARM64" >>> mm: rmap: Allow platforms without mm_cpumask to defer TLB flush >>> mm: rmap: Extend tlbbatch APIs to fit new platforms >>> arm64: support batched/deferred tlb shootdown during page reclamation >>> >>> Documentation/features/arch-support.txt | 1 - >>> .../features/vm/TLB/arch-support.txt | 2 +- >>> arch/arm/Kconfig | 1 + >>> arch/arm64/Kconfig | 1 + >>> arch/arm64/include/asm/tlbbatch.h | 12 ++++++++++ >>> arch/arm64/include/asm/tlbflush.h | 23 +++++++++++++++++-- >>> arch/loongarch/Kconfig | 1 + >>> arch/mips/Kconfig | 1 + >>> arch/openrisc/Kconfig | 1 + >>> arch/powerpc/Kconfig | 1 + >>> arch/riscv/Kconfig | 1 + >>> arch/s390/Kconfig | 1 + >>> arch/um/Kconfig | 1 + >>> arch/x86/Kconfig | 1 + >>> arch/x86/include/asm/tlbflush.h | 3 ++- >>> mm/Kconfig | 3 +++ >>> mm/rmap.c | 14 +++++++---- >>> 17 files changed, 59 insertions(+), 9 deletions(-) >>> create mode 100644 arch/arm64/include/asm/tlbbatch.h >>> >> -- >> Best Regards! >> Xin Hao >> > > Thanks > Barry > . >