From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A9B6EB64DC for ; Thu, 29 Jun 2023 16:31:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DAAC78D0002; Thu, 29 Jun 2023 12:31:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D33C08D0001; Thu, 29 Jun 2023 12:31:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD49D8D0002; Thu, 29 Jun 2023 12:31:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A7EFE8D0001 for ; Thu, 29 Jun 2023 12:31:58 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 757D340360 for ; Thu, 29 Jun 2023 16:31:58 +0000 (UTC) X-FDA: 80956327116.06.719BC38 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id ED9C940048 for ; Thu, 29 Jun 2023 16:31:46 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf11.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688056307; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PhADCJ4K3AGZ+NLYnaC2ZyPdXUnlqRSriww5+TxUrHc=; b=L26fqCVbW6XFFzuGLf3ZPIegi70tCgsbbEXfUg3mQUSh52fmbBZz/wRoth6l51L9MZSe1i V7aLAHfFtEA9TuMt45Zxu8b8y1kABuD7uftrYBYMYOsP0TY6LMAWm5reDrT9tzNLHNdTrN bPYvlX3fvWaNwpgrNm4U71t2/UmMIXA= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf11.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688056307; a=rsa-sha256; cv=none; b=okN/mIkOX+puk7iwBoRh6t1qeQuYskAdnmQdkZ0OSaqk5NOXBT6LYGwQYEdA66TM4qLBwf 5PTEyI7cD+7ZPGfvu8TXCTO89DW9rEkYzo6wf/FhXc2NQoflUQElAiJ2pjzQzGYMo75nby 34M8zFmNqhbbv8SAHtHQE4ogF3+UxiE= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 53E4A61516; Thu, 29 Jun 2023 16:31:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 38922C433C0; Thu, 29 Jun 2023 16:31:39 +0000 (UTC) Date: Thu, 29 Jun 2023 17:31:36 +0100 From: Catalin Marinas To: Yicong Yang Cc: akpm@linux-foundation.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, mark.rutland@arm.com, ryan.roberts@arm.com, will@kernel.org, anshuman.khandual@arm.com, linux-doc@vger.kernel.org, corbet@lwn.net, peterz@infradead.org, arnd@arndb.de, punit.agrawal@bytedance.com, linux-kernel@vger.kernel.org, darren@os.amperecomputing.com, yangyicong@hisilicon.com, huzhanyuan@oppo.com, lipeifeng@oppo.com, zhangshiming@oppo.com, guojian@oppo.com, realmz6@gmail.com, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, Barry Song <21cnbao@gmail.com>, wangkefeng.wang@huawei.com, xhao@linux.alibaba.com, prime.zeng@hisilicon.com, Jonathan.Cameron@huawei.com, Barry Song , Nadav Amit , Mel Gorman Subject: Re: [RESEND PATCH v9 2/2] arm64: support batched/deferred tlb shootdown during page reclamation/migration Message-ID: References: <20230518065934.12877-1-yangyicong@huawei.com> <20230518065934.12877-3-yangyicong@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230518065934.12877-3-yangyicong@huawei.com> X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: ED9C940048 X-Stat-Signature: 97uefhzm5esqxg4yhzxdxj3ma5ra9bzr X-HE-Tag: 1688056306-238656 X-HE-Meta: U2FsdGVkX18tfv20wR7ngNue7MGiCI9PCDtvLe2WD6y+iCZ7nA2hYnGJGO79X01cB6eUk3NoxeoqYiicJOuFYDgAjICG4u03XW0eKV4d+UKbOUpvn2hO0FoxJykh63hXYqnHySTM7LWe32PtAs6OoTvatjyLTALrYZNntSUL0J8DSQkNAq7y3QgbvDKjRNEvq15x7XSWTqOU/+fPyEakPXFafKEL3HUKZxLZP5lUuQ2nMdxqknEqwnL0/dxItHc2Un1N+gmpg61lFXTmu6BE6Ej+VCtFbcjrwBe7vDMZKheiYMu1MeSiO39omwUNEaPkcZr05jvJOCB8qJGtey1icE8WSP5cd8kwl14DmDOsDHs3aMmvowLjlEX1hAAF3ck5V26jNP81gTcH0Yie2wr/MzYM7c70wng2Tv1hSDP4/Kc0KEWuJ7MmvLreJATGri57ywX0ErjMMjwj8v+FY2MSaf0rCXHxY9BNEtSvC8ZCa4YVl9msv1FobOLdsqR3ez9ok2PL/XpdD41rm6Y4tjx7zaB9sA7+rU+k+WP7iX5hCP53o8xOx+zsByP0ionOhv9OUxsQjophxdWcl37WvP9lDLB6KJrPijtD7hv1RmWIWYvZhEr8MCvAoqcT7SUyXQRrg2pTOohIs2IrhBgIonWNORqx+lD3xFpzwjnKSrX59GSW9eD2rEXDb3fUZk5c0ZnHBecsH0D5LUQ6sk0Y63zMjNMXNcAPtiJ/9kp4EOS2F865mSKIOlAUO4TvNY2SU/d7u6O9c2FQTIn+XS4YLBBxKGboDCBvHfp4tq/22q9rwgeMQ8fk3hIETEeggHt15HytQCQ+ULCh3DBZVmG/YSiM6aOfjO9GugV9t72UIiixvgIpslo0hzHCtBIWQBPlydBxwtdDyu09XQge38CsBxTrUDGM17qPRxZIVg5u7uaH5HOxOaniNowl8JTrjBxubCTUWfstrWJBPai2Wh1I1M+ t5lAGSGG ZA/TqSt5b6u/5sI+p8MUFvouom90+/h8zOmFkL6oLfw8RWg/bXSF6+dmeyO7oporI5ADmW8VXPcZBbEoNuEwuyHFGarslY9f3z4RFB2TOMR2KfNQmTwNKrLxB/q32cpMH8nI4shy/XWdx/b05qh5Aomr3ndzyYaTJCOida65zjgEDParyMEaok/giZhWuAMLxIS7kxQYEWY38VRiFyMOJl1pzxtzQfFMUsQgYwbM4wQ/pORx4oa9ViHKAe4xgzNyUPa8HoowKUa4ffL4RYMTehthk7tQQdA0wMALr6fQFDy6Rw2QjUSlJ6fK/VxNC0QgnNVBvQnKoZdT7RosvDP2sWK45dG/J/ILQ/N4Di9E2SMQZEpz3psry6/VJszkDjYURqtw5HVgtIKZfBuNnUSrB2vvGKIZlWqcbxM/tf4IfRgZzJpvJs6EyLOV4grTNYyhR5d5w02cAlaTHGM/PSsIVNz5yVXk7xgCfrwl5/eX4hsTrLGSFQAo7cyq682Zxs8pzOr4tU5SYfe4d3jBnrUWBWgNZObKEHZThbygbG0S187HV/JOtn/kNLnlwW9Tgo/XwtVT1s84h/JLT9xPkmPLqG7z5WA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, May 18, 2023 at 02:59:34PM +0800, Yicong Yang wrote: > From: Barry Song > > on x86, batched and deferred tlb shootdown has lead to 90% > performance increase on tlb shootdown. on arm64, HW can do > tlb shootdown without software IPI. But sync tlbi is still > quite expensive. [...] > .../features/vm/TLB/arch-support.txt | 2 +- > arch/arm64/Kconfig | 1 + > arch/arm64/include/asm/tlbbatch.h | 12 ++++ > arch/arm64/include/asm/tlbflush.h | 33 ++++++++- > arch/arm64/mm/flush.c | 69 +++++++++++++++++++ > arch/x86/include/asm/tlbflush.h | 5 +- > include/linux/mm_types_task.h | 4 +- > mm/rmap.c | 12 ++-- First of all, this patch needs to be split in some preparatory patches introducing/renaming functions with no functional change for x86. Once done, you can add the arm64-only changes. Now, on the implementation, I had some comments on v7 but we didn't get to a conclusion and the thread eventually died: https://lore.kernel.org/linux-mm/Y7cToj5mWd1ZbMyQ@arm.com/ I know I said a command line argument is better than Kconfig or some random number of CPUs heuristics but it would be even better if we don't bother with any, just make this always on. Barry had some comments around mprotect() being racy and that's why we have flush_tlb_batched_pending() but I don't think it's needed (or, for arm64, it can be a DSB since this patch issues the TLBIs but without the DVM Sync). So we need to clarify this (see Barry's last email on the above thread) and before attempting new versions of this patchset. With flush_tlb_batched_pending() removed (or DSB), I have a suspicion such implementation would be faster on any SoC irrespective of the number of CPUs. -- Catalin