From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D579FEB64D9 for ; Thu, 29 Jun 2023 17:27:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1AFE88D0002; Thu, 29 Jun 2023 13:27:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 160F28D0001; Thu, 29 Jun 2023 13:27:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0273F8D0002; Thu, 29 Jun 2023 13:27:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E8E158D0001 for ; Thu, 29 Jun 2023 13:27:07 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9B085140251 for ; Thu, 29 Jun 2023 17:27:07 +0000 (UTC) X-FDA: 80956466094.08.D88489C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf15.hostedemail.com (Postfix) with ESMTP id D8770A0007 for ; Thu, 29 Jun 2023 17:27:05 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; spf=pass (imf15.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688059626; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=npR9dIRYM5rqxDospUYhZHVBgHoelI9HYi9qSuYjj+E=; b=r2XxBoEdJMB6PH7pozaR8t7zMGlJZ8mjqtGjZR7sq23Of5jn5ryLHS1WJOsbDdDjkZQB+0 zM8Oj/qAqKdahIhs0J3xbCKhU5mpURgpQoJpMl/8MnlaYQelSiqOt3FQbNcKW78LFQAFem R1NcqR7OruujoLsuyQ31o7okgj/+9GQ= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; spf=pass (imf15.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688059626; a=rsa-sha256; cv=none; b=ME93nzqd7pwHySO4RKo6W9EWb9uv+GQzSz67tnQMAld7a6vd72APDTU4Q24ypRjxOIciqU cOLDfUIcKlV1Meltc7K8BR5esR0sdawjKaK2nioUmEJK1JEPJ3S0eVMnbSoZhbOdmPqMsz Y154czOcor9ILZpy3PfViKqhvIyf0OU= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CD8FF6153C; Thu, 29 Jun 2023 17:27:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ABA6FC433C0; Thu, 29 Jun 2023 17:26:58 +0000 (UTC) Date: Thu, 29 Jun 2023 18:26:55 +0100 From: Catalin Marinas To: Yicong Yang Cc: akpm@linux-foundation.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, mark.rutland@arm.com, ryan.roberts@arm.com, will@kernel.org, anshuman.khandual@arm.com, linux-doc@vger.kernel.org, corbet@lwn.net, peterz@infradead.org, arnd@arndb.de, punit.agrawal@bytedance.com, linux-kernel@vger.kernel.org, darren@os.amperecomputing.com, yangyicong@hisilicon.com, huzhanyuan@oppo.com, lipeifeng@oppo.com, zhangshiming@oppo.com, guojian@oppo.com, realmz6@gmail.com, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, Barry Song <21cnbao@gmail.com>, wangkefeng.wang@huawei.com, xhao@linux.alibaba.com, prime.zeng@hisilicon.com, Jonathan.Cameron@huawei.com, Barry Song , Nadav Amit , Mel Gorman Subject: Re: [RESEND PATCH v9 2/2] arm64: support batched/deferred tlb shootdown during page reclamation/migration Message-ID: References: <20230518065934.12877-1-yangyicong@huawei.com> <20230518065934.12877-3-yangyicong@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: D8770A0007 X-Rspam-User: X-Stat-Signature: 6pzgjd5o5cuzhnp8yrdhuks3k7doweam X-Rspamd-Server: rspam01 X-HE-Tag: 1688059625-280322 X-HE-Meta: U2FsdGVkX189zNtMpXOnDzuE74632XFhGjS325wl9FqALGfvvcJgSIOu4ATShnKf4MFvihJXSCIZYqTpdj+d5fpmOKRMr1tlbHMcniBZWz6r7huMIspJ30PmbPdXM8hh6zBODXeqd32ia+mnib61qhOPx9RspQVmUA374IUsc1mRAQBuI92FIzz3WFn/wEdws0QG5nwhDBrnW935y8Ysy1zOjT/kELdnphawMrA6UvXMDwu9SPU05S+qma5QABDw1bbKbBE8cdo+ZHC8uGMecsiN4Yi6K9znAdOTf6oDVWp0Xe0pjyEgbDP2xYmzuqHbL6YJI7cvAUD4OdSYZ5K5oOrcAPVr73pgkyNNbOUnt5DhAtptnoYt/QF/wq1+ySeC/rPhgh4Ck+xSSu0XKfUln+3vIVpjQj9H4qrUCrTSrxPKPxJaej50jmn0xnGfNgw/ak8H+ED8agBDBZR8G7ZSql3fVm4QxBQG+AckuBWp3VLtVKon+Vy53R/5IB+5ifH7grBzxB9EV52vGTqvnhz6ElAtNTybwwF9WYctLoT3z3LvKXpUHKUbGS1Fh9SWNX70+0p3744P6dg2+5ap7tG8xmWd1xkWanEUi2FiIyl9atevmksuy+KOwqNCdldTBv0wOSP8ZtQgo4z1sUu0gMcpMg0qZqusLMDqiChPZ3cOoa54oMGdZ0EsWQHMOTfD1NGo/YXC6LNK+obXMqMSAkp793pj+TA8b+ZX2DrRtF2FerDZzYr7++2iRablyvYAjtMvZ5eyeV5H36z+B5q1owVEIAHO+0g8bG/2lUMoepk0O7CvEE/1S9ppVTmufpCCPMupAlBn7vDSoVFUwUvS7wlpl0AHn+1kz/B3ncgU8JWbHWgINcR3fa8TdnD/uNg3CI618FmZiQeM4v5PAsAyBgkR7bYG+qq0/pOVJxoHV6waQqcgnbqFGXjEFt6SBWVC+oiHlhczbpbS5JqlWFKLGBX I/0e0iOp oEren2vozCGyJ43dnRT7mm2F0pf2TtWsMN0iSUJCzqGWb37qf/2VyMQnOX6nqn4mzR3FssDqXnXxQP7+wrTAvYk4ApugsO7k4zwoUn7U3nbyD7gPwfIfb3oCRdeFl20GBrddEmXJ2KeZJMTyqeq6PuP7lInvHGcv8Klck1Z/chsYPQtcbrxOrIiaUOg8qTlqAC7kOkpSRJVgRm4zeJ9TV7mB4Fkjf4a4qbpP9DqumW2HhrBETjrTQjfnO7/MnzUpuTvxF7UkTBRQ2ljxZ6ZIIi1BmxNTkqlCqmbWOwoaBg3/ExyZuaTI3P7xb5FrH0jn4j+Rpi6yallN4HE91JF1JDfOD6y0k0FemiVGmQFV0yogzBuzHFE1mrl2upBWPB6W2be2LO/Z728wqINOf8hPiHvC2hNacfsExLX5x9IwRMY73K8PATg0pSgsR/k85PDuNBlEGQy5OT63wXPvL9ZTIfeGlnQd4byY+spI0FHkRaK0JjeSgZyP85YztX22tsDtE5eeIXVEtYt/f9A+Seilg6gwW9lZX4OZyw5T8qdjn85FQwvjlcSs6WHOUn8g99j7Cl7yCrfShGG0QnQnvsVIkDMD8VQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Jun 29, 2023 at 05:31:36PM +0100, Catalin Marinas wrote: > On Thu, May 18, 2023 at 02:59:34PM +0800, Yicong Yang wrote: > > From: Barry Song > > > > on x86, batched and deferred tlb shootdown has lead to 90% > > performance increase on tlb shootdown. on arm64, HW can do > > tlb shootdown without software IPI. But sync tlbi is still > > quite expensive. > [...] > > .../features/vm/TLB/arch-support.txt | 2 +- > > arch/arm64/Kconfig | 1 + > > arch/arm64/include/asm/tlbbatch.h | 12 ++++ > > arch/arm64/include/asm/tlbflush.h | 33 ++++++++- > > arch/arm64/mm/flush.c | 69 +++++++++++++++++++ > > arch/x86/include/asm/tlbflush.h | 5 +- > > include/linux/mm_types_task.h | 4 +- > > mm/rmap.c | 12 ++-- > > First of all, this patch needs to be split in some preparatory patches > introducing/renaming functions with no functional change for x86. Once > done, you can add the arm64-only changes. > > Now, on the implementation, I had some comments on v7 but we didn't get > to a conclusion and the thread eventually died: > > https://lore.kernel.org/linux-mm/Y7cToj5mWd1ZbMyQ@arm.com/ > > I know I said a command line argument is better than Kconfig or some > random number of CPUs heuristics but it would be even better if we don't > bother with any, just make this always on. Barry had some comments > around mprotect() being racy and that's why we have > flush_tlb_batched_pending() but I don't think it's needed (or, for > arm64, it can be a DSB since this patch issues the TLBIs but without the > DVM Sync). So we need to clarify this (see Barry's last email on the > above thread) and before attempting new versions of this patchset. With > flush_tlb_batched_pending() removed (or DSB), I have a suspicion such > implementation would be faster on any SoC irrespective of the number of > CPUs. I think I got the need for flush_tlb_batched_pending(). If try_to_unmap() marks the pte !present and we have a pending TLBI, change_pte_range() will skip the TLB maintenance altogether since it did not change the pte. So we could be left with stale TLB entries after mprotect() before TTU does the batch flushing. We can have an arch-specific flush_tlb_batched_pending() that can be a DSB only on arm64 and a full mm flush on x86. -- Catalin