From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA7BCC6FA92 for ; Wed, 21 Sep 2022 01:50:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 76A3380008; Tue, 20 Sep 2022 21:50:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 719DC6B0073; Tue, 20 Sep 2022 21:50:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B9B080008; Tue, 20 Sep 2022 21:50:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 4E4306B0072 for ; Tue, 20 Sep 2022 21:50:37 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1D79D1C5CAF for ; Wed, 21 Sep 2022 01:50:37 +0000 (UTC) X-FDA: 79934413314.23.2C6FE51 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) by imf15.hostedemail.com (Postfix) with ESMTP id CE2EBA000A for ; Wed, 21 Sep 2022 01:50:36 +0000 (UTC) Received: by mail-ej1-f50.google.com with SMTP id u9so10395232ejy.5 for ; Tue, 20 Sep 2022 18:50:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=6BQ+n+4VW6hHwhXqAlkDoSlNz/771IN91baTY6cW3mc=; b=Hsxb1h+EMvvkdsefMmznixkb9yPc3ROIaG+LUt0+8GdgrzGiBW3AgAI0RjPjnW1a93 DY3sKYF7CvzXAe00TGJRDY77eMBmAM0480SdlHfjxG4lyc5EbwrTXa15468ciTechEYT WGETfvgoH53XN8KYelN7OAjCyWHBudaSxMJ2L8rTgdvY1WUYvP7Q8KL14RK4KdBht206 uGl//hA7A5//q7f7eaYrI0DnwnygQF2ZaOGzjx7l19o32olfp0XYIWQPV7GeFwqwofTY VpclnNyeiusjibhMe9Hq1ilMiZtIFMoCJZSuw0/xwB2H0iVevrwbGzuhW+Knat88VluT 5vvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=6BQ+n+4VW6hHwhXqAlkDoSlNz/771IN91baTY6cW3mc=; b=X7Xu02WUtX/G4GItUJqJmxOIneSgTQrPRb2zVuCIiVkOED0y/c9kUlFDelJ/y0SVCk eLeXy54+y16eFPfflnKwzVmZVo6vORSnPe0N4Tzv4vej6ieA4cQZJa60A21Yqo6hGXq3 C7mDlm5PYmQkCJP/Dy48XxgxIhfceKKr9KdjpGX+MhkPa4C+Hy0ap/5N3OlBbkE4BJ6a JOJpcsa+u4Vo+2L4a4h2b3rHW/BqilwCu+1iQ9kX8x8RVoEZrhFegQgTs036sFEUCILe RcfeZ6Dt3V4xO0394YMuk4vy8Wo/uEo1tyINiX1U3O7ZwIQiqbLimtzFuOpP4HuXn4Cx uIHQ== X-Gm-Message-State: ACrzQf2OlD+0Ro7PCmtV0RAczIshncIBOseMzSZkU6mRCefnEfqzVy3s tzM1vw8E91kcvSxgL01KEPIsZ/Goyf0kijZIsVc= X-Google-Smtp-Source: AMsMyM4BpJG31Gs91JHdJmZGt8F5S3yEOSSf/UDOFiYo+s9qWWQwy3OlMqr27Xca6ZiQ4/GcrUZCaWrZRKbnRXNpOoI= X-Received: by 2002:a17:907:3e89:b0:779:4869:b799 with SMTP id hs9-20020a1709073e8900b007794869b799mr17924838ejc.91.1663725035280; Tue, 20 Sep 2022 18:50:35 -0700 (PDT) MIME-Version: 1.0 References: <20220822082120.8347-1-yangyicong@huawei.com> <20220822082120.8347-5-yangyicong@huawei.com> <302febae-508c-d73e-8676-d51752946645@arm.com> <8c4f103b-8f04-d0ad-b30a-2db7e52b36a3@arm.com> In-Reply-To: <8c4f103b-8f04-d0ad-b30a-2db7e52b36a3@arm.com> From: Barry Song <21cnbao@gmail.com> Date: Wed, 21 Sep 2022 13:50:24 +1200 Message-ID: Subject: Re: [PATCH v3 4/4] arm64: support batched/deferred tlb shootdown during page reclamation To: Anshuman Khandual Cc: Yicong Yang , akpm@linux-foundation.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, catalin.marinas@arm.com, will@kernel.org, linux-doc@vger.kernel.org, corbet@lwn.net, peterz@infradead.org, arnd@arndb.de, linux-kernel@vger.kernel.org, darren@os.amperecomputing.com, yangyicong@hisilicon.com, huzhanyuan@oppo.com, lipeifeng@oppo.com, zhangshiming@oppo.com, guojian@oppo.com, realmz6@gmail.com, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, wangkefeng.wang@huawei.com, xhao@linux.alibaba.com, prime.zeng@hisilicon.com, Barry Song , Nadav Amit , Mel Gorman Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663725036; a=rsa-sha256; cv=none; b=Ksirkb+pRPWkueVkkaTEtY9nqbzqeuat0JP4+ALCKBk2GIokd4kTGNXsdxju6A/lYvLBly L9Hbxh/qXDCU+WZk7bp1sUe4VjXLz6gqGWSMjvcE8kE9l2qCWphsohCVUtgsOOy/pVzfpv +yhgYoJP6/dtdElJYOSJ97OS5ZFTx9k= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Hsxb1h+E; spf=pass (imf15.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663725036; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6BQ+n+4VW6hHwhXqAlkDoSlNz/771IN91baTY6cW3mc=; b=N/Je0VC4M3+opXRtwyNC2z9aT7FdZCwE9H3v90gLWEkaLeJWj5Qc5V2GSN0ReX+4IMTSE2 XqY5H2ca33uRTaHu93ci8pFofFA87gbOd9x3OmjODmEg6xQnMlC4BMBS2vgaxDHoT1O1w2 yUJIDYtmUSMJVwxaMezRNIGgQpkT2co= X-Rspamd-Queue-Id: CE2EBA000A X-Rspam-User: Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Hsxb1h+E; spf=pass (imf15.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam11 X-Stat-Signature: unhbgbth5p98eks1apj9h997rmu6iuj3 X-HE-Tag: 1663725036-335561 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Sep 20, 2022 at 8:45 PM Anshuman Khandual wrote: > > > > On 9/20/22 09:09, Barry Song wrote: > > On Tue, Sep 20, 2022 at 3:00 PM Anshuman Khandual > > wrote: > >> > >> > >> On 8/22/22 13:51, Yicong Yang wrote: > >>> +static inline bool arch_tlbbatch_should_defer(struct mm_struct *mm) > >>> +{ > >>> + return true; > >>> +} > >> > >> This needs to be conditional on systems, where there will be performance > >> improvements, and should not just be enabled all the time on all systems. > >> num_online_cpus() > X, which does not hold any cpu hotplug lock would be > >> a good metric ? > > > > for a small system, i don't see how this patch will help, e.g. cpus <= 4; > > so we can actually disable tlb-batch on small systems. > > Do not subscribe ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH based on NR_CPUS ? > That might not help much as the default value is 256 for NR_CPUS. > > OR > > arch_tlbbatch_should_defer() checks on > > 1. online cpus (dont enable batched TLB if <= X) > 2. ARM64_WORKAROUND_REPEAT_TLBI (dont enable batched TLB) > > > just need to check if we will have any race condition since hotplug will > > make the condition true and false dynamically. > > If should_defer_flush() evaluate to be false, then ptep_clear_flush() > clears and flushes the entry right away. This should not race with other > queued up TLBI requests, which will be flushed separately. Wondering how > there can be a race here ! Right. How about we make something as below? static inline bool arch_tlbbatch_should_defer(struct mm_struct *mm) { /* for a small system very small number of CPUs, TLB shootdown is cheap */ if (num_online_cpus() <= 4 || unlikely(this_cpu_has_cap(ARM64_WORKAROUND_REPEAT_TLBI))) return false; #ifdef CONFIG_ARM64_WORKAROUND_REPEAT_TLBI if (unlikely(this_cpu_has_cap(ARM64_WORKAROUND_REPEAT_TLBI))) return false; #endif return true; } Thanks Barry