From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55CB0C77B7F for ; Tue, 16 May 2023 10:05:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D3345280008; Tue, 16 May 2023 06:05:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CE3D5280001; Tue, 16 May 2023 06:05:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BAC71280008; Tue, 16 May 2023 06:05:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id AD755280001 for ; Tue, 16 May 2023 06:05:39 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B2D7840201 for ; Tue, 16 May 2023 10:05:38 +0000 (UTC) X-FDA: 80795686356.04.E1E1721 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf28.hostedemail.com (Postfix) with ESMTP id BC156C0003 for ; Tue, 16 May 2023 10:05:36 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=agA3p7sg; spf=pass (imf28.hostedemail.com: domain of bhe@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=bhe@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684231536; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ms0GeZSN+Pyh2iBTX2jaVNvVybqQDgxDiZGPMEWHKsg=; b=ss75MavHi9uyGFqUvhoUuTVroRHM+2ySKT8/gFF9PsQ+kFGh+ZIi0XznhkyNjG7kk4gojl Q4WoIDOUKaBflxmyrNO8oq9KsOWYZZo8w78jjPPfeqoIcpl4ewGOV/o9tluaYap1ynrShq elt9N7LmcFssDeYazrBLmb43LcxnGsQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684231536; a=rsa-sha256; cv=none; b=w/GPftOX0+Df6JHf5ErEJV07WX/+5zJ/KQDDUJ4GTTWxJRJb+AvIusD7r6ESmksLSr8Crh 9F4h1ga5wHZrJ6cXvuCiybdi89Cmbdvoq+7V2CQTLfi3MoFHEJwwFZrZ3HnqfPGnjv25a5 Ty37bBzTpa2uV2kivVRsByWZzOnvf3s= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=agA3p7sg; spf=pass (imf28.hostedemail.com: domain of bhe@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=bhe@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684231535; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ms0GeZSN+Pyh2iBTX2jaVNvVybqQDgxDiZGPMEWHKsg=; b=agA3p7sgjSsoEnZgP4BxcszMhaRJvNQGUJaTrIviOdylPrJ/4nfZJiQg9/HaRLvJCf+dax wv72T/8UI70QTAatsHpTI2XqEJn39d5mCFSyyiVtUFedSpfpcbWAVtlUwFFhNMEXk7ORDL ErLO1ICUqZuGeKXx4t1LUjyD2ZGe6dA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-369-IeJjDBWgMMm3dic2cSXE9w-1; Tue, 16 May 2023 06:05:32 -0400 X-MC-Unique: IeJjDBWgMMm3dic2cSXE9w-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 475A5101A47E; Tue, 16 May 2023 10:05:31 +0000 (UTC) Received: from localhost (ovpn-13-34.pek2.redhat.com [10.72.13.34]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 458B940C6EC4; Tue, 16 May 2023 10:05:29 +0000 (UTC) Date: Tue, 16 May 2023 18:05:26 +0800 From: Baoquan He To: Thomas Gleixner Cc: "Russell King (Oracle)" , Andrew Morton , linux-mm@kvack.org, Christoph Hellwig , Uladzislau Rezki , Lorenzo Stoakes , Peter Zijlstra , John Ogness , linux-arm-kernel@lists.infradead.org, Mark Rutland , Marc Zyngier , x86@kernel.org Subject: Re: Excessive TLB flush ranges Message-ID: References: <87a5y5a6kj.ffs@tglx> <87353x9y3l.ffs@tglx> <87zg658fla.ffs@tglx> <87r0rg93z5.ffs@tglx> <87ilcs8zab.ffs@tglx> <87fs7w8z6y.ffs@tglx> <874joc8x7d.ffs@tglx> MIME-Version: 1.0 In-Reply-To: <874joc8x7d.ffs@tglx> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Rspamd-Queue-Id: BC156C0003 X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: wnm53acr5qwcptgiu88y41ssfm79tkfd X-HE-Tag: 1684231536-562802 X-HE-Meta: U2FsdGVkX1/mZ9mklV4GOqR/V3nryaKp2WgUryDto4lAMcS8x6s3QeIwa7vzzKtFFO7tudWownXmCZpCxf0y0h8wH6/f+0SHFs8CiI/RWErGiZwdlXRVtLx0OMWIv72HW3piX/dC1L5GBn0/YFuM8L4Gw8KTEIIEZ5qoLbkLFvlr/cfOnsl3U/4ZFGPoAxCNVBouQCsFTIL0yYY4jaQzAqUYyNXoFEOTos4NXrpVVZGvSbcHLjqTpbrWYrdvaSWxr796KPZPiPu9ocy6jkS0X7cvDqBhw22mkVhfnE++eTUpy9v568i5kODvFR55kdfzIJyhWXuW+G29/C63A2NtCz99kXxz1oTgN4TAGkvgU3zKRZps02tDbOCwkWmF6QWzDWO5PZ2/3Fe9JVxzwPOjIQWC6Eggqtwx6GLAuBMebMuQCVDNdgOzWnHBcYUM6FJ9kxkB/KoltySmsNNcl8bLrH2+QxRZOFqq4rkVmh5yu24NMk+iAAjR/7iB7KzNGOyl4wBfIN0oCmPWjyb2YOlhiGToMrNbNMlWKe1THO3hBPB1QJ/8NZqGY7KIrBwCfaAGYyhjwKdWUfg/VWsTn4drAhW/6+IbJyaS2IT4UQcNp7DD5OgNYfbNX763yKy/mro0jCNfxglq/goTeBKisXpWebrlfIbZGoyHRizIbt2iztoqybJsGBa28qylySiTfjZxY7wEHZlGC7I27TlgLqNmpiFZfbsbZEHPSwEoKZhcFwvNdTQfQrHWB3EJBdF1B6c9dfS3GJwpByaWVXwyaMKtjVvdmwF+tCHr9oDzkjKeyrWJ7rYXyCXf/Dn9UcVhuJLtqj5xz4fn3hmFTi2hZl8dmfonVYPMEeXsCIspHsti5TWrMyAxHqGNYB8R0DcCRWybaNqBGlKEPLi4ETaYAfFJs5M3mOxM+JqDvPuw4WVpoHXscA0yRAQV/S4FHSj9Uabn38xDdME39pKbGv0uoRl 2zqwAzeL VIvdBQL77WtFW6G8adpt+wTB4bHH4httityxgyEV473xZrKl4e3VAhF59g9JUicXZ+ElLTHFTzXsoVwg6C8TRrozhiPEVhhgeMlm2w/2PpovBw6IFtwNZ0c0wszOCAjnZVfeGcZtFl9tL5eOu8eEl4ujx8Ka9aBXWQEyE+UR1aW9ieGI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 05/16/23 at 11:03am, Thomas Gleixner wrote: ...... > --- a/mm/vmalloc.c > +++ b/mm/vmalloc.c > @@ -1724,7 +1724,8 @@ static void purge_fragmented_blocks_allc > */ > static bool __purge_vmap_area_lazy(unsigned long start, unsigned long end) > { > - unsigned long resched_threshold; > + unsigned long resched_threshold, num_entries = 0, num_alias_entries = 0; > + struct vmap_area alias_va = { .va_start = start, .va_end = end }; Note that the start and end passed in are not only direct map which is alias of va. It is the range which has done merging on direct map range and all dirty range of vbq in _vm_unmap_aliases(). We may need to append below draft code on your patch to at least flush the direct map range separately. diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 7672c2422f0c..beaaa2f983d3 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -1722,13 +1722,15 @@ static void purge_fragmented_blocks_allcpus(void); /* * Purges all lazily-freed vmap areas. */ -static bool __purge_vmap_area_lazy(unsigned long start, unsigned long end) +static bool __purge_vmap_area_lazy(struct *range) { unsigned long resched_threshold, num_entries = 0, num_alias_entries = 0; - struct vmap_area alias_va = { .va_start = start, .va_end = end }; + struct vmap_area alias_va = { .va_start = range[0].start, .va_end = range[0].end }; + struct vmap_area dirty_va = { .va_start = range[1].start, .va_end = range[1].end }; unsigned int num_purged_areas = 0; struct list_head local_purge_list; struct vmap_area *va, *n_va; + unsigned long start = ULONG_MAX, end = 0; lockdep_assert_held(&vmap_purge_lock); @@ -1737,6 +1739,10 @@ static bool __purge_vmap_area_lazy(unsigned long start, unsigned long end) list_replace_init(&purge_vmap_area_list, &local_purge_list); spin_unlock(&purge_vmap_area_lock); + start = alias_va.va_start; + end = alias_va.va_end; + start = min(start, dirty_va.va_start); + end = min(start, dirty_va.va_end); start = min(start, list_first_entry(&local_purge_list, struct vmap_area, list)->va_start); end = max(end, list_last_entry(&local_purge_list, struct vmap_area, list)->va_end); @@ -1752,6 +1758,10 @@ static bool __purge_vmap_area_lazy(unsigned long start, unsigned long end) list_add(&alias_va.list, &local_purge_list); } + if (dirty_va.va_end > dirty_va.va_start) { + num_alias_entries = (dirty_va.va_end - dirty_va.va_start) >> PAGE_SHIFT; + list_add(&dirty_va.list, &local_purge_list); + } flush_tlb_kernel_vas(&local_purge_list, num_entries + num_alias_entries); if (num_alias_entries) @@ -2236,15 +2246,18 @@ static void vb_free(unsigned long addr, unsigned long size) spin_unlock(&vb->lock); } -static void _vm_unmap_aliases(unsigned long start, unsigned long end, int flush) +static void _vm_unmap_aliases(unsigned long dm_start, unsigned long dm_end, int flush) { int cpu; + struct range range[2]; + unsigned long start = ULONG_MAX, end = 0; if (unlikely(!vmap_initialized)) return; might_sleep(); + range[0] = {dm_start, dm_end}; for_each_possible_cpu(cpu) { struct vmap_block_queue *vbq = &per_cpu(vmap_block_queue, cpu); struct vmap_block *vb; @@ -2269,6 +2282,7 @@ static void _vm_unmap_aliases(unsigned long start, unsigned long end, int flush) rcu_read_unlock(); } + range[1] = {start, end}; mutex_lock(&vmap_purge_lock); purge_fragmented_blocks_allcpus(); if (!__purge_vmap_area_lazy(start, end) && flush)