From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECCD8C3DA7E for ; Tue, 30 Jul 2024 11:38:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4154C6B007B; Tue, 30 Jul 2024 07:38:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C4D86B0083; Tue, 30 Jul 2024 07:38:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 28C306B0085; Tue, 30 Jul 2024 07:38:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0AD816B007B for ; Tue, 30 Jul 2024 07:38:23 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 67F19C028A for ; Tue, 30 Jul 2024 11:38:22 +0000 (UTC) X-FDA: 82396220844.12.84E21DA Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) by imf13.hostedemail.com (Postfix) with ESMTP id 78DCB2000A for ; Tue, 30 Jul 2024 11:38:20 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=CFIbHIOe; spf=pass (imf13.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.45 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722339446; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lP9ZC3R5hMrSLBrjIcRWHj9UaDtclwyOij0udu20d1U=; b=1eQ0/Hugz+2LCQFdVVgoITRyal2YXYrsJ6Wyj3qa9zVY7Ihshmq3cAHWSG8GumB59xrF9u R1KmEcv2zapGkx25RZoMUDTcE2dVam54Yry8CfXlLQpacplu6436zn3T0irsYRY2Tl2ojD FNglWchBBKUNe00AA5aCGs5yHhb2LVM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722339446; a=rsa-sha256; cv=none; b=2i50uwwsSvf67Hwb431r4S+xafVoy21KXnoYg26r7isD+CtQaTk0vGIFac9/x4R/h1FzVT USp7GNc2LRmrOln1uycOP9UAXfa/mPfWSCKI1Z4kON3uFSs1vLd8GmFwKN96cG99o3VmN5 X3o7wfun70VgxMTWBIzli9ubwgj2gz4= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=CFIbHIOe; spf=pass (imf13.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.45 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-lf1-f45.google.com with SMTP id 2adb3069b0e04-52efbc57456so4688794e87.1 for ; Tue, 30 Jul 2024 04:38:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722339498; x=1722944298; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=lP9ZC3R5hMrSLBrjIcRWHj9UaDtclwyOij0udu20d1U=; b=CFIbHIOeykdFmwtgB+GJ30kV8Oqiux2rCwEsCsgem4swt1X0cvoALqARzaj4Wgm/Qs qXGDNTdLh2AyNmC6NztEq6raAmBi0q7BLYbuHLpna/0i5cQobC1vOUVfmE5QkqzFtnLf DMljdPc7IxqiF9EMNSkA7mjxVX19HhIZtzt5uic4EziSoYmql1CSwuSE1niERqHzmztv kdxoBqyMp1POVKR0W64XWRtRp5/VKtqhDcjOG/1IpLd9gwvdHIOP6VcvQwtwlZXOV0Ic CLxCTNkjv7boIk0cnirsXRoy6+JF7QjFepVc5Enn/OGnG+wsTBvF1MK/2RqXOX+qGQo/ /LjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722339498; x=1722944298; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=lP9ZC3R5hMrSLBrjIcRWHj9UaDtclwyOij0udu20d1U=; b=sJSjKNcOgv0wm0lx+yZ26VxjV5vlYNtW9dCQfBfO7YSxpdp+jIXjLLXQNDmG+ddibB Y4c4qCHL/uagvDGP5g+tEx1neZGAgYQtD78Ss1J8xRmFQDbEDFKYbjOtkabhyV8wnugD bUvf1kd7yljxWwG8HHG7EQlGq6LD8OGOFbY4huDnm2Sj6Z5xyGhcXadvx1Axj3GOA6fw w/Ce4SS+XvUUVxe/jB4SGsUOlSOGNc2ISy3r4i5JsENE75GD2QJeM5FhWHAUKiTlrp/B Xkc9pL3jJ9FwVjb7wnbV0srJLwjHBXEnZQavPaet7RUa1Nuz1XWzUpXvwCcEb143OuHv aGiw== X-Forwarded-Encrypted: i=1; AJvYcCWj24DcVUSDjlAbvWw4wu1A/VkjVGA3rLbVq49V7vnAfDmePOiQbGBn05mRB5aXKfJhfIgmVHgwA3sGU+IuE2PuSzM= X-Gm-Message-State: AOJu0Yzjcy3HOM0OHxJVyzhzfOEJzb8TpTDGVpbuBWtt6+5MzCKxvLr/ yatwU/1MNhtMTcOg8iD/NJ3c8qN3ZnZd5H5EvPi5YooN2QImnhZF X-Google-Smtp-Source: AGHT+IEhL0AF4VJATyVfHZMXtaBFUppKExZeQWXMI1cwWGjKNu0oR/QUccYAXymk39Z4fcAa1bkkqg== X-Received: by 2002:a19:7404:0:b0:52c:d80e:55a5 with SMTP id 2adb3069b0e04-5309b2c3071mr6147385e87.41.1722339498077; Tue, 30 Jul 2024 04:38:18 -0700 (PDT) Received: from pc636 (host-90-235-1-92.mobileonline.telia.com. [90.235.1.92]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-52fd5c08ec2sm1876073e87.127.2024.07.30.04.38.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jul 2024 04:38:17 -0700 (PDT) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Tue, 30 Jul 2024 13:38:15 +0200 To: Adrian Huang Cc: urezki@gmail.com, ahuang12@lenovo.com, akpm@linux-foundation.org, andreyknvl@gmail.com, bhe@redhat.com, dvyukov@google.com, glider@google.com, hch@infradead.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, ryabinin.a.a@gmail.com, sunjw10@lenovo.com, vincenzo.frascino@arm.com Subject: Re: [PATCH 1/1] mm/vmalloc: Combine all TLB flush operations of KASAN shadow virtual address into one operation Message-ID: References: <20240730093630.5603-1-ahuang12@lenovo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240730093630.5603-1-ahuang12@lenovo.com> X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 78DCB2000A X-Stat-Signature: tczraogotdbnu76ujze6eij6e5hnwmim X-HE-Tag: 1722339500-505564 X-HE-Meta: U2FsdGVkX19ABL4+LK+BsYZE36/xQ/rLXEWEyd6hN4XzlJfB6SMTOrwOQQZj/Bp32GZL4e7IwyKbVU9cjutJW4PdZS6yAwLlf66J7/3voT8Jl+vFQLbSjOOZTuO6toD1vBGa60mKiVE817KDP18Gd09A2o5tKA68NoVSFECLumdxjk5U8rTXk+FKVDsUCbW847FKaLhGSrYkr3mkjVTNVbR+6vzBS1y75KBN6BCydklBGyRcH1j517zAyMgbIGRG8O0anb7lbf6as728ynt5gaI4jsEM4uEFfV0qWHBOjGvRlf7F6u8hj+3JizcsRaLrq22OxOiZEM5FLGUrO10SrzJtZpAiPdm+RhHUPv5SpKlWEPk21hBmbEk3MFR4P4l/WoDTJX8nsLWBZcjuRMEseIRoZym7jpDlEPf4QlV7ZpuGzBnkTNELIKWK/vDhEMFCEfgoSkQGeCncWtYypNkNsvjI6k8c3yxAt7oiaP2mvBWrYqPh9Ct7DAFqEm4UeuYzmpETCQHfg9Rc5y7S6rylCB0mJ1llR7AP0ejVydNhDX126BOix6yTru14iFRbVaAb9uM4h1RfIaPQHe7ZFboFlJV7eya/bYqtoLtOeZKzyTfL+N7mecmpSXvNrcsI+lh2Hny0xkMBwkLXHGvqxhE9xHlt4NKAhA+tmZN4F0dqNVdxueLEMU1zoHG429FE/pkIAMoxGrFW2ukSFJx6oVSZ8RolvN8iwGO4vJKxWrqxVJsbXtMn044CNifsh9BXvk/xW7+jrVt+xJIHq/gFODodvHYRCy9ROeTibkLyMGvQMIKMcoquqQUl0nJKdUPYelmc8821qpYKPLbJaDLtvhLMpMtn9R+HK12XYYI0e4WZGgbIHGGIUed8bTgniPxpBwegvNAFLfcZ+OkhA8Vnzr6eudzjM9zwx2ScAuM5C7LnxNT3QTSF2IZ1HByamYVKgH26+YghjYUinHYXd14Zzqu vbeq1F9N FTQ7khuR1X37nre9qW2yZ3BN488m6PY12++NrfjNPT8Es2RpLWasLixZrubjB/s+pDBwa8naXzEFaDFzUB4Ys2dUWjfNBqYJ5q7JbKZzqfwSHMktgTbyBxP4ei4l28rgxqaNHr8g7mk+Rr7ROQ+9EjSAyQGpTJlP+q+5K1FgvLU6/hA1KUUW0J7Ad1nVfRMVhY1h1RPWX5dNsOFEkPf1S2uKbW8pg9XKeqQI+2MouaApJ74uhAwUrr294QYnzTj1VrxvgTch1lX4FjKQGQNIP6ynVXtcI8lMXTX/r2NOpmQ7/TCleU8fz/mEEf7vWhIhy++/tu5VsyuKBkQDDUpK1i1RgRRH8/7MXt8tZhFQHPFQO1RAkllOPVZ70UQuIDGJo3Hxm/UcqzOrm2xizqr8dY6fnQ4FUuFygLCPgcm8AKpgpXqumuGmD43d+dlLTovAVELkDuxlO/mvc4wC1cfc/1r1lGYVj31VmJxxO4jfiTry35sTjndsqLO8nlkKg9sAC+tiW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: > On Mon, Jul 29, 2024 at 7:29 PM Uladzislau Rezki wrote: > > It would be really good if Adrian could run the "compiling workload" on > > his big system and post the statistics here. > > > > For example: > > a) v6.11-rc1 + KASAN. > > b) v6.11-rc1 + KASAN + patch. > > Sure, please see the statistics below. > > Test Result (based on 6.11-rc1) > =============================== > > 1. Profile purge_vmap_node() > > A. Command: trace-cmd record -p function_graph -l purge_vmap_node make -j $(nproc) > > B. Average execution time of purge_vmap_node(): > > no patch (us) patched (us) saved > ------------- ------------ ----- > 147885.02 3692.51 97% > > C. Total execution time of purge_vmap_node(): > > no patch (us) patched (us) saved > ------------- ------------ ----- > 194173036 5114138 97% > > [ftrace log] Without patch: https://gist.github.com/AdrianHuang/a5bec861f67434e1024bbf43cea85959 > [ftrace log] With patch: https://gist.github.com/AdrianHuang/a200215955ee377288377425dbaa04e3 > > 2. Use `time` utility to measure execution time > > A. Command: make clean && time make -j $(nproc) > > B. The following result is the average kernel execution time of five-time > measurements. ('sys' field of `time` output): > > no patch (seconds) patched (seconds) saved > ------------------ ---------------- ----- > 36932.904 31403.478 15% > > [`time` log] Without patch: https://gist.github.com/AdrianHuang/987b20fd0bd2bb616b3524aa6ee43112 > [`time` log] With patch: https://gist.github.com/AdrianHuang/da2ea4e6aa0b4dcc207b4e40b202f694 > I meant another statistics. As noted here https://lore.kernel.org/linux-mm/ZogS_04dP5LlRlXN@pc636/T/#m5d57f11d9f69aef5313f4efbe25415b3bae4c818 i came to conclusion that below place and lock: static void exit_notify(struct task_struct *tsk, int group_dead) { bool autoreap; struct task_struct *p, *n; LIST_HEAD(dead); write_lock_irq(&tasklist_lock); ... keeps IRQs disabled, so it means that the purge_vmap_node() does the progress but it can be slow. CPU_1: disables IRQs trying to grab the tasklist_lock CPU_2: Sends an IPI to CPU_1 waits until the specified callback is executed on CPU_1 Since CPU_1 has disabled IRQs, serving an IPI and completion of callback takes time until CPU_1 enables IRQs back. Could you please post lock statistics for kernel compiling use case? KASAN + patch is enough, IMO. This just to double check whether a tasklist_lock is a problem or not. Thanks! -- Uladzislau Rezki