From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D624C3DA7E for ; Tue, 30 Jul 2024 16:42:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BDC676B007B; Tue, 30 Jul 2024 12:42:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B8B336B0082; Tue, 30 Jul 2024 12:42:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A2B7D6B0083; Tue, 30 Jul 2024 12:42:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 850526B007B for ; Tue, 30 Jul 2024 12:42:14 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 39AF81603B8 for ; Tue, 30 Jul 2024 16:42:14 +0000 (UTC) X-FDA: 82396986588.03.2C6861E Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com [209.85.167.46]) by imf30.hostedemail.com (Postfix) with ESMTP id 20B7480006 for ; Tue, 30 Jul 2024 16:42:11 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="U0r9/Mqm"; spf=pass (imf30.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.46 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722357705; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xPM0aMJJv5fPv90OOd8ioocukIL23WeQ7CLJAnS2lW8=; b=BiLxIM0pWgZ6TQCAA2eDiHfY0XoOZAPnDm4IQSeYbIfCv3LhvZG6NJDQd9at16BqAJniMT eFex/Mp94L4dQLNDAoWL0VBvcbPjIXTc1pbjFxKbrQaaEnlCijASHhcN+oWdjy99NjLJ54 PQP+TJzUlZPNAvqn8jRGGVYs/d3w2r0= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="U0r9/Mqm"; spf=pass (imf30.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.46 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722357705; a=rsa-sha256; cv=none; b=R6mr1TRalswc/ZI0CacZmpDlvi+LPspOYCA31HzQddIbn41exE+AOnrAx7FudZWsaNiOQ9 fIbTYKCJQxI8u93okb0QFpMG0L+2FO8eq4Wv3EHzWO3zdh2Na+p3ZDYxxQfC0cjZDZTcnq 3CyTKgFJyqYELyEeksQ6xWIEwTRiQlI= Received: by mail-lf1-f46.google.com with SMTP id 2adb3069b0e04-52fc4388a64so8262034e87.1 for ; Tue, 30 Jul 2024 09:42:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722357730; x=1722962530; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:date:from:from:to :cc:subject:date:message-id:reply-to; bh=xPM0aMJJv5fPv90OOd8ioocukIL23WeQ7CLJAnS2lW8=; b=U0r9/MqmZVjbXK+dzwwPSv8iRLoYFdN7+4KfWTDFEZiT5oSZYyr2F0Cu2qLRd0wKvv op/OUhCjZaTImSSUZ8tvNUTDeBl9U7RKzL/kxZN2Dg5+zkpijsiEB35gqtKQJ3By+RsD 9imzp9ex1FyNtGKcvUj1LR/zissl/2wdXeFYLwJy787R/sD3uCr7VOUUsfr6aiDUfCqr N3RXhJCesQrJ4kYJTpwXXl/y6BXXofi3UvG8gPF0uK44oz7B3rFUnMStG5YgBLjjczd1 GuGy3Cjc43FH6E04PxOrknNx+jaEC6MqR6eXQgV7RVx6DIgDlsykhC/Bl5fknVvT5sT8 VmUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722357730; x=1722962530; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:date:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xPM0aMJJv5fPv90OOd8ioocukIL23WeQ7CLJAnS2lW8=; b=LiUxJ7Z+PVlFGGXcuxGKoOvpMqL8O1dqF65KcFrUw2Lkm7Vnt2+fb7ODoCWLrswH9Q nmqyhC9eCGkifN5OZ4tyAW5zqUBdodIN7+MIpY1vs7jp7Mu4lMpHXSXCzl+dvKjZmVjN PNl4lOywSy8+ncUtwXlzysFcMwf78vBqz88zBbOoVLl2PrzcUmW+rP6QC7CHbs09X6Nv sQtCCOT3+87ABSamyhi8/3Rq7N/0EEwetTnk4oyvahOpOOCLR3zOcH0zqZmY56YEIR0D Bk8kxxsQe2+7OAFneFA2vPfjAhVYowxmU/7NAIXH6SJ3+DIXt4eselLBWOL5Uthxy0xH RlQA== X-Forwarded-Encrypted: i=1; AJvYcCUVPBlX9ly0J9hDWvsAgubLBKljBhpzKQntIcxV5CZNc3nyaa4JSvEYEsKjai5WmRuAluFwJX4v/TcVu/o+FbLubi8= X-Gm-Message-State: AOJu0Yz215nvB6SB00tecBEeCXn9Vy+apozwjYIKbiRxBg6N1//IRFQ8 ya/AJ3bpsHryycFfcmB32DiwdVf43U7BG8TSoNJAgmdDWvqXROhJ X-Google-Smtp-Source: AGHT+IF/jr9bYuURe1Nq0+SHPOrvajknC/lPYU8z6eVyf6oJs098kdQ2WdeeUeuo0rhRY1wCJexCiA== X-Received: by 2002:a05:6512:1cf:b0:52e:9ab9:da14 with SMTP id 2adb3069b0e04-5309b280728mr7253046e87.31.1722357729826; Tue, 30 Jul 2024 09:42:09 -0700 (PDT) Received: from pc636 (host-90-235-1-92.mobileonline.telia.com. [90.235.1.92]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-52fd5c19579sm1939593e87.182.2024.07.30.09.42.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jul 2024 09:42:09 -0700 (PDT) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Tue, 30 Jul 2024 18:42:06 +0200 To: Huang Adrian Cc: Uladzislau Rezki , ahuang12@lenovo.com, akpm@linux-foundation.org, andreyknvl@gmail.com, bhe@redhat.com, dvyukov@google.com, glider@google.com, hch@infradead.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, ryabinin.a.a@gmail.com, sunjw10@lenovo.com, vincenzo.frascino@arm.com Subject: Re: [PATCH 1/1] mm/vmalloc: Combine all TLB flush operations of KASAN shadow virtual address into one operation Message-ID: References: <20240730093630.5603-1-ahuang12@lenovo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 20B7480006 X-Stat-Signature: zp35hraexmeg78q1nqhqjfcpgqr5fmsz X-HE-Tag: 1722357731-497403 X-HE-Meta: U2FsdGVkX19GkGgHcd8WwPPQKDVWWLueZ9/1fC1dTMTNPwEPMf+DJhQY6QvDnSvKBnBXEwTY50KtSqmB239+B/J/WBHSYZUlWSjy5IDl6aI16ZchhoQKyx2Of+fNPch5c3a59b1ijuIE3qmmASu6nkTTt2cEsKfo5e9THe1h68n5kgpuWN7cuNSovcal0VVd3VdeKetWXuwquzN0ib1QG0lq3X0XkMDzTX/nQ+NMePbQdAS/KBoCOZURGa1tDoNSlHVeWSCGjvo+0oKagMqtEOeL6BKEOkhIEOF/Fs624EfmARaclm3tr9cSQBh0HU46/XhbcAP4UZVhpUL1W9COWdLnOmj6tDn9p/rQW//VXHoGTZwGv2SC2g2qcZBcSxlRjeyGaXFrA5Cd3crUdPvykGMCIzIdz+Mp/MsAO3Dxphe+kJRnolmNmHfDdyALfuRnl5lqRAOgdjhv/I6B2x/EhHMRY0lVGdG7DvyrGYJTNq28J674TE4GxRhT8Xhk1a2uK+Yii7FFqeO/8Y0nt3QB69NgT+T/M/nDAYAOkeX/vW9avOinq9jt6y/72fD6USCHd6JRQD0gMWKoGbD9PlkbaS/CAJRpGgNEqoxdbXthmuOKqNJdmijTCoa/jBp2GvlirUu9fK9GTjTvVbbbNjG7OwWplA2UR86iIv0JRUXejBcGH6fDx7ybyoO8gBgPAqscWMTt6fmD6zVCfu0l0xkAR/D4kEXyXtQnKjeKloIc31Cce/rKbI+wSgAHMgK9Vxhs+ux4haxvg8O7Cy+SEeAh4CvCSn13lcrsdHpaAvF8DvFeVw7KudgYeeLDuLeM5WcJ3cYLnMpuPCQnZnTb8geNLqha3MKeP/BAMs643ch2KHxXwAi88WMCrP+2ZkB+UJAF1q46KCjRCx8xnKOltjJ1AHMQJlmr3ojtULT5hD2ACXLhjyVFpV9JeDj2gPZD2wzQKFaWTozL8eHC13wT8vy SvZW6CZJ /wGFHXkmJtlwAgW3cGWVcuM2FPH1viOOCrBAJsXJkGJtibjN00iTxI6uEy0dmqjicYMp9QM2e8ZYd74Zcg4i49isq5WpPTt3ASWSL/8fVK/xFiExiS7COhucmGv0W4hRFr+7kp11H6dyNA/nehbdIA/geLfJLPuSCQxKAQc+XFHGOgmXO+lya/qn8sT+XU3zERt9Qwd04YxFIT8JLlj42aLtaCilmZiIxWGZfbDgM5LLyYGG3cuKA9bjWLEIEoP54La8HXo440r642e/1/XYbuWYXCd2kgGep1FX+l+VuXh/UMzeMU+ujpUpZWiX5hbWqRShyHGZzYyPrn7oIBHkneTr9fMeZIKpdquMnNj8OanAQpBXg/B6qS+Bss1kqVeKrsr6o0KvseTSKllBG6ORxQBQo2otpnfcKFx1szNX1p7ksiRRnCln0UcpNti7WkilHLVfTojoAA60r2zz0NXGM5nCo3Y/J5xMcgAxWA2qyg+ONZsovLEvkUF2JLExtL24cyUVYzaEh/a/6BvgjGlfKJ7MxEKASVOWbEbuy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jul 31, 2024 at 12:27:27AM +0800, Huang Adrian wrote: > On Tue, Jul 30, 2024 at 7:38 PM Uladzislau Rezki wrote: > > > > > On Mon, Jul 29, 2024 at 7:29 PM Uladzislau Rezki wrote: > > > > It would be really good if Adrian could run the "compiling workload" on > > > > his big system and post the statistics here. > > > > > > > > For example: > > > > a) v6.11-rc1 + KASAN. > > > > b) v6.11-rc1 + KASAN + patch. > > > > > > Sure, please see the statistics below. > > > > > > Test Result (based on 6.11-rc1) > > > =============================== > > > > > > 1. Profile purge_vmap_node() > > > > > > A. Command: trace-cmd record -p function_graph -l purge_vmap_node make -j $(nproc) > > > > > > B. Average execution time of purge_vmap_node(): > > > > > > no patch (us) patched (us) saved > > > ------------- ------------ ----- > > > 147885.02 3692.51 97% > > > > > > C. Total execution time of purge_vmap_node(): > > > > > > no patch (us) patched (us) saved > > > ------------- ------------ ----- > > > 194173036 5114138 97% > > > > > > [ftrace log] Without patch: https://gist.github.com/AdrianHuang/a5bec861f67434e1024bbf43cea85959 > > > [ftrace log] With patch: https://gist.github.com/AdrianHuang/a200215955ee377288377425dbaa04e3 > > > > > > 2. Use `time` utility to measure execution time > > > > > > A. Command: make clean && time make -j $(nproc) > > > > > > B. The following result is the average kernel execution time of five-time > > > measurements. ('sys' field of `time` output): > > > > > > no patch (seconds) patched (seconds) saved > > > ------------------ ---------------- ----- > > > 36932.904 31403.478 15% > > > > > > [`time` log] Without patch: https://gist.github.com/AdrianHuang/987b20fd0bd2bb616b3524aa6ee43112 > > > [`time` log] With patch: https://gist.github.com/AdrianHuang/da2ea4e6aa0b4dcc207b4e40b202f694 > > > > > I meant another statistics. As noted here https://lore.kernel.org/linux-mm/ZogS_04dP5LlRlXN@pc636/T/#m5d57f11d9f69aef5313f4efbe25415b3bae4c818 > > i came to conclusion that below place and lock: > > > > > > static void exit_notify(struct task_struct *tsk, int group_dead) > > { > > bool autoreap; > > struct task_struct *p, *n; > > LIST_HEAD(dead); > > > > write_lock_irq(&tasklist_lock); > > ... > > > > > > keeps IRQs disabled, so it means that the purge_vmap_node() does the progress > > but it can be slow. > > > > CPU_1: > > disables IRQs > > trying to grab the tasklist_lock > > > > CPU_2: > > Sends an IPI to CPU_1 > > waits until the specified callback is executed on CPU_1 > > > > Since CPU_1 has disabled IRQs, serving an IPI and completion of callback > > takes time until CPU_1 enables IRQs back. > > > > Could you please post lock statistics for kernel compiling use case? > > KASAN + patch is enough, IMO. This just to double check whether a > > tasklist_lock is a problem or not. > > Sorry for the misunderstanding. > > Two experiments are shown as follows. I saw you think KASAN + patch is > enough. But, in case you need another one. ;-) > > a) v6.11-rc1 + KASAN > > The result is different from yours, so I ran two tests (make sure the > soft lockup warning was triggered). > > Test #1: waittime-max = 5.4ms > > ... > class name con-bounces contentions waittime-min waittime-max > waittime-total waittime-avg acq-bounces acquisitions > holdtime-min holdtime-max holdtime-total holdtime-avg > ... > tasklist_lock-W: 118762 120090 0.44 > 5443.22 24807413.37 206.57 429757 569051 > 2.27 3222.00 69914505.87 122.86 > tasklist_lock-R: 108262 108300 0.41 > 5381.34 23613372.10 218.04 489132 541541 > 0.20 5543.40 10095470.68 18.64 > --------------- > tasklist_lock 44594 [<0000000099d3ea35>] > exit_notify+0x82/0x900 > tasklist_lock 32041 [<0000000058f753d8>] > release_task+0x104/0x3f0 > tasklist_lock 99240 [<000000008524ff80>] > __do_wait+0xd8/0x710 > tasklist_lock 43435 [<00000000f6e82dcf>] > copy_process+0x2a46/0x50f0 > --------------- > tasklist_lock 98334 [<0000000099d3ea35>] > exit_notify+0x82/0x900 > tasklist_lock 82649 [<0000000058f753d8>] > release_task+0x104/0x3f0 > tasklist_lock 2 [<00000000da5a7972>] > mm_update_next_owner+0xc0/0x430 > tasklist_lock 26708 [<00000000f6e82dcf>] > copy_process+0x2a46/0x50f0 > ... > > > Test #2:waittime-max = 5.7ms > > ... > class name con-bounces contentions waittime-min waittime-max > waittime-total waittime-avg acq-bounces acquisitions > holdtime-min holdtime-max holdtime-total holdtime-avg > ... > tasklist_lock-W: 121742 123167 0.43 > 5713.02 25252257.61 205.02 432111 569762 > 2.25 3083.08 70711022.74 124.11 > tasklist_lock-R: 111479 111523 0.39 > 5050.50 24557264.88 220.20 491404 542221 > 0.20 5611.81 10007782.09 18.46 > --------------- > tasklist_lock 102317 [<000000008524ff80>] > __do_wait+0xd8/0x710 > tasklist_lock 44606 [<00000000f6e82dcf>] > copy_process+0x2a46/0x50f0 > tasklist_lock 45584 [<0000000099d3ea35>] > exit_notify+0x82/0x900 > tasklist_lock 32969 [<0000000058f753d8>] > release_task+0x104/0x3f0 > --------------- > tasklist_lock 100498 [<0000000099d3ea35>] > exit_notify+0x82/0x900 > tasklist_lock 27401 [<00000000f6e82dcf>] > copy_process+0x2a46/0x50f0 > tasklist_lock 85473 [<0000000058f753d8>] > release_task+0x104/0x3f0 > tasklist_lock 650 [<000000004d0b9f6b>] > tty_open_proc_set_tty+0x23/0x210 > ... > > > > b) v6.11-rc1 + KASAN + patch: waittime-max = 5.7ms > > ... > class name con-bounces contentions waittime-min waittime-max > waittime-total waittime-avg acq-bounces acquisitions > holdtime-min holdtime-max holdtime-total holdtime-avg > ... > tasklist_lock-W: 108876 110087 0.33 > 5688.64 18622460.43 169.16 426740 568715 > 1.94 2930.76 62560515.48 110.00 > tasklist_lock-R: 99864 99909 0.43 > 5868.69 17849478.20 178.66 487654 541328 > 0.20 5709.98 9207504.90 17.01 > --------------- > tasklist_lock 91655 [<00000000a622e532>] > __do_wait+0xd8/0x710 > tasklist_lock 41100 [<00000000ccf53925>] > exit_notify+0x82/0x900 > tasklist_lock 8254 [<00000000093ccded>] > tty_open_proc_set_tty+0x23/0x210 > tasklist_lock 39542 [<00000000a0e6bf4d>] > copy_process+0x2a46/0x50f0 > --------------- > tasklist_lock 90525 [<00000000ccf53925>] > exit_notify+0x82/0x900 > tasklist_lock 76934 [<00000000cb7ca00c>] > release_task+0x104/0x3f0 > tasklist_lock 23723 [<00000000a0e6bf4d>] > copy_process+0x2a46/0x50f0 > tasklist_lock 18223 [<00000000a622e532>] > __do_wait+0xd8/0x710 > ... > > Thank you for posting this! So tasklist_lock is not a problem. I assume you have a full output of lock_stat. Could you please paste it for v6.11-rc1 + KASAN? Thank you! -- Uladzislau Rezki