From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24A5FC4345F for ; Fri, 12 Apr 2024 15:04:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AD3FF6B008A; Fri, 12 Apr 2024 11:04:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A5A936B008C; Fri, 12 Apr 2024 11:04:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D4966B0093; Fri, 12 Apr 2024 11:04:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 691296B008A for ; Fri, 12 Apr 2024 11:04:39 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 24584A0E4A for ; Fri, 12 Apr 2024 15:04:39 +0000 (UTC) X-FDA: 82001201478.15.804F684 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf13.hostedemail.com (Postfix) with ESMTP id B0E082002B for ; Fri, 12 Apr 2024 15:04:35 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=idVPleDK; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=z2aFTdgp; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=idVPleDK; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=z2aFTdgp; dmarc=none; spf=pass (imf13.hostedemail.com: domain of jack@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712934276; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0hVNMsNyA9PVRoUtXJetJJKHKtLZbHkxz6VQTgln4Sg=; b=K7+3iTVjeDpfiUrtf5p116KFk3gWcMNWKCGJzk/Pk+aqZzb1efinqM/t/LA3CvgaB6jP5X CzUQBWdjPLZN1Cec1WQIm7bFqKOezbQ00C+mxKm4ONNbobU2LUQ0jbRZWG701/bADSFApb XdRUEABWgtpBci63nZkfyL9TeLcrhFY= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=idVPleDK; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=z2aFTdgp; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=idVPleDK; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=z2aFTdgp; dmarc=none; spf=pass (imf13.hostedemail.com: domain of jack@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712934276; a=rsa-sha256; cv=none; b=5CkX7p8LHmlII1wPnlrEpenBxxTHxVH0wKAQVJOCL236uEDuu5NYSOpPv3OOuGZWZ3jJf3 dASghTU+AGOnIer3u2cwzGTCCngLMbtzPLOtWchm8JPCNP8GTTlgDKKQlzPlgXWjPLqWHp IFixmoAr5hVmugyt/TN1KzCWUcyh+jE= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 1057B383D5; Fri, 12 Apr 2024 15:04:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1712934273; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0hVNMsNyA9PVRoUtXJetJJKHKtLZbHkxz6VQTgln4Sg=; b=idVPleDK4tHiOH4KWes89evLQb70n1rWE8u60beyYqfj5gg9NCRCYdMMVZkPuCVirTvEyU Pxizy+xebD4YuHJ3eOhPdKaSQRwot071S0Ezfn9Q01oVM7gDTAnw5lECStDdbQfySeC/QK zOUDf9dQ38FlazP6D6YBojcw+92QSwk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1712934273; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0hVNMsNyA9PVRoUtXJetJJKHKtLZbHkxz6VQTgln4Sg=; b=z2aFTdgpGW5elC1TOnk9T0X7YKBQovfCBqd3ygf4z30Sjl+1c2j5hRMCstRmlQS7CzApZ8 CuHZ2MJDjNX8HZAg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1712934273; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0hVNMsNyA9PVRoUtXJetJJKHKtLZbHkxz6VQTgln4Sg=; b=idVPleDK4tHiOH4KWes89evLQb70n1rWE8u60beyYqfj5gg9NCRCYdMMVZkPuCVirTvEyU Pxizy+xebD4YuHJ3eOhPdKaSQRwot071S0Ezfn9Q01oVM7gDTAnw5lECStDdbQfySeC/QK zOUDf9dQ38FlazP6D6YBojcw+92QSwk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1712934273; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0hVNMsNyA9PVRoUtXJetJJKHKtLZbHkxz6VQTgln4Sg=; b=z2aFTdgpGW5elC1TOnk9T0X7YKBQovfCBqd3ygf4z30Sjl+1c2j5hRMCstRmlQS7CzApZ8 CuHZ2MJDjNX8HZAg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 020E71368B; Fri, 12 Apr 2024 15:04:33 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id BWx7AIFNGWbLfQAAD6G6ig (envelope-from ); Fri, 12 Apr 2024 15:04:33 +0000 Received: by quack3.suse.cz (Postfix, from userid 1000) id 6E6EAA071E; Fri, 12 Apr 2024 17:04:32 +0200 (CEST) Date: Fri, 12 Apr 2024 17:04:32 +0200 From: Jan Kara To: Amir Goldstein Cc: Jan Kara , kernel test robot , oe-lkp@lists.linux.dev, lkp@intel.com, Linux Memory Management List , linux-fsdevel@vger.kernel.org, ying.huang@intel.com, feng.tang@intel.com, fengwei.yin@intel.com Subject: Re: [linux-next:master] [fsnotify] a5e57b4d37: stress-ng.full.ops_per_sec -17.3% regression Message-ID: <20240412150432.4tt3w26fsfifwx5k@quack3> References: <202404101624.85684be8-oliver.sang@intel.com> <20240411115408.266zydqiwalko5k3@quack3> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: B0E082002B X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: debd9dff8wjk5t3yeumddun3m6yho1hy X-HE-Tag: 1712934275-9792 X-HE-Meta: U2FsdGVkX1+ZflWRWZ7ciOQm1/GMK6kixuF/R02We/6eAArzxnFvzuyfkSsmTtb0dLpXcBMddI9s5ehZoI0c7bDee2XEbqqnhkO2ZOUm9jAwzcyTjpdHwQfHAaWxBqTCO97eNd3OGotQwoXXS/EwM72AKDMT5LgxI1tvMiGdTnXOHjksl6n4eyymV6iRhAihEUDxYMVOD4fdH2tdGUWcjsEhaxc2GhHdc+oMnpIzw3rWmRSPeGKWYqz/SaLuTWJ1ZMjSeGrTSR3Ek0roCgDMECcv6Rsv0jrQKjM1vJjYcNcO+LNK0GbyKFal8DQTWs+o6XkdwyLLGNyuus/nKCXv8cd7JdSPryTZItiDNAeRn0VtrMPiDRZBXs17Q5Q9uTHz5Inklp71QhYif5vjArvClUBYies39IV2ECJrLqajnsubZeKyJS60vLKxhHQSD6mlj2auyPWi6hyEU9VSW13RSETF3hwVX9nxRHByhye+IQZfjtVWFwgTJvjk9TBKGPrvQohF7wmAuYkf1Q6WFYDfLTL+0/3FEA1wAuOWocw+HP4Mkkk65xUsjjYnc5VaO88MQocF3wrZ+l/Z22NCmEbKsIApuFXIwr5C2D/IuiM/ejCqcxht7yaiCNzUdd9D8FT9AzK02l1ar1EP70jrPQxP64ZfC1X2Ei2vRkIjM0qqhHEjTUKGv5Oh3BjpB8LJVZ74U12OT8i9ed4vIJ9rv5nXTnr60UW29VO/Kf9OKSJ83PLx1F3kSvf4txR7kAt/KJIMsm7KFXsEcwjxfV+TOLxIW0tZwcH2EYf8Ntzeu9vg/b3QBh3j/fjo/srL7VA5YEwepobYO9d82O9OoSvl7nkC0NRQGy7N+BsxRH/qxFhCmWWdkuUqVfijkfxt4ZT4RJ+J4PMohPWD69pOpeUc/Y5Ei3RGRpMbSpS8Ceo/DP+dGiq2FfxcRPBYOsph50HdPLIx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu 11-04-24 19:22:29, Amir Goldstein wrote: > On Thu, Apr 11, 2024 at 2:54 PM Jan Kara wrote: > > > > On Thu 11-04-24 12:23:34, Amir Goldstein wrote: > > > On Thu, Apr 11, 2024 at 4:42 AM kernel test robot wrote: > > > > for "[amir73il:fsnotify-sbconn] [fsnotify] 629f30e073: unixbench.throughput 5.8% improvement" > > > > (https://lore.kernel.org/all/202403141505.807a722b-oliver.sang@intel.com/) > > > > you requested us to test unixbench for this commit on different branches and we > > > > observed consistent performance improvement. > > > > > > > > now we noticed this commit is merged into linux-next/master, we still > > > > observed similar unixbench improvement, however, we also captured a > > > > stress-ng regression now. below details FYI. > > > > > > > > Hello, > > > > > > > > kernel test robot noticed a -17.3% regression of stress-ng.full.ops_per_sec on: > > > > > > > > > > > > commit: a5e57b4d370c6d320e5bfb0c919fe00aee29e039 ("fsnotify: optimize the case of no permission event watchers") > > > > > > Odd. This commit does add an extra fsnotify_sb_has_priority_watchers() > > > inline check for reads and writes, but the inline helper > > > fsnotify_sb_has_watchers() > > > already exists in fsnotify_parent() and it already accesses fsnotify_sb_info. > > > > > > It seems like stress-ng.full does read/write/mmap operations on /dev/full, > > > so the fsnotify_sb_info object would be that of devtmpfs. > > > > > > I think that the permission events on special files are not very relevant, > > > but I am not sure. > > > > > > Jan, any ideas? > > > > So I'm not 100% sure but this load simply seems to run 'stress-ng' with all > > the syscalls it is able to exercise (one per CPU if I'm right). Hum... > > looking at perf numbers I've noticed changes like: > > > > 0.43 ą 3% -0.2 0.21 ą 5% perf-profile.self.cycles-pp.__fsnotify_parent > > 0.00 +2.8 2.79 ą 5% perf-profile.self.cycles-pp.fsnotify_open_perm > > > > or > > > > 1.77 ą 12% +1.9 3.64 ą 8% perf-profile.calltrace.cycles-pp.rw_verify_area.vfs_read.__x64_sys_pread64.do_syscall_64.entry_SYSCALL_64_after_hwframe > > 1.71 ą 15% +1.9 3.64 ą 9% perf-profile.calltrace.cycles-pp.rw_verify_area.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe > > 0.00 +2.8 2.79 ą 5% perf-profile.calltrace.cycles-pp.fsnotify_open_perm.do_dentry_open.do_open.path_openat.do_filp_open > > > > So the savings in __fsnotify_parent() don't really outweight the costs in > > fsnotify_file()... I can see stress-ng exercises also inotify so maybe > > there's some contention on the counters which is causing the regression now > > that we have more of them? > > > > BTW, I'm not sure how you've arrived at the conclusing the test is using > > /dev/full. For all I can tell the e.g. the stress-mmap test is using a file > > in a subdir of CWD. > > > > Oh, I just saw the file stress-full.c in stress-ng and wrongly assumed that > test stress-ng.full refers to this code. > > Where do I find the code for this test? Ah, now that I've investigated the LKP details again, you're indeed right. repro-script shows how stress-ng is run and when I do that with cloned stress-ng repository, it is the test using /dev/full. So with that I'm not sure why patch adds so much cost to fsnotify_file()... Honza -- Jan Kara SUSE Labs, CR