Greeting, FYI, we noticed a -7.0% regression of will-it-scale.per_process_ops due to commit: commit: 92093b1bdf48286aefc8da09db3f28dab01f0449 ("srcu: Automatically determine size-transition strategy at boot") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: will-it-scale on test machine: 128 threads 2 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz with 256G memory with following parameters: nr_task: 16 mode: process test: open1 cpufreq_governor: performance ucode: 0xd000280 test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale If you fix the issue, kindly add following tag Reported-by: kernel test robot Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/process/16/debian-10.4-x86_64-20200603.cgz/lkp-icl-2sp2/open1/will-it-scale/0xd000280 commit: 9903af70f6 ("srcu: Make srcu_size_state_name static") 92093b1bdf ("srcu: Automatically determine size-transition strategy at boot") 9903af70f616d6d4 92093b1bdf48286aefc8da09db3 ---------------- --------------------------- %stddev %change %stddev \ | \ 3835166 -7.0% 3566809 will-it-scale.16.processes 239697 -7.0% 222925 will-it-scale.per_process_ops 3835166 -7.0% 3566809 will-it-scale.workload 15486741 -5.3% 14664776 proc-vmstat.numa_hit 15370774 -5.3% 14548606 proc-vmstat.numa_local 15497430 -5.3% 14675449 proc-vmstat.pgalloc_normal 15523434 -5.3% 14701758 proc-vmstat.pgfree 24577 ± 4% -10.3% 22052 ± 6% softirqs.CPU10.RCU 24213 ± 6% -9.2% 21997 ± 7% softirqs.CPU3.RCU 33754 ± 15% +29.1% 43573 ± 27% softirqs.CPU39.SCHED 24805 ± 4% -10.6% 22168 ± 6% softirqs.CPU5.RCU 23623 ± 5% -9.5% 21388 ± 7% softirqs.CPU64.RCU 25011 ± 5% -11.2% 22204 ± 6% softirqs.CPU65.RCU 25013 ± 3% -13.3% 21689 ± 6% softirqs.CPU74.RCU 4.645e+09 -6.7% 4.333e+09 perf-stat.i.branch-instructions 16545058 -7.5% 15310915 perf-stat.i.cache-misses 88146134 -7.5% 81538205 perf-stat.i.cache-references 1.91 +7.2% 2.05 perf-stat.i.cpi 166.86 -5.1% 158.31 ± 4% perf-stat.i.cpu-migrations 2756 +7.4% 2960 perf-stat.i.cycles-between-cache-misses 6.959e+09 -6.7% 6.495e+09 perf-stat.i.dTLB-loads 3.931e+09 -6.7% 3.668e+09 perf-stat.i.dTLB-stores 2.352e+10 -6.7% 2.193e+10 perf-stat.i.instructions 0.52 -6.8% 0.49 perf-stat.i.ipc 0.89 ± 3% -3.5% 0.86 perf-stat.i.major-faults 769.55 -7.6% 711.13 perf-stat.i.metric.K/sec 121.36 -6.7% 113.25 perf-stat.i.metric.M/sec 3926814 ± 2% -7.7% 3622899 ± 2% perf-stat.i.node-loads 6219114 -7.9% 5729919 ± 2% perf-stat.i.node-stores 1.91 +7.4% 2.05 perf-stat.overall.cpi 2715 +8.2% 2939 perf-stat.overall.cycles-between-cache-misses 0.52 -6.9% 0.49 perf-stat.overall.ipc 4.63e+09 -6.7% 4.318e+09 perf-stat.ps.branch-instructions 16488361 -7.5% 15259725 perf-stat.ps.cache-misses 87845545 -7.5% 81266787 perf-stat.ps.cache-references 166.22 -5.1% 157.72 ± 4% perf-stat.ps.cpu-migrations 6.935e+09 -6.7% 6.474e+09 perf-stat.ps.dTLB-loads 3.918e+09 -6.7% 3.656e+09 perf-stat.ps.dTLB-stores 2.344e+10 -6.7% 2.186e+10 perf-stat.ps.instructions 3913490 ± 2% -7.7% 3610694 ± 2% perf-stat.ps.node-loads 6198148 -7.9% 5711133 ± 2% perf-stat.ps.node-stores 7.113e+12 -6.8% 6.626e+12 perf-stat.total.instructions 2.37 -0.1 2.25 perf-profile.calltrace.cycles-pp.kmem_cache_alloc.__alloc_file.alloc_empty_file.path_openat.do_filp_open 1.54 ± 2% -0.1 1.42 ± 2% perf-profile.calltrace.cycles-pp.link_path_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open 2.07 -0.1 1.96 ± 2% perf-profile.calltrace.cycles-pp.getname_flags.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.25 -0.1 1.15 ± 2% perf-profile.calltrace.cycles-pp.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 1.28 ± 2% -0.1 1.20 ± 2% perf-profile.calltrace.cycles-pp.rcu_do_batch.rcu_core.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt 0.76 -0.1 0.69 ± 3% perf-profile.calltrace.cycles-pp.complete_walk.do_open.path_openat.do_filp_open.do_sys_openat2 0.68 ± 2% -0.1 0.61 ± 2% perf-profile.calltrace.cycles-pp.try_to_unlazy.complete_walk.do_open.path_openat.do_filp_open 0.70 ± 4% -0.1 0.63 ± 3% perf-profile.calltrace.cycles-pp.terminate_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open 0.81 ± 2% -0.1 0.74 ± 3% perf-profile.calltrace.cycles-pp.filp_close.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 0.78 ± 2% -0.1 0.72 ± 3% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.apparmor_file_open.security_file_open.do_dentry_open.do_open 0.76 ± 2% -0.1 0.71 ± 4% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.apparmor_file_open.security_file_open.do_dentry_open 0.61 ± 2% -0.1 0.56 ± 3% perf-profile.calltrace.cycles-pp.alloc_fd.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.63 ± 2% -0.0 0.58 ± 3% perf-profile.calltrace.cycles-pp.fput_many.filp_close.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.75 ± 2% -0.0 0.71 ± 2% perf-profile.calltrace.cycles-pp.kmem_cache_alloc.security_file_alloc.__alloc_file.alloc_empty_file.path_openat 12.82 +0.2 13.05 perf-profile.calltrace.cycles-pp.security_file_alloc.__alloc_file.alloc_empty_file.path_openat.do_filp_open 11.95 +0.3 12.23 perf-profile.calltrace.cycles-pp.apparmor_file_alloc_security.security_file_alloc.__alloc_file.alloc_empty_file.path_openat 14.70 +0.3 15.04 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 14.60 +0.3 14.94 perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 14.21 +0.4 14.58 perf-profile.calltrace.cycles-pp.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe 21.98 +0.4 22.41 perf-profile.calltrace.cycles-pp.ima_file_check.do_open.path_openat.do_filp_open.do_sys_openat2 21.82 +0.4 22.26 perf-profile.calltrace.cycles-pp.security_current_getsecid_subj.ima_file_check.do_open.path_openat.do_filp_open 21.77 +0.4 22.21 perf-profile.calltrace.cycles-pp.apparmor_current_getsecid_subj.security_current_getsecid_subj.ima_file_check.do_open.path_openat 12.81 +0.5 13.28 perf-profile.calltrace.cycles-pp.__fput.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64 11.05 +0.6 11.64 perf-profile.calltrace.cycles-pp.security_file_free.__fput.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode 11.00 +0.6 11.60 perf-profile.calltrace.cycles-pp.apparmor_file_free_security.security_file_free.__fput.task_work_run.exit_to_user_mode_prepare 3.34 -0.3 3.09 perf-profile.children.cycles-pp.rcu_core 3.32 -0.3 3.07 perf-profile.children.cycles-pp.rcu_do_batch 3.56 -0.2 3.32 perf-profile.children.cycles-pp.__softirqentry_text_start 3.78 -0.2 3.58 perf-profile.children.cycles-pp.kmem_cache_alloc 3.24 -0.2 3.08 ± 2% perf-profile.children.cycles-pp.irq_exit_rcu 1.75 -0.1 1.63 perf-profile.children.cycles-pp.kmem_cache_free 1.60 -0.1 1.48 ± 2% perf-profile.children.cycles-pp.link_path_walk 1.27 -0.1 1.16 ± 2% perf-profile.children.cycles-pp.__x64_sys_close 2.09 -0.1 1.98 ± 2% perf-profile.children.cycles-pp.getname_flags 1.14 -0.1 1.04 perf-profile.children.cycles-pp.__might_resched 1.06 ± 2% -0.1 0.98 ± 2% perf-profile.children.cycles-pp.dput 0.78 -0.1 0.71 ± 3% perf-profile.children.cycles-pp.complete_walk 0.73 ± 4% -0.1 0.66 ± 3% perf-profile.children.cycles-pp.terminate_walk 0.84 ± 2% -0.1 0.77 ± 3% perf-profile.children.cycles-pp.filp_close 0.71 ± 2% -0.1 0.64 ± 2% perf-profile.children.cycles-pp.try_to_unlazy 0.68 ± 2% -0.1 0.62 ± 3% perf-profile.children.cycles-pp.inode_permission 0.79 -0.1 0.73 ± 2% perf-profile.children.cycles-pp.mod_objcg_state 0.65 ± 2% -0.1 0.60 ± 3% perf-profile.children.cycles-pp.alloc_fd 0.55 ± 3% -0.1 0.50 ± 2% perf-profile.children.cycles-pp.__legitimize_path 0.91 -0.1 0.86 ± 2% perf-profile.children.cycles-pp.__entry_text_start 0.60 ± 2% -0.0 0.56 perf-profile.children.cycles-pp.__slab_free 0.60 ± 2% -0.0 0.55 ± 3% perf-profile.children.cycles-pp.file_free_rcu 0.89 ± 2% -0.0 0.84 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.55 ± 3% -0.0 0.51 ± 3% perf-profile.children.cycles-pp.lookup_fast 0.60 ± 2% -0.0 0.56 ± 4% perf-profile.children.cycles-pp.call_rcu 0.50 ± 2% -0.0 0.46 ± 4% perf-profile.children.cycles-pp._raw_spin_lock 0.46 ± 3% -0.0 0.42 ± 3% perf-profile.children.cycles-pp.may_open 0.40 -0.0 0.36 ± 6% perf-profile.children.cycles-pp.close_fd 0.55 ± 3% -0.0 0.51 ± 4% perf-profile.children.cycles-pp.walk_component 0.35 ± 3% -0.0 0.31 ± 5% perf-profile.children.cycles-pp.pick_file 0.27 ± 5% -0.0 0.24 ± 4% perf-profile.children.cycles-pp.__legitimize_mnt 0.23 ± 5% -0.0 0.21 ± 3% perf-profile.children.cycles-pp.lockref_get_not_dead 0.27 ± 3% -0.0 0.24 ± 2% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.22 ± 3% -0.0 0.20 ± 4% perf-profile.children.cycles-pp.syscall_enter_from_user_mode 12.86 +0.2 13.09 perf-profile.children.cycles-pp.security_file_alloc 12.00 +0.3 12.28 perf-profile.children.cycles-pp.apparmor_file_alloc_security 14.96 +0.3 15.28 perf-profile.children.cycles-pp.syscall_exit_to_user_mode 14.75 +0.3 15.09 perf-profile.children.cycles-pp.exit_to_user_mode_prepare 14.26 +0.4 14.63 perf-profile.children.cycles-pp.task_work_run 22.02 +0.4 22.44 perf-profile.children.cycles-pp.ima_file_check 21.85 +0.4 22.29 perf-profile.children.cycles-pp.security_current_getsecid_subj 21.81 +0.4 22.24 perf-profile.children.cycles-pp.apparmor_current_getsecid_subj 12.96 +0.5 13.43 perf-profile.children.cycles-pp.__fput 11.04 +0.6 11.64 perf-profile.children.cycles-pp.apparmor_file_free_security 11.07 +0.6 11.68 perf-profile.children.cycles-pp.security_file_free 2.09 -0.1 1.96 ± 2% perf-profile.self.cycles-pp.__alloc_file 1.00 -0.1 0.92 perf-profile.self.cycles-pp.__might_resched 0.87 ± 2% -0.1 0.80 ± 2% perf-profile.self.cycles-pp.kmem_cache_free 1.18 -0.1 1.11 ± 4% perf-profile.self.cycles-pp.kmem_cache_alloc 0.81 -0.1 0.75 ± 2% perf-profile.self.cycles-pp.do_dentry_open 0.89 ± 2% -0.1 0.84 ± 2% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.60 ± 2% -0.0 0.55 perf-profile.self.cycles-pp.__slab_free 0.39 ± 2% -0.0 0.35 ± 4% perf-profile.self.cycles-pp.do_sys_openat2 0.59 ± 2% -0.0 0.54 ± 3% perf-profile.self.cycles-pp.file_free_rcu 0.46 ± 2% -0.0 0.42 ± 3% perf-profile.self.cycles-pp._raw_spin_lock 0.41 ± 3% -0.0 0.37 ± 3% perf-profile.self.cycles-pp.lockref_put_or_lock 0.32 ± 2% -0.0 0.29 perf-profile.self.cycles-pp.__d_lookup_rcu 0.25 ± 4% -0.0 0.22 ± 4% perf-profile.self.cycles-pp.__legitimize_mnt 0.28 ± 2% -0.0 0.25 ± 4% perf-profile.self.cycles-pp.task_work_run 0.27 ± 3% -0.0 0.25 ± 3% perf-profile.self.cycles-pp.alloc_fd 0.26 ± 2% -0.0 0.24 perf-profile.self.cycles-pp.__mod_memcg_lruvec_state 0.20 ± 5% -0.0 0.18 ± 2% perf-profile.self.cycles-pp.lockref_get_not_dead 20.96 +0.3 21.24 perf-profile.self.cycles-pp.apparmor_file_open 11.43 +0.3 11.71 perf-profile.self.cycles-pp.apparmor_file_alloc_security 21.06 +0.5 21.54 perf-profile.self.cycles-pp.apparmor_current_getsecid_subj 10.64 +0.6 11.25 perf-profile.self.cycles-pp.apparmor_file_free_security Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang