From: Jeff Layton <jlayton@kernel.org>
To: kernel test robot <oliver.sang@intel.com>
Cc: Thomas Gleixner <tglx@linutronix.de>,
oe-lkp@lists.linux.dev, lkp@intel.com,
Linux Memory Management List <linux-mm@kvack.org>,
Christian Brauner <brauner@kernel.org>,
Josef Bacik <josef@toxicpanda.com>,
"Darrick J. Wong" <djwong@kernel.org>,
linux-xfs@vger.kernel.org, ying.huang@intel.com,
feng.tang@intel.com, fengwei.yin@intel.com
Subject: Re: [linux-next:master] [xfs] 3062a738d7: filebench.sum_operations/s -85.0% regression
Date: Mon, 30 Sep 2024 14:21:30 -0400 [thread overview]
Message-ID: <3ae3693f35018e73cc6f629cb88c0a5e305e3137.camel@kernel.org> (raw)
In-Reply-To: <202409292200.d8132f52-oliver.sang@intel.com>
This is a known problem.
I have a fix that moves the floor handing into the timekeeper, but
Thomas said he had a better way to do this, so I haven't resent them
yet.
The patches in Christian's tree are out of date, so it may be best to
just drop them for now until I have the newer set ready.
Thanks,
Jeff
On Sun, 2024-09-29 at 22:36 +0800, kernel test robot wrote:
> hi, Jeff Layton,
>
> we reported
> "[jlayton:mgtime] [xfs] 4edee232ed: fio.write_iops -34.9% regression"
> in
> https://lore.kernel.org/all/202406141453.7a44f956-oliver.sang@intel.com/
>
> you asked us to supply further information at that time.
>
> now we noticed this commit is in linux-next/master, and we observed the
> regression for a different test - filebench. FYI.
>
>
>
> Hello,
>
> kernel test robot noticed a -85.0% regression of filebench.sum_operations/s on:
>
>
> commit: 3062a738d73c866bf50df13bc47a2223b7b47d87 ("xfs: switch to multigrain timestamps")
> https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
>
> testcase: filebench
> test machine: 96 threads 2 sockets Intel(R) Xeon(R) Platinum 8260L CPU @ 2.40GHz (Cascade Lake) with 128G memory
> parameters:
>
> disk: 1HDD
> fs: xfs
> fs2: nfsv4
> test: filemicro_rwritefsync.f
> cpufreq_governor: performance
>
>
>
>
> If you fix the issue in a separate patch/commit (i.e. not just a new version of
> the same patch/commit), kindly add following tags
> > Reported-by: kernel test robot <oliver.sang@intel.com>
> > Closes: https://lore.kernel.org/oe-lkp/202409292200.d8132f52-oliver.sang@intel.com
>
>
> Details are as below:
> -------------------------------------------------------------------------------------------------->
>
>
> The kernel config and materials to reproduce are available at:
> https://download.01.org/0day-ci/archive/20240929/202409292200.d8132f52-oliver.sang@intel.com
>
> =========================================================================================
> compiler/cpufreq_governor/disk/fs2/fs/kconfig/rootfs/tbox_group/test/testcase:
> gcc-12/performance/1HDD/nfsv4/xfs/x86_64-rhel-8.3/debian-12-x86_64-20240206.cgz/lkp-csl-2sp3/filemicro_rwritefsync.f/filebench
>
> commit:
> 42ba4ae657 ("Documentation: add a new file documenting multigrain timestamps")
> 3062a738d7 ("xfs: switch to multigrain timestamps")
>
> 42ba4ae65752b8cb 3062a738d73c866bf50df13bc47
> ---------------- ---------------------------
> %stddev %change %stddev
> \ | \
> 1.48 -13.5% 1.28 ± 5% iostat.cpu.iowait
> 4.302e+10 ± 2% -21.9% 3.361e+10 ± 2% cpuidle..time
> 2316977 -10.5% 2072537 cpuidle..usage
> 763659 ± 17% -33.4% 508644 ± 15% numa-numastat.node1.local_node
> 817625 ± 14% -30.4% 568838 ± 11% numa-numastat.node1.numa_hit
> 0.32 ± 12% -0.0 0.27 ± 6% perf-profile.children.cycles-pp.idle_cpu
> 0.31 ± 12% -0.0 0.26 ± 6% perf-profile.self.cycles-pp.idle_cpu
> 0.03 ± 88% +128.5% 0.08 ± 6% perf-sched.sch_delay.avg.ms.schedule_timeout.transaction_kthread.kthread.ret_from_fork
> 0.03 ± 88% +128.5% 0.08 ± 6% perf-sched.sch_delay.max.ms.schedule_timeout.transaction_kthread.kthread.ret_from_fork
> 523.13 -18.9% 424.32 ± 2% uptime.boot
> 48962 -18.9% 39691 ± 2% uptime.idle
> 208.10 ± 7% -67.4% 67.75 ± 52% numa-vmstat.node0.nr_mlock
> 816180 ± 14% -30.4% 567697 ± 11% numa-vmstat.node1.numa_hit
> 762194 ± 17% -33.4% 507502 ± 15% numa-vmstat.node1.numa_local
> 4823448 +10.8% 5344826 meminfo.Cached
> 244108 ± 6% -41.9% 141806 ± 29% meminfo.Dirty
> 2242446 +23.2% 2763206 meminfo.Inactive
> 1552268 +34.0% 2080318 meminfo.Inactive(file)
> 1.49 -0.2 1.29 ± 5% mpstat.cpu.all.iowait%
> 0.10 ± 2% +0.0 0.12 ± 4% mpstat.cpu.all.irq%
> 0.01 ± 4% +0.0 0.02 ± 2% mpstat.cpu.all.soft%
> 0.03 ± 2% +0.0 0.04 ± 2% mpstat.cpu.all.usr%
> 2603 ± 2% +16.6% 3036 ± 2% vmstat.io.bo
> 1.43 -13.6% 1.23 ± 7% vmstat.procs.b
> 2058 -14.7% 1756 vmstat.system.cs
> 5528 +21.9% 6738 ± 2% vmstat.system.in
> 2.10 -84.9% 0.32 ± 21% filebench.sum_bytes_mb/s
> 16385 -85.0% 2456 ± 13% filebench.sum_operations
> 273.04 -85.0% 40.94 ± 13% filebench.sum_operations/s
> 0.00 ± 14% +7.2e+05% 24.14 ± 13% filebench.sum_time_ms/op
> 273.00 -85.0% 41.00 ± 13% filebench.sum_writes/s
> 447.44 ± 2% -21.9% 349.26 ± 2% filebench.time.elapsed_time
> 447.44 ± 2% -21.9% 349.26 ± 2% filebench.time.elapsed_time.max
> 2343344 +207.9% 7214762 ± 9% filebench.time.file_system_outputs
> 8762 ± 2% -80.1% 1747 ± 6% filebench.time.voluntary_context_switches
> 269483 ± 6% -17.0% 223745 ± 8% sched_debug.cpu.clock.avg
> 269496 ± 6% -17.0% 223755 ± 8% sched_debug.cpu.clock.max
> 269474 ± 6% -17.0% 223735 ± 8% sched_debug.cpu.clock.min
> 268974 ± 6% -17.0% 223268 ± 8% sched_debug.cpu.clock_task.avg
> 269263 ± 6% -17.0% 223549 ± 8% sched_debug.cpu.clock_task.max
> 261595 ± 6% -17.5% 215932 ± 8% sched_debug.cpu.clock_task.min
> 8873 ± 4% -12.9% 7731 ± 5% sched_debug.cpu.curr->pid.max
> 1033 ± 3% -9.4% 936.02 ± 5% sched_debug.cpu.curr->pid.stddev
> 6038 ± 5% -27.7% 4366 ± 7% sched_debug.cpu.nr_switches.avg
> 977.53 ± 7% -17.2% 809.02 ± 8% sched_debug.cpu.nr_switches.min
> 269486 ± 6% -17.0% 223746 ± 8% sched_debug.cpu_clk
> 268914 ± 6% -17.0% 223174 ± 8% sched_debug.ktime
> 270076 ± 6% -16.9% 224334 ± 8% sched_debug.sched_clk
> 15708 -6.1% 14746 ± 2% proc-vmstat.nr_active_anon
> 817011 +120.3% 1799485 ± 9% proc-vmstat.nr_dirtied
> 61152 ± 6% -42.0% 35491 ± 30% proc-vmstat.nr_dirty
> 1206117 +10.8% 1335949 proc-vmstat.nr_file_pages
> 388315 +33.9% 519818 proc-vmstat.nr_inactive_file
> 18531 +1.0% 18721 proc-vmstat.nr_kernel_stack
> 16540 -3.8% 15909 proc-vmstat.nr_mapped
> 213.82 ± 4% -53.7% 99.00 ± 14% proc-vmstat.nr_mlock
> 25295 -6.6% 23625 proc-vmstat.nr_shmem
> 24447 -1.6% 24066 proc-vmstat.nr_slab_reclaimable
> 817011 +120.3% 1799485 ± 9% proc-vmstat.nr_written
> 15708 -6.1% 14746 ± 2% proc-vmstat.nr_zone_active_anon
> 388315 +33.9% 519818 proc-vmstat.nr_zone_inactive_file
> 139175 -19.7% 111713 ± 7% proc-vmstat.nr_zone_write_pending
> 1299984 -13.1% 1129235 proc-vmstat.numa_hit
> 1200290 -14.2% 1029746 proc-vmstat.numa_local
> 2273118 ± 2% -10.2% 2042156 proc-vmstat.pgalloc_normal
> 1203036 -18.6% 979402 ± 2% proc-vmstat.pgfault
> 1743917 ± 2% -13.4% 1509546 proc-vmstat.pgfree
> 1171848 -8.8% 1069087 proc-vmstat.pgpgout
> 56737 -18.4% 46304 ± 2% proc-vmstat.pgreuse
> 2.40 ± 2% +7.8% 2.59 ± 3% perf-stat.i.MPKI
> 49439015 +14.9% 56804851 ± 2% perf-stat.i.branch-instructions
> 4.34 +0.1 4.47 perf-stat.i.branch-miss-rate%
> 2634429 +20.4% 3171479 ± 2% perf-stat.i.branch-misses
> 5.07 ± 2% +0.4 5.46 ± 3% perf-stat.i.cache-miss-rate%
> 545304 ± 4% +22.0% 665397 ± 5% perf-stat.i.cache-misses
> 7567339 +11.9% 8468261 perf-stat.i.cache-references
> 2021 -15.8% 1702 perf-stat.i.context-switches
> 2.24 +3.8% 2.33 perf-stat.i.cpi
> 4.391e+08 +14.4% 5.022e+08 ± 2% perf-stat.i.cpu-cycles
> 102.06 +1.4% 103.52 perf-stat.i.cpu-migrations
> 2.401e+08 +15.0% 2.761e+08 ± 2% perf-stat.i.instructions
> 0.01 ± 6% -100.0% 0.00 perf-stat.i.metric.K/sec
> 5.33 +0.3 5.58 perf-stat.overall.branch-miss-rate%
> 7.21 ± 4% +0.7 7.86 ± 5% perf-stat.overall.cache-miss-rate%
> 49259568 +14.8% 56567448 ± 2% perf-stat.ps.branch-instructions
> 2625073 +20.3% 3158787 ± 2% perf-stat.ps.branch-misses
> 543464 ± 4% +22.0% 662949 ± 5% perf-stat.ps.cache-misses
> 7540908 +11.8% 8433223 perf-stat.ps.cache-references
> 2017 -15.8% 1697 perf-stat.ps.context-switches
> 4.373e+08 +14.3% 4.999e+08 ± 2% perf-stat.ps.cpu-cycles
> 101.82 +1.4% 103.21 perf-stat.ps.cpu-migrations
> 2.392e+08 +14.9% 2.75e+08 ± 2% perf-stat.ps.instructions
> 1.072e+11 -10.2% 9.632e+10 perf-stat.total.instructions
>
>
>
>
> Disclaimer:
> Results have been estimated based on internal Intel analysis and are provided
> for informational purposes only. Any difference in system hardware or software
> design or configuration may affect actual performance.
>
>
--
Jeff Layton <jlayton@kernel.org>
next prev parent reply other threads:[~2024-09-30 18:21 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-09-29 14:36 kernel test robot
2024-09-30 18:21 ` Jeff Layton [this message]
2024-10-01 15:09 ` Christian Brauner
2024-10-01 15:29 ` Jeff Layton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3ae3693f35018e73cc6f629cb88c0a5e305e3137.camel@kernel.org \
--to=jlayton@kernel.org \
--cc=brauner@kernel.org \
--cc=djwong@kernel.org \
--cc=feng.tang@intel.com \
--cc=fengwei.yin@intel.com \
--cc=josef@toxicpanda.com \
--cc=linux-mm@kvack.org \
--cc=linux-xfs@vger.kernel.org \
--cc=lkp@intel.com \
--cc=oe-lkp@lists.linux.dev \
--cc=oliver.sang@intel.com \
--cc=tglx@linutronix.de \
--cc=ying.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox