Re: [linux-next:master] [block/mq] 574e7779cf: fio.write_iops -72.9% regression

[Oliver Sang <oliver.sang@intel.com>]

[-- Attachment #1: Type: text/plain, Size: 2538 bytes --]

hi, Jens Axboe,

On Thu, Feb 01, 2024 at 06:40:07AM -0700, Jens Axboe wrote:
> On 2/1/24 12:18 AM, Oliver Sang wrote:
> > hi, Jens Axboe,
> > 
> > On Wed, Jan 31, 2024 at 11:42:46AM -0700, Jens Axboe wrote:
> >> On 1/31/24 11:17 AM, Bart Van Assche wrote:
> >>> On 1/31/24 07:42, kernel test robot wrote:
> >>>> kernel test robot noticed a -72.9% regression of fio.write_iops on:
> >>>>
> >>>>
> >>>> commit: 574e7779cf583171acb5bf6365047bb0941b387c ("block/mq-deadline: use separate insertion lists")
> >>>> https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
> >>>>
> >>>> testcase: fio-basic
> >>>> test machine: 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory
> >>>> parameters:
> >>>>
> >>>>     runtime: 300s
> >>>>     disk: 1HDD
> >>>>     fs: xfs
> >>>>     nr_task: 100%
> >>>>     test_size: 128G
> >>>>     rw: write
> >>>>     bs: 4k
> >>>>     ioengine: io_uring
> >>>>     direct: direct
> >>>>     cpufreq_governor: performance
> >>>
> >>> The actual test is available in this file:
> >>> https://download.01.org/0day-ci/archive/20240131/202401312320.a335db14-oliver.sang@intel.com/repro-script
> >>>
> >>> I haven't found anything in that file for disabling merging. Merging
> >>> requests decreases IOPS. Does this perhaps mean that this test is
> >>> broken?
> >>
> >> It's hard to know as nothing in this email or links include the actual
> >> output of the job...
> > 
> > I attached a dmesg and 2 outputs while running tests on 574e7779cf.
> > not sure if they are helpful?
> 
> Both fio outputs is all I need, but I only see one of them attached?

while we running fio, there are below logs captured:
fio
fio.output
fio.task
fio.time

I tar them in fio.tar.gz as attached.
you can get them by 'tar xzvf fio.tar.gz'


> 
> >> But if it's fio IOPS, then those are application side and don't
> >> necessarily correlate to drive IOPS due to merging. Eg for fio iops,
> >> if it does 4k sequential and we merge to 128k, then the fio perceived
> >> iops will be 32 times larger than the device side.
> >>
> >> I'll take a look, but seems like there might be something there. By
> >> inserting into the other list, the request is also not available for
> >> merging. And the test in question does single IOs at the time.
> > 
> > if you have any debug patch want us to run, please just let us know.
> > it will be our great pleasure!
> 
> Thanks, might take you up on that, probably won't have time for this
> until next week however.
> 
> -- 
> Jens Axboe
> 

[-- Attachment #2: fio.tar.gz --]
[-- Type: application/gzip, Size: 2208 bytes --]