From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1773C4828F for ; Thu, 1 Feb 2024 14:31:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E21EF6B0083; Thu, 1 Feb 2024 09:31:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DD15D6B0085; Thu, 1 Feb 2024 09:31:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C99B16B0089; Thu, 1 Feb 2024 09:31:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id B79DE6B0083 for ; Thu, 1 Feb 2024 09:31:20 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 8683B1401C4 for ; Thu, 1 Feb 2024 14:31:20 +0000 (UTC) X-FDA: 81743472720.27.79318BF Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf01.hostedemail.com (Postfix) with ESMTP id E8F1B40008 for ; Thu, 1 Feb 2024 14:31:16 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=USs4fgU5; dmarc=none; spf=pass (imf01.hostedemail.com: domain of axboe@kernel.dk designates 209.85.215.182 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706797877; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/dTVX6pxClkgwrpo39ncDD6CaHIctKZeaWD9GIB9NW4=; b=JrLJihBvSKZszcoeEkgclXaTm3mcfINWO4o47qiJCBP9f7hrzAiM3N3NHUQ8GlwIN1fw74 Da/Pls+1R1NTzsS7FSqp0bne/pqxBWIR3yG45sJfEz3vNpoPoSUvoYyHH3BIwCTYCACBLf CE/BTpwgO8KtF0DX/WZMqW6RmtX35YU= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=USs4fgU5; dmarc=none; spf=pass (imf01.hostedemail.com: domain of axboe@kernel.dk designates 209.85.215.182 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706797877; a=rsa-sha256; cv=none; b=VUZSuRJWPyjVRZFdwEp+YDQKkHM4OeaJ6vAx8sZqnLqzKdYFvAZeiLmPz+N0kbyxCQu9I1 R4X2Im7ftw1T6gP4qA9zXlhSTqj13Xp70Q8yT9y5nMFwGBmRvQ5U16A4NMaDEndmCJ1z9P xRd4JUirQUkBtI7HdeDDhknJeoRX8ok= Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-5cfb8126375so209850a12.1 for ; Thu, 01 Feb 2024 06:31:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1706797876; x=1707402676; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=/dTVX6pxClkgwrpo39ncDD6CaHIctKZeaWD9GIB9NW4=; b=USs4fgU5uBRiBeGvWSf95R6yGsVvzUKgeAKC6VdVRUNb9CjboBYiM2xiUyIVnT9f0L zKnrEqQYPuTm+5YNYndFXjWqoSmZrf4PDlMQyMP1imGxfRB8gnfLi/L5EPBoWOxPQ6dd bNsws33ATMBrIERNH7SQo857EEJSGOrODEsU3rR2CTwqvgfguC3sEsmk3NqHB/kRzGTm tSAB6xPbq6uTTMGBe2e2DXVsPoAn1HvK6585Vyt9005DaxQO/dcfbpCikFXrgNBbYiqv qDL4Z9smCAdE8P1dg8h5ksLcE6vKc67JeZOZOeKG3bH8INEII9OJTDwEo4JHH2sw4f/j i/bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706797876; x=1707402676; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/dTVX6pxClkgwrpo39ncDD6CaHIctKZeaWD9GIB9NW4=; b=hNDd3b1uTeXJ6WDIj0DoOki8KCoae/ios2SOfiHhkaq+RbDlNo1YQ9GGTDuvKyjxLM pQAYpjLcVKnIrivPXi1iAqEhddiQBpkIkzJlLAXFaIx9BfaC8KL7EuZn3DqswYuVzJst blpsUQ0fvhOIXkebo/sT5tPbvfs5P7w/wXem/cNN5jl/YNYcWPi7tYXupXy0H27Z17BB 1cDsJlHwmC21OoMHAAdUNu3J7SwFFUokrJVreLEWeOCWfCfmCPbjkIPSNGYbVtaXbCdN 5UavNexOzJFY7iqT9+deDo1ld92R2VmXb1K1iPLhhn2GVqjdjbQuygRKVMjmUSe5FFvY SW1Q== X-Forwarded-Encrypted: i=0; AJvYcCU3kPizEBDGXSRr8JIUIMmq/KsEdT4pkYUIjFYturX40Z3iYH20kZYUF0JpwI9O71Mc+GaOJZxGSN0z9S+rgTlO3tg= X-Gm-Message-State: AOJu0YwXH91epgFTIKucUPDhVlr6aNTCXUCEewKsCyFLo2P3tkl09zBi p+Dodo6VQ0CzIklri14avbGh9O++9mxMiQec5Tv1e5z1tIwoQNcCmd/qtaBYZaZCgtWw3uFVLsH Wr5o= X-Google-Smtp-Source: AGHT+IGjr3PB2u1hCazqyl/UID6CCPDJrrHd/ws2fUwZZdQsBc8VTptmDs1ULWba0p866rTSTOBJpQ== X-Received: by 2002:a6b:c9d8:0:b0:7bf:cc4d:ea53 with SMTP id z207-20020a6bc9d8000000b007bfcc4dea53mr5626454iof.0.1706797854684; Thu, 01 Feb 2024 06:30:54 -0800 (PST) X-Forwarded-Encrypted: i=0; AJvYcCVkvQdv9mMB4YBGEqQrVv/NU9tOrIDvVNLdF8HpP1cvgR5XF4fhdQ9uPu6z/meAnFpyTizrBa0QM2ZrPid9NDyqCL4TPyhZQq+WdClsBrISSLctRgpAUSOGd2hX5C1NzHZBKbiKivk1f/Y6UWNeJWIR+W+UahWqqBKo0XYJM0fIE0rv7WYUsUrS4hmPJvXy6y+1m3bcePJRIV0fdgp0aU3YCTZWc9W5ZgFx7uxfVeIUnRuo1RFjznhexTesohTPph2bCdlcg1KpgWXXu8b18loX3FyMgLlr7HZVe2Q6KM7+AEWxgKHyuKBH60sdVVCx6HnS39bx0IYJuf+tSkN8WQ== Received: from [192.168.1.116] ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id g8-20020a6b7608000000b007bffb82bfcfsm2252911iom.11.2024.02.01.06.30.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Feb 2024 06:30:54 -0800 (PST) Message-ID: Date: Thu, 1 Feb 2024 07:30:53 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [linux-next:master] [block/mq] 574e7779cf: fio.write_iops -72.9% regression Content-Language: en-US To: Oliver Sang Cc: Bart Van Assche , oe-lkp@lists.linux.dev, lkp@intel.com, Linux Memory Management List , Oleksandr Natalenko , Johannes Thumshirn , linux-block@vger.kernel.org, ying.huang@intel.com, feng.tang@intel.com, fengwei.yin@intel.com References: <202401312320.a335db14-oliver.sang@intel.com> <9d2f99d8-ecd2-413e-b910-18e05239a2b8@kernel.dk> From: Jens Axboe In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: E8F1B40008 X-Stat-Signature: 5nzohmdnjkm4g743ugqpkdzonuod1jim X-HE-Tag: 1706797876-509485 X-HE-Meta: U2FsdGVkX1+sfsaLuSswjuywLJBfoBaUkBGtaDXJiMQIBafXhHQUrLe53RLWWKQWXc7kNDy/bMsEcDoxpaBlHctvmr+KB0YQWnoCFvqk8mOHPXIY4+Yy6AEYUQwr8XKZQi2Rdjb9Yt1VrbxDbAuxKMg7NUXs/aoP5zJGX8I6Q70piHoAZ/to51SxpUm6FbzQ85JJyOyaBMBq5d6HVLlQ2ajFkQNnz9TPgdtVpADKYhENfFAHaJIzJX4xuQ1NSwSNRePu8uqwgdy2yLkotkkW5PBzAbtoyxHQyldaTDmk4ac8EQrU0GQI79V/ZpP7R54uKs9EnFplkMk9KM2OXcRao15g3sa1H57zE/JQocOl+wTVEH/h/AqaxjQyXLKbaxhl8Yd44LNvl6hSHrLlxKENgc8epRPVZBeQk6Zli9KR764b7GQ0pjIKwQOKgKioeMOnTHEsemVWKQ1ePCJEfT6M8PsyHwfyUh5xsOB4iLi2JH6/HIhncUEv6BcnyYccciiGP8XqP49M4WuIOGp8MqhoNs11qBL52MrIU9Xze/9eY+DwRQdH68DQXLzYiWGTegenNsXKP+nOwmCbZJP4WnoLRlaTzSk45xeTjqTb8qMnVVOGF8NOyJCDW8I/A3hV+NLWLgPbcFBDJMwmdLNMGu1OuD2+UwnOjEGW1yMK5W4G/um4pOebsU2H42t6Ebo5t/4l263l/wgnTILXE4mD9K0CP5UVpEWa/updLc3+KE8DprkmKibVFvnDPG6Wzrl/ZBg4T7ZdCT9bZie6M2EKUxtSdY0NUIPo8Nkc8xq5MYzFoD0LkMzwjsmB/qEBRhovNJkb1TBnEuSwHbSzC50z5H810Bp9DYUJ3MiACilXALBOJyJoGGMyoohH8ai6Bc2AuTI1lKxh6RkaFlqYH6zO5dnFKHGf8snghMunwcbeqk77+LOF4jvKTbGuwP5+KSmJ0vJkNRdP6/Uf0LPl2oJOeEB rkGkzPhe jqjFNTCqREKUuY/lbywcL/N9Nes0wuwE9lR+etfXiO39WzOZdR9MWb330maYSWVw9W725woWAAiCQlocfz/OUL2nGo0f3AxS0G7SU3DRmkg0Ur/gmiQmnkYCxUdvNGg1sUBPIL7aBcZPNqy9M6p+bKfx7mGuRaHtZgSFkmscYBy+V9qWu2zkczHtwaQjdwWAvD9vu04tu8qlC6qZwzRIspL/sUeN9Gt//WRgHuafopVKwSCJeFbYlstWQupGlY8iCjxaeowjiYwhCaJMHtWMc0bx9fJK2SCnUIsWLMxwAVPGqXbP+XkQ8SD/SI/FcsSJebG4AQ2+MK4mN9+hd31vIwnePMrSatJox15pN6saVTmwbqZJN4Vh0YRpvtob0SXAod9J4Zha/eEcRsBpKTVwIecwMv2hSrWhfGuOTjVqjMVYE9CReKr6ySNNNFbEUznoryD9CJnoUWfJXAesuXmdvvoY0UiY15alijz0WM8iCcBpou/hDCZjgdUtkxjPYUYdK8TxDi/uozsT8lW0jYcUfkiGZ0Dkw+Q1DxAr/Q7Q9BEZdjDZKWEZgrhwzbgb4jSubjoduN0//OdpD4vpdmXcsxyu18ObTeGcB3lGMK2nrtS/bLKY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2/1/24 7:03 AM, Oliver Sang wrote: > hi, Jens Axboe, > > On Thu, Feb 01, 2024 at 06:40:07AM -0700, Jens Axboe wrote: >> On 2/1/24 12:18 AM, Oliver Sang wrote: >>> hi, Jens Axboe, >>> >>> On Wed, Jan 31, 2024 at 11:42:46AM -0700, Jens Axboe wrote: >>>> On 1/31/24 11:17 AM, Bart Van Assche wrote: >>>>> On 1/31/24 07:42, kernel test robot wrote: >>>>>> kernel test robot noticed a -72.9% regression of fio.write_iops on: >>>>>> >>>>>> >>>>>> commit: 574e7779cf583171acb5bf6365047bb0941b387c ("block/mq-deadline: use separate insertion lists") >>>>>> https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master >>>>>> >>>>>> testcase: fio-basic >>>>>> test machine: 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory >>>>>> parameters: >>>>>> >>>>>> runtime: 300s >>>>>> disk: 1HDD >>>>>> fs: xfs >>>>>> nr_task: 100% >>>>>> test_size: 128G >>>>>> rw: write >>>>>> bs: 4k >>>>>> ioengine: io_uring >>>>>> direct: direct >>>>>> cpufreq_governor: performance >>>>> >>>>> The actual test is available in this file: >>>>> https://download.01.org/0day-ci/archive/20240131/202401312320.a335db14-oliver.sang@intel.com/repro-script >>>>> >>>>> I haven't found anything in that file for disabling merging. Merging >>>>> requests decreases IOPS. Does this perhaps mean that this test is >>>>> broken? >>>> >>>> It's hard to know as nothing in this email or links include the actual >>>> output of the job... >>> >>> I attached a dmesg and 2 outputs while running tests on 574e7779cf. >>> not sure if they are helpful? >> >> Both fio outputs is all I need, but I only see one of them attached? > > while we running fio, there are below logs captured: > fio > fio.output > fio.task > fio.time > > I tar them in fio.tar.gz as attached. > you can get them by 'tar xzvf fio.tar.gz' Right, but I need BOTH outputs - one from before the commit and the one on the commit. The report is a regression, hence there must be both a good and a bad run output... This looks like just the same output again, I can't really do much with just one output. -- Jens Axboe