From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DA64C2BB41 for ; Tue, 16 Aug 2022 16:38:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BCF5A6B0075; Tue, 16 Aug 2022 12:38:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B7EDB6B0078; Tue, 16 Aug 2022 12:38:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A1F9E6B007B; Tue, 16 Aug 2022 12:38:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 91FBA6B0075 for ; Tue, 16 Aug 2022 12:38:47 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 709A54091B for ; Tue, 16 Aug 2022 16:38:47 +0000 (UTC) X-FDA: 79806014694.26.5116A04 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by imf11.hostedemail.com (Postfix) with ESMTP id 5440A401CB for ; Tue, 16 Aug 2022 16:38:46 +0000 (UTC) Received: from fraeml702-chm.china.huawei.com (unknown [172.18.147.206]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4M6cD02PJ1z67lT9; Wed, 17 Aug 2022 00:33:52 +0800 (CST) Received: from lhrpeml500003.china.huawei.com (7.191.162.67) by fraeml702-chm.china.huawei.com (10.206.15.51) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2375.24; Tue, 16 Aug 2022 18:38:43 +0200 Received: from [10.48.154.245] (10.48.154.245) by lhrpeml500003.china.huawei.com (7.191.162.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Tue, 16 Aug 2022 17:38:42 +0100 Message-ID: <28d6e48b-f52f-9467-8260-262504a1a1ff@huawei.com> Date: Tue, 16 Aug 2022 17:38:43 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.6.1 Subject: Re: [ata] 0568e61225: stress-ng.copy-file.ops_per_sec -15.0% regression To: Damien Le Moal , Oliver Sang CC: Christoph Hellwig , "Martin K. Petersen" , LKML , "Linux Memory Management List" , , , , , , , References: <1f498d4a-f93f-ceb4-b713-753196e5e08d@opensource.wdc.com> <3451fa5a-6229-073f-ae18-0c232cd48ed5@huawei.com> <2e9cf5a6-c043-5ccf-e363-097c6c941891@huawei.com> <43eaa104-5b09-072c-56aa-6289569b0015@opensource.wdc.com> From: John Garry In-Reply-To: <43eaa104-5b09-072c-56aa-6289569b0015@opensource.wdc.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.48.154.245] X-ClientProxiedBy: lhrpeml500006.china.huawei.com (7.191.161.198) To lhrpeml500003.china.huawei.com (7.191.162.67) X-CFilter-Loop: Reflected ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660667927; a=rsa-sha256; cv=none; b=dPOSevoGrA906qH0r62mMOeL89vVi+sGc7ETO+UrMEtDtt79Rq7u4ErAwu0+1WYNMB8otQ tf/mbFsniQimiejBu8q95q5Vb/Rbgs7FxB1pEpGjP90EgycdaAfmt1S8++sppRX0fSH412 RCmHfqpwS4DthHNZPnSKUO4cTEZz9/0= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf11.hostedemail.com: domain of john.garry@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=john.garry@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660667927; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lGq8TCI30xd5qGlaknNe8y9AUoiI9fI+4z+Co5G/kXI=; b=RD0yqQ0AICE35ACFYI8FjPPX5fSL+autUMK4fdFzaraXzPlobH60wgJUlRef4eM+R6ue9k BoDmKwBIiz6XT+dFd6HgZ6XLs5GzYXvoIgT5oQ0huhwNyIa4bnM+x+3jw6fEl/qBCGMDQ2 T0i9hwwpcUKbGLbxUd5gqETU7s8SoaM= X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5440A401CB X-Rspam-User: Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf11.hostedemail.com: domain of john.garry@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=john.garry@huawei.com X-Stat-Signature: w594nqgwk35pt9sr3qbx5q9398ksfmr7 X-HE-Tag: 1660667926-408745 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 16/08/2022 16:42, Damien Le Moal wrote: > On 2022/08/16 3:35, John Garry wrote: >> On 16/08/2022 07:57, Oliver Sang wrote: >>>>> For me, a complete kernel log may help. >>>> and since only 1HDD, the output of the following would be helpful: >>>> >>>> /sys/block/sda/queue/max_sectors_kb >>>> /sys/block/sda/queue/max_hw_sectors_kb >>>> >>>> And for 5.19, if possible. >>> for commit >>> 0568e61225 ("ata: libata-scsi: cap ata_device->max_sectors according to shost->max_sectors") >>> >>> root@lkp-icl-2sp1 ~# cat /sys/block/sda/queue/max_sectors_kb >>> 512 >>> root@lkp-icl-2sp1 ~# cat /sys/block/sda/queue/max_hw_sectors_kb >>> 512 >>> >>> for both commit >>> 4cbfca5f77 ("scsi: scsi_transport_sas: cap shost opt_sectors according to DMA optimal limit") >>> and v5.19 >>> >>> root@lkp-icl-2sp1 ~# cat /sys/block/sda/queue/max_sectors_kb >>> 1280 >>> root@lkp-icl-2sp1 ~# cat /sys/block/sda/queue/max_hw_sectors_kb >>> 32767 >>> >> >> thanks, I appreciate this. >> >> From the dmesg, I see 2x SATA disks - I was under the impression that >> the system only has 1x. >> >> Anyway, both drives show LBA48, which means the large max hw sectors at >> 32767KB: >> [ 31.129629][ T1146] ata6.00: 1562824368 sectors, multi 1: LBA48 NCQ >> (depth 32) >> >> So this is what I suspected: we are capped from the default shost max >> sectors (1024 sectors). >> >> This seems like the simplest fix for you: >> >> --- a/include/linux/libata.h >> +++ b/include/linux/libata.h >> @@ -1382,7 +1382,8 @@ extern const struct attribute_group >> *ata_common_sdev_groups[]; >> .proc_name = drv_name, \ >> .slave_destroy = ata_scsi_slave_destroy, \ >> .bios_param = ata_std_bios_param, \ >> - .unlock_native_capacity = ata_scsi_unlock_native_capacity >> + .unlock_native_capacity = ata_scsi_unlock_native_capacity,\ >> + .max_sectors = ATA_MAX_SECTORS_LBA48 > > This is crazy large (65535 x 512 B sectors) and never result in that being > exposed as the actual max_sectors_kb since other limits will apply first > (mapping size). Here is how I read values from above for max_sectors_kb and max_hw_sectors_kb: v5.19 + 0568e61225 : 512/512 v5.19 + 0568e61225 + 4cbfca5f77 : 512/512 v5.19: 1280/32767 They are want makes sense to me, at least. Oliver, can you confirm this? Thanks! On this basis, it appears that max_hw_sectors_kb is getting capped from scsi default @ 1024 sectors by commit 0568e61225. If it were getting capped by swiotlb mapping limit then that would give us 512 sectors - this value is fixed. So for my SHT change proposal I am just trying to revert to previous behaviour in 5.19 - make max_hw_sectors_kb crazy big again. > > The regression may come not from commands becoming tiny, but from the fact that > after the patch, max_sectors_kb is too large, I don't think it is, but need confirmation. >causing a lot of overhead with > qemu swiotlb mapping and slowing down IO processing. > > Above, it can be seen that we ed up with max_sectors_kb being 1280, which is the > default for most scsi disks (including ATA drives). That is normal. But before > that, it was 512, which likely better fits qemu swiotlb and does not generate Again, I don't think this this is the case. Need confirmation. > overhead. So the above fix will not change anything I think... Thanks, John