From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F2F9C54E58 for ; Thu, 21 Mar 2024 08:13:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 52B4E6B007B; Thu, 21 Mar 2024 04:13:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4D95B6B0082; Thu, 21 Mar 2024 04:13:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A06C6B0083; Thu, 21 Mar 2024 04:13:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2B2716B007B for ; Thu, 21 Mar 2024 04:13:05 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id C0D26805AF for ; Thu, 21 Mar 2024 08:13:04 +0000 (UTC) X-FDA: 81920330688.23.7E43CF4 Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) by imf07.hostedemail.com (Postfix) with ESMTP id DC3074000D for ; Thu, 21 Mar 2024 08:13:00 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of shikemeng@huaweicloud.com designates 45.249.212.56 as permitted sender) smtp.mailfrom=shikemeng@huaweicloud.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711008782; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hnQCO6NN/iGLItqvEHmACo2gN2tgT2T9gutQI25myug=; b=QFFQcPdsrvnMvV8bCS0wQa9LoQ70f12XARTd+l0+9eHLRY7CEDSNnxMhVjpL4YsMAX2dZe TYT7teXghgXKFO/v5x28VgLfdgZsSOeZoFtBAWubKm9G+6Q5xk1LyZjNApWiR2eFlAg9Jj jYN4bA5h1w72XNyor+kKF0XjTYsW1/M= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of shikemeng@huaweicloud.com designates 45.249.212.56 as permitted sender) smtp.mailfrom=shikemeng@huaweicloud.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711008783; a=rsa-sha256; cv=none; b=CvQJwNyFwK0ZiHb5Wc6oPdBVQ/OzDFNObb4Vq6FJUiqVsMjK+RRbPs5vhN5w8oqlT1+82n 6HLQ7CdhJvBs5saueWaJuVG5PUvxQFHCIXt1Fk3wgBfJW2QM5ugDgR/vzXeSQ5NQ4g7SO9 wiIMlYTKw1/Kpjt93FHtzwfFAXAFLJI= Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4V0dVn2WrYz4f3jd8 for ; Thu, 21 Mar 2024 16:12:49 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 5F6991A017A for ; Thu, 21 Mar 2024 16:12:55 +0800 (CST) Received: from [10.174.178.129] (unknown [10.174.178.129]) by APP1 (Coremail) with SMTP id cCh0CgCHoQoE7PtlB3B5Hg--.2097S2; Thu, 21 Mar 2024 16:12:54 +0800 (CST) Subject: Re: [PATCH 0/6] Improve visibility of writeback To: Jan Kara Cc: akpm@linux-foundation.org, tj@kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, willy@infradead.org, bfoster@redhat.com, dsterba@suse.com, mjguzik@gmail.com, dhowells@redhat.com, peterz@infradead.org References: <20240320110222.6564-1-shikemeng@huaweicloud.com> <20240320172240.7buswiv7zj2m5odg@quack3> From: Kemeng Shi Message-ID: <44e3b910-8b52-5583-f8a9-37105bf5e5b6@huaweicloud.com> Date: Thu, 21 Mar 2024 16:12:52 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <20240320172240.7buswiv7zj2m5odg@quack3> Content-Type: text/plain; charset=gbk Content-Transfer-Encoding: 7bit X-CM-TRANSID:cCh0CgCHoQoE7PtlB3B5Hg--.2097S2 X-Coremail-Antispam: 1UD129KBjvJXoWxJF1xGry7CF47XF4rWF15urg_yoW5tF4Dpa 95Cw1Utr48Z34xArsakF1aqryYy3yUXFy3Xr92vFWxCrn0gr15trWvg3yFy3W5ZrZxAFy3 JFsxZryvqr4vvaUanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUvab4IE77IF4wAFF20E14v26r4j6ryUM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28lY4IEw2IIxxk0rwA2F7IY1VAKz4 vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7Cj xVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x 0267AKxVW0oVCq3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG 6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFV Cjc4AY6r1j6r4UM4x0Y48IcVAKI48JM4IIrI8v6xkF7I0E8cxan2IY04v7Mxk0xIA0c2IE e2xFo4CEbIxvr21l42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxV Aqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1q 6r43MIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIxAIcVC0I7IYx2IY6x kF7I0E14v26r4j6F4UMIIF0xvE42xK8VAvwI8IcIk0rVWrJr0_WFyUJwCI42IY6I8E87Iv 67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyT uYvjxUrR6zUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-Rspamd-Queue-Id: DC3074000D X-Rspam-User: X-Stat-Signature: 699wf7n6qzufc77o1m44b4wqq4utmss1 X-Rspamd-Server: rspam01 X-HE-Tag: 1711008780-580043 X-HE-Meta: U2FsdGVkX1/GIISH7sY5F4vLgNPDQW0gYMJkl9dcM1D3gyvMh654+wxvnga3sqxp+ak1CdQnEpoojY1ukNr0jIHV2St0OJ+5lxEP64UZpwCP6YiC5okIqTsAjrvJ0P2Vq4sIpSM2iy4snZywzwiqjbJpjgXkfSKSVI/zUtGAnqwLeInQukYGgohxvBQqe7jon3zkcQ6l9NCGj9v1qdHDFsoZgFCAqxIy9gw2yXAt550B5lVPutytg1jQRNUupR3AzcxN+MG0AZx0c+gBRnuzuEntGcdkvpOzFOjst3q+ctPFijiELJ/etf1xGDmtXKfVzqHC4Ds2qiqsOVrqcWgMF1aomYllbCJS7EbTylFsjTfE4enUV8b67L28+sAAOePjmR7xGnaOv3LPiJfoxt7p7bwl6LLEJ8IrsqlaPJoXBSt80L1YVtTVq/cS7TU6uiRouKItAbLD2+6CEtHXa6TclfSqMrkqQ5UdHRnWt8elHi8sEEelwMnZzxanBmgViMERaDMlvYueIPhjwepqKjoE2LQjcXv81WTiJYVCzrZ09CPzfwzvA0fLNcW9FetJUQNA3BytWEx6SrjFtPZF92HwdTSUscpQtDgM56BQByvVhZ96yLSl9TxhiOJ9Upw6KrvpZpXxoZNm3hbXF1RWp473ZLAoq+5pX65nxBY+kHx95RlsxHd4Kw93Babgg+vkyjN2HIk9DFTNKhx32bE+XE8CvvlCLns1zX9BhH8q692Bgi3EJto2X36swc2RrvZi33sP/EuTVmAnF0uhUk2KDznj3QLnY/BgpIpSurIReh4uGgDm1wW9g9jiekvjjU8a1iV/zlCsy1yBEGa8Qt5T3/QJDgUr4086xVeDXg4tRZGD5WYQxSgAfrRkLTKZIySypPYSxFxTchbg/3lvsfbqVyX3ydZx/lTIYhpfuHuovYwCYdOVVI6zuiAl8HM7+5o4q8oV4eTGX/ahOwqUjfpDXiF W7O+Getn BXaONfGUWXxhqnkpOMW8yGyOodjtpxWabLDUejDstdTCWcMEs9xw8H+XdDsh9CBMHDPKiHq+eWDURS0a2RL5uPwzB/AQHeTglS6ReGA2u6dL9Ab20/6hVCn2h0cQVHLWw5rZ8xmAsbkdHyepNLelS9vmpivJ9NTn6IebY338303kyh/OHB5TP3T6W2whXe6Qhvz0j0kuVcQhatWT0LD0jx8bAq0V8hF5ik1qRKWoy5J7m5Bkt4Imf/4JzFP/21M1nl8CO3EvgmsQpGvoX4VrWOLvxCQY/ppOck/9xld1slBhl8RRRCj8+/Cu+hFdtMB6Jqyp/uTWUCSnBq7NYnzPO62VUOFu+SCLknBTa X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: on 3/21/2024 1:22 AM, Jan Kara wrote: > On Wed 20-03-24 19:02:16, Kemeng Shi wrote: >> This series tries to improve visilibity of writeback. Patch 1 make >> /sys/kernel/debug/bdi/xxx/stats show writeback info of whole bdi >> instead of only writeback info in root cgroup. Patch 2 add a new >> debug file /sys/kernel/debug/bdi/xxx/wb_stats to show per wb writeback >> info. Patch 4 add wb_monitor.py to monitor basic writeback info >> of running system, more info could be added on demand. Rest patches >> are some random cleanups. More details can be found in respective >> patches. Thanks! >> >> Following domain hierarchy is tested: >> global domain (320G) >> / \ >> cgroup domain1(10G) cgroup domain2(10G) >> | | >> bdi wb1 wb2 >> >> /* all writeback info of bdi is successfully collected */ >> # cat /sys/kernel/debug/bdi/252:16/stats: >> BdiWriteback: 448 kB >> BdiReclaimable: 1303904 kB >> BdiDirtyThresh: 189914124 kB >> DirtyThresh: 195337564 kB >> BackgroundThresh: 32516508 kB >> BdiDirtied: 3591392 kB >> BdiWritten: 2287488 kB >> BdiWriteBandwidth: 322248 kBps >> b_dirty: 0 >> b_io: 0 >> b_more_io: 2 >> b_dirty_time: 0 >> bdi_list: 1 >> state: 1 >> >> /* per wb writeback info is collected */ >> # cat /sys/kernel/debug/bdi/252:16/wb_stats: >> cat wb_stats >> WbCgIno: 1 >> WbWriteback: 0 kB >> WbReclaimable: 0 kB >> WbDirtyThresh: 0 kB >> WbDirtied: 0 kB >> WbWritten: 0 kB >> WbWriteBandwidth: 102400 kBps >> b_dirty: 0 >> b_io: 0 >> b_more_io: 0 >> b_dirty_time: 0 >> state: 1 >> WbCgIno: 4284 >> WbWriteback: 448 kB >> WbReclaimable: 818944 kB >> WbDirtyThresh: 3096524 kB >> WbDirtied: 2266880 kB >> WbWritten: 1447936 kB >> WbWriteBandwidth: 214036 kBps >> b_dirty: 0 >> b_io: 0 >> b_more_io: 1 >> b_dirty_time: 0 >> state: 5 >> WbCgIno: 4325 >> WbWriteback: 224 kB >> WbReclaimable: 819392 kB >> WbDirtyThresh: 2920088 kB >> WbDirtied: 2551808 kB >> WbWritten: 1732416 kB >> WbWriteBandwidth: 201832 kBps >> b_dirty: 0 >> b_io: 0 >> b_more_io: 1 >> b_dirty_time: 0 >> state: 5 >> >> /* monitor writeback info */ >> # ./wb_monitor.py 252:16 -c >> writeback reclaimable dirtied written avg_bw >> 252:16_1 0 0 0 0 102400 >> 252:16_4284 672 820064 9230368 8410304 685612 >> 252:16_4325 896 819840 10491264 9671648 652348 >> 252:16 1568 1639904 19721632 18081952 1440360 >> >> >> writeback reclaimable dirtied written avg_bw >> 252:16_1 0 0 0 0 102400 >> 252:16_4284 672 820064 9230368 8410304 685612 >> 252:16_4325 896 819840 10491264 9671648 652348 >> 252:16 1568 1639904 19721632 18081952 1440360 >> ... > > So I'm wondering: Are you implementing this just because this looks > interesting or do you have a real need for the functionality? Why? Hi Jan, I added debug files to test change in [1] which changes the way how dirty background threshold of wb is calculated. Without debug files, we could only monitor writeback to imply that threshold is corrected. In current patchset, debug info has not included dirty background threshold yet, I will add it when discution of calculation of dirty background threshold in [1] is done. The wb_monitor.py is suggested by Tejun in [2] to improve visibility of writeback. The script is more convenient than trace to monitor writeback behavior of the running system. Thanks [1] https://lore.kernel.org/lkml/a747dc7d-f24a-08bd-d969-d3fb35e151b7@huaweicloud.com/ [2] https://lore.kernel.org/lkml/ZcUsOb_fyvYr-zZ-@slm.duckdns.org/ > > Honza >