From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81D0BC25B10 for ; Tue, 7 May 2024 01:16:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BCF9B6B007B; Mon, 6 May 2024 21:16:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B58AE6B0082; Mon, 6 May 2024 21:16:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D1996B0083; Mon, 6 May 2024 21:16:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7F4E86B007B for ; Mon, 6 May 2024 21:16:51 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D891480984 for ; Tue, 7 May 2024 01:16:50 +0000 (UTC) X-FDA: 82089835380.22.BF1F956 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) by imf25.hostedemail.com (Postfix) with ESMTP id 68B07A000C for ; Tue, 7 May 2024 01:16:47 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=none; spf=none (imf25.hostedemail.com: domain of shikemeng@huaweicloud.com has no SPF policy when checking 45.249.212.51) smtp.mailfrom=shikemeng@huaweicloud.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715044608; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xn/eMhvc2fhroSiA+kGnszdY90Zy2nkbpCeGQIu5KOk=; b=Gu6emXgL8VEYr4UohbBmYgZxZ0DTuyr6BENTEbXxVciTcyhYY7B7F6MNeujPa+/3zQ2fk1 q9PG/nfKr+2jZApiBh2xUUep1XtTdL1HwEk+3I4n/oHOSVZixTfUIDkOoqD0A7x/Gabyz5 RlgMXcm+RYLzoB5xGDOvZDoOFV3c8FQ= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; dmarc=none; spf=none (imf25.hostedemail.com: domain of shikemeng@huaweicloud.com has no SPF policy when checking 45.249.212.51) smtp.mailfrom=shikemeng@huaweicloud.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715044608; a=rsa-sha256; cv=none; b=7zqBpCNzoRSeiMcCYL971Mi86rcjQ7AfAz2LudJvogtMwJs6pfY534xh3+6FZ717Mpa1eV njJnlLkhgmuPGj5BghofkEcV6YlPLK+NidqHRfDIUdV6LMJ6bReIUzblDLBbWEyKzpTla+ EGJJIR975U06YnCs1rX9UvOr3I8TiMQ= Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4VYL2l46pkz4f3mHZ for ; Tue, 7 May 2024 09:16:31 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 5B0DB1A0572 for ; Tue, 7 May 2024 09:16:41 +0800 (CST) Received: from [10.174.178.129] (unknown [10.174.178.129]) by APP1 (Coremail) with SMTP id cCh0CgDXGRP3gDlmE2ndLw--.2612S2; Tue, 07 May 2024 09:16:41 +0800 (CST) Subject: Re: [PATCH v2 2/4] mm: correct calculation of wb's bg_thresh in cgroup domain To: Jan Kara Cc: willy@infradead.org, akpm@linux-foundation.org, tj@kernel.org, hcochran@kernelspring.com, axboe@kernel.dk, mszeredi@redhat.com, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20240425131724.36778-1-shikemeng@huaweicloud.com> <20240425131724.36778-3-shikemeng@huaweicloud.com> <20240503093056.6povgn2shvqzpedj@quack3> From: Kemeng Shi Message-ID: <12bf104e-aeac-67a5-6e5a-bc7bdbfe4d79@huaweicloud.com> Date: Tue, 7 May 2024 09:16:39 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <20240503093056.6povgn2shvqzpedj@quack3> Content-Type: text/plain; charset=gbk Content-Transfer-Encoding: 7bit X-CM-TRANSID:cCh0CgDXGRP3gDlmE2ndLw--.2612S2 X-Coremail-Antispam: 1UD129KBjvJXoWxZw1Uur15CF4ruryrWw4kZwb_yoWrWF17pF WkJF12yr48Jr1xCrsIgayqqry8Jw4ftFW7XF9xt347tr13GF18KF17CFs0gFW5AF13GFWf ZFsxu3s7Xr1Dt3DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUv2b4IE77IF4wAFF20E14v26r4j6ryUM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28lY4IEw2IIxxk0rwA2F7IY1VAKz4 vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7Cj xVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x 0267AKxVW0oVCq3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG 6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFV Cjc4AY6r1j6r4UM4x0Y48IcVAKI48JM4IIrI8v6xkF7I0E8cxan2IY04v7Mxk0xIA0c2IE e2xFo4CEbIxvr21l42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxV Aqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1q 6r43MIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIxAIcVC0I7IYx2IY6x kF7I0E14v26r1j6r4UMIIF0xvE42xK8VAvwI8IcIk0rVWrZr1j6s0DMIIF0xvEx4A2jsIE 14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Jr0_GrUvcSsGvfC2KfnxnUUI43ZEXa 7IU1zuWJUUUUU== X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-Rspamd-Server: rspam01 X-Stat-Signature: h7bchyixmaj5oru69fc94dpn5rgq6hyn X-Rspam-User: X-Rspamd-Queue-Id: 68B07A000C X-HE-Tag: 1715044607-384464 X-HE-Meta: U2FsdGVkX18qO96lgBSSUhnrl/oafHFrXVrN1roJgEWu534L820KGlcJ764geom3HBDmTH8EP+djTGQanwG/Yu0ZR6QluPn+pALTVpBJxoi4weEdk39HHbKy/SjY1K1LYZoDG1cv4nI5pkUkKynITSMBqXdfrCOpvclUHlBZmm0szSG3zW7giCyJogSRgcpi/NW9QoSM/XkplaNjfVoFXFK8xv96gB7EHO2vuLgSNdlOfbjZCjhKxrrNuHj9b02akjBiSLj2eFRhM8Q1SFJ7d7Uiv/IRnbA1qwxKXhYtmoFojB3S592Gg/z2PIlrT71cudw3pjzmcwhHWPDy002ToL1YRD1RP4zoovZBPoxJMMRFUpiP2nTr6NzHtSZVxzwd9bZWqDt8ENjC2EwBfIUsHr+priYVgYqfB9f4CLuXUDlQyMP+A+nEYERa37JLU3f8s/Ngr7ySKbinIxLAMGSG6Wfgect3yW7d4b30zb9ORlSWuPujjQ/T18L/+qmoEaxbpFon827N8j05kNp5xNOfwQyWkARcNcaFFijI3kz6XpQA9bX3ykdpSKkgSkIqj1fNND3EHDg7v6LusNMCxKQ09kKSn2EylIkdUmGnu5Hg2xdO/FX1EtSE/D2gt3c5gYwQ1ZQibKc+G2m//VGlPDNCGu7P8h8HR+Z2X+/XwVpC5GmMOWtEO7M/ANoHK/XQkGfjH9x4qiNl6Z2j2Lg+Et6dJOZOHaPWSxY0FUtqDvMPFCQB7G9C6CpKCTJKPPqdApMmMwCDD80/pTUpuZFgv4ljDL5Xrm00FVVqme2DThf5FQNhj8cb/Mc781EFFNPh7c+toTKE2q24cPpJl/tL7q40TxILHH/1crIcMFPvSCL3BH+RV5Z0Y9jF0vtYOm5yzu0p1IJaql70JtNdtNfh65Utuvg2QaceQpKamVOsrExpYNOlOb+5kNa+1Z+V1SWCR1I/A0nW0I3ubmp3k7+UPcp SFWauhJv 1UkQ80HI3xudiHXK7YwalCN628b38pluxfKi6PyTNIBbuN1Zah9a6pH/8MVt3cfCEocCmqD/WBNaoETpgI4A9NWacxERugrFLS7x5vMPBR3sysUGkNnx+SLCDnqiCe2/cpmo8sitYrBbYeZ7Xb2pY/mZXnC/DMEeRpZjOK0opwxw96Qu5aYYEQnuObleSX+7P39kTMTcRorsokt/UfYxWzvmZyu7gsVa5zNF/ljzOocuDzamnwEQ4zPVOrA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Jan, on 5/3/2024 5:30 PM, Jan Kara wrote: > On Thu 25-04-24 21:17:22, Kemeng Shi wrote: >> The wb_calc_thresh is supposed to calculate wb's share of bg_thresh in >> global domain. To calculate wb's share of bg_thresh in cgroup domain, >> it's more reasonable to use __wb_calc_thresh in which way we calculate >> dirty_thresh in cgroup domain in balance_dirty_pages(). >> >> Consider following domain hierarchy: >> global domain (> 20G) >> / \ >> cgroup domain1(10G) cgroup domain2(10G) >> | | >> bdi wb1 wb2 >> Assume wb1 and wb2 has the same bandwidth. >> We have global domain bg_thresh > 2G, cgroup domain bg_thresh 1G. >> Then we have: >> wb's thresh in global domain = 2G * (wb bandwidth) / (system bandwidth) >> = 2G * 1/2 = 1G >> wb's thresh in cgroup domain = 1G * (wb bandwidth) / (system bandwidth) >> = 1G * 1/2 = 0.5G >> At last, wb1 and wb2 will be limited at 0.5G, the system will be limited >> at 1G which is less than global domain bg_thresh 2G. > > This was a bit hard to understand for me so I'd rephrase it as: > > wb_calc_thresh() is calculating wb's share of bg_thresh in the global > domain. However in case of cgroup writeback this is not the right thing to > do. Consider the following domain hierarchy: > > global domain (> 20G) > / \ > cgroup1 (10G) cgroup2 (10G) > | | > bdi wb1 wb2 > > and assume wb1 and wb2 have the same bandwidth and the background threshold > is set at 10%. The bg_thresh of cgroup1 and cgroup2 is going to be 1G. Now > because wb_calc_thresh(mdtc->wb, mdtc->bg_thresh) calculates per-wb > threshold in the global domain as (wb bandwidth) / (domain bandwidth) it > returns bg_thresh for wb1 as 0.5G although it has nobody to compete against > in cgroup1. > > Fix the problem by calculating wb's share of bg_thresh in the cgroup > domain. Thanks for improving the changelog. As this was merged into -mm and mm-unstable tree, I'm not sure if a new patch is needed. If there is anything I should do, please let me konw. Thanks. > >> Test as following: >> /* make it easier to observe the issue */ >> echo 300000 > /proc/sys/vm/dirty_expire_centisecs >> echo 100 > /proc/sys/vm/dirty_writeback_centisecs >> >> /* run fio in wb1 */ >> cd /sys/fs/cgroup >> echo "+memory +io" > cgroup.subtree_control >> mkdir group1 >> cd group1 >> echo 10G > memory.high >> echo 10G > memory.max >> echo $$ > cgroup.procs >> mkfs.ext4 -F /dev/vdb >> mount /dev/vdb /bdi1/ >> fio -name test -filename=/bdi1/file -size=600M -ioengine=libaio -bs=4K \ >> -iodepth=1 -rw=write -direct=0 --time_based -runtime=600 -invalidate=0 >> >> /* run fio in wb2 with a new shell */ >> cd /sys/fs/cgroup >> mkdir group2 >> cd group2 >> echo 10G > memory.high >> echo 10G > memory.max >> echo $$ > cgroup.procs >> mkfs.ext4 -F /dev/vdc >> mount /dev/vdc /bdi2/ >> fio -name test -filename=/bdi2/file -size=600M -ioengine=libaio -bs=4K \ >> -iodepth=1 -rw=write -direct=0 --time_based -runtime=600 -invalidate=0 >> >> Before fix, the wrttien pages of wb1 and wb2 reported from >> toos/writeback/wb_monitor.py keep growing. After fix, rare written pages >> are accumulated. >> There is no obvious change in fio result. >> >> Fixes: 74d369443325 ("writeback: Fix performance regression in wb_over_bg_thresh()") >> Signed-off-by: Kemeng Shi > > Besides the changelog rephrasing the change looks good. Feel free to add: > > Reviewed-by: Jan Kara > > Honza > >> --- >> mm/page-writeback.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/mm/page-writeback.c b/mm/page-writeback.c >> index 2a3b68aae336..14893b20d38c 100644 >> --- a/mm/page-writeback.c >> +++ b/mm/page-writeback.c >> @@ -2137,7 +2137,7 @@ bool wb_over_bg_thresh(struct bdi_writeback *wb) >> if (mdtc->dirty > mdtc->bg_thresh) >> return true; >> >> - thresh = wb_calc_thresh(mdtc->wb, mdtc->bg_thresh); >> + thresh = __wb_calc_thresh(mdtc, mdtc->bg_thresh); >> if (thresh < 2 * wb_stat_error()) >> reclaimable = wb_stat_sum(wb, WB_RECLAIMABLE); >> else >> -- >> 2.30.0 >>