From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD7EBC07E95 for ; Tue, 13 Jul 2021 10:36:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5D8416120A for ; Tue, 13 Jul 2021 10:36:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5D8416120A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 68DFC6B0095; Tue, 13 Jul 2021 06:36:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4190A6B009F; Tue, 13 Jul 2021 06:36:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2BA7C6B009B; Tue, 13 Jul 2021 06:36:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0048.hostedemail.com [216.40.44.48]) by kanga.kvack.org (Postfix) with ESMTP id EA2176B0096 for ; Tue, 13 Jul 2021 06:36:32 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 037EF180CD5EC for ; Tue, 13 Jul 2021 10:36:32 +0000 (UTC) X-FDA: 78357210624.25.22B9BCD Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf08.hostedemail.com (Postfix) with ESMTP id A4F6C30000AE for ; Tue, 13 Jul 2021 10:36:31 +0000 (UTC) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 8FE842009C; Tue, 13 Jul 2021 10:36:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1626172590; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cwd920rhpGUTHdIQz5gT6BgkDjsjvRiDvhpMr0jbX2c=; b=PK+wKQqu8M5zxp5XGd7ArSno1osTikUcbM/yIwK/bve/LE8SubRWyHv6s/lnll+aeYaiyh h7TsPWIDKTnoOW5VJDnScCLlSvf7yA1gD4TaJL7VEe3wejeLuJ18+uCTqQuxgmtGzfxFPi IUxWtxQNW8VKXoUNDoUXDGm3j4d2IzE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1626172590; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cwd920rhpGUTHdIQz5gT6BgkDjsjvRiDvhpMr0jbX2c=; b=k6p3gtsJwnlUTX2i4i39S4uGZeip0sBVvZ9QAXaV6luEgxau9jrkprEowtTrOb1b7CaDdP wXcqTC7iZsiQ2GAQ== Received: from quack2.suse.cz (unknown [10.100.224.230]) by relay2.suse.de (Postfix) with ESMTP id 838A3A3B89; Tue, 13 Jul 2021 10:36:30 +0000 (UTC) Received: by quack2.suse.cz (Postfix, from userid 1000) id 540B01E0BC4; Tue, 13 Jul 2021 12:36:30 +0200 (CEST) From: Jan Kara To: Andrew Morton Cc: , Michael Stapelberg , Wu Fengguang , Jan Kara Subject: [PATCH 5/5] writeback: Use READ_ONCE for unlocked reads of writeback stats Date: Tue, 13 Jul 2021 12:36:29 +0200 Message-Id: <20210713103630.19490-5-jack@suse.cz> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713103347.8364-1-jack@suse.cz> References: <20210713103347.8364-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5429; h=from:subject; bh=xHVEPemuC9U/YgdswQrKaWj1x38XIRUUUZFGkA2MFV8=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBg7WysQ6z44m7peGKIJAk+6VTDZjLXtUed7sdirR2J fVdpqQ6JATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYO1srAAKCRCcnaoHP2RA2eE3CA C6ARFY69Q9bQ5/2f8P4hBsJyeM+2nbCL72Rvp+a3gyd8ZOhNSP7ETBwOs+5O7/+0dpQwLjGVhgrCu4 rLxy0OcSIZDMQ5Fv7VAACof7oYEZcfYQuTNlzf7RrvLO5IzFgIkkvN6FWQmwTeIKbqgfIHW+Lbe4c3 ouJDlKE5Z0J3sEVEQOxdh0yLGnD9JihxqPdknc8w+LV0X/PetQuEQXsMZXOv8OXQ0C09HDiqXrhKZF dldd4kQSrKpQi1SESNABaICm8ah3UAUx7RtgQSjfvigkwMjKwprdtVINzVOPMwgSOipExMpF2VxqJf 69qF4GMT/7ffsrbqhAglOgT/GdCB6l X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A4F6C30000AE X-Stat-Signature: jbsip6pzn9rkfyp11681pf1djyhdh87b Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=PK+wKQqu; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=k6p3gtsJ; dmarc=none; spf=pass (imf08.hostedemail.com: domain of jack@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=jack@suse.cz X-HE-Tag: 1626172591-297120 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We do some unlocked reads of writeback statistics like avg_write_bandwidth, dirty_ratelimit, or bw_time_stamp. Generally we are fine with getting somewhat out-of-date values but actually getting different values in various parts of the functions because the compiler decided to reload value from original memory location could confuse calculations. Use READ_ONCE for these unlocked accesses and WRITE_ONCE for the updates to be on the safe side. Signed-off-by: Jan Kara --- mm/page-writeback.c | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/mm/page-writeback.c b/mm/page-writeback.c index ae5ace94e86e..2b69cd2b5db6 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -183,7 +183,7 @@ static struct fprop_local_percpu *wb_memcg_completion= s(struct bdi_writeback *wb) static void wb_min_max_ratio(struct bdi_writeback *wb, unsigned long *minp, unsigned long *maxp) { - unsigned long this_bw =3D wb->avg_write_bandwidth; + unsigned long this_bw =3D READ_ONCE(wb->avg_write_bandwidth); unsigned long tot_bw =3D atomic_long_read(&wb->bdi->tot_write_bandwidth= ); unsigned long long min =3D wb->bdi->min_ratio; unsigned long long max =3D wb->bdi->max_ratio; @@ -892,7 +892,7 @@ static long long pos_ratio_polynom(unsigned long setp= oint, static void wb_position_ratio(struct dirty_throttle_control *dtc) { struct bdi_writeback *wb =3D dtc->wb; - unsigned long write_bw =3D wb->avg_write_bandwidth; + unsigned long write_bw =3D READ_ONCE(wb->avg_write_bandwidth); unsigned long freerun =3D dirty_freerun_ceiling(dtc->thresh, dtc->bg_th= resh); unsigned long limit =3D hard_dirty_limit(dtc_dom(dtc), dtc->thresh); unsigned long wb_thresh =3D dtc->wb_thresh; @@ -1115,7 +1115,7 @@ static void wb_update_write_bandwidth(struct bdi_wr= iteback *wb, &wb->bdi->tot_write_bandwidth) <=3D 0); } wb->write_bandwidth =3D bw; - wb->avg_write_bandwidth =3D avg; + wb->avg_write_bandwidth =3D WRITE_ONCE(avg); } =20 static void update_dirty_limit(struct dirty_throttle_control *dtc) @@ -1324,7 +1324,7 @@ static void wb_update_dirty_ratelimit(struct dirty_= throttle_control *dtc, else dirty_ratelimit -=3D step; =20 - wb->dirty_ratelimit =3D max(dirty_ratelimit, 1UL); + wb->dirty_ratelimit =3D WRITE_ONCE(max(dirty_ratelimit, 1UL)); wb->balanced_dirty_ratelimit =3D balanced_dirty_ratelimit; =20 trace_bdi_dirty_ratelimit(wb, dirty_rate, task_ratelimit); @@ -1336,11 +1336,12 @@ static void __wb_update_bandwidth(struct dirty_th= rottle_control *gdtc, { struct bdi_writeback *wb =3D gdtc->wb; unsigned long now =3D jiffies; - unsigned long elapsed =3D now - wb->bw_time_stamp; + unsigned long elapsed; unsigned long dirtied; unsigned long written; =20 spin_lock(&wb->list_lock); + elapsed =3D now - wb->bw_time_stamp; dirtied =3D percpu_counter_read(&wb->stat[WB_DIRTIED]); written =3D percpu_counter_read(&wb->stat[WB_WRITTEN]); =20 @@ -1361,7 +1362,7 @@ static void __wb_update_bandwidth(struct dirty_thro= ttle_control *gdtc, =20 wb->dirtied_stamp =3D dirtied; wb->written_stamp =3D written; - wb->bw_time_stamp =3D now; + wb->bw_time_stamp =3D WRITE_ONCE(now); spin_unlock(&wb->list_lock); } =20 @@ -1385,7 +1386,7 @@ static void wb_bandwidth_estimate_start(struct bdi_= writeback *wb) spin_lock(&wb->list_lock); wb->dirtied_stamp =3D wb_stat(wb, WB_DIRTIED); wb->written_stamp =3D wb_stat(wb, WB_WRITTEN); - wb->bw_time_stamp =3D now; + wb->bw_time_stamp =3D WRITE_ONCE(now); spin_unlock(&wb->list_lock); } } @@ -1410,7 +1411,7 @@ static unsigned long dirty_poll_interval(unsigned l= ong dirty, static unsigned long wb_max_pause(struct bdi_writeback *wb, unsigned long wb_dirty) { - unsigned long bw =3D wb->avg_write_bandwidth; + unsigned long bw =3D READ_ONCE(wb->avg_write_bandwidth); unsigned long t; =20 /* @@ -1432,8 +1433,8 @@ static long wb_min_pause(struct bdi_writeback *wb, unsigned long dirty_ratelimit, int *nr_dirtied_pause) { - long hi =3D ilog2(wb->avg_write_bandwidth); - long lo =3D ilog2(wb->dirty_ratelimit); + long hi =3D ilog2(READ_ONCE(wb->avg_write_bandwidth)); + long lo =3D ilog2(READ_ONCE(wb->dirty_ratelimit)); long t; /* target pause */ long pause; /* estimated next pause */ int pages; /* target nr_dirtied_pause */ @@ -1713,12 +1714,12 @@ static void balance_dirty_pages(struct bdi_writeb= ack *wb, if (dirty_exceeded && !wb->dirty_exceeded) wb->dirty_exceeded =3D 1; =20 - if (time_is_before_jiffies(wb->bw_time_stamp + + if (time_is_before_jiffies(READ_ONCE(wb->bw_time_stamp) + BANDWIDTH_INTERVAL)) __wb_update_bandwidth(gdtc, mdtc, true); =20 /* throttle according to the chosen dtc */ - dirty_ratelimit =3D wb->dirty_ratelimit; + dirty_ratelimit =3D READ_ONCE(wb->dirty_ratelimit); task_ratelimit =3D ((u64)dirty_ratelimit * sdtc->pos_ratio) >> RATELIMIT_CALC_SHIFT; max_pause =3D wb_max_pause(wb, sdtc->wb_dirty); @@ -2368,7 +2369,8 @@ int do_writepages(struct address_space *mapping, st= ruct writeback_control *wbc) * but if there's constant writeback being submitted, this makes sure * writeback bandwidth is updated once in a while. */ - if (time_is_before_jiffies(wb->bw_time_stamp + BANDWIDTH_INTERVAL)) + if (time_is_before_jiffies(READ_ONCE(wb->bw_time_stamp) + + BANDWIDTH_INTERVAL)) wb_update_bandwidth(wb); return ret; } --=20 2.26.2