From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35AF3C11F67 for ; Tue, 13 Jul 2021 10:47:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DF3B4610CB for ; Tue, 13 Jul 2021 10:47:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DF3B4610CB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 439CE8D0003; Tue, 13 Jul 2021 06:47:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2FED38D0001; Tue, 13 Jul 2021 06:47:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14E358D0003; Tue, 13 Jul 2021 06:47:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0226.hostedemail.com [216.40.44.226]) by kanga.kvack.org (Postfix) with ESMTP id BC5D98D0001 for ; Tue, 13 Jul 2021 06:47:19 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C83442A00B for ; Tue, 13 Jul 2021 10:47:18 +0000 (UTC) X-FDA: 78357237756.10.07A6B26 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf13.hostedemail.com (Postfix) with ESMTP id 692E91005F63 for ; Tue, 13 Jul 2021 10:47:18 +0000 (UTC) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 4087B222B0; Tue, 13 Jul 2021 10:47:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1626173236; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l9rP1blpDrDqcFXoxgp2IAn88SNzPDEU8zo1Ko48dJ8=; b=ljyV+MCK5zvz7qc+1jNVFDLnTlCZSRJqFr4f7kLf2RVQb3OekzyV9BoKGdmdkXFFYT0f4H hiZDC5B/OyuERchcmWl4++bdEcBcbJj/Cdh7zm4pva9Qi68lGRUmTibyAkRHBKoMLirQE6 epahu0YFldBAfm8RHCnnvi/f0mjHQZc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1626173236; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l9rP1blpDrDqcFXoxgp2IAn88SNzPDEU8zo1Ko48dJ8=; b=pYtPVeEtk32aUg4c0KDC1e5FxAZj1+gFtjd80a5mGcJ1zxe8iDg34EXMyqfJZrLmXPmOrF U8HdIXpsir1DI+Bw== Received: from quack2.suse.cz (unknown [10.100.224.230]) by relay2.suse.de (Postfix) with ESMTP id 358FBA3B8C; Tue, 13 Jul 2021 10:47:16 +0000 (UTC) Received: by quack2.suse.cz (Postfix, from userid 1000) id 14AB71E0BC4; Tue, 13 Jul 2021 12:47:16 +0200 (CEST) From: Jan Kara To: Andrew Morton Cc: , Michael Stapelberg , Wu Fengguang , Jan Kara Subject: [PATCH 5/5] writeback: Use READ_ONCE for unlocked reads of writeback stats Date: Tue, 13 Jul 2021 12:47:11 +0200 Message-Id: <20210713104716.22868-5-jack@suse.cz> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713104519.16394-1-jack@suse.cz> References: <20210713104519.16394-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5425; h=from:subject; bh=L40Wzy9Mw5aXTD49yVlWjdxFK0dic9iV+AdMKLBrT80=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBg7W8ulsYXBA61mTuNaxfOofv26TYd5hLwGgQCCU98 33C4fOmJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYO1vLgAKCRCcnaoHP2RA2RI8B/ 9qO2eWdyekhlHYKcbC//XuicY9q2yQRiVdvQAfgVAj8+Dj8WNHsZqslzz/GqcSCvaZG8Ww3fTyXihL e5sl4/0sPbmQ8gBwGys8bRfQl07R2fuxX1D7TywG0EfCyk29M2hyIFqvRdNPtwxDxHz5AwjcGTwaxi lnQVW8mnAR4JR7HSFWkVUhc83OMvMMgm/xzZg5c5KOMpz2MKfuE1aGqtCkrbo0jbYRSfRghmnpuTk2 TeU+MVVxCuAlU72zKL09eDM2a5jfYkdyvTH5QJOH/KwgU4TOXckoJ6p2mTQHAp4DUdvfkATojWHIKN USKrSjFXpXKicxHGkXky7EdROO6hLZ X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=ljyV+MCK; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=pYtPVeEt; spf=pass (imf13.hostedemail.com: domain of jack@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=jack@suse.cz; dmarc=none X-Rspamd-Server: rspam02 X-Stat-Signature: gygushfrydaxb1jsux5mobabzez3prej X-Rspamd-Queue-Id: 692E91005F63 X-HE-Tag: 1626173238-326351 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We do some unlocked reads of writeback statistics like avg_write_bandwidth, dirty_ratelimit, or bw_time_stamp. Generally we are fine with getting somewhat out-of-date values but actually getting different values in various parts of the functions because the compiler decided to reload value from original memory location could confuse calculations. Use READ_ONCE for these unlocked accesses and WRITE_ONCE for the updates to be on the safe side. Signed-off-by: Jan Kara --- mm/page-writeback.c | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/mm/page-writeback.c b/mm/page-writeback.c index ae5ace94e86e..57b98ea365e2 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -183,7 +183,7 @@ static struct fprop_local_percpu *wb_memcg_completion= s(struct bdi_writeback *wb) static void wb_min_max_ratio(struct bdi_writeback *wb, unsigned long *minp, unsigned long *maxp) { - unsigned long this_bw =3D wb->avg_write_bandwidth; + unsigned long this_bw =3D READ_ONCE(wb->avg_write_bandwidth); unsigned long tot_bw =3D atomic_long_read(&wb->bdi->tot_write_bandwidth= ); unsigned long long min =3D wb->bdi->min_ratio; unsigned long long max =3D wb->bdi->max_ratio; @@ -892,7 +892,7 @@ static long long pos_ratio_polynom(unsigned long setp= oint, static void wb_position_ratio(struct dirty_throttle_control *dtc) { struct bdi_writeback *wb =3D dtc->wb; - unsigned long write_bw =3D wb->avg_write_bandwidth; + unsigned long write_bw =3D READ_ONCE(wb->avg_write_bandwidth); unsigned long freerun =3D dirty_freerun_ceiling(dtc->thresh, dtc->bg_th= resh); unsigned long limit =3D hard_dirty_limit(dtc_dom(dtc), dtc->thresh); unsigned long wb_thresh =3D dtc->wb_thresh; @@ -1115,7 +1115,7 @@ static void wb_update_write_bandwidth(struct bdi_wr= iteback *wb, &wb->bdi->tot_write_bandwidth) <=3D 0); } wb->write_bandwidth =3D bw; - wb->avg_write_bandwidth =3D avg; + WRITE_ONCE(wb->avg_write_bandwidth, avg); } =20 static void update_dirty_limit(struct dirty_throttle_control *dtc) @@ -1324,7 +1324,7 @@ static void wb_update_dirty_ratelimit(struct dirty_= throttle_control *dtc, else dirty_ratelimit -=3D step; =20 - wb->dirty_ratelimit =3D max(dirty_ratelimit, 1UL); + WRITE_ONCE(wb->dirty_ratelimit, max(dirty_ratelimit, 1UL)); wb->balanced_dirty_ratelimit =3D balanced_dirty_ratelimit; =20 trace_bdi_dirty_ratelimit(wb, dirty_rate, task_ratelimit); @@ -1336,11 +1336,12 @@ static void __wb_update_bandwidth(struct dirty_th= rottle_control *gdtc, { struct bdi_writeback *wb =3D gdtc->wb; unsigned long now =3D jiffies; - unsigned long elapsed =3D now - wb->bw_time_stamp; + unsigned long elapsed; unsigned long dirtied; unsigned long written; =20 spin_lock(&wb->list_lock); + elapsed =3D now - wb->bw_time_stamp; dirtied =3D percpu_counter_read(&wb->stat[WB_DIRTIED]); written =3D percpu_counter_read(&wb->stat[WB_WRITTEN]); =20 @@ -1361,7 +1362,7 @@ static void __wb_update_bandwidth(struct dirty_thro= ttle_control *gdtc, =20 wb->dirtied_stamp =3D dirtied; wb->written_stamp =3D written; - wb->bw_time_stamp =3D now; + WRITE_ONCE(wb->bw_time_stamp, now); spin_unlock(&wb->list_lock); } =20 @@ -1385,7 +1386,7 @@ static void wb_bandwidth_estimate_start(struct bdi_= writeback *wb) spin_lock(&wb->list_lock); wb->dirtied_stamp =3D wb_stat(wb, WB_DIRTIED); wb->written_stamp =3D wb_stat(wb, WB_WRITTEN); - wb->bw_time_stamp =3D now; + WRITE_ONCE(wb->bw_time_stamp, now); spin_unlock(&wb->list_lock); } } @@ -1410,7 +1411,7 @@ static unsigned long dirty_poll_interval(unsigned l= ong dirty, static unsigned long wb_max_pause(struct bdi_writeback *wb, unsigned long wb_dirty) { - unsigned long bw =3D wb->avg_write_bandwidth; + unsigned long bw =3D READ_ONCE(wb->avg_write_bandwidth); unsigned long t; =20 /* @@ -1432,8 +1433,8 @@ static long wb_min_pause(struct bdi_writeback *wb, unsigned long dirty_ratelimit, int *nr_dirtied_pause) { - long hi =3D ilog2(wb->avg_write_bandwidth); - long lo =3D ilog2(wb->dirty_ratelimit); + long hi =3D ilog2(READ_ONCE(wb->avg_write_bandwidth)); + long lo =3D ilog2(READ_ONCE(wb->dirty_ratelimit)); long t; /* target pause */ long pause; /* estimated next pause */ int pages; /* target nr_dirtied_pause */ @@ -1713,12 +1714,12 @@ static void balance_dirty_pages(struct bdi_writeb= ack *wb, if (dirty_exceeded && !wb->dirty_exceeded) wb->dirty_exceeded =3D 1; =20 - if (time_is_before_jiffies(wb->bw_time_stamp + + if (time_is_before_jiffies(READ_ONCE(wb->bw_time_stamp) + BANDWIDTH_INTERVAL)) __wb_update_bandwidth(gdtc, mdtc, true); =20 /* throttle according to the chosen dtc */ - dirty_ratelimit =3D wb->dirty_ratelimit; + dirty_ratelimit =3D READ_ONCE(wb->dirty_ratelimit); task_ratelimit =3D ((u64)dirty_ratelimit * sdtc->pos_ratio) >> RATELIMIT_CALC_SHIFT; max_pause =3D wb_max_pause(wb, sdtc->wb_dirty); @@ -2368,7 +2369,8 @@ int do_writepages(struct address_space *mapping, st= ruct writeback_control *wbc) * but if there's constant writeback being submitted, this makes sure * writeback bandwidth is updated once in a while. */ - if (time_is_before_jiffies(wb->bw_time_stamp + BANDWIDTH_INTERVAL)) + if (time_is_before_jiffies(READ_ONCE(wb->bw_time_stamp) + + BANDWIDTH_INTERVAL)) wb_update_bandwidth(wb); return ret; } --=20 2.26.2