From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D713DC4167B for ; Wed, 6 Dec 2023 21:56:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 46E186B0089; Wed, 6 Dec 2023 16:56:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 41E5F6B0093; Wed, 6 Dec 2023 16:56:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E6696B0096; Wed, 6 Dec 2023 16:56:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1E4366B0089 for ; Wed, 6 Dec 2023 16:56:04 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id EE1B3A0F79 for ; Wed, 6 Dec 2023 21:56:03 +0000 (UTC) X-FDA: 81537751806.25.4E8EF6B Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by imf01.hostedemail.com (Postfix) with ESMTP id 1EDD240006 for ; Wed, 6 Dec 2023 21:56:00 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=mit.edu header.s=outgoing header.b=VpqBDJ31; dmarc=pass (policy=none) header.from=mit.edu; spf=pass (imf01.hostedemail.com: domain of tytso@mit.edu designates 18.9.28.11 as permitted sender) smtp.mailfrom=tytso@mit.edu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701899761; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/7ZJSSTMxPqENXQP9mS+r24/FmYE5UmiqpIimksAM+4=; b=4Tv7sA5xg/ZPQuT3/rGGOImS748uyrhFsASWxKyiPguIvZpY1SunPqJLbOWv9z0+7/UI7j ytDpp6YEYtppdHhQ4UNoXDaLiycObZeUNCHu68WNVAHzv0LR7Thivro4c+tVge/5AYnxlS DSCM/f3nuhcdCogQYmmpEhPGbHYqNIA= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=mit.edu header.s=outgoing header.b=VpqBDJ31; dmarc=pass (policy=none) header.from=mit.edu; spf=pass (imf01.hostedemail.com: domain of tytso@mit.edu designates 18.9.28.11 as permitted sender) smtp.mailfrom=tytso@mit.edu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701899761; a=rsa-sha256; cv=none; b=hAvYgSeLEbUuPXAylM1gWSyBPqup6SRx8gkFCbatTPOjON6bhF9gTGtrt2gF0QhfSQPQgi 5QzXeBQLxDB0IQPZqP+BLvsVibAJXaT3nDEskQbS4jFj+A/0EFw8thpb+e1hDFX/gpyabx UOQsW1dBH2YOZZmkquSz2xPoyRix4HY= Received: from cwcc.thunk.org (pool-173-48-122-214.bstnma.fios.verizon.net [173.48.122.214]) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 3B6LtTXD027632 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Dec 2023 16:55:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mit.edu; s=outgoing; t=1701899732; bh=/7ZJSSTMxPqENXQP9mS+r24/FmYE5UmiqpIimksAM+4=; h=Date:From:Subject:Message-ID:MIME-Version:Content-Type; b=VpqBDJ31l4kNlgq06ADF3MorBzkDiJiBFy1L2qe+RuVQX+gdSbBPZFoc9TKp1X+c9 f2Pn2b68MCJJNOLAJaz3SpEZHeR7+AbZLdTiVDYpR2FA7J03/4jsz6Px4d87Q8F54G GjEnMN+xvLTFDPJF1LOOE7HXfGjJNaue20Cr/PIzpoR/8znfIsMcdA4bvzxnlxX6DP savcig+eBkWNj2D6T+3E7mYtTiKo7/iKe+Hirv6T5fQJPM8BGAv0cFFQ6mYud82zKT t5AnVRBSyrq/Z2xdjpLPHw2KRvZw5/+3GuxOQKp0B0IMovT6bMl73p09jbFrtllaga 05aRnThSHFqTA== Received: by cwcc.thunk.org (Postfix, from userid 15806) id 38F7315C057B; Wed, 6 Dec 2023 16:55:29 -0500 (EST) Date: Wed, 6 Dec 2023 16:55:29 -0500 From: "Theodore Ts'o" To: Baokun Li Cc: Jan Kara , linux-mm@kvack.org, linux-ext4@vger.kernel.org, adilger.kernel@dilger.ca, willy@infradead.org, akpm@linux-foundation.org, ritesh.list@gmail.com, linux-kernel@vger.kernel.org, yi.zhang@huawei.com, yangerkun@huawei.com, yukuai3@huawei.com Subject: Re: [PATCH -RFC 0/2] mm/ext4: avoid data corruption when extending DIO write race with buffered read Message-ID: <20231206215529.GK509422@mit.edu> References: <20231202091432.8349-1-libaokun1@huawei.com> <20231204121120.mpxntey47rluhcfi@quack3> <20231205041755.GG509422@mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: 1EDD240006 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: qnfzzm5p85c3maduiai97i9jnfa7arny X-HE-Tag: 1701899760-587597 X-HE-Meta: U2FsdGVkX199i8CSKHsAQWI+6Yj2HrJJ3tB6U/6b0vcN/S8PkYWtKlwTUZPK1lTyWddBaWem6uRAAvEX8ajhNf+PAYJD32a5avaS8P34CseIUewPQCn/zZlASTifCdJcEHro0mMElpKudmsAlb54Myqp3iw3xJwUisBJp7E1BhrpJkEakblAVJgcgQR0Sc4hWwe0hGnHZP7/5VsOPKy6JZAmX1T31QdpGhdlfQ8LL0Pv1mmgsAqpwl2RDz6JXPJJIlObTrUQ/7IU9a0luT4XgaaK0OsJHQQoTNep60qbsBIN7es1srxSMCs8BPEpZZSnJDFl8zhwtiyXjCyVZRcf943JpkEQj3OD0ZKMlzVX7rZgmXXgFNmIJvP/YStQPqRFvyd/0TO2aoBap/elV60Q0G18onvYUM6YqDfdragfkv0Pc7fjNU8jXSOTbM6yKmHRVCbva1rHUQb9F8P6cP3C77vusOs19Q5vrCTyPufEdk+riOO6R3nRzejf8DSktdMMCPWdZmvYW+oXRiT5aSo3xJrW1CmByL8JqBEEbpn3y5MKjCTXP0LEELCmKc1J+yI+kvEDcuCx0tEiMgpi365h40MjEAuRtHBAc7p+ui0XyQHlmmtvSwFlYQ/8D/8LCZuZgOzGzIWunDmGsg+MLRqfVZqRRFAD2Iau6zhnro2DM/+EG6ba17cNJJ0ZKthvpRclfAcXXVKrfbLnsXeZcfZubt+OoB9gcsp1OPOe/zLj2TQ9vsoJGxHCtg22Hiu0SBZQ+etYbrK/qPZr2e37W1UlclJAlDExpHOwOQmkEDzAob4WDpKlyUtqTjhtgqRj5OsxBLRXslOiMyRGq2EmI6wvlVEzmGCK1eYMigRqVPGLMLI8+UaHC8Pd9xIiVeIhi/ROycVIlY6pkQ51hHNwlmxBdMi/AtpktzuHVvfol8pqYscEgKO7vJB1lUjmEhiDeAqHKN1ntiQ5rOjB3I+84uP Kt4v+a8J yA/AUNaRxfLPBaZk8F4Cz+pteStJ5kQqjnkSpWBkKDEJU3hWfcw5erg9R+3snbhc3hLhSAaUfrGLzTaAkbVejlxyGOALFoiFIKRtyK7GQvwhQMBMVs5YU+xT02TrKaYHJEzY5w7PdOFV+7TCSmx8LHfeLuTn4UTZH/5IAvQAIIoPnQzW0RiSi2zP/SZVc76N6Z5knWeUCjR+YqgSYpYMFdn0Xf3beLUAh14pMsMT82q5VpnbpbkLD3XbOjTl6iGZV4QYkkzJnQ7Vm/XRiB8XregEtiSNCp4Fwo/fBPskz4u9VBbG0B7TF3BScrA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Dec 05, 2023 at 09:19:03PM +0800, Baokun Li wrote: > Thank you very much for your detailed explanation! > But the downstream users do have buffered reads to read the relay log > file, as I confirmed with bpftrace. Here's an introduction to turning on > relay logging, but I'm not sure if you can access this link: > https://blog.csdn.net/javaanddonet/article/details/112596148 Well, if a mysql-supplied program is trying read the relay log using a buffered read, when it's being written using Direct I/O, that's a bug in mysql, and it should be reported as such to the mysql folks. It does look like there is a newer way of doing replication which doesn't rely on messign with log files. From: https://dev.mysql.com/doc/refman/8.0/en/replication.html MySQL 8.0 supports different methods of replication. The traditional method is based on replicating events from the source's binary log, and requires the log files and positions in them to be synchronized between source and replica. The newer method based on global transaction identifiers (GTIDs) is transactional and therefore does not require working with log files or positions within these files, which greatly simplifies many common replication tasks. Replication using GTIDs guarantees consistency between source and replica as long as all transactions committed on the source have also been applied on the replica. For more information about GTIDs and GTID-based replication in MySQL, see Section 17.1.3, “Replication with Global Transaction Identifiers”. For information on using binary log file position based replication, see Section 17.1, “Configuring Replication”. Perhaps you can try and see how mysql handles GTID-based replication using bpftrace? Cheers, - Ted