From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E435CC46CD2 for ; Sat, 30 Dec 2023 19:16:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E1CF26B00D0; Sat, 30 Dec 2023 14:16:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DCC4A6B0117; Sat, 30 Dec 2023 14:16:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C6BC86B0198; Sat, 30 Dec 2023 14:16:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B16046B00D0 for ; Sat, 30 Dec 2023 14:16:41 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 774E580266 for ; Sat, 30 Dec 2023 19:16:41 +0000 (UTC) X-FDA: 81624441402.06.54E0BD4 Received: from s1.sapience.com (s1.sapience.com [72.84.236.66]) by imf30.hostedemail.com (Postfix) with ESMTP id 8BD0680002 for ; Sat, 30 Dec 2023 19:16:38 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=sapience.com header.s=dk-ed25519-220413 header.b=1nhNBXQv; dkim=pass header.d=sapience.com header.s=dk-rsa-220413 header.b=TO0XcrPD; dmarc=pass (policy=reject) header.from=sapience.com; spf=pass (imf30.hostedemail.com: domain of lists@sapience.com designates 72.84.236.66 as permitted sender) smtp.mailfrom=lists@sapience.com; arc=pass ("sapience.com:s=arc6-rsa-220412:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703963798; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1pXyoFmz1lTgRA1zLZsIStCGbbxT6t//SqJToS9e4ck=; b=ncsHUmL9NX2FDe/7TMZvZsHJbFA2a/XT6fXUlPtwUV+/BrzmnsHUaUQrjM9J3DEV9P1NOG tnZi3MSoaeREJdZNnhQ4Yv8fvNHGRXHqIiqX8Nq/UONecoCNbX/oeOuIn2RSas0LMhMfGX 2dPCL9TIFeYEx5+zxgoNobYXilWBivs= ARC-Authentication-Results: i=2; imf30.hostedemail.com; dkim=pass header.d=sapience.com header.s=dk-ed25519-220413 header.b=1nhNBXQv; dkim=pass header.d=sapience.com header.s=dk-rsa-220413 header.b=TO0XcrPD; dmarc=pass (policy=reject) header.from=sapience.com; spf=pass (imf30.hostedemail.com: domain of lists@sapience.com designates 72.84.236.66 as permitted sender) smtp.mailfrom=lists@sapience.com; arc=pass ("sapience.com:s=arc6-rsa-220412:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1703963798; a=rsa-sha256; cv=pass; b=zEP/KMGJh8lLOXFkHrWQQAjtrAV1XuaGKmgvfQUJBa1shMl5+ejSavRGFOhAjq0wqBM2aO nuuQonkycJPijDceT+dKT+cNdKemT5hz7pHl4rllRWypkIQQtLlXiuOqs/AqxRf2Z6DzQn xWQIXHGac/ahqAe8Gz41JkWBbs3XR08= Received: from srv8.prv.sapience.com (srv8.prv.sapience.com [x.x.x.x]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by s1.sapience.com (Postfix) with ESMTPS id 8D4C3480A25; Sat, 30 Dec 2023 14:16:37 -0500 (EST) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=sapience.com; i=@sapience.com; q=dns/txt; s=dk-ed25519-220413; t=1703963797; h=message-id : subject : from : to : cc : date : in-reply-to : references : content-type : content-transfer-encoding : mime-version : from; bh=1pXyoFmz1lTgRA1zLZsIStCGbbxT6t//SqJToS9e4ck=; b=1nhNBXQvm4m6bQhdzTmfQ4+kduqcgCuKhTJY6bvKIppUy7h+dYSUn+wsCGyBCodC5YXsN QUfscGjXQ8pvUUxCg== ARC-Seal: i=1; a=rsa-sha256; d=sapience.com; s=arc6-rsa-220412; t=1703963797; cv=none; b=Kg6/hIVi7se7LPvhdKoewOhSp+tq1dfMyh3PRgyqEcEOt0Fi+dUkxZYgJFNqq51zwbO7wScs40MGRb9W6bx2rfFng0HL03NDFvZykhTckkznmaIVUZ3FHBLrpIr6mY0TabLnJ4SGf1NtfzpvEjp+nlAgQS53mgm5ky6CLsw7+BGWMJrOvaQPnpwDzr8N9yYzzswGoinoTdDWcOURltMPF1qPWz1kjVKHgH3D5745wTnkwUEWEvzezYbaX69Pg1bbpfxYEjikAXUqjxvg5p9JBpJQQ5PeN06aYdrq43wUoi8zyPMffdi93Aj5GB0DlYCaxQtNmtTSexDR2gVRmAPGow== ARC-Message-Signature: i=1; a=rsa-sha256; d=sapience.com; s=arc6-rsa-220412; t=1703963797; c=relaxed/simple; bh=ZSEN9WdxrSB4Yv9vUSNhbHsmt9jvdauFgZdtTySjXfk=; h=DKIM-Signature:DKIM-Signature:Message-ID:Subject:From:To:Cc:Date: In-Reply-To:References:Autocrypt:Content-Type: Content-Transfer-Encoding:User-Agent:MIME-Version; b=bdYm9PaJadbQKNI+IJrAJrwBxpPKpNoVNmS7JsZF62sTQ/4+iOS+pEvs/ZoVFNQjKISzjSFWuurh2YssYNukMp2IDuajrnTTkhnVXXE0kmoLFr65QreIQAtNKJ5f8MrXoxs0oa2fDVrnSfc4pKHUeEy4ayeLXdoVCjbv28JLSijqjzIUgnaUPqdjWffGjR9ZVxhtYOv0J73/UT362dpsNqQxxgfVIv2lGWAorIBE9Z0pznM6UEU5Ea9kUbN9GwG+9LD47gma6F0H5ZL40h4IFg5URS/DYajVl4zR3SN0iOIUxC6qd2SMhlY5aAgzCNzWhG6Kr6iz5X8Fd2gcnRSjWA== ARC-Authentication-Results: i=1; arc-srv8.sapience.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sapience.com; i=@sapience.com; q=dns/txt; s=dk-rsa-220413; t=1703963797; h=message-id : subject : from : to : cc : date : in-reply-to : references : content-type : content-transfer-encoding : mime-version : from; bh=1pXyoFmz1lTgRA1zLZsIStCGbbxT6t//SqJToS9e4ck=; b=TO0XcrPDBH0phrT/CuAAmEBwOZF9/2Xe0fXfYM4NBrkY4eEogWlhSOBm80m6xcavh4NXw qbqCGUSYJ52ZVi9d0aAnZ+zmDDcUQdIbhcoYMj96dVCyKp7e9BdRKcAa4osO9gz62VzYcrV Mto16gEYCBPHvo96hIXBaq+5r0+zz1B3As7Pnjri8WZcM6MK+lvGlX72aZiEbqKpouGGHYJ xMo3mxJpfJETVmUZi0KJWuvTPNNRGixF11yphbaXZ2t4oe9pPLEOmRDn3lQCxBOOpq5QiEF YlHYJKR/LameUzj5i9EyPUMjsnYYADhyGSd6h7FFyS7pPeV0/p3CxAC2V+lw== Message-ID: <0a03664f3adba63600bddebae5b5fd299d4ebbf1.camel@sapience.com> Subject: Re: 6.6.8 stable: crash in folio_mark_dirty From: Genes Lists To: Matthew Wilcox Cc: linux-kernel@vger.kernel.org, Andrew Morton , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Date: Sat, 30 Dec 2023 14:16:36 -0500 In-Reply-To: References: <8bb29431064fc1f70a42edef75a8788dd4a0eecc.camel@sapience.com> Autocrypt: addr=lists@sapience.com; prefer-encrypt=mutual; keydata=mQENBFx5lqsBCACnoqPWNljBVS+eM+AiVpFn25csowDVqjLKEmCxJ0CWD/jj4h0opv2278TgyPrAny9Zw0Vrtb38ho6i0LqKE4mptFDxp/aRP9v+ywE8Ax/XCaOYeQ2u8eKKI4iLyjFdlzETpMfF4Xs5HKklugmhPxqcQs1S+4pzrQoch5z0aBi8A3MiGc9QVZEjfiK5NmRgibg3cHXMGcuODeijoYWDjcU/Y+yTyQ9MG4p95pUMPYBEVufR+Gaksk4xULurOlciphQC8oO9uV9X9f8CvEOBilM1eo98NL0egnOWg4hJgcBgHJ6vjalW7v+3gjoyLxbqzRDTjp63nMAF08C5RAY+ylCdABEBAAG0IEdlbmVzIExpc3RzIDxsaXN0c0BzYXBpZW5jZS5jb20+iQFUBBMBCAA+FiEEc/TBPYz5PhmLF5p8HgSPILb7+v0FAlx5lqwCGwMFCQHhM4AFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQHgSPILb7+v38fgf/T0t7OSNSi6AfSdZrt1Uy04mY7A9MqLhRn00gJMXQMiZRdlkFROUXLkLYLiZUVJRLCsXXDGHMl44EsWddwwTqx0cVGBjErT2AkKnN+o72Uc+fyI5XCMjgQafFIHXVcOBmbM9QjruL6zgbmPO8EtndHGsQ7pVhN3ysZxQfcN+wxdaufXXi8ZQzNvCMyE9dVnVWBpKRSxgN1L33SyjPnuJIdGCYz1ZIgfyUyB3AF/dK8ZQ05NW8TxFcsOF1lsku4WINNBMkd+WaSaxQ+lsCPhRiw+aL87HWccUI4wnNOdvo76f1Hork6Abd6S0UKlBDrknaNsFERvErSguAmqFMmIxhd5gzBF0mPRkWCSsGAQQB2kcPAQEHQMMxX5qfptJXZdr4Jm7Twv1ze/5Ob7HKQA6twZkcuUe8tC1NYWlsIExpc3RzIChMMCAyMDE5MDcxMCkgPGxpc3RzQ HNhcGllb mNlLmNvbT6IlgQTFggAPgIbAQULCQgHAgYVCgkICwIEFgIDAQIeAQIXgBYhBOWDKFMXGxIEDrzjCnPnZQr+j8UABQJjyX6KBQkZb0RxAAoJEHPnZQr+j8UAHP8BAPw25oPdQzhl5ZdDltRimmUBkA5e3x4mzDikRgul/3pqAPsEEeRSMwZ1fz6122qPC0v4dLUpHl4D7GMa8732SJyAArgzBF0mPVgWCSsGAQQB2kcPAQEHQB+8yWRF3SUA9RiXZR3wH7WZJ3IH09ZiCLamrdomXGXmiPUEGBYIACYCGwIWIQTlgyhTFxsSBA684wpz52UK/o/FAAUCY8l+sQUJF44Q2QCBdiAEGRYIAB0WIQRByXNdQO2KDRJ2iXo5BdB0L6Ze2wUCXSY9WAAKCRA5BdB0L6Ze2+tHAQDDg1To6YZVGo2AorjOaX4H54E08Avh+qttW+8Ly8YwGAD/eEZvKAxdDj9SvNGqtJh+76Q/fso19FpN+CRYWc+9wgMJEHPnZQr+j8UAwo4A/184tkQWY8y8K1kumSuhiSl6tbXSHNNKA3g/dsxns0UgAQC1MAJo2MJzCrIV5pyVnmHfiNDFPctA2G1RiLo/9TPnB7g4BF0mPYwSCisGAQQBl1UBBQEBB0BAXvdh6o1D7dZqc5tLM3t5KMVbxdtbTY07YFCtLbCNCgMBCAeIfgQYFggAJgIbDBYhBOWDKFMXGxIEDrzjCnPnZQr+j8UABQJjyX6xBQkXjhClAAoJEHPnZQr+j8UAgpIA/0trNf9NrQszYGZ1cq27doPaDeWDbp5HwQFdCcxTuIIRAP9x4Ia8td4dbauGMK343i+xViosCxWwh49r1gqZULEtDg== Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.50.2 MIME-Version: 1.0 X-Rspamd-Queue-Id: 8BD0680002 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: fo8ygd8nydijpi4gpps4izq1n4k8d5he X-HE-Tag: 1703963798-927290 X-HE-Meta: U2FsdGVkX18sN5ofBy7vY0gC+u1speZCKo8lhCwa1gY4MgK440gcZiAOA+GWhO2rbCYjx6CbuWGrrEfVYXWKKAe69f8MwAANk/5H5UZ9GdyR1PEC99a1VKfVQ1yd028CBFAXHkmkOslvl3RSJxAxSiNQuruvPvknZw/AvHkvgSX1CUWTpgEkX4/Hy+55bbD4zNVbqyC9rMXEJ3/PixJ2/76fAa0lmliC4Ogkd3BmO0reFsXXYoKp05JY6wLVY0yk2sWIDhIKOYREE98ph+ZkH+Lgkprq05qvwl2HSiRfnES05DbZNMWzJys0O0SAsVrJbV6Ldbm9rWwZKN6/n2iJYkf/3H9Cnyy/EmgTpm5CbmcagT8zED284G7ka7dlKI8LjYlY4e3xx5KvO/8/Wr3+66WvkkFGJG6q8G+8i9d68ZvV0WkTyEk1eGNu1yIYdQbit69suUl0QZrQXEgfSwb04W9hoLZaYrcJTSYFA4VYYIpGtW5FuAniLh2vilVeTAdk4vYoo7Aub/HzcMjvgjcRYdikey8WeTKI0M0nzSvNDMrEUPxz7XIxfxw8Ulj2Svff2x4Mdq3ZM09U5JQdtlPPQBBRp5GjfmxD5mfSM6yR4poTECwSVQZEEqNlXncqOSoMGQ4PGonWpMD/g8Q8iuq25/eHxyWvxGwRMw2APEhX23MFNE31BwdUmtZSTEiHxXo2px41tP2tbUZ6QVLcu4VVYynvdscpei95adEAkQuIjtuQP3DOmuJvvzJSDsKrQe4tzWTnNhwAG09/od9KsoC5DxAuSTIGlN4yXhjY0emH3/2N1FBZHBSVt0BvQRw9kAevf62CTF8JaS/NUpGGsOZro9ZmzAaThd4CVecVFnKzDhj2tbK1iVHC73FRYZ2ItixI20lDm41n/NOnQ5wSqKI4cAvDMTl4jj64zyGxGjX4K7yClpz7ke7Vrisc9ljJP/3H6Zfe/u1tFogzoOn5rdC NUip5M/C 8X40b52ar6ALm/4ApD7W1NiGyuA60BLTZOSI8ofoD1WUhDijSmoUX1exwncIrAalUDqsp2gzAr0Tgc4AjJ89DMYkHMLvYpsx/KvQXEF2q/+5sJieGalpFkKWp0PeAMikC30d87VGQNZ4UPssE5JPVuhPJkIIwdkzBBQyf+5c+1QGf7OHIGRl7TCY4MYO+MLeBUOw3t6ZkZU1yHit8Bfq/O9D3kdOOul7eE2YMEhocA1hhm8bByooOcjZihP8MrP/7/B0OPe8qFHosh+VPUAVjiZMWj99rMn9f5qhD9YdOGCcX+AiYefsUEySkTW2FRcn8eHF2yzFDKdA634P2E2996b3VFzO08gDfJfmPxIvQ6MZqzk+Jy58vtWTsgm4CnuSHMSMDmg1w0NV6K1+hNSH6bBAcNLVjx9JVHEEmwZ3E3xtduus= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, 2023-12-30 at 18:02 +0000, Matthew Wilcox wrote: >=20 > Thanks for the report.=C2=A0 Apologies, I'm on holiday until the middle o= f > the week so this will be extremely terse. >=20 Enjoy =F0=9F=99=82 > >=20 > Dec 30 07:00:36 s6 kernel: CPU: 0 PID: 521524 Comm: rsync Not tainted > So rsync is exiting.=C2=A0 Do you happen to know what rsync is doing? > . There are 2 rsyncs I can think of: (a) rsync from another server (s8) pushing files over the local network to this machine (s6). rsync writes to the raid drives on s6. s8 says the rsync completed successfully at 3:04 am (about 4 hours prior to this error at 7.00 am).=20 (b) There is also a script running inotify which uses rsync to keep the spare root drive sync'ed. System had update at 5:48 am of a few packages, and that would have caused an rsync from root on nvme to sapre on sdg. Most likely this is this one that triggered around 7 am. This one runs:=20 /usr/bin/rsync --open-noatime --no-specials --delete --atimes - axHAX --times > t looks llike rsync has a page from the block device mmaped?=C2=A0 I'll > have > to investigate this properly when I'm back.=C2=A0 If you haven't heard > from > me in a week, please ping me. Thank you. >=20 > (I don't think I caused this, but I think I stand a fighting chance > of > tracking down what the problem is, just not right now). This may or may not be related, but this same machine crashed during an rsync same as (a) above (i.e. s8 pushing files to the raid6 disks on s6) about 3 weeks ago - then was on 6.6.4 kernel. In that case the error was in md code. https://lore.kernel.org/lkml/e2d47b6c-3420-4785-8e04-e5f217d09a46@leemhuis.= info/T/ Thank you again, gene