From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 645F3C6FA8F for ; Wed, 30 Aug 2023 13:56:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 86D06280054; Wed, 30 Aug 2023 09:56:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 81D4F280052; Wed, 30 Aug 2023 09:56:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70B94280054; Wed, 30 Aug 2023 09:56:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 62638280052 for ; Wed, 30 Aug 2023 09:56:50 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 28F86160240 for ; Wed, 30 Aug 2023 13:56:50 +0000 (UTC) X-FDA: 81180921780.05.E84ACF8 Received: from domac.alu.hr (domac.alu.unizg.hr [161.53.235.3]) by imf08.hostedemail.com (Postfix) with ESMTP id A09EF160012 for ; Wed, 30 Aug 2023 13:56:47 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=alu.unizg.hr header.s=mail header.b="0B/i0UGw"; dkim=pass header.d=alu.unizg.hr header.s=mail header.b=AQ3Drmvv; dmarc=pass (policy=none) header.from=alu.unizg.hr; spf=pass (imf08.hostedemail.com: domain of mirsad.todorovac@alu.unizg.hr designates 161.53.235.3 as permitted sender) smtp.mailfrom=mirsad.todorovac@alu.unizg.hr ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693403808; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Blz2AgXurrDWBCLAuEzQutNHZWq8CzREmIH+fe82FfU=; b=H1E0HKm1lp7nV1JE7E8miWLIpTw5HasyU9dhQ7iEBJRC91yzZBijCZf80KwJixaYcIYffd 7KwNPPB47cCP0Miag3jSWqjSNo6A1FpfUX4wdLRqhEH6XilxCKc510r6rjwRDKE78x2Vo0 gzzRviaSIrBYz2bAU5+JzUmKEk8dncw= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=alu.unizg.hr header.s=mail header.b="0B/i0UGw"; dkim=pass header.d=alu.unizg.hr header.s=mail header.b=AQ3Drmvv; dmarc=pass (policy=none) header.from=alu.unizg.hr; spf=pass (imf08.hostedemail.com: domain of mirsad.todorovac@alu.unizg.hr designates 161.53.235.3 as permitted sender) smtp.mailfrom=mirsad.todorovac@alu.unizg.hr ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693403808; a=rsa-sha256; cv=none; b=WDIzTlRzS/dMMvH59WBY1J0CBM/hBTQZj08HpMJbULqncdE/zkr8p9iXv1TXyzOF+6fFCT LWQSdT/oDDwJmNtQTb3erpQKf1yVddCGQfY9+La3Z/94lwgvvN10pTbiP3yCVR6gfPj7YT eOEpV/8WCrRPK+QYOsqDEqPs6EOLsOE= Received: from localhost (localhost [127.0.0.1]) by domac.alu.hr (Postfix) with ESMTP id 2102860174; Wed, 30 Aug 2023 15:56:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1693403804; bh=hnLk+K6rr9gwJrJ0ChjSxYnCl7KkTNurvlFVkmbyUF8=; h=Date:Subject:From:To:Cc:References:In-Reply-To:From; b=0B/i0UGwGBgC7Lvt7X8+TLSXKan3yOZJIKfWm/HRYhSK9JQR6kne1B9lpKXPvNvKQ I/vHHLMdbCeBq99sZqZ1FSfRTQOkX8i+ipRl6nQT06Ur5DbmCVBYfx548TdesHD8i7 9k0k7gktLvQfL4kvkkty0baafhNNuG1SJ8gFdyoZsHPKUJywYylcC6ICINh9yfc51Q p3szv81RA58NBxnhmp8oLHLHD1n1QBjg/vxpDhW+LtFfh5v2SyoS7K6HMqvbphwAwF /cWykf/p62Q9IdQH33pEKF8nT7SesCxQt5NiO4717WH+HmBa/8sBSdvW/PexAmOdAK rOjYGVzgIaisw== X-Virus-Scanned: Debian amavisd-new at domac.alu.hr Received: from domac.alu.hr ([127.0.0.1]) by localhost (domac.alu.hr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C4sc4Is0z4B9; Wed, 30 Aug 2023 15:56:40 +0200 (CEST) Received: from [193.198.186.200] (pc-mtodorov.slava.alu.hr [193.198.186.200]) by domac.alu.hr (Postfix) with ESMTPSA id 1CE9660173; Wed, 30 Aug 2023 15:56:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=alu.unizg.hr; s=mail; t=1693403800; bh=hnLk+K6rr9gwJrJ0ChjSxYnCl7KkTNurvlFVkmbyUF8=; h=Date:Subject:From:To:Cc:References:In-Reply-To:From; b=AQ3DrmvvOoIYmibsT0w2sGRQlNpYFYJcHNZ0s4z1y2CtstsTVuwjHSVylL345ECEs y/GLU3VD5+vyIFGkHc6302abOzJeT9yYFkAoLrdhC01B39QGSiqZQQ2EoHLPs2UdrK sp/tzs6K8qaSg3sfsDX3kA81MhU+r98NaXZ6Lb0zrUubAgSvvsBNKNepUvXIruBFwj g0NZHBGQ36UTpxAbcbbDeXwPnZZTNDJ/VfLKUmTpxFSm+KhoXf8hvQ8W9DXaWYKeJu vvsoXueyPycxeDzzo8sVLptXfjFATpruGyKfaneEEuCegVg1xCnfow9uIWxr8SwQAo ogyOaK+XNe5zA== Message-ID: Date: Wed, 30 Aug 2023 15:56:39 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: BUG: KCSAN: data-race in folio_batch_move_lru / mpage_read_end_io Content-Language: en-US, hr From: Mirsad Todorovac To: Matthew Wilcox Cc: linux-kernel@vger.kernel.org, Andrew Morton , linux-mm@kvack.org, Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org References: <5f60813c-c52b-5c08-27c7-490b7d28c598@alu.unizg.hr> In-Reply-To: <5f60813c-c52b-5c08-27c7-490b7d28c598@alu.unizg.hr> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: A09EF160012 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 3iaxj47x5ie37gtjq4m8bxdd4sbmih7h X-HE-Tag: 1693403807-852889 X-HE-Meta: U2FsdGVkX1/4vE/QTuz/M1TDRp+1hcd3LCGT+DoreCxs25U9D9dk3/K1uZwpjUY+byD9YWF6QKcPkOHWMyYKpKA+aK+2TjhrN/RmL7ztpRLDrktC4CMZkUazRgIiJc0dahi/HB7hQHFIGKourCUppFy9iLUeZLxxxAn78qaf3jn260v3uBHYcLF7XDX/UIcMqJXe4MpioEhV+UqcDrCBRfOUMeMsg6pmMGqpf6LnwESG4nX54ir5NJFgSpjPCTxImRLtsSdwYApTKyrQvF+5IHCwvQXzMSf2+z/JUkPT74lV5q4srAV7+KrB4nAi1K54kXBrRp+lOV0YiLpfAt53GukCeYvn6we95DU7e7FTKlN+ZntR8JgKWq9sVTXBtlCmmtaj5nVYkJCgYuLBM8tcBkoqUEsN9sGve7t5jPtjv9TKvFBTyRMR/d/YMhSiO4LOkcgxPLIvBEdmuLABjr04DV1TPzAZ1vlpPSPfOMN3C2H0tBUwx8/V/v55+tOKKSoZIogQfdG+hv0hQW5zoQxXZGyLF4kbpxhbfcJnx4jUo863rEGz/hektvpqJGRtMYn99eTpqzvS1gNjpXuLlApIYaev5lPG6m3cHZbD5RA44p0UdESFs3QOrK3nV29pgeIjrC87HlFSHjaI1J3UI7jbuPjaVd/eEc8sBss7TeO7/kK02UwnuaNRPfH4AkgulFTN8d+BWZiBIqFrO+uIiY2qSGFRA0J4oQquqfJlOGkc8tt6krzDZ71w/PqDllcZWikHwtUIbjIk5KUYOcCkdACGL9EXfEvcKAhRl8q/ckRg/Rfpd6AG+DTAs+GnY3LlphQsltMupSnMdsXW8x3xMj8D8QgsftU4zG51xZAFbhRMGtAQEvF7Y82LwYCl0D8/TyN+qdQBaTQmJ0eWjKi+cuvx/i9kp3bnP+vZ+FGEdBOHVzHDIOTvpM6s/FphnIzdkbp8xPqVK6oqZ6GSr7yBGQC bP8sh6lv JffBYWi4m0YRBJEo8V1+H7roVwk+9QsiNIpMLPplNkwPbRAfvxdubXwIkxUw7KlpMSPWLIHxQlHhieVwDkdZneMVWXlJY5SBojVlfa6tBZTyQ1CNz8WEjpT9tH4h8cbwZlOMxrETMnbhNwJJdEkolB/IQOLU3Wa64kODKtA6RbCZ6Mjczla6mpySL31OgyMzGY2LSJUoV6brw9g4hGeeELtYJXzsUD5jYDW6n X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, Mr. Matthew, On 8/29/23 21:13, Matthew Wilcox wrote: > On Mon, Aug 28, 2023 at 11:14:23PM +0200, Mirsad Todorovac wrote: >> In the vanilla torvalds tree 6.5 kernel on the Ubuntu 22.04 system, KCSAN found another data race: > > KCSAN is wrong. Thank you for evaluating this bug report to such a detail. Well, I ain't giving up on KCSAN anyway, because it found some real life data races. To express a data race more graphically, it is very unpleasant when the other core changes the data from underneath you or it magically and unexpectedly changes in the course of some work ... 🙁 >> [ 34.102069] write (marked) to 0xffffef9a44978bc0 of 8 bytes by interrupt on cpu 28: >> [ 34.108569] mpage_read_end_io (/home/marvin/linux/kernel/linux_torvalds/./arch/x86/include/asm/bitops.h:55 /home/marvin/linux/kernel/linux_torvalds/./include/asm-generic/bitops/instrumented-atomic.h:29 /home/marvin/linux/kernel/linux_torvalds/./include/linux/page-flags.h:739 /home/marvin/linux/kernel/linux_torvalds/fs/mpage.c:55) > > bio_for_each_folio_all(fi, bio) { > if (err) > folio_set_error(fi.folio); > else > folio_mark_uptodate(fi.folio); > folio_unlock(fi.folio); > } > It's noting the write to folio->flags in folio_mark_uptodate(). You can > see it's locked. Also, the folio is under I/O. Yes, from folio_unlock(fi.folio), it appears that somewhere it was locked. But finding where it was locked is beyond my understanding ATM. I see folio_put() in other places, but it seems to increase refcount only, I did not where it is locked, but this is probably just me ... >> [ 34.115221] read to 0xffffef9a44978bc0 of 8 bytes by task 348 on cpu 12: >> [ 34.121702] folio_batch_move_lru (/home/marvin/linux/kernel/linux_torvalds/./include/linux/mm.h:1814 /home/marvin/linux/kernel/linux_torvalds/./include/linux/mm.h:1824 /home/marvin/linux/kernel/linux_torvalds/./include/linux/memcontrol.h:1636 /home/marvin/linux/kernel/linux_torvalds/./include/linux/memcontrol.h:1659 /home/marvin/linux/kernel/linux_torvalds/mm/swap.c:216) > > Here, it's noting the read to folio->flags that's part of page_to_nid(). > >> [ 34.121713] folio_batch_add_and_move (/home/marvin/linux/kernel/linux_torvalds/mm/swap.c:235) >> [ 34.121724] folio_add_lru (/home/marvin/linux/kernel/linux_torvalds/./arch/x86/include/asm/preempt.h:95 /home/marvin/linux/kernel/linux_torvalds/mm/swap.c:518) >> [ 34.121735] folio_add_lru_vma (/home/marvin/linux/kernel/linux_torvalds/mm/swap.c:538) >> [ 34.121746] do_anonymous_page (/home/marvin/linux/kernel/linux_torvalds/mm/memory.c:4146) > > Here we can see the page is freshly allocated. > > So KCSAN has three things wrong here. One is that the write to > folio_mark_uptodate() is setting a bit, that is nowhere near the bits > that are used for the node ID. It can't know that; it doesn't track > writes at that granularity. > > The second thing is that the node bits in folio->flags are immutable. > They're set at boot (or memory hotplug). There is never a race risk when > reading them. Presumably there needs to be some kind of annotation to > tell KCSAN that this is always safe. > > The third thing is that these two accesses cannot race. The write is > to a folio which is under I/O, so cannot be freed. The read is to a > folio which has just been allocated, so cannot be under I/O. This is > some kind of failure of KCSAN. Based on your insight, I will assume that the bug report is resolved. Thank you again for your time. Best regards, Mirsad Todorovac -- Mirsad Todorovac Sistem inženjer Grafički fakultet | Akademija likovnih umjetnosti Sveučilište u Zagrebu System engineer Faculty of Graphic Arts | Academy of Fine Arts University of Zagreb, Republic of Croatia