From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3C13C02196 for ; Fri, 7 Feb 2025 07:24:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4CA6E6B007B; Fri, 7 Feb 2025 02:24:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 47A296B0082; Fri, 7 Feb 2025 02:24:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36A676B0083; Fri, 7 Feb 2025 02:24:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 178716B007B for ; Fri, 7 Feb 2025 02:24:04 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 869EE816E8 for ; Fri, 7 Feb 2025 07:24:03 +0000 (UTC) X-FDA: 83092309566.18.10A6B44 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by imf27.hostedemail.com (Postfix) with ESMTP id 62FCB40008 for ; Fri, 7 Feb 2025 07:24:00 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=Hs5MSU5m; spf=pass (imf27.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738913042; a=rsa-sha256; cv=none; b=I5/vRb2fInpI9DMHehbQvtgCar96SQ0b5RhfUhYZmXVxkVq30RddTMnAdC9JHrujU4uK2z UziaMYuzTzwLZClauCKt/SEploSI2nR8o6U/7anf+9FoEH5X4NPfSomRq+5r1LslVKItTW uhixUfV4SFRTnV9fpZV8XgsI5zqDWR0= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=Hs5MSU5m; spf=pass (imf27.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738913042; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ox0wPzJ21VXBrJQcKW4ys1F7+QgqqSC9CHDYhDL6SQ4=; b=RhXk7Z6kb/Rjp3COG5Amso6L+9dHVXW+HEdSdAkAUfbg0e1lQG3YoCrolDcypkWZEFzl1s pUlo4a/mrwTfKFK7x39390lmFFqep9sad5LeMPj1I/UEirdTJOz18Y/IgdDFJwpyFyNlSZ DjLZ4mgd/KmXCsf4qdCtnx1/jddzo4Y= DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1738913037; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type; bh=Ox0wPzJ21VXBrJQcKW4ys1F7+QgqqSC9CHDYhDL6SQ4=; b=Hs5MSU5m60i+wd3fnVoBrDOM5Ah83S2BPtwM8lzm8PWdlMcLiEfvLykCbNyBRisC3DOfcG1vsrDz6yK+qOThw0J1QopMyui97QFJrJ+gQM8ZLqcPpevz1nEYcFfotuRnGnKc0U2kojyfBIkFWPfYy+PkDZicX3YBe+QdZV4WkBo= Received: from 30.74.144.113(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WOyZ-L._1738913034 cluster:ay36) by smtp.aliyun-inc.com; Fri, 07 Feb 2025 15:23:55 +0800 Message-ID: Date: Fri, 7 Feb 2025 15:23:54 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Hang when swapping huge=within_size tmpfs from zram To: Lance Yang Cc: "Alex Xu (Hello71)" , linux-mm@kvack.org, Daniel Gomez , Barry Song , David Hildenbrand , Hugh Dickins , Kefeng Wang , Matthew Wilcox , Ryan Roberts , linux-kernel@vger.kernel.org, Andrew Morton References: <1738717785.im3r5g2vxc.none.ref@localhost> <1738717785.im3r5g2vxc.none@localhost> <25e2d5e4-8214-40de-99d3-2b657181a9fd@linux.alibaba.com> <5dd39b03-c40e-4f34-bf89-b3e5a12753dc@linux.alibaba.com> From: Baolin Wang In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Stat-Signature: k3z36zpjqyondgpr5coz3orz6695kqjd X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 62FCB40008 X-Rspam-User: X-HE-Tag: 1738913040-638922 X-HE-Meta: U2FsdGVkX18Y46zoZ07AVkLiT5SKH2Ho88gkRPPb7S5ESJnslTLsUGYT8vVYXkua4dOvR3iqbAziMFJJROy/38B3YAmNSTz5exlWexR5aBiM40vdfv7RAAJ2NWG9kyDYIBLk6m2XH78on/lCIeFYJ7uVXXZFvOun9N98Zc4Cfkrnzid65GKJNMmGQ/CZ1wydT+F335ahX4Z79xpX+Z4FhyOFlnRNlfG/PI2CwxeJqdQL2mmFkl43hLFpW1XKxDf2Oq6cdusXsHY5bx2gnxgP089gRFQLwektCsFUFRbutYUhwaN+AmCrzYe4MQgqkPBJsWM+Oe6s/nsfG6H5FAUt2+yjKooBv6CVCRSMfCsgBBK8uYcjNsGnZhl6ZQ/1JpybKf6PL7uoM4QvGse3hNZeiSWE55p7P5wu8acPM+zLUm9O5bAo0jOak0FINMgW1+SqvMpx+l1pjhBMCZueW1HPEVUpIH5EaOU3nosz4lr9EJ+I12I5ewRNNuqaGa5cIZgaLFc54uLx1ilK+CqxhVktZDwYK+pqkkiXa5jA+SdqTRnKQEETmNNl+ubpkBBPMeukAMRxhqeYSpp1rgU238ZvpWPC3C7IN0azIseJ8pnQU0bvAL4k1XqxCKL6G8ZIkQ6i47Cat/aSfzFQrxdwCh9s8yMaEBEmgQrPphyP8OyNqSN4CKKlQYRCb5XnznqeHbPOwJqT+KyiPKbBPmvarr8yt1MkL7URVBGSs4Gjy06ovWoNC2jBg4P/v4zlKMmvMEouR3KQmBFxlG7KIWevM7w/tBeKWA574ClskzU8GRhrqvhqukJIK+wnyBSd6/RYrMpezD2s6TvfRWSZqTNCkJcDDX9o4BLszD2ROrEwxVAe/5/rUfs4CY7Y7GL9117ZfJJj/ctOgu8L9wiTHuiPK5dN0urNWh1C+RPIA5/IURNlT3YuvlvzsNFKSf3YEb56qWPRJQkmOKDTqgPdOPbsGtd b1x3a73A /vSXZ8g4UkGW7RAFL7MjnTtUgzFwbV+APuwaHit8p/mW2n6Pz1uGlFwt19E23YjoRfv6m+YIxUmK/PBS84Lfx2eeCVLg4fz3zgdciPw0oVZBAlkeMZIIdq6hxphqPEgJ1C4lWbNHpHb1MW8RFklJAWD8GvOQmtRhxIxw+caIwRmk/+XvdxSKwzn0HjW58miSmKIOkqexV4voG8Tao8ld7mbscpeKd8WZvFCljIqWxoSwPiIKMmncZhOtx50VQw5meYetGZPrR1FnXz2Oh77AZgHxC9+3JUsPvsW2L2szHaygllCR/592Kl1p1QhBZSN8VtkOK6rDbhreCHhAJbIam2yygGqOxXBBG5M7+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2025/2/5 22:39, Lance Yang wrote: > On Wed, Feb 5, 2025 at 2:38 PM Baolin Wang > wrote: >> >> >> >> On 2025/2/5 09:55, Baolin Wang wrote: >>> Hi Alex, >>> >>> On 2025/2/5 09:23, Alex Xu (Hello71) wrote: >>>> Hi all, >>>> >>>> On 6.14-rc1, I found that creating a lot of files in tmpfs then deleting >>>> them reliably hangs when tmpfs is mounted with huge=within_size, and it >>>> is swapped out to zram (zstd/zsmalloc/no backing dev). I bisected this >>>> to acd7ccb284b "mm: shmem: add large folio support for tmpfs". >>>> >>>> When the issue occurs, rm uses 100% CPU, cannot be killed, and has no >>>> output in /proc/pid/stack or wchan. Eventually, an RCU stall is >>>> detected: >>> >>> Thanks for your report. Let me try to reproduce the issue locally and >>> see what happens. >>> >>>> rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: >>>> rcu: Tasks blocked on level-0 rcu_node (CPUs 0-11): P25160 >>>> rcu: (detected by 10, t=2102 jiffies, g=532677, q=4997 ncpus=12) >>>> task:rm state:R running task stack:0 pid:25160 >>>> tgid:25160 ppid:24309 task_flags:0x400000 flags:0x00004004 >>>> Call Trace: >>>> >>>> ? __schedule+0x388/0x1000 >>>> ? kmem_cache_free.part.0+0x23d/0x280 >>>> ? sysvec_apic_timer_interrupt+0xa/0x80 >>>> ? asm_sysvec_apic_timer_interrupt+0x16/0x20 >>>> ? xas_load+0x12/0xc0 >>>> ? xas_load+0x8/0xc0 >>>> ? xas_find+0x144/0x190 >>>> ? find_lock_entries+0x75/0x260 >>>> ? shmem_undo_range+0xe6/0x5f0 >>>> ? shmem_evict_inode+0xe4/0x230 >>>> ? mtree_erase+0x7e/0xe0 >>>> ? inode_set_ctime_current+0x2e/0x1f0 >>>> ? evict+0xe9/0x260 >>>> ? _atomic_dec_and_lock+0x31/0x50 >>>> ? do_unlinkat+0x270/0x2b0 >>>> ? __x64_sys_unlinkat+0x30/0x50 >>>> ? do_syscall_64+0x37/0xe0 >>>> ? entry_SYSCALL_64_after_hwframe+0x50/0x58 >>>> >>>> >>>> Let me know what information is needed to further troubleshoot this >>>> issue. >> >> Sorry, I can't reproduce this issue, and my testing process is as follows: >> 1. Mount tmpfs with huge=within_size >> 2. Create and write a tmpfs file >> 3. Swap out the large folios of the tmpfs file to zram >> 4. Execute 'rm' command to remove the tmpfs file > > I’m unable to reproduce the issue as well, and am following steps similar > to Baolin's process: > > 1) Mount tmpfs with the huge=within_size option and enable swap (using > zstd/zsmalloc without a backing device). > 2) Create and write over 10,000 files in the tmpfs. > 3) Swap out the large folios of these tmpfs files to zram. > 4) Use the rm command to delete all the files from the tmpfs. > > Testing with both 2MiB and 64KiB large folio sizes, and with > shmem_enabled=within_size, but everything works as expected. Thanks Lance for confirming again. Alex, could you give more hints on how to reproduce this issue?