From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6606AC54791 for ; Sun, 10 Mar 2024 16:31:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 78E006B0072; Sun, 10 Mar 2024 12:31:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 73E4E6B0074; Sun, 10 Mar 2024 12:31:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6054E6B0075; Sun, 10 Mar 2024 12:31:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 507A46B0072 for ; Sun, 10 Mar 2024 12:31:32 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id CB5F8C0386 for ; Sun, 10 Mar 2024 16:31:31 +0000 (UTC) X-FDA: 81881669982.08.4CB090D Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf19.hostedemail.com (Postfix) with ESMTP id EF5FE1A001A for ; Sun, 10 Mar 2024 16:31:29 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf19.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710088290; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3EycFxreD44zSwrnzNNbhm/sLw2PpjCyjNL8p/S4uYA=; b=8NDGsqzbA8MWZea/OJICZ+FW1zwDKii269wRFD3JZCvTinY4b70tuivYpa89fXYtbv+0F2 rzONnTRMaegKTpzKs2peRQv+IMmxs9VGeJCG/ZU6N6WSPJXTBlDfkPUK2loFeqss1n/W/x On2emEgKpcwS9r4/qvHs3LwDpfptJL4= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf19.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710088290; a=rsa-sha256; cv=none; b=PkbpJuwMlhhSYJ2FmObzFDzyBL1Nep1y9je1n8+hNUeJaJ5jMJ9B59WhpHK7o7euSSzBlH w9YUEi4s5M+fUXIG8tvGhDvJR4+m1GavyQVjGnpR15lLHGXuKNHty5XhFNnxZnkuov4egE NBKZ2sdAOWWQ7j1fmLPb3nxOcuTCaGU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 36F1F113E; Sun, 10 Mar 2024 09:32:05 -0700 (PDT) Received: from [10.57.68.196] (unknown [10.57.68.196]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3A2B33F73F; Sun, 10 Mar 2024 09:31:28 -0700 (PDT) Message-ID: <02e820c2-8a1d-42cc-954b-f9e041c4417a@arm.com> Date: Sun, 10 Mar 2024 16:31:25 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 10/18] mm: Allow non-hugetlb large folios to be batch processed Content-Language: en-GB To: Matthew Wilcox Cc: Andrew Morton , linux-mm@kvack.org References: <20240227174254.710559-1-willy@infradead.org> <20240227174254.710559-11-willy@infradead.org> <367a14f7-340e-4b29-90ae-bc3fcefdd5f4@arm.com> <8cd67a3d-81a7-4127-9d17-a1d465c3f9e8@arm.com> From: Ryan Roberts In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: EF5FE1A001A X-Stat-Signature: hphpwwekh4cnms3fiy6rj3t86skeunrj X-Rspam-User: X-HE-Tag: 1710088289-381092 X-HE-Meta: U2FsdGVkX1/A8K4mYPqtXAYbCmlYnKz/rPTdRuumejDYjgJ1u4TWE5tC6Y52Q6lxOONuI1ZbeDccjBFZ5D0hTIgbcTRSOUl1d5rQ5xYnMeFVgvCI2qURL/FXnZ/fzdO1it2/FTICFHQjFDUYaQpMOsBtVslc+SaJ62Ip1gT2Tz0sXet4V27yXJqQz2ekA6C5Erv/xtc5dwc+KiGEGM7fZAxMeAG73rIvj7jpwl/pT0tRPoIwZU+90Gl4m/ByBplTcAgfkdWPWH/HRM85zbJI6zvzC2jBNo1u3uJYAgpkfyOn8HBLGjj8qV1rwE9x/8vqp9NfFmiNn5syqJFeI6SLsfFOBha4tkvsqd0dyXGR1uP63lCuB6C7CCxhkO8/PYun9iSmq594OMekNN0Sbm/UmNFlkVtXYpA6WjWSLUsFYyrGh3w8a9CBkE8cD2Ido0xwPg9yeGVArPqxCtK/jJgb1sIZ1gy4KNWLifekL1F59tNuPrxoWsycnZVsh+IfSc9q1vdlsb9QhB6wOEs+uasIFrV7y015LilbHi4LTFvSRyeOfRX0pdD1Vd0kjeIELDi2Wi4tETrfk58aGV/hvQiipRZE4DdlxDcteTRzuLw+9nXpbrTH1IY9rXb8DpRcCIaI2VAv6uj3NqdGhoskoGnhtdCMTFVleTzAV2LKQGBryUE3vW+a6OgUqvMkPryWI2DVCRP8R9x+XVAFAxJt93Q2SlpoaLEr/OET/KlgeQERNAK9yvh+/gTvDfWX2u7sAOJaWpUD7fgAB1tlz4wUn3Yv2GCxwm8i10ZzgaaoIXpFIC9VtnMq9FU3WPmGpcRch3hRm4SnUTfDqBN7LfyyEwc8C8swkGlTIYL+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 10/03/2024 11:11, Matthew Wilcox wrote: > On Sun, Mar 10, 2024 at 11:01:06AM +0000, Ryan Roberts wrote: >>> So after my patch, instead of calling (in order): >>> >>> page_cache_release(folio); >>> folio_undo_large_rmappable(folio); >>> mem_cgroup_uncharge(folio); >>> free_unref_page() >>> >>> it calls: >>> >>> __page_cache_release(folio, &lruvec, &flags); >>> mem_cgroup_uncharge_folios() >>> folio_undo_large_rmappable(folio); >> >> I was just looking at this again, and something pops out... >> >> You have swapped the order of folio_undo_large_rmappable() and >> mem_cgroup_uncharge(). But folio_undo_large_rmappable() calls >> get_deferred_split_queue() which tries to get the split queue from >> folio_memcg(folio) first and falls back to pgdat otherwise. If you are now >> calling mem_cgroup_uncharge_folios() first, will that remove the folio from the >> cgroup? Then we are operating on the wrong list? (just a guess based on the name >> of the function...) > > Oh my. You've got it. This explains everything. Thank you! I've just taken today's mm-unstable, added your official patch to fix the ordering and applied my large folio swap-out series on top (v4, which I haven't posted yet). In testing that, I'm seeing another oops :-( That's exactly how I discovered the original problem, and was hoping that with your fix, this would unblock me. Given I can only repro this when my changes are on top, I guess my code is most likely buggy, but perhaps you can take a quick look at the oops and tell me what you think? [ 96.372503] BUG: Bad page state in process usemem pfn:be502 [ 96.373336] page: refcount:0 mapcount:0 mapping:000000005abfa8d5 index:0x0 pfn:0xbe502 [ 96.374341] aops:0x0 ino:fffffc0001f940c8 [ 96.374893] flags: 0x7fff8000000000(node=0|zone=0|lastcpupid=0xffff) [ 96.375653] page_type: 0xffffffff() [ 96.376071] raw: 007fff8000000000 0000000000000000 fffffc0001f94090 ffff0000c99ee860 [ 96.377055] raw: 0000000000000000 0000000000000000 00000000ffffffff 0000000000000000 [ 96.378650] page dumped because: non-NULL mapping [ 96.379828] Modules linked in: binfmt_misc nls_iso8859_1 dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua sch_fq_codel drm efi_pstore ip_tables x_tables autofs4 xfs btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor xor_neon raid6_pq libcrc32c raid1 raid0 crct10dif_ce ghash_ce sha2_ce virtio_net sha256_arm64 net_failover sha1_ce virtio_blk failover virtio_scsi virtio_rng aes_neon_bs aes_neon_blk aes_ce_blk aes_ce_cipher [ 96.386802] CPU: 13 PID: 4713 Comm: usemem Not tainted 6.8.0-rc5-ryarob01-swap-out-v4 #2 [ 96.387691] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015 [ 96.388887] Call trace: [ 96.389348] dump_backtrace+0x9c/0x128 [ 96.390213] show_stack+0x20/0x38 [ 96.390688] dump_stack_lvl+0x78/0xc8 [ 96.391163] dump_stack+0x18/0x28 [ 96.391545] bad_page+0x88/0x128 [ 96.391893] get_page_from_freelist+0xa94/0x1bc0 [ 96.392407] __alloc_pages+0x194/0x10b0 [ 96.392833] alloc_pages_mpol+0x98/0x278 [ 96.393278] vma_alloc_folio+0x74/0xd8 [ 96.393674] __handle_mm_fault+0x7ac/0x1470 [ 96.394146] handle_mm_fault+0x70/0x2c8 [ 96.394575] do_page_fault+0x100/0x530 [ 96.395013] do_translation_fault+0xa4/0xd0 [ 96.395476] do_mem_abort+0x4c/0xa8 [ 96.395869] el0_da+0x30/0xa8 [ 96.396229] el0t_64_sync_handler+0xb4/0x130 [ 96.396735] el0t_64_sync+0x1a8/0x1b0 [ 96.397133] Disabling lock debugging due to kernel taint [ 112.507052] Adding 36700156k swap on /dev/ram0. Priority:-2 extents:1 across:36700156k SS [ 113.131515] ------------[ cut here ]------------ [ 113.132190] UBSAN: array-index-out-of-bounds in mm/vmscan.c:1654:14 [ 113.132892] index 7 is out of range for type 'long unsigned int [5]' [ 113.133617] CPU: 9 PID: 528 Comm: kswapd0 Tainted: G B 6.8.0-rc5-ryarob01-swap-out-v4 #2 [ 113.134705] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015 [ 113.135500] Call trace: [ 113.135776] dump_backtrace+0x9c/0x128 [ 113.136218] show_stack+0x20/0x38 [ 113.136574] dump_stack_lvl+0x78/0xc8 [ 113.136964] dump_stack+0x18/0x28 [ 113.137322] __ubsan_handle_out_of_bounds+0xa0/0xd8 [ 113.137885] isolate_lru_folios+0x57c/0x658 [ 113.138352] shrink_lruvec+0x5b4/0xdf8 [ 113.138751] shrink_node+0x3f0/0x990 [ 113.139152] balance_pgdat+0x3d0/0x810 [ 113.139579] kswapd+0x268/0x568 [ 113.139936] kthread+0x118/0x128 [ 113.140289] ret_from_fork+0x10/0x20 [ 113.140686] ---[ end trace ]--- The UBSAN issue reported for mm/vmscan.c:1654 is: nr_skipped[folio_zonenum(folio)] += nr_pages; nr_skipped is a stack array of 5 elements. So I guess folio_zonemem(folio) is returning 7. That comes from the flags. I guess this is most likely just a side effect of the corrupted folio due to someone writing to it while its on the free list?