From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C53ACC54798 for ; Wed, 6 Mar 2024 02:19:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 255856B007E; Tue, 5 Mar 2024 21:19:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2065E6B0080; Tue, 5 Mar 2024 21:19:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0CE8A6B0081; Tue, 5 Mar 2024 21:19:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id F19AA6B007E for ; Tue, 5 Mar 2024 21:19:58 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 9F3FCC0B52 for ; Wed, 6 Mar 2024 02:19:58 +0000 (UTC) X-FDA: 81865008876.19.38A9B8F Received: from mail-qv1-f51.google.com (mail-qv1-f51.google.com [209.85.219.51]) by imf25.hostedemail.com (Postfix) with ESMTP id 96231A0009 for ; Wed, 6 Mar 2024 02:19:56 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=mWEqJYWY; spf=pass (imf25.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.51 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709691596; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=acf+30rZOU0mvAbyQhYXFdJzkgjEIoHhA9P8NXsNW7E=; b=EFLMwiKsMO+00xe5j9KwQlbkfhKvMwme3Q+Fz7i/nHS6PUXmVVgJkVGijeKu734mrkrHWm SM4aYYPBARO8J++GlRjPt/EM+K1/CrNuK5wzIXXuElxDIJPdWpwSuzVrGdcfullqzxo3d3 0Mg8EZzPgxjO2XjNvCVy2Wi5i1HiqII= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709691596; a=rsa-sha256; cv=none; b=Aa79OapUBXgIRliMz7+6mqzMRH+7CcLj3G7kuxTIuIGarZgrd4uHFvwLZAa+SXtxa7GzcT kcZWskwu09lF9N0GA+C1xAW1heA5KjdLYCd0juGn+jRgSQFbVB6LZv3E5kzDJSYTrc8KTU D06gVYVF78VV6+LJsNTXoEumLn4PTV0= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=mWEqJYWY; spf=pass (imf25.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.51 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org Received: by mail-qv1-f51.google.com with SMTP id 6a1803df08f44-68facd7ea4aso3308646d6.0 for ; Tue, 05 Mar 2024 18:19:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1709691595; x=1710296395; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=acf+30rZOU0mvAbyQhYXFdJzkgjEIoHhA9P8NXsNW7E=; b=mWEqJYWYkjzTBUjVOKa1LwoWTeEFXOlHFmEzF8N2oLRZghScLTgPyhk0eGGL+Iy8eE xNW4NCPIkesVzO/5JURH/Q6swLS0191V/WsgMLGa3WFVRuJiQDbv4Jdy0JSWuyLD0m3y I1wjmxBErWTSvFpmD9WtXuSql/nmQCj20eGTlhOHzg00XLhsoKRpsvKL8v0nro1pFwlW FzyP92eUeAS3NSg93BkvRHWvG8kO+cvqo0o+C5urO2U5U5FYUCVMPa+wKz/49ihWLdRp ELKrz8zXmAzcY0noz6seCUZEeFHrVmuEKWK0CBxdODYlDlKRUqoYAUj6f+Ypr1nSHhGD /rSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709691595; x=1710296395; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=acf+30rZOU0mvAbyQhYXFdJzkgjEIoHhA9P8NXsNW7E=; b=kebqDLsKGXYM7g6IzzQmuIzW8z/zMVA12tSJfkmQQqKS/qU3cDpSub9ouLNtG9d46U Jvxn0Cj9+P2tMnOivUiq2ubPJcs60SBvaLEYvv2Bo7cEn53KlEU1gKy321+FXf9o9Lr4 YtK7O9lute3PMxGsgGcUCEpwCbYdxYoFDn57XSw9MzAAYkpoYxLjy+1XtYcc/jVvkxoF 42smzxgMukY4UJBtI+lvt6nItqaJQD8NHrEJeRyUznkvKpXthSABO1GzGDIfV7w7Uz0B bCMqYOrLsKPtL9jOJFIrHig7rVJf6sTLBuLsSPU6rckMwG8khfMg+NLdFPgkL5w7s5Mn KJnQ== X-Forwarded-Encrypted: i=1; AJvYcCXtJI2WB4QYj/BH0/DYtY7JMLE4ez8Zw2S/TJtjlnWgQE3aYoHflXGM9q+0P5RjGFvlYWPpKLa9oz5I3uRR7JX9vBU= X-Gm-Message-State: AOJu0YwUzbc7p+uWsYnISxlXT1Phy98r5f+4WjlszwLTRqVqdejrUcKq l4q3NHjpYkd72QdBQHjJSNStDUhcg+aFd85HXbwyXk1eonh6e+fA3Eu5zlK4kAY= X-Google-Smtp-Source: AGHT+IEmyP0orYTDcGtYCnkFo+3YjfLkmaNOkkJK6Ir3myHyg0AcpXpcmUhE2WdPcJpjeWnyKDNU1A== X-Received: by 2002:a0c:dc04:0:b0:690:68a:a141 with SMTP id s4-20020a0cdc04000000b00690068aa141mr6220950qvk.24.1709691595613; Tue, 05 Mar 2024 18:19:55 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id pd6-20020a056214490600b006907801a000sm2629696qvb.26.2024.03.05.18.19.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Mar 2024 18:19:54 -0800 (PST) Date: Tue, 5 Mar 2024 21:19:50 -0500 From: Johannes Weiner To: Ryan Roberts Cc: Andrew Morton , David Hildenbrand , "Huang, Ying" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org Subject: Re: [PATCH v1] mm: swap: Fix race between free_swap_and_cache() and swapoff() Message-ID: <20240306021950.GA801254@cmpxchg.org> References: <20240305151349.3781428-1-ryan.roberts@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240305151349.3781428-1-ryan.roberts@arm.com> X-Rspamd-Queue-Id: 96231A0009 X-Rspam-User: X-Stat-Signature: bmxdkmhokx4bhkdz99m33gfwqk87metf X-Rspamd-Server: rspam03 X-HE-Tag: 1709691596-668916 X-HE-Meta: U2FsdGVkX1/QLj2DmMKyIgIetkK4xQRBvB8z8cyVA7F22QWSbzigpRSATw6OM6CZjE19NgiKisiXLEOstjmUeQcJUbx/jTp/v+4ApX62s2A0wginV8kxQSlsEjWnMdJK+diyTUiYsUZOB10a6P3F8yB9nvrXxgvB9YnjHBXyM/1lyhZfKA94u1VheR4Y1owX0T5dGNFNAYv7H55WxK/vZqieZdyWybA8gK5uEx4+LDnYJEKE1+yzfywX5s+hEuQExgyV9qCRJO09j4JrnKtuUbXCJmufjz7ZjEZUa0q12rSAlo62bIqj782iv6PLA4GtVGI9ZEuMVnwEOIBqA9UAGFs8gEuLG7LHTd4W/gbwpEJvejnRS8on/CYaNi0PIkurPXhtit4SA7PTL6w1/l99BTiLAq9AlUR8Jl/ToGOL1zVza9Ye1xgxOay8GCcsieZ6Xh7KF9yrhiyVEkJwbEk9+CMGO/WX53NNuES+E91tEiGvuNXVbInqh0Pw6lRxPuA5FRgMIRNBWO6upxluuGu5PQAbDsoDo9haP1NJNIEaQ6UNxmqEE3/8rhfYE9pcQAwAEdWsq488dL1Wb/m/12mUEJW1LVcLoF+6C3MmIRuA1fNSdTiq1Yi2KGa98NLZq4ZiQMoxIpcSAUcDeqDmekbGAZGCdErJFqfS5Y33+pdVXSRJLHNr0J66ApQ6/v5BsvkfJvFMVfd/FxeL6DfNBiGY4Lc0WQKS+JGLP9ptR+NeDjZVJytIsTDBbK3Xbj52SHdAOZvh1aopoKn8nnATDRRA/smSEY9ib+WwBL9STQIvENOkrEppNUd9FfmGvt+e1M0T3GuBRJJkYksF/5NoO3tTTUm0mv+Qv3F6CyFBVav/hZg74zRnbyN58hvH3iitp5ceXdrWrpXAmSSEsUpqSZumefGFMJvjB65EZZ2DB8S3bgqzLX1/TBZAFX3Y2MjJMuHrJjMtPsoXGyGRvXKkZea raSUm18o zhYs0DrnbFyLkFEskusCcHShQ4px0fhtqXSXp9Kl+O96YnAypdZr+U2C1CQf5tfbQGWpYqDO2iI/4gkuSvZ0Riw2FjNa37llyb33kn0pzpGAbGynSlxmpDHLO5pdcFiEu2JP49fzxdtMJNEQFMk7pFTudgoeCtLes8tVv3R6KMsy2mf7sML5OFGsIx7iWju/s9MWu0wRNrEGK5rIYsvxyLhVaENHCrE+DD1r9FzHl57JViaSWJH+qIsUH6L5hSyW60ljh2cd1rlXD0iyzJduyUbIi9Kw8GXaknl5btyvEZ118yNhCwzftcY+hH9mYvBRhSp1E+Nhi9adEiI09ua7V55MfkO0xl/Jhk/iELPnrKwVQ+IEW4741XXiSstoxDP+c0DpV1lPe1JQ3hhlrAHYe9ZD9qGquwi5EQzDKrrg3YMk+Ftrup+fvsHnNXxcv9LHp+kY+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Ryan, On Tue, Mar 05, 2024 at 03:13:49PM +0000, Ryan Roberts wrote: > There was previously a theoretical window where swapoff() could run and > teardown a swap_info_struct while a call to free_swap_and_cache() was > running in another thread. This could cause, amongst other bad > possibilities, swap_page_trans_huge_swapped() (called by > free_swap_and_cache()) to access the freed memory for swap_map. > > This is a theoretical problem and I haven't been able to provoke it from > a test case. But there has been agreement based on code review that this > is possible (see link below). > > Fix it by using get_swap_device()/put_swap_device(), which will stall > swapoff(). There was an extra check in _swap_info_get() to confirm that > the swap entry was valid. This wasn't present in get_swap_device() so > I've added it. I couldn't find any existing get_swap_device() call sites > where this extra check would cause any false alarms. Unfortunately, I found one, testing current mm-everything: [ 189.420777] get_swap_device: Unused swap offset entry 000641ae [ 189.426648] ------------[ cut here ]------------ [ 189.431290] WARNING: CPU: 3 PID: 369 at mm/swapfile.c:1301 get_swap_device+0x2da/0x3f0 [ 189.439242] CPU: 3 PID: 369 Comm: cachish Not tainted 6.8.0-rc5-00527-g19d98776f227-dirty #30 [ 189.447791] Hardware name: Micro-Star International Co., Ltd. MS-7B98/Z390-A PRO (MS-7B98), BIOS 1.80 12/25/2019 [ 189.457998] RIP: 0010:get_swap_device+0x2da/0x3f0 [ 189.462721] Code: a8 03 75 2a 65 48 ff 08 e9 36 ff ff ff 4c 89 e9 48 c7 c2 40 fd 91 83 48 c7 c6 c0 f9 91 83 48 c7 c7 60 ee 91 83 e8 26 2f af ff <0f> 0b eb af 4c 8d 6b 08 48 b8 00 00 00 00 00 fc ff df 4c 89 ea 48 [ 189.481497] RSP: 0000:ffffc90000cff8a8 EFLAGS: 00010282 [ 189.486760] RAX: 0000000000000032 RBX: ffff8881262eee00 RCX: 0000000000000000 [ 189.493909] RDX: 0000000000000001 RSI: ffffffff83a1e620 RDI: 0000000000000001 [ 189.501054] RBP: 1ffff9200019ff15 R08: 0000000000000001 R09: fffff5200019fee1 [ 189.508202] R10: ffffc90000cff70f R11: 0000000000000001 R12: ffffc900018d51ae [ 189.515346] R13: 00000000000641ae R14: 0000000000000000 R15: 00000000000641af [ 189.522494] FS: 00007f7120263680(0000) GS:ffff88841c380000(0000) knlGS:0000000000000000 [ 189.530591] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 189.536373] CR2: 00007f6e659a2ea3 CR3: 0000000046860004 CR4: 00000000003706f0 [ 189.543516] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 189.550661] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 189.557811] Call Trace: [ 189.560276] [ 189.562393] ? __warn+0xc4/0x250 [ 189.565647] ? get_swap_device+0x2da/0x3f0 [ 189.569761] ? report_bug+0x348/0x440 [ 189.573444] ? handle_bug+0x6d/0x90 [ 189.576951] ? exc_invalid_op+0x13/0x40 [ 189.580810] ? asm_exc_invalid_op+0x16/0x20 [ 189.585019] ? get_swap_device+0x2da/0x3f0 [ 189.589142] ? get_swap_device+0x2da/0x3f0 [ 189.593255] ? __pfx_get_swap_device+0x10/0x10 [ 189.597717] __read_swap_cache_async+0x9f/0x630 [ 189.602281] ? __pfx___read_swap_cache_async+0x10/0x10 [ 189.607439] ? __mod_memcg_lruvec_state+0x238/0x4f0 [ 189.612344] ? __pfx_swp_swap_info+0x10/0x10 [ 189.616652] swap_cluster_readahead+0x2cd/0x510 [ 189.621206] ? __pfx_swap_cluster_readahead+0x10/0x10 [ 189.626279] ? swap_cache_get_folio+0xcd/0x360 [ 189.630760] ? __count_memcg_events+0x10a/0x370 [ 189.635318] shmem_swapin_folio+0x2f2/0xc60 [ 189.639525] ? __pfx__raw_spin_lock+0x10/0x10 [ 189.643908] ? __pte_offset_map+0x19/0x1d0 [ 189.648024] shmem_get_folio_gfp+0x307/0xe30 [ 189.652323] ? __schedule+0x9f0/0x1fe0 [ 189.656096] ? __pfx_shmem_get_folio_gfp+0x10/0x10 [ 189.660923] ? filemap_map_pages+0x999/0xe60 [ 189.665211] shmem_fault+0x1d9/0x810 [ 189.668834] ? __pfx_shmem_fault+0x10/0x10 [ 189.672954] ? __pfx_filemap_map_pages+0x10/0x10 [ 189.677590] __do_fault+0xed/0x390 [ 189.681012] __handle_mm_fault+0x1ba1/0x2e80 [ 189.685297] ? __pfx___handle_mm_fault+0x10/0x10 [ 189.689933] ? __pfx_down_read_trylock+0x10/0x10 [ 189.694570] ? __pfx_hrtimer_nanosleep+0x10/0x10 [ 189.699215] handle_mm_fault+0xe0/0x560 [ 189.703074] ? __pfx_restore_fpregs_from_fpstate+0x10/0x10 [ 189.708620] do_user_addr_fault+0x2ba/0x9d0 [ 189.712828] exc_page_fault+0x54/0x90 [ 189.716508] asm_exc_page_fault+0x22/0x30 [ 189.720535] RIP: 0033:0x5640dc2d72b5 [ 189.724131] Code: 98 48 ba 00 00 00 00 03 00 00 00 48 89 d6 48 2b 75 a0 ba 00 00 00 00 48 f7 f6 48 89 d1 48 89 ca 48 8b 45 a0 48 01 d0 48 01 d8 <0f> b6 00 bf e8 03 00 00 e8 1e fe ff ff 48 83 45 a8 01 48 8d 45 d0 [ 189.742922] RSP: 002b:00007ffc227e3f60 EFLAGS: 00010206 [ 189.748165] RAX: 00007f6e659a2ea3 RBX: 00007f6e2007e000 RCX: 0000000045924ea3 [ 189.755311] RDX: 0000000045924ea3 RSI: 0000000300000000 RDI: 00007f71202586a0 [ 189.762483] RBP: 00007ffc227e3fe0 R08: 00007f7120258074 R09: 00007f71202580a0 [ 189.769633] R10: 0000000000019458 R11: 00000000008aa400 R12: 0000000000000000 [ 189.776781] R13: 00007ffc227e4128 R14: 00007f712029d000 R15: 00005640dc2d9dd8 [ 189.783928] [ 189.786126] ---[ end trace 0000000000000000 ]--- [ 285.827888] get_swap_device: Unused swap offset entry 0018403f [ 320.699306] get_swap_device: Unused swap offset entry 000b001b [ 354.031339] get_swap_device: Unused swap offset entry 000681a9 [ 364.958435] get_swap_device: Unused swap offset entry 001f4055 [ 364.976235] get_swap_device: Unused swap offset entry 001f4057 [ 365.530174] get_swap_device: Unused swap offset entry 000d415c [ 394.223792] get_swap_device: Unused swap offset entry 001540d0 [ 394.317299] get_swap_device: Unused swap offset entry 000341d9 [ 394.341727] get_swap_device: Unused swap offset entry 0006c07e [ 396.062365] get_swap_device: Unused swap offset entry 000541a4 [ 396.068262] get_swap_device: Unused swap offset entry 000541a7 [ 402.629551] get_swap_device: Unused swap offset entry 00294021 [ 436.740622] get_swap_device: Unused swap offset entry 00334155 [ 436.758527] get_swap_device: Unused swap offset entry 001b417c swap_cluster_readahead() calls __read_swap_cache_async() on a range of made-up swap entries around the faulting slot. The device and the range (si->max) are valid, but the specific entry might not be in use. __read_swap_cache_async() instead relies on swapcache_prepare() returning -ENOENT to catch this and skip gracefully. Confirmed that reverting the patch makes the warnings go away.