From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FFCCC5AE59 for ; Sat, 31 May 2025 06:36:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6C9A66B0158; Sat, 31 May 2025 02:36:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A1926B0159; Sat, 31 May 2025 02:36:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B74B6B015D; Sat, 31 May 2025 02:36:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3C9A96B0158 for ; Sat, 31 May 2025 02:36:09 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 947748115A for ; Sat, 31 May 2025 06:36:08 +0000 (UTC) X-FDA: 83502243216.10.E869226 Received: from mail-ua1-f54.google.com (mail-ua1-f54.google.com [209.85.222.54]) by imf27.hostedemail.com (Postfix) with ESMTP id A284C40003 for ; Sat, 31 May 2025 06:36:06 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="jxND5wa/"; spf=pass (imf27.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.54 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748673366; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qxAnNwf04k1daTyHuUhp/+U0tlJ5knXJTN7hN8ijbm4=; b=264XYhkfLhqWfU2+HrZNPWZwL60j94xGP6V4FvWwAJ4UHnOSfohy0AywhP+ecTPezE9NSz s3gXd0EWxPcjFfEZEjOVX+vvJ7j58c6nGdmDP0aA1nC/FDdcGWSZtrXU9R6lyPHEAYb6yO zTxDvUArJnqSpzFpNIP5iXc0v7Hdusk= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="jxND5wa/"; spf=pass (imf27.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.54 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748673366; a=rsa-sha256; cv=none; b=lXnw/0ZZUMQ8U1Mb7nEEcn5t0wZmXf5nFUEhsh4B9VenTX0oBjeyHEqdd+zom2Za9/hQOv 4kevrJPglkZ7y8DqimmwBo4wOZQGZGJisPLFe2k0NTWdvUjoA1dw11sdwIirEk+Bmznz/k yzOqu3duMR4h3W4LQhveZImpey5gYac= Received: by mail-ua1-f54.google.com with SMTP id a1e0cc1a2514c-87df4fd380bso532823241.0 for ; Fri, 30 May 2025 23:36:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748673366; x=1749278166; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=qxAnNwf04k1daTyHuUhp/+U0tlJ5knXJTN7hN8ijbm4=; b=jxND5wa/zWyvux0y6KmlvPEtyatwuNqZ8GK+L6Zjcgp5+neuSL2NBM9R7mmSydyLwI 5+z1M9cUlNMw3KKDgwI3wv6/RJ+96i4jzU/Lt+fd4qROhGm9t993TYauEuSt6DSo9sdP dHspOj0J+C4Vd8ZXQ0mVB0i5+FH5P+C6htWXF1Fa1ZDAX448lEiBy/W84K9EnpNp1XIt YheETj2hdMSIygmgqpxV/sCDzrWsm3ZlzzGgg0CUjqAJ5mGeqCW8J0Wj2lIjIkxbcPRk ANTXUlUpJpS3blCkfA5OA/4G1WxqbZ8o+N19MlR/uGnRPVBTS/7WWREk/+0ii8B43eXS O4sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748673366; x=1749278166; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qxAnNwf04k1daTyHuUhp/+U0tlJ5knXJTN7hN8ijbm4=; b=wRcRWiiWU2VhrWaSsZt+W0yv37OQjZMZUrOlkBBSXfZBXmAfPt6LP6tR7U2lZJ9G9c YfYuEku9rfUIlpBRWb13+KPq4TN02DG/MxmRkD3J/MTKKaPgiYqswB/LWwWpQTpw/WaD 6c4fRNw+ykweTHU212S8PlUymBn0e3PwSFsAx2K7kJfKG1srP1pPsazTvQHtYa1pPCG7 s1KQfj2OqQcuBgYhENxtaod8WCezfiTshRyhh3dgArTX2qQlroPSSXIPsyAapmNMsq+D qm3rXOOThTDolBZUr1r0ifwtF7SyVUGtGdZq3JUdrhiWpAuupuo44EZefwbh8NGbYlqR PCwQ== X-Forwarded-Encrypted: i=1; AJvYcCW6c4W3YaJR//fWynPR65Hnx4KyJSw0qdzWgbMaElIZ1GXRZ6Gn9X3yumFkbGqZYPQbNljdTS/zxw==@kvack.org X-Gm-Message-State: AOJu0YxpT1a7tiUYKAoF1FIgXjYlVFyDAnUkLcYeIWg5UAqxBTtParl4 bRWBjHlzdZsAKZj1ailE+oCs9l4+3SmVpbAGqk0dQtidrHBMVTIGx9SUg7X3fAUnWmcd9+8sxDy gpzHqg1B4ZKH2SIjcoslcYQbWDbSc0QY= X-Gm-Gg: ASbGncsUdMctSqEZYb0WOXd2KDlWk+AUZTIOkOidDyQFEK172pZlJ5j3006Agm6Tdi4 kim/WXsyC/49V8bAXn49ZLNUZculRvSohBPbG5EyVvYonWky7g5lO4J7HEzPQ1vtDtzb89Lfigj 9YFYcZx2/L6onBDk1uehGo71Zw8uyoBCx+OA== X-Google-Smtp-Source: AGHT+IFoFH9kRyoo67Aj4j7sLe/VRhCJ+NyYgJ+kdpibA364NNkrfUx+kDIf2nV6F+Nf9J3DCvPMTi+MJfD/icdKAsE= X-Received: by 2002:a05:6102:6d2:b0:4df:8259:eaa with SMTP id ada2fe7eead31-4e701bae8a0mr364647137.5.1748673365645; Fri, 30 May 2025 23:36:05 -0700 (PDT) MIME-Version: 1.0 References: <20250530201710.81365-1-ryncsn@gmail.com> In-Reply-To: From: Barry Song <21cnbao@gmail.com> Date: Sat, 31 May 2025 18:35:54 +1200 X-Gm-Features: AX0GCFsKja1UXSjAtb35_iM4kGOhGAzatuZyI2XERD_MsOjTIh0b4SoyXztRzgA Message-ID: Subject: Re: [PATCH] mm: userfaultfd: fix race of userfaultfd_move and swap cache To: Kairui Song Cc: Lokesh Gidra , linux-mm@kvack.org, Andrew Morton , Peter Xu , Suren Baghdasaryan , Andrea Arcangeli , David Hildenbrand , stable@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: A284C40003 X-Stat-Signature: fuokpwk6kgp5jsfryx8igxdrn4ai7mji X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1748673366-76265 X-HE-Meta: U2FsdGVkX19wyz1cTW87xyiTPz7pqpgiCNEHDO72/YnVMcVTwQ8ebtdDR2zQ5Wbh10k5XG3/YN/IGKzTnzrWCnegMd8NgIZwLi/QfaST7LmAKEPIIpvJbicH6N+wkgZPzZmFsaFNa1PiD5USuaeESsErylmJVDTPmZBSOPTqSgbzTrrgAbQv8pLVfmomnNGc2Pg6SqjIwV/WZxfZQT9D6r/b0Sqv3Gg88LYDM4qr0QuAGeDRE2YptJUiqn7p5xCIx2kailvoBQRbHFfe/hyPvkWslTdJXe6ZYhHGMkT5K7ptHKUbrZv7x+IGar4ZgLJOPaJa+Hi9QrwzbcULav/4BX3PqM2TM+WCXPl5wgRH+PM9MDjQVX9W30uwg+GS/jKcw47v4rdImrzRXdl9u8d+Lp+BGfRr++oRz5KkQfn19xhXEHyNi6w7+ax7yuvKCb9wddLnA9iUGwcSRFMsxmdTuSFosh0IH6oFilhfmf0AIq4MA7pIfHKj82cgpbV/xVGooQVOrf3XUmuJDe/eAKBgnOTev15c5P3rtfHBpv3D+XCQ1CHvCh35QA+l4aNsfOozWmvBRDFh/MPssdfhp4TwMo6/+he7iGnzxyCW3iJjyA6KhDv2qqBBEBOqVL0n3zA/4sPiZ/8gyzuAJ4zLrisyDwkIDIZ2Vr4bToQZTsmgGLKsK41y0ADeg5Ci4qKnJtt0Bk+pOJj18TLzCE9pA9QYD3Q4ppHyWKmFqMedE4NBBj1wmFPvAlSWTF3D4DQiOBU+5JAco/0XX4FH5KcrbfQHNEXnhjTlL5hSr53jnp+dsfAOiYVq7ZFwFAdyja007l5jioptMxkbcaXqcmhOmE6hfSZ2A5R23rhVBI1UvHLDNN7nFbDwB7tg/hCXHKtn4nOhNYGaLiWkzNzUnEVCYZV8oV7o43wfvPX/gqu9nYrGoUIcqEgVqjY5vihEn5/OazhW6piXPOGFFK9sqoE4YxY AZKtFAbq iq9/zy+rxXVos8mUnE9PLK3bNTNBQITnTy+oAzmB0xUesoFzzZVFpyLtmWhcV37792bt4JDuSbWviUf9LMPrDxx1dc7PToNyj11+NE+c9UlGDQU34bmCYWX3IfuTtzmRWlAP+tia26II55fyUw4M8Zxh9AMhjvqET1XbJhVU6SxU089eoBKZUj/qTh8d15I/u4NdtcJ1XRnSJZM3wSb/U5vhArw3xgz7Wx/ICbGl1hEi/qt+jBR4zmlcB7Xt5cNJAgTmDuJt3MY1GVGIG1IpH3EGwgyUlLUIhcq3DGNvZdH9EguSxHSG+rlJ4N7+fuLsJRgpJU9IaIS+XL8oZ3JDcp2eDsMgXcdFJ/Fn1pMIVcZ1agg/xv3ZjieSba/Mo646xwdTzHVa8z6LzGrf/G0+wJxfbbZJ3ezoTCfFb/hcFy+EFHFzGBlWlVxqX6c+X/j6dR2gU8SwxhJ6/k2g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, May 31, 2025 at 6:25=E2=80=AFPM Kairui Song wrot= e: > > On Sat, May 31, 2025 at 11:39=E2=80=AFAM Barry Song <21cnbao@gmail.com> w= rote: > > > > On Sat, May 31, 2025 at 11:40=E2=80=AFAM Lokesh Gidra wrote: > > > > > > On Fri, May 30, 2025 at 1:17=E2=80=AFPM Kairui Song wrote: > > > > > > > > From: Kairui Song > > > > > > > > On seeing a swap entry PTE, userfaultfd_move does a lockless swap c= ache > > > > lookup, and try to move the found folio to the faulting vma when. > > > > Currently, it relies on the PTE value check to ensure the moved fol= io > > > > still belongs to the src swap entry, which turns out is not reliabl= e. > > > > > > > > While working and reviewing the swap table series with Barry, follo= wing > > > > existing race is observed and reproduced [1]: > > > > > > > > ( move_pages_pte is moving src_pte to dst_pte, where src_pte is a > > > > swap entry PTE holding swap entry S1, and S1 isn't in the swap cac= he.) > > > > > > > > CPU1 CPU2 > > > > userfaultfd_move > > > > move_pages_pte() > > > > entry =3D pte_to_swp_entry(orig_src_pte); > > > > // Here it got entry =3D S1 > > > > ... < Somehow interrupted> ... > > > > > > > > // folio A is just a new allocat= ed folio > > > > // and get installed into src_pt= e > > > > > > > > // src_pte now points to folio A= , S1 > > > > // has swap count =3D=3D 0, it c= an be freed > > > > // by folio_swap_swap or swap > > > > // allocator's reclaim. > > > > > > > > // folio B is a folio in another= VMA. > > > > > > > > // S1 is freed, folio B could us= e it > > > > // for swap out with no problem. > > > > ... > > > > folio =3D filemap_get_folio(S1) > > > > // Got folio B here !!! > > > > ... < Somehow interrupted again> ... > > > > > > > > // Now S1 is free to be used aga= in. > > > > > > > > // Now src_pte is a swap entry p= te > > > > // holding S1 again. > > > > folio_trylock(folio) > > > > move_swap_pte > > > > double_pt_lock > > > > is_pte_pages_stable > > > > // Check passed because src_pte =3D=3D S1 > > > > folio_move_anon_rmap(...) > > > > // Moved invalid folio B here !!! > > > > > > > > The race window is very short and requires multiple collisions of > > > > multiple rare events, so it's very unlikely to happen, but with a > > > > deliberately constructed reproducer and increased time window, it c= an be > > > > reproduced [1]. > > > > > > Thanks for catching and fixing this. Just to clarify a few things > > > about your reproducer: > > > 1. Is it necessary for the 'race' mapping to be MAP_SHARED, or > > > MAP_PRIVATE will work as well? > > > 2. You mentioned that the 'current dir is on a block device'. Are you > > > indicating that if we are using zram for swap then it doesn't > > > reproduce? > > > > > > > > > > > It's also possible that folio (A) is swapped in, and swapped out ag= ain > > > > after the filemap_get_folio lookup, in such case folio (A) may stay= in > > > > swap cache so it needs to be moved too. In this case we should also= try > > > > again so kernel won't miss a folio move. > > > > > > > > Fix this by checking if the folio is the valid swap cache folio aft= er > > > > acquiring the folio lock, and checking the swap cache again after > > > > acquiring the src_pte lock. > > > > > > > > SWP_SYNCRHONIZE_IO path does make the problem more complex, but so = far > > > > we don't need to worry about that since folios only might get expos= ed to > > > > swap cache in the swap out path, and it's covered in this patch too= by > > > > checking the swap cache again after acquiring src_pte lock. > > > > > > > > Fixes: adef440691ba ("userfaultfd: UFFDIO_MOVE uABI") > > > > Closes: https://lore.kernel.org/linux-mm/CAMgjq7B1K=3D6OOrK2OUZ0-tq= Czi+EJt+2_K97TPGoSt=3D9+JwP7Q@mail.gmail.com/ [1] > > > > Signed-off-by: Kairui Song > > > > --- > > > > mm/userfaultfd.c | 26 ++++++++++++++++++++++++++ > > > > 1 file changed, 26 insertions(+) > > > > > > > > diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c > > > > index bc473ad21202..a1564d205dfb 100644 > > > > --- a/mm/userfaultfd.c > > > > +++ b/mm/userfaultfd.c > > > > @@ -15,6 +15,7 @@ > > > > #include > > > > #include > > > > #include > > > > +#include > > > I guess you mistakenly left it from your reproducer code :) > > > > #include > > > > #include > > > > #include "internal.h" > > > > @@ -1086,6 +1087,8 @@ static int move_swap_pte(struct mm_struct *mm= , struct vm_area_struct *dst_vma, > > > > spinlock_t *dst_ptl, spinlock_t *src_ptl, > > > > struct folio *src_folio) > > > > { > > > > + swp_entry_t entry; > > > > + > > > > double_pt_lock(dst_ptl, src_ptl); > > > > > > > > if (!is_pte_pages_stable(dst_pte, src_pte, orig_dst_pte, or= ig_src_pte, > > > > @@ -1102,6 +1105,19 @@ static int move_swap_pte(struct mm_struct *m= m, struct vm_area_struct *dst_vma, > > > > if (src_folio) { > > > > folio_move_anon_rmap(src_folio, dst_vma); > > > > src_folio->index =3D linear_page_index(dst_vma, dst= _addr); > > > > + } else { > > > > + /* > > > > + * Check again after acquiring the src_pte lock. Or= we might > > > > + * miss a new loaded swap cache folio. > > > > + */ > > > > + entry =3D pte_to_swp_entry(orig_src_pte); > > > > + src_folio =3D filemap_get_folio(swap_address_space(= entry), > > > > + swap_cache_index(entr= y)); > > > > > > Given the non-trivial overhead of filemap_get_folio(), do you think i= t > > > will work if filemap_get_filio() was only once after locking src_ptl? > > > Please correct me if my assumption about the overhead is wrong. > > > > not quite sure as we have a folio_lock(src_folio) before move_swap_pte(= ). > > can we safely folio_move_anon_rmap + src_folio->index while not holding > > folio lock? > > I think no, we can't even make sure the folio is still in the swap > cache, so it can be a freed folio that does not belong to any VMA > while not holding the folio lock. Right, but will the following be sufficient, given that we don=E2=80=99t re= ally care about the folio=E2=80=94only whether there=E2=80=99s new cache? if (READ_ONCE(si->swap_map[offset]) & SWAP_HAS_CACHE) { double_pt_unlock(dst_ptl, src_ptl); return -EAGAIN; } Thanks Barry