From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43AE1C5B552 for ; Sat, 31 May 2025 07:11:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4E3C6B017C; Sat, 31 May 2025 03:11:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CFF3E6B0181; Sat, 31 May 2025 03:11:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C157F6B0184; Sat, 31 May 2025 03:11:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A2ED66B017C for ; Sat, 31 May 2025 03:11:43 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5A7A25F247 for ; Sat, 31 May 2025 07:11:43 +0000 (UTC) X-FDA: 83502332886.28.3892DDF Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf02.hostedemail.com (Postfix) with ESMTP id 692D580006 for ; Sat, 31 May 2025 07:11:41 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=1WXYS4NH; spf=pass (imf02.hostedemail.com: domain of lokeshgidra@google.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=lokeshgidra@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748675501; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hJUPLZSZKX8rYufeGcr9Ae45p6/9NH9FDi+v384RGEw=; b=8M+o8uktYalqK2BF13inuAShiRKsTKFhMd02oGShUhZF5ULdPx6KafVAM38KrKmIkhg7wK kcJT+gJPv3YdXvTShb4w3EhBPYMWRpdj4RsW52cwTFllreJqPItrKe8Z71htQa2nvQlHop VJXHOJC+rNYTCN+9NmEzUrZDmmfUljk= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=1WXYS4NH; spf=pass (imf02.hostedemail.com: domain of lokeshgidra@google.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=lokeshgidra@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748675501; a=rsa-sha256; cv=none; b=JtBRvmqFAO1ovGaw+1LSatLHurAtWDJgn6KAiLiwYGcEcHa8JV1b/TGYwwE5vd1baEICkG 9gp6fkxNzNoeaGFjJ+C060Zq9hZlTHcRBADXe/sZtjfJXJcfu9g+TaWZX+p8L9vkS4H2uP xCuAETmfN14MelxafUs1Li/ertt8J9U= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-231ba6da557so82715ad.1 for ; Sat, 31 May 2025 00:11:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1748675500; x=1749280300; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=hJUPLZSZKX8rYufeGcr9Ae45p6/9NH9FDi+v384RGEw=; b=1WXYS4NHKrzwsM0Mk72EWTA3v2fvrQpsanjCP5Iam7MLYj1cQBmFtmrKbSuVz/7EBN bNTUIVVTi6BALf5C1iiCd567RZfC38bBePSJpKs9v9GiUwott59XesBlwDIjAbPmA+em KZOJ/Mo+s9d+CPm/KE5q38SVU9AEsj4V1ReHV5e95Uf0ztJulXiNpVHCHgySalVTSzcJ r6XYhrISCkRe5KhLErjhNfnGLFUTJqDRU0M7GEuIZ08v27F91MHEZYQgnTwHTTWjjkZ/ hCfZs539cU7DF4luQ5IN5FK1Erhc8Lz4Eago4+HJVs2P9eZR8M4G7XCrHGuUk2XytEv+ ghUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748675500; x=1749280300; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hJUPLZSZKX8rYufeGcr9Ae45p6/9NH9FDi+v384RGEw=; b=As8u4SYBIyW4ou5vS1D2kh65cDe+F77eIqhpa98Q1ZfSJt3wUlGYzrmE4Cxf/Z2gAR JoZM+r58hVc1CPwQNlS/PAyHY/plW3b3PX/E56u34IrLrwMFPZmDxMG7Wdqn3rJcTX9w +2KjU7aGWn370CVhwzLb1Nv3NtdxJJpklOVHkivjiDHdkRyDexZg4WUK8s22nf2UKswx HWbomdgKAJNJl2opPWUH6bC36dupQOYkZaxPOY6n9rmpH5yNZP560X5uXy5NhegR0R0m 2+6g2/QDxVbPx2kpY1Z5Zz/trGrFSBEEc4+D+O8aNdY4DlqvhADhiLk1dt5ElHOrZcTZ 8wRg== X-Forwarded-Encrypted: i=1; AJvYcCUPdizk/RoYcsJFqr+2D6EeBj8xruzOrNi07C543otERR9wNWwORxCXASv6QorJ8wBHOgQAKhmvfg==@kvack.org X-Gm-Message-State: AOJu0YwbsMv5OzRzc5gqFLG02aIf8IO7LDL8fRE4RiwLFuJG9pgW+GZA Cf3C11KrG5IAyK3YMffDTn7peBDPWEt5XzCy1X/AGUNh5RjwTTttZ6Ae76eHdbdFmaTQR+Y48w7 kGlP/ZmASdSCoqO/FQFCjEuSVUuOE4V8B7XP28vOL X-Gm-Gg: ASbGncvLs2GlJwgAA78cFCIw8mwigWbGIC1TbPaZnZEcZ3k5h8EXxfapb91AaWK1K+1 SAtvTX9R4NXpLSOUgzNnZfaL7rvfsP/zXftbY+x75FVOgzWKonPkVz3bijKS3nBO2p/Hlaf4Buo /rB92H7Eyl9x6OZc/j2k/lnAgNjXAUjsb0PqiiY6v+yH118JM8NfxpNp2DLnymFwQbNxhFN90wu 2DnH+QT0M3Y5Xc= X-Google-Smtp-Source: AGHT+IFL2G6R9UYfr0o8oK/7I4HkDQML7Z++7jpPt0Grez3RcIK5mSoEa/a8KBvtmIkNzJbgoKpZnCjzVgIrrFgtNbM= X-Received: by 2002:a17:902:da8e:b0:234:14ff:541f with SMTP id d9443c01a7336-2355af16345mr998905ad.21.1748675499856; Sat, 31 May 2025 00:11:39 -0700 (PDT) MIME-Version: 1.0 References: <20250530201710.81365-1-ryncsn@gmail.com> In-Reply-To: From: Lokesh Gidra Date: Sat, 31 May 2025 00:11:24 -0700 X-Gm-Features: AX0GCFu-U44uG2WWotTuwSQuH2rZxfvOvvkcAntijfqzmYHrTd196Ke7FpZ3cCU Message-ID: Subject: Re: [PATCH] mm: userfaultfd: fix race of userfaultfd_move and swap cache To: Barry Song <21cnbao@gmail.com> Cc: Kairui Song , linux-mm@kvack.org, Andrew Morton , Peter Xu , Suren Baghdasaryan , Andrea Arcangeli , David Hildenbrand , stable@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 692D580006 X-Stat-Signature: 9ejyywboufe9box7dccxwgx9nbbjhb3i X-Rspam-User: X-HE-Tag: 1748675501-416076 X-HE-Meta: U2FsdGVkX19PPCAvrCRaiO3/btiRLsEMTf4TY5/3pf55Plln/fatrou8oM4FfiVeOai9xtwv2+xxueIbXP3E3uNC1M9kGiRt2deW0MVB41KGlTL6922pRCyPULBl0vMljqooa+iaTN4l9gNtPPJeoKgwo9gmB/ihVu1dHE7av2UU0kd6PEq5VR1Iwa1ZHrM6Cie4n4Z3xxQFBjNItjBRzrBiaMfVxsIPM+f50JlzwZW3zWBYh4BxEdA3ZQq8lgdAM0m69kxJ84tk0SA9qJBs4stvwlW3jLhlYrTPnCm18KHsyXaeNVf0f1bnpPDEB/TH1vAXRk4F/FdyTvWIjhU4xvyw32ac2VnrkLPjZU9OcpRbrSEIofwLXp/16EyVdxSaxik9kCEK4v5h/iArHVz5/XkcJ7sRyBeygorqYYEkk6T8CrrI9j6WKFt7Stb5TtPkKZDg3bj5rfyCdstSXIjAxvMC+e9RaqZmTV4VZHnc2a5caTqwKjL4Oo3u7HSjHjKIv5l/GfsiVpar91MQsPDraox8q+MdEmfBCabC5XzpkQQ9eDZveTPtUi5qBvE4+UEjsFPveHGjXPiwIml63pXwPp/rH8qYsMXw9Dy0A3WsyLajtIXvzJX4oxGiLdvttBgkM5BYCBu+rrN7B8pcw+HcgM7PKzSvHEp4Jpj4bXKZXEo4W0i08peB2ghftonThRAsjEH88TJpS32xSZPf1FnKgGu1T2/s4CUZOENxgBoOlBAx42VeR9Zs5gEOtcoWxNeOV/p8tg2a79ArwMTD5w0upmfnTBCOaU9Fl5WI3EVX9bf8gvmuTFcTdqKb/68Ozw58im/PL6LUDxsfFkQQBnNocD8MbRN/xkMgNeCfSXSn86nihU6GXbac/rPxSjfPcKSYxyELFsd+kOAWeRfDv0HCOOfmwis3IjzirWh6JOaPswsXWPWlUW8jtAWGhApbQwXhT/6trh2DuJulkA8sBnN nsf6zP1b T6n91Bojm77oSpaXO2PFb1WHWW1bD0qyjSUv1Y25AXJDTURtSoKncjW15ddujWgcEDMP/cdwCJYTU8LLrzLc2o7qd7y1NvdkqADgXm3BHTsS+274pGr+F7tZDHdEeiI1YAdCrtOh/5bKsXQVc2MMPr1LXdqdaTdgoKgDVOgpCc9uNbSGMJ1EgKe2P0/R+V5HdLzW+OHlgdJ7lRiFqs5aYThp1LVRTZVuNHyRr4UcFwX1vijWxSEvSP3Vv3WcfbXXXk3SWsU+u1Pf4FZZbo1a48ZPkniROTAZpHZ7GWLI3s5vVBbYzCEZvS1qNYmfLSHTwr91mPntjDkWgRMA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, May 31, 2025 at 12:06=E2=80=AFAM Barry Song <21cnbao@gmail.com> wro= te: > > On Sat, May 31, 2025 at 7:00=E2=80=AFPM Kairui Song wr= ote: > > > > On Sat, May 31, 2025 at 2:36=E2=80=AFPM Barry Song <21cnbao@gmail.com> = wrote: > > > > > > On Sat, May 31, 2025 at 6:25=E2=80=AFPM Kairui Song wrote: > > > > > > > > On Sat, May 31, 2025 at 11:39=E2=80=AFAM Barry Song <21cnbao@gmail.= com> wrote: > > > > > > > > > > On Sat, May 31, 2025 at 11:40=E2=80=AFAM Lokesh Gidra wrote: > > > > > > > > > > > > On Fri, May 30, 2025 at 1:17=E2=80=AFPM Kairui Song wrote: > > > > > > > > > > > > > > From: Kairui Song > > > > > > > > > > > > > > On seeing a swap entry PTE, userfaultfd_move does a lockless = swap cache > > > > > > > lookup, and try to move the found folio to the faulting vma w= hen. > > > > > > > Currently, it relies on the PTE value check to ensure the mov= ed folio > > > > > > > still belongs to the src swap entry, which turns out is not r= eliable. > > > > > > > > > > > > > > While working and reviewing the swap table series with Barry,= following > > > > > > > existing race is observed and reproduced [1]: > > > > > > > > > > > > > > ( move_pages_pte is moving src_pte to dst_pte, where src_pte = is a > > > > > > > swap entry PTE holding swap entry S1, and S1 isn't in the sw= ap cache.) > > > > > > > > > > > > > > CPU1 CPU2 > > > > > > > userfaultfd_move > > > > > > > move_pages_pte() > > > > > > > entry =3D pte_to_swp_entry(orig_src_pte); > > > > > > > // Here it got entry =3D S1 > > > > > > > ... < Somehow interrupted> ... > > > > > > > > > > > > > > // folio A is just a new a= llocated folio > > > > > > > // and get installed into = src_pte > > > > > > > > > > > > > > // src_pte now points to f= olio A, S1 > > > > > > > // has swap count =3D=3D 0= , it can be freed > > > > > > > // by folio_swap_swap or s= wap > > > > > > > // allocator's reclaim. > > > > > > > > > > > > > > // folio B is a folio in a= nother VMA. > > > > > > > > > > > > > > // S1 is freed, folio B co= uld use it > > > > > > > // for swap out with no pr= oblem. > > > > > > > ... > > > > > > > folio =3D filemap_get_folio(S1) > > > > > > > // Got folio B here !!! > > > > > > > ... < Somehow interrupted again> ... > > > > > > > > > > > > > > // Now S1 is free to be us= ed again. > > > > > > > > > > > > > > // Now src_pte is a swap e= ntry pte > > > > > > > // holding S1 again. > > > > > > > folio_trylock(folio) > > > > > > > move_swap_pte > > > > > > > double_pt_lock > > > > > > > is_pte_pages_stable > > > > > > > // Check passed because src_pte =3D=3D S1 > > > > > > > folio_move_anon_rmap(...) > > > > > > > // Moved invalid folio B here !!! > > > > > > > > > > > > > > The race window is very short and requires multiple collision= s of > > > > > > > multiple rare events, so it's very unlikely to happen, but wi= th a > > > > > > > deliberately constructed reproducer and increased time window= , it can be > > > > > > > reproduced [1]. > > > > > > > > > > > > Thanks for catching and fixing this. Just to clarify a few thin= gs > > > > > > about your reproducer: > > > > > > 1. Is it necessary for the 'race' mapping to be MAP_SHARED, or > > > > > > MAP_PRIVATE will work as well? > > > > > > 2. You mentioned that the 'current dir is on a block device'. A= re you > > > > > > indicating that if we are using zram for swap then it doesn't > > > > > > reproduce? > > > > > > > > > > > > > > > > > > > > It's also possible that folio (A) is swapped in, and swapped = out again > > > > > > > after the filemap_get_folio lookup, in such case folio (A) ma= y stay in > > > > > > > swap cache so it needs to be moved too. In this case we shoul= d also try > > > > > > > again so kernel won't miss a folio move. > > > > > > > > > > > > > > Fix this by checking if the folio is the valid swap cache fol= io after > > > > > > > acquiring the folio lock, and checking the swap cache again a= fter > > > > > > > acquiring the src_pte lock. > > > > > > > > > > > > > > SWP_SYNCRHONIZE_IO path does make the problem more complex, b= ut so far > > > > > > > we don't need to worry about that since folios only might get= exposed to > > > > > > > swap cache in the swap out path, and it's covered in this pat= ch too by > > > > > > > checking the swap cache again after acquiring src_pte lock. > > > > > > > > > > > > > > Fixes: adef440691ba ("userfaultfd: UFFDIO_MOVE uABI") > > > > > > > Closes: https://lore.kernel.org/linux-mm/CAMgjq7B1K=3D6OOrK2O= UZ0-tqCzi+EJt+2_K97TPGoSt=3D9+JwP7Q@mail.gmail.com/ [1] > > > > > > > Signed-off-by: Kairui Song > > > > > > > --- > > > > > > > mm/userfaultfd.c | 26 ++++++++++++++++++++++++++ > > > > > > > 1 file changed, 26 insertions(+) > > > > > > > > > > > > > > diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c > > > > > > > index bc473ad21202..a1564d205dfb 100644 > > > > > > > --- a/mm/userfaultfd.c > > > > > > > +++ b/mm/userfaultfd.c > > > > > > > @@ -15,6 +15,7 @@ > > > > > > > #include > > > > > > > #include > > > > > > > #include > > > > > > > +#include > > > > > > I guess you mistakenly left it from your reproducer code :) > > > > > > > #include > > > > > > > #include > > > > > > > #include "internal.h" > > > > > > > @@ -1086,6 +1087,8 @@ static int move_swap_pte(struct mm_stru= ct *mm, struct vm_area_struct *dst_vma, > > > > > > > spinlock_t *dst_ptl, spinlock_t *src= _ptl, > > > > > > > struct folio *src_folio) > > > > > > > { > > > > > > > + swp_entry_t entry; > > > > > > > + > > > > > > > double_pt_lock(dst_ptl, src_ptl); > > > > > > > > > > > > > > if (!is_pte_pages_stable(dst_pte, src_pte, orig_dst_p= te, orig_src_pte, > > > > > > > @@ -1102,6 +1105,19 @@ static int move_swap_pte(struct mm_str= uct *mm, struct vm_area_struct *dst_vma, > > > > > > > if (src_folio) { > > > > > > > folio_move_anon_rmap(src_folio, dst_vma); > > > > > > > src_folio->index =3D linear_page_index(dst_vm= a, dst_addr); > > > > > > > + } else { > > > > > > > + /* > > > > > > > + * Check again after acquiring the src_pte lo= ck. Or we might > > > > > > > + * miss a new loaded swap cache folio. > > > > > > > + */ > > > > > > > + entry =3D pte_to_swp_entry(orig_src_pte); > > > > > > > + src_folio =3D filemap_get_folio(swap_address_= space(entry), > > > > > > > + swap_cache_inde= x(entry)); > > > > > > > > > > > > Given the non-trivial overhead of filemap_get_folio(), do you t= hink it > > > > > > will work if filemap_get_filio() was only once after locking sr= c_ptl? > > > > > > Please correct me if my assumption about the overhead is wrong. > > > > > > > > > > not quite sure as we have a folio_lock(src_folio) before move_swa= p_pte(). > > > > > can we safely folio_move_anon_rmap + src_folio->index while not h= olding > > > > > folio lock? > > > > > > > > I think no, we can't even make sure the folio is still in the swap > > > > cache, so it can be a freed folio that does not belong to any VMA > > > > while not holding the folio lock. > > > > > > Right, but will the following be sufficient, given that we don=E2=80= =99t really > > > care about the folio=E2=80=94only whether there=E2=80=99s new cache? > > > > > > if (READ_ONCE(si->swap_map[offset]) & SWAP_HAS_CACHE) { > > > double_pt_unlock(dst_ptl, src_ptl); > > > return -EAGAIN; > > > } > > > > The problem is reading swap_map without locking the cluster map seems > > unstable, and has strange false positives, a swapin will set this bit > > first, while not adding the folio to swap cache or even when skipping > > the swap cache, that seems could make it more complex. > > As long as it's a false positive and not a false negative, I think it's > acceptable=E2=80=94especially if we're concerned about the overhead of > filemap_get_folio. The probability is extremely low (practically close > to 0%), but we still need to call filemap_get_folio for every swap PTE. > That's exactly my concern too. A retry or EAGAIN on rare false positives are acceptable. But adding an additional call to filemap_get_folio, that too with PTL for both src and dst locked is not cheap. Consider that on a multi-threaded application, there could be many threads blocked on the same PTL. So keeping that critical section as short as possible is desirable. > Thanks > Barry