From: Peter Xu <peterx@redhat.com>
To: linux-mm@kvack.org, linux-kernel@vger.kernel.org
Cc: Rik van Riel <riel@surriel.com>,
Muchun Song <songmuchun@bytedance.com>,
Andrew Morton <akpm@linux-foundation.org>,
peterx@redhat.com, James Houghton <jthoughton@google.com>,
Nadav Amit <nadav.amit@gmail.com>,
Andrea Arcangeli <aarcange@redhat.com>,
David Hildenbrand <david@redhat.com>,
Miaohe Lin <linmiaohe@huawei.com>,
Mike Kravetz <mike.kravetz@oracle.com>
Subject: [PATCH RFC v2 05/12] mm/hugetlb: Make userfaultfd_huge_must_wait() safe to pmd unshare
Date: Thu, 17 Nov 2022 20:10:18 -0500 [thread overview]
Message-ID: <20221118011025.2178986-6-peterx@redhat.com> (raw)
In-Reply-To: <20221118011025.2178986-1-peterx@redhat.com>
We can take the hugetlb walker lock, here taking vma lock directly.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
fs/userfaultfd.c | 18 ++++++++++++++----
1 file changed, 14 insertions(+), 4 deletions(-)
diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index 07c81ab3fd4d..a602f008dde5 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -376,7 +376,8 @@ static inline unsigned int userfaultfd_get_blocking_state(unsigned int flags)
*/
vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason)
{
- struct mm_struct *mm = vmf->vma->vm_mm;
+ struct vm_area_struct *vma = vmf->vma;
+ struct mm_struct *mm = vma->vm_mm;
struct userfaultfd_ctx *ctx;
struct userfaultfd_wait_queue uwq;
vm_fault_t ret = VM_FAULT_SIGBUS;
@@ -403,7 +404,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason)
*/
mmap_assert_locked(mm);
- ctx = vmf->vma->vm_userfaultfd_ctx.ctx;
+ ctx = vma->vm_userfaultfd_ctx.ctx;
if (!ctx)
goto out;
@@ -493,6 +494,13 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason)
blocking_state = userfaultfd_get_blocking_state(vmf->flags);
+ /*
+ * This stablizes pgtable for hugetlb on e.g. pmd unsharing. Need
+ * to be before setting current state.
+ */
+ if (is_vm_hugetlb_page(vma))
+ hugetlb_vma_lock_read(vma);
+
spin_lock_irq(&ctx->fault_pending_wqh.lock);
/*
* After the __add_wait_queue the uwq is visible to userland
@@ -507,13 +515,15 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason)
set_current_state(blocking_state);
spin_unlock_irq(&ctx->fault_pending_wqh.lock);
- if (!is_vm_hugetlb_page(vmf->vma))
+ if (!is_vm_hugetlb_page(vma))
must_wait = userfaultfd_must_wait(ctx, vmf->address, vmf->flags,
reason);
else
- must_wait = userfaultfd_huge_must_wait(ctx, vmf->vma,
+ must_wait = userfaultfd_huge_must_wait(ctx, vma,
vmf->address,
vmf->flags, reason);
+ if (is_vm_hugetlb_page(vma))
+ hugetlb_vma_unlock_read(vma);
mmap_read_unlock(mm);
if (likely(must_wait && !READ_ONCE(ctx->released))) {
--
2.37.3
next prev parent reply other threads:[~2022-11-18 1:10 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-18 1:10 [PATCH RFC v2 00/12] mm/hugetlb: Make huge_pte_offset() thread-safe for " Peter Xu
2022-11-18 1:10 ` [PATCH RFC v2 01/12] mm/hugetlb: Let vma_offset_start() to return start Peter Xu
2022-11-18 1:10 ` [PATCH RFC v2 02/12] mm/hugetlb: Move swap entry handling into vma lock for fault Peter Xu
2022-11-18 1:35 ` Peter Xu
2022-11-18 1:10 ` [PATCH RFC v2 03/12] mm/hugetlb: Don't wait for migration entry during follow page Peter Xu
2022-11-18 1:10 ` [PATCH RFC v2 04/12] mm/hugetlb: Add pgtable walker lock Peter Xu
2022-11-18 1:10 ` Peter Xu [this message]
2022-11-18 1:10 ` [PATCH RFC v2 06/12] mm/hugetlb: Protect huge_pmd_share() with " Peter Xu
2022-11-18 1:17 ` Peter Xu
2022-11-18 1:10 ` [PATCH RFC v2 07/12] mm/hugetlb: Use hugetlb walker lock in hugetlb_follow_page_mask() Peter Xu
2022-11-18 1:10 ` [PATCH RFC v2 08/12] mm/hugetlb: Use hugetlb walker lock in follow_hugetlb_page() Peter Xu
2022-11-18 1:10 ` [PATCH RFC v2 09/12] mm/hugetlb: Use hugetlb walker lock in hugetlb_vma_maps_page() Peter Xu
2022-11-18 1:10 ` [PATCH RFC v2 10/12] mm/hugetlb: Use hugetlb walker lock in walk_hugetlb_range() Peter Xu
2022-11-18 1:11 ` [PATCH RFC v2 11/12] mm/hugetlb: Use hugetlb walker lock in page_vma_mapped_walk() Peter Xu
2022-11-18 1:11 ` [PATCH RFC v2 12/12] mm/hugetlb: Introduce hugetlb_walk() Peter Xu
2022-11-23 9:40 ` [PATCH RFC v2 00/12] mm/hugetlb: Make huge_pte_offset() thread-safe for pmd unshare David Hildenbrand
2022-11-23 15:09 ` Peter Xu
2022-11-23 18:21 ` Mike Kravetz
2022-11-23 18:56 ` Peter Xu
2022-11-23 19:31 ` David Hildenbrand
2022-11-25 9:43 ` David Hildenbrand
2022-11-25 13:55 ` Peter Xu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221118011025.2178986-6-peterx@redhat.com \
--to=peterx@redhat.com \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=david@redhat.com \
--cc=jthoughton@google.com \
--cc=linmiaohe@huawei.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mike.kravetz@oracle.com \
--cc=nadav.amit@gmail.com \
--cc=riel@surriel.com \
--cc=songmuchun@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox