From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48D0CC4332F for ; Fri, 9 Dec 2022 17:01:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C6DEA8E0008; Fri, 9 Dec 2022 12:01:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BF7238E0001; Fri, 9 Dec 2022 12:01:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A716E8E0008; Fri, 9 Dec 2022 12:01:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9768D8E0001 for ; Fri, 9 Dec 2022 12:01:27 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6040F1413BD for ; Fri, 9 Dec 2022 17:01:27 +0000 (UTC) X-FDA: 80223383814.20.31E1D14 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf10.hostedemail.com (Postfix) with ESMTP id 0A22EC0034 for ; Fri, 9 Dec 2022 17:01:24 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=g7dzTzuI; spf=pass (imf10.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670605285; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aMLwOwXmHw3aEdY2dAtBVU7yezK1ZrIU2NcKyDsSofc=; b=GqiC9GG6mukZPHbQEXrnvvCfYPu8yRaFp9E4UmMQU5WEILGPQFEWZfOTWD4UaNzTvG1U5Z im8/MH4MqooYM9c/3vEHgSKokmJiFVzcEyAto5SztxskKsoqafTUnwaTHEHrli7WIo9Wlz iC/tWkgVpqdzW/x74LzpaoyxDdp7aXo= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=g7dzTzuI; spf=pass (imf10.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670605285; a=rsa-sha256; cv=none; b=THUtJsvmzdhCNwX+pt/9QiERvJdpRokYm8LigsH/S3nZkscWgJxLySrLybl1J7ixBvQYnB EoJfer3jT2iXkw4ZYElNQSzFz3GCK8RCwoGYEGKRPKRmeRZPVwd0kBiG4stRSgJnO/tH8E 65KEN5nYarUziOwbrlAbUp+pcMjORBw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670605284; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aMLwOwXmHw3aEdY2dAtBVU7yezK1ZrIU2NcKyDsSofc=; b=g7dzTzuIStyhUqbX5O2PvBP0ufK6hU7vMNljbd3UAFtAFPn3YwGQ6Z/OZeCmQ1eFuHRWTR tafqkgQspdy/FVwdoTKNLA1Jns6Gdn26ZYLaiLuRTCCUBFPZuwuLEW2Q4/wfI6yze8dK2T eignUeerhCZtpVbFtPshV2pgeQ7hbRU= Received: from mail-oo1-f72.google.com (mail-oo1-f72.google.com [209.85.161.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-35-pWm7OXQ9Ot-2PGyKKiH9Qw-1; Fri, 09 Dec 2022 12:01:22 -0500 X-MC-Unique: pWm7OXQ9Ot-2PGyKKiH9Qw-1 Received: by mail-oo1-f72.google.com with SMTP id c6-20020a4ad206000000b004a33f36aa4dso1583243oos.21 for ; Fri, 09 Dec 2022 09:01:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aMLwOwXmHw3aEdY2dAtBVU7yezK1ZrIU2NcKyDsSofc=; b=7E9MQpOmolV6kWQyd3el9kDmmT84fQpR0Qb/03D34qsrNQUCG9FaYD5XQ5RihjT3zt MCEijxv+uxQq+M3z6W+CWIUbQw9QNuVhJn4Vi1nN+tA4YsOtQf91AEUbAP1zvCK0ReZo ta2796yvl/Q5ZlDtrCdNnTZw7sHeHldfPC0WNmnpBkYRLJkIqhT36zJlPGbXe0SdVKxX 5lFdyNUhat84nSywpwtqB3NGi1RZI1Luxc1NCb4LnpzcaAPyctYwen9oVp6yCmdFF5vw qezdLTD2KyyjTQr9R7uzy/WVgd26dyXLoRAr4XhTOMUGV5nEwxVgVFI8CSZPRYA2ZWdg vWFQ== X-Gm-Message-State: ANoB5plv50NxZNN64dzeMuLlZB9zLfccQaICkAclZXk+hhnRhLMN8GHY JN6vtrIdXni3LuO2lJA1QvMZZkeTWY40H6CjCJ47+2MNahFK2zkOgrm+NBkMb/SnQiI6/cPTIfR dE5zoGWHytR/6F002Ht5E/06zKlRvRpLXA0eTd9ai5JzBHcuMTglZtx+jFti6 X-Received: by 2002:a9d:77c1:0:b0:66d:c8a2:b9b with SMTP id w1-20020a9d77c1000000b0066dc8a20b9bmr3276936otl.12.1670605281248; Fri, 09 Dec 2022 09:01:21 -0800 (PST) X-Google-Smtp-Source: AA0mqf4oQHCNJfUC59F5uvhM+SYrQAPCUwZXG/cB2hzsrwqoVJGknQCDLk0levxupMtmfDVgjRq8Jw== X-Received: by 2002:a9d:77c1:0:b0:66d:c8a2:b9b with SMTP id w1-20020a9d77c1000000b0066dc8a20b9bmr3276898otl.12.1670605280840; Fri, 09 Dec 2022 09:01:20 -0800 (PST) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id q7-20020a05620a0d8700b006cf38fd659asm178907qkl.103.2022.12.09.09.01.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Dec 2022 09:01:20 -0800 (PST) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Nadav Amit , peterx@redhat.com, Andrea Arcangeli , Jann Horn , John Hubbard , Mike Kravetz , James Houghton , Rik van Riel , Muchun Song Subject: [PATCH v3 5/9] mm/hugetlb: Make userfaultfd_huge_must_wait() safe to pmd unshare Date: Fri, 9 Dec 2022 12:00:56 -0500 Message-Id: <20221209170100.973970-6-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221209170100.973970-1-peterx@redhat.com> References: <20221209170100.973970-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 0A22EC0034 X-Stat-Signature: 3gzhfyd4d66e9ou6nz3gt7i643dccy6x X-HE-Tag: 1670605284-114513 X-HE-Meta: U2FsdGVkX1/yrRveH2rCVqN0JUGBpNoXD46NBCjssaIJfUI1Sja/X78QIEMaPv+1wXa0BKLy6LcJrIsTv5+qFb8jNcEFVFhH1+i1jdFmAOehBVmBvRWxSjmPnA847vR3QsmR1ZDbVr5jxL9u6uWV8rc1Jy6/As7tfzGDIVYayh80MrOv/dBlhCwBHX64SEXYRlCQ8D6wAI4MEnTbSwoUUetea/ojE7PWCWoDJYIv+gReHExek3suB8A7ytuGtCV34KLz4+up7Ecj5CMY/OHAai7lgYgmoxl/wYla0mKEJD81URUzhYsrrt6pmhqevlAy6ic0yUJAOOn71s80ZZ31g1bQvhqhCFGFIxZ+6HlPeMQSZwnXlSQ1qBs4fTPMQn9rV6VlwIns0QABtN/CFw13/P9N2YUOhZ6vY9q266D6bxgMqD9mi22OK2snDog3GlFt12xVw+KBNrwf9jTMJB96VSAu6/yOeWcqCd+/nBKJ/pmATL5FAM8bmXtZ/QrathMc/+UU5v7/SvqiScbnsVyDacWW0sl1VaqN/eNY3tUWiU9lYxdwN4AX0zudJ+J+kKR30B75CeY61vj4dXV+HcQYy/p7YOfRfjFoiJKFmCPW0OTNKT0B2Zt+HTg6xhGhoZfyhal1mMELbqdaccz1NhCC3S/JCH+2yVJNUTst15Ba7yhxlvyLEbiESSrE3jdFMd/YIQZi4n7L1OBzhS+HhMLSuzZ9UapsqMzXRiA1OlSQEQvMBKBivHZNKG3q3j59RMliJ2ws0YX9kzN1ISy4C4clRPF6fMrpGlJAbbBiOvV/atGaPjxdBfwnZjdMbBSfpZvym9CoxPXOzKh0su4pGqobMdQxJy7RQSiTi9QZSh5zdUh6WMkSQtEfcOcdx2jXYx6W8sxXcyEgu10zyHF6dgcNIlkbF50Gv6gBaOv7Kd+wSCFKOK7/NlIiqadnbFHyjGbANovgkibpEeiNmbaTlda 5V8Qp/5d yXRTVESB+nET0gblSnd6qm2Gn2n9DipLctiSnDallUiGZR40AczDWn0M28clGWqlowR2vVs39bUO8XBjNbMC4ydXXzGbCJ2B5h/67/Q495TaNrYGCiMeGtsnh5O+R1ApMVQ2FerzT9ODsm66Iorz7d2vIhW/OP19nXbLwMjig+i9eb5j+o5qMttAf6FvYgaGUvG/9rV/2csDHfDuCaxU4HRgsfgmgh+cOsjE7k6SsTTzhm4fMWP+PwoyA9Yy1uXMX5tfiRi4Ez+Fz+gk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We can take the hugetlb walker lock, here taking vma lock directly. Reviewed-by: David Hildenbrand Reviewed-by: Mike Kravetz Reviewed-by: John Hubbard Signed-off-by: Peter Xu --- fs/userfaultfd.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 07c81ab3fd4d..969f4be967c6 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -376,7 +376,8 @@ static inline unsigned int userfaultfd_get_blocking_state(unsigned int flags) */ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) { - struct mm_struct *mm = vmf->vma->vm_mm; + struct vm_area_struct *vma = vmf->vma; + struct mm_struct *mm = vma->vm_mm; struct userfaultfd_ctx *ctx; struct userfaultfd_wait_queue uwq; vm_fault_t ret = VM_FAULT_SIGBUS; @@ -403,7 +404,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) */ mmap_assert_locked(mm); - ctx = vmf->vma->vm_userfaultfd_ctx.ctx; + ctx = vma->vm_userfaultfd_ctx.ctx; if (!ctx) goto out; @@ -493,6 +494,15 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) blocking_state = userfaultfd_get_blocking_state(vmf->flags); + /* + * Take the vma lock now, in order to safely call + * userfaultfd_huge_must_wait() later. Since acquiring the + * (sleepable) vma lock can modify the current task state, that + * must be before explicitly calling set_current_state(). + */ + if (is_vm_hugetlb_page(vma)) + hugetlb_vma_lock_read(vma); + spin_lock_irq(&ctx->fault_pending_wqh.lock); /* * After the __add_wait_queue the uwq is visible to userland @@ -507,13 +517,15 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) set_current_state(blocking_state); spin_unlock_irq(&ctx->fault_pending_wqh.lock); - if (!is_vm_hugetlb_page(vmf->vma)) + if (!is_vm_hugetlb_page(vma)) must_wait = userfaultfd_must_wait(ctx, vmf->address, vmf->flags, reason); else - must_wait = userfaultfd_huge_must_wait(ctx, vmf->vma, + must_wait = userfaultfd_huge_must_wait(ctx, vma, vmf->address, vmf->flags, reason); + if (is_vm_hugetlb_page(vma)) + hugetlb_vma_unlock_read(vma); mmap_read_unlock(mm); if (likely(must_wait && !READ_ONCE(ctx->released))) { -- 2.37.3