From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3266C001B2 for ; Fri, 16 Dec 2022 15:52:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5B2AB8E0006; Fri, 16 Dec 2022 10:52:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 450428E0002; Fri, 16 Dec 2022 10:52:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 22DC18E0006; Fri, 16 Dec 2022 10:52:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 06D158E0002 for ; Fri, 16 Dec 2022 10:52:29 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 961B740669 for ; Fri, 16 Dec 2022 15:52:28 +0000 (UTC) X-FDA: 80248611576.20.92AF16B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf26.hostedemail.com (Postfix) with ESMTP id A025914000E for ; Fri, 16 Dec 2022 15:52:26 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Fq9Y01sw; spf=pass (imf26.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671205946; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oQxWaXmf/BdMLd8AroQPz0fxFz+2cpe/6LbLYuumgUQ=; b=X80zj/nuYu6XJoPRxoPRXvDLDjsPcXBnnXdL+Tr8bB2oZWMGHdp2kimItlyhrNask+UEdO 8jYeVJAdOoQfUsUiezyxqqkI7uro/3MAE5JzY348RmZYWFx+7WCXGuGAuSexpT1Z2pxUKa uaENcHfvEn8YnmROE/wzS2MQ0Dv61dI= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Fq9Y01sw; spf=pass (imf26.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671205946; a=rsa-sha256; cv=none; b=5mugqQptBE0W8XM7AeOuZIVPQNRDvSncJjJdh2CmPbnpyhMZ1tIdlO2HHeVHWvpH2xeXBe A8llRpwF9bhovOJjVnjNDsSswzSn+F8b8Qz4ZbUgNUyRJbvKT2t3kGrwLtGAXjqL6OPxFE I6i/JzZ+XvAneLBfG8MF8HLNPNHHeto= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1671205946; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oQxWaXmf/BdMLd8AroQPz0fxFz+2cpe/6LbLYuumgUQ=; b=Fq9Y01swPG7tPVzWweQ9JfC9o7xRGfxd3pXVTlPhd9+ulDNKdPNsq3eUWFVnd3I3wI51Bo giFMNgR3iYUGmxTQCzkikm3uOx334czRiJRW39weuwuDvXekF0ggPf6O715fVDk8dSG9lN TB2SgDZV/SINBRvc9ywd0mariP54MbM= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-78-nyX4EDnUNouD5ljbR3wW4w-1; Fri, 16 Dec 2022 10:52:20 -0500 X-MC-Unique: nyX4EDnUNouD5ljbR3wW4w-1 Received: by mail-qk1-f200.google.com with SMTP id q20-20020a05620a0d9400b006fcaa1eac9bso2055771qkl.23 for ; Fri, 16 Dec 2022 07:52:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oQxWaXmf/BdMLd8AroQPz0fxFz+2cpe/6LbLYuumgUQ=; b=lhk8MrY44HWAXgM1TTZ1S7DOxyt//hXzRtQJSSnA2kACF9gGMnYe48CDX/CZ6leZCr kxjv4WR/7Odnx4VfZXwJl8ewsEqs1UoGfqqElhv0BWBiwqx6X3v80syFtwx/rDUe3JJC JgIQzJesOmaPy1I422V1zeJBs60rOYk5szb6pqvFPf/EJrmxZn/BTq+1kMmqSlk0kjWF HqkK93/P2dPJS56nDTlaBhC+6fXxhHADPbo5FguJezz0biCoJwiVQQXnJ4vJtEgd+AMq CGrdlE4pVFcX3lXO2tNmeate+m+ZFZKDk+xU3acTWH0Zf//IVNfv/V/zkqFcYSLhKFe2 vliA== X-Gm-Message-State: ANoB5plwXzNjb9pvc+gwYrZH02uPtrhbFDWL6jlUuYlcK8xtDQJoohvV jwtVzE7s4+lEyd+AD+Motdccm1SIYy8QuZV5t9abJJo1XFUBi5JaUA4q/b6ES7RzCCdP4DPBHDV Ao3i3KByjpbU= X-Received: by 2002:a05:622a:4c11:b0:3a5:2f80:5b08 with SMTP id ey17-20020a05622a4c1100b003a52f805b08mr46852438qtb.1.1671205939596; Fri, 16 Dec 2022 07:52:19 -0800 (PST) X-Google-Smtp-Source: AA0mqf7dhghRe2b9Hr/nL0Vtk5vE2Zf1v91HSpvXyjbmaYYF/40Yyuz87fscH2VZ1GmjeQmVyYE2LA== X-Received: by 2002:a05:622a:4c11:b0:3a5:2f80:5b08 with SMTP id ey17-20020a05622a4c1100b003a52f805b08mr46852414qtb.1.1671205939371; Fri, 16 Dec 2022 07:52:19 -0800 (PST) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-45-70-31-26-132.dsl.bell.ca. [70.31.26.132]) by smtp.gmail.com with ESMTPSA id x13-20020ac8120d000000b003a6847d6386sm1483787qti.68.2022.12.16.07.52.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Dec 2022 07:52:18 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: David Hildenbrand , John Hubbard , Muchun Song , Mike Kravetz , Nadav Amit , Andrea Arcangeli , Rik van Riel , peterx@redhat.com, Miaohe Lin , Jann Horn , James Houghton , Andrew Morton Subject: [PATCH v4 5/9] mm/hugetlb: Make userfaultfd_huge_must_wait() safe to pmd unshare Date: Fri, 16 Dec 2022 10:52:17 -0500 Message-Id: <20221216155217.2043700-1-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221216155100.2043537-1-peterx@redhat.com> References: <20221216155100.2043537-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII"; x-default=true X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A025914000E X-Stat-Signature: zc9dyd5g3znfzkt7iww3sh3dwdorfwg3 X-Rspam-User: X-HE-Tag: 1671205946-229045 X-HE-Meta: U2FsdGVkX193RbXp5kyqgdNjfr1mTMeXs4LKqALNx5LjmtdGssK/1Zf2Teo4KmfEQ5jS8YeOHiaaC02zQXBF/PkQc0Lf6/YW5p2UEW0wRncxffqxzn4LyntxXTv9tAQtVyzvn/rOwFbEMmPaytbObw+rMAULMCcHS+XEnwF1dRvjqbs8+EynuBVM0Q9FTrFjO5MoFy7QvV4bLgZ3eRwOEDhJumGLzefJZXjPPlCpnNGirIGVw9vdmIkciiAJuYMMMKBvvYd1BwzmV4ApYdrGRu7LT/R+Autdbp9451rFlrdD0x4VO+2QBZISwSB06yYiGnd+NfcLM3xPyOcA/4MqrlRZcGOp7ZrTk/VmTFkesDhlh+YG6m4fctzpOrzPgZnqQyp/7AO4Dda2se1KS9+prrUxzdF/Re4wkJvp18vC1TkAeiJETrZGTwNfYrmJJsrgwXEoeyb0z3vrZ6sSu1QOA1wLJZCNemyyD4ju23H0Grlv/+S9zqhYFzpvew/UhdB34fWCZziGNI+zsx8+OAW701x7LaQBs9dtKetNG//IAyhvDyqshdUi9XdYSld0F6cBgObCX84pfRRbMUm97mdkSGkR7InZHI5rlr0+orLHONAbQAIwV/IC4zYzqC9trUz//2Uw0PXiGKFYta2jzL9Ar0T2AzY1iXLMKienlVYdEJoyBHSvEug6eFe8nNc529FWL5sUdgVqcRidqmkfATPRo2P8lRTdOzYDJ4hWnnucc+pXDItUQzs9Wu4mhVQM8nnU8txiQxPPZbDDdfs75ADbndms6qDGV/WiHp4Mj8zpfh5RALzfOU71glPSLsc/7ERwBTHNvZoTSJcBxxD5fKui/5Jo1srjLWK5dtBm5WzFeQtz4/6HWgJ3onycaP3PISXFFe254p+AtcAUgEgOZOWUVRz/oxAdoihoH8XrHXh8L6cHaVw5Nzo16eSw7/8mysVYkb5L+8IdpEPtJEZRxqO qJ4AY4MW F6MDPR5xfX0jFn+VeRX/8Il+ObiPD154QJImpBqAG1obz0NibfxXZWS9U22+i+nrBiqcYzaX+LJfBU9tuzOiBe1TDeNQRVdmPW6GzGUAVB+CnIt46GHQ2AkJd6NZyZz85PzToaHpBgcGSmHLb5XX1xEenqY4fLmxCIMdBuSa29sB0VfosXQrhhCNYOOWMVyHBZii3Qh4uhf5ozV1P7zFrwWN9WXaSiqrR5lup X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We can take the hugetlb walker lock, here taking vma lock directly. Reviewed-by: David Hildenbrand Reviewed-by: Mike Kravetz Reviewed-by: John Hubbard Signed-off-by: Peter Xu --- fs/userfaultfd.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 98ac37e34e3d..887e20472051 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -376,7 +376,8 @@ static inline unsigned int userfaultfd_get_blocking_state(unsigned int flags) */ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) { - struct mm_struct *mm = vmf->vma->vm_mm; + struct vm_area_struct *vma = vmf->vma; + struct mm_struct *mm = vma->vm_mm; struct userfaultfd_ctx *ctx; struct userfaultfd_wait_queue uwq; vm_fault_t ret = VM_FAULT_SIGBUS; @@ -403,7 +404,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) */ mmap_assert_locked(mm); - ctx = vmf->vma->vm_userfaultfd_ctx.ctx; + ctx = vma->vm_userfaultfd_ctx.ctx; if (!ctx) goto out; @@ -493,6 +494,15 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) blocking_state = userfaultfd_get_blocking_state(vmf->flags); + /* + * Take the vma lock now, in order to safely call + * userfaultfd_huge_must_wait() later. Since acquiring the + * (sleepable) vma lock can modify the current task state, that + * must be before explicitly calling set_current_state(). + */ + if (is_vm_hugetlb_page(vma)) + hugetlb_vma_lock_read(vma); + spin_lock_irq(&ctx->fault_pending_wqh.lock); /* * After the __add_wait_queue the uwq is visible to userland @@ -507,13 +517,15 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) set_current_state(blocking_state); spin_unlock_irq(&ctx->fault_pending_wqh.lock); - if (!is_vm_hugetlb_page(vmf->vma)) + if (!is_vm_hugetlb_page(vma)) must_wait = userfaultfd_must_wait(ctx, vmf->address, vmf->flags, reason); else - must_wait = userfaultfd_huge_must_wait(ctx, vmf->vma, + must_wait = userfaultfd_huge_must_wait(ctx, vma, vmf->address, vmf->flags, reason); + if (is_vm_hugetlb_page(vma)) + hugetlb_vma_unlock_read(vma); mmap_read_unlock(mm); if (likely(must_wait && !READ_ONCE(ctx->released))) { -- 2.37.3