From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 816B8EB64DA for ; Thu, 20 Jul 2023 21:28:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D096B280160; Thu, 20 Jul 2023 17:28:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C928128004C; Thu, 20 Jul 2023 17:28:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0C59280160; Thu, 20 Jul 2023 17:28:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9DDB528004C for ; Thu, 20 Jul 2023 17:28:13 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 75A7D16026E for ; Thu, 20 Jul 2023 21:28:13 +0000 (UTC) X-FDA: 81033278466.12.5ADDFBE Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf25.hostedemail.com (Postfix) with ESMTP id 71AC6A0021 for ; Thu, 20 Jul 2023 21:28:10 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=nXxLfgmB; spf=pass (imf25.hostedemail.com: domain of isaku.yamahata@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=isaku.yamahata@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689888490; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9Hy/9zTVZrlg7bbFpxkjr0lhxcNTNGSKdXmN/n9YFLY=; b=Oy7ruAZrf6W70rn3SL4wTzDn13HTnZKhwISMRe7H7NDblSEr9DSVeA3npVfvs5l9VBfyNt 3FbMMtiDNtLNVQlmX7DI1h7k3wiouzNLhFcKr6N5ejsEXAR0YGpMLotL0hin8zuTvFtOXV DrzbqKdFdrWDHcL7Ark4YxuRQkD564Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689888490; a=rsa-sha256; cv=none; b=uKxYPsuAn8sBZciHa050ioFiKH6YrqimGo3+QvZzKLQjxdAysJo7OJvg8bWqtZktFGn+Ov m4LHaUtC2sYw/BU2yrMP2wgPQsw7tLa5Rl24BEKErvPtuqrZl6zmGzVUEZZcqf5BxY6PUP A0aL6XuvXR6W905/MSE9enjntj0My+g= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=nXxLfgmB; spf=pass (imf25.hostedemail.com: domain of isaku.yamahata@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=isaku.yamahata@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-666e97fcc60so882182b3a.3 for ; Thu, 20 Jul 2023 14:28:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689888489; x=1690493289; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=9Hy/9zTVZrlg7bbFpxkjr0lhxcNTNGSKdXmN/n9YFLY=; b=nXxLfgmBt92XEDE8OP3OcE0nisAlc/Gi9Zse4iq73APm5A373sb6ySF1KTfOUa+QuX ia3Xy59KQUF7gIB0hPoPjhSopV94TeiwDt/LztWRB6Qyz1jHNY1OozvVaE5RMQ9UdGw5 PMMWNQSNpVIxaxHNS4LWKSkQHzq5Z3SDiNLSXhdgHAc7DXvTGgH+4VKo2M8xFGbIHCXh 337Ve685ARtbWN8k2WWVvoDis4FS9zWIZ1LrAJA1fxWvbDhCuW9Dnu/E9PeGYaOXlPVE kd4H7n4i3GLSfis0TrFmdjayzB65DGsbK8pheZGzu8z8qpliVRrp9gRaSAXgpDekRLO2 A4Nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689888489; x=1690493289; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=9Hy/9zTVZrlg7bbFpxkjr0lhxcNTNGSKdXmN/n9YFLY=; b=H8FzYEBd6QbeoOLsBrP2N1yc+jLQ1UkVZnwQJpCct05IJhhjfavnA0kD1SK/MBq5M1 QjJDoE3QShCRQpvVgyTY/4BUYZehHX40hi2kENnZvMPVJLwfBZqJYSYBdr9HOstBKksW SEGYXZRVMYkvNpNnb7z5SBr27CWo74UTkJlWRviE1lbH1QRdnmDUr7D+RmSqjy5N6Tti bdykOCxwIpQ3AHJndGPT2CROLlIvYhrRwXzuglg8mYMsZKQJVA6vaE6XRrNkwKRXfjg4 3+OzuSmBIXP0U6dahbZeD0LrKCv1YrD89T/zOYYmF2dHrbhXiE6wvXehjCT23UNXL2Xa bjYQ== X-Gm-Message-State: ABy/qLYCwszwjDGo+IRQqk9MP2pNnPGSl22+uTFzjM0KHaDq2ep0PHrk rGDs35r0CqgqeuyexsNXBPI= X-Google-Smtp-Source: APBJJlG/e53j4EpED5Jwtfn4LkeoExV6FWkBmpJOuNFRNxz/SI2h0UOe64NVRwIQoof3O2DGaGflqQ== X-Received: by 2002:a05:6a20:2583:b0:135:10fd:31b0 with SMTP id k3-20020a056a20258300b0013510fd31b0mr109795pzd.15.1689888488716; Thu, 20 Jul 2023 14:28:08 -0700 (PDT) Received: from localhost ([192.55.54.50]) by smtp.gmail.com with ESMTPSA id n2-20020a62e502000000b006826df9e286sm1637942pff.143.2023.07.20.14.28.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Jul 2023 14:28:08 -0700 (PDT) Date: Thu, 20 Jul 2023 14:28:06 -0700 From: Isaku Yamahata To: Sean Christopherson Cc: Paolo Bonzini , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , "Matthew Wilcox (Oracle)" , Andrew Morton , Paul Moore , James Morris , "Serge E. Hallyn" , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-security-module@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Peng , Fuad Tabba , Jarkko Sakkinen , Yu Zhang , Vishal Annapurve , Ackerley Tng , Maciej Szmigiero , Vlastimil Babka , David Hildenbrand , Quentin Perret , Michael Roth , Wang , Liam Merwick , Isaku Yamahata , "Kirill A . Shutemov" Subject: Re: [RFC PATCH v11 12/29] KVM: Add KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory Message-ID: <20230720212806.GG25699@ls.amr.corp.intel.com> References: <20230718234512.1690985-1-seanjc@google.com> <20230718234512.1690985-13-seanjc@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20230718234512.1690985-13-seanjc@google.com> X-Rspamd-Queue-Id: 71AC6A0021 X-Rspam-User: X-Stat-Signature: qg19idcng78ppiq4i6pdaysapaxyjdjs X-Rspamd-Server: rspam03 X-HE-Tag: 1689888490-459214 X-HE-Meta: U2FsdGVkX1/CsX+epQRq4gQXzBOlwJ3zIqys45U9SyR+S4+ycDFHoFEy7KIKQ/mjNB/+QgrqyNNkm2pL3fwCwdWaZgMjyTpwJChTi56ekypJ/Kcb0zb+MvetLBzYnLdU+x7MlF3j/LCLMFDWKiWrsL//LM8c7/t4qceWoyuGLblGt5dtJhBaOpHJJ+keAOK1O4KZE4DvFLckUAZbTBz3gDLtneVxSZRKRVAuGTnPInDC57yqiKYqwNi9kwRs7YPsPwpl7PHcEVNldOOfYz0rw5zDKje0PKTIE+zTZi6dW9zNaUVPnRqGPqWpGGEypQoMXK6/zU09tC6uHZK0oQiT9UpN0WE5E2zg4/ztM3r82GgPIm6aqc/LQoEnZoRQMyQASA85bLkTj/JZGrHhRgRqjtFEto+TLkSl1cAxtPLRGQ/Z50jn1LDppg2wE4Cw4y81Wvx311MJBCOGVjctJoKtBMc7R9xuwt320kwzWjyB/TWV8rX3QVYCjeRuexY9nwhm+PZfu/ZOv9FlyVCmwAk/ETlZ/D+tHN05YQGjJSa/oe5Ehpeu/jwesoTO0iW9kpv84m1zCTLzUBTMZT9CesnT5mMHdBUUANNwyidzN4cGCEhDl+WnppWdTzq3NsqTZNwGP51uJBYO3sFu7virsjhWcyZOSqTtHbpHteJWXjT2TakuUaiPaQL1tMRHtwx1mj2gSjy+7svZupAEHWUjC9nDuoi7Py3ACjOXvc4Hn8JzFqyjzrMx5WgQl97ABL3yg/Ltr39DtH8rNqkZlVwBu2EPAoqsOY7wzItLigRvCExo6kQSGmBoiwPuctMtBODdNTw+quSv0qa38aV5yf2rQJ91j7Kq55VdsI9tdeRdgFHpnqkW4v3B5upY8670MRu0cIj1l1Lrt9GZlqGnUHglwSGwF5XvNnKGpUF0VngvWNYQnJqkqublnXKqUs0QowWujEwFUxau1y0k3lHu5nWl9IN VtdwANf8 uYDYRok5uTtHhVAkyO8+3I0fidPAmeiqmGUuDbnqW5VlXd5wbfAPGYk/hw2RjdvoIefMaovNI9KoTwr47/ec1aU5dzmNPtYxnRhyQ5ygHti3+SrjUmxKnzqyEj/W7w72ps1FEQHPYilwmm5WjoaJso9Orh3UTrzl9JGhZYvCS3/sML7IYV92/OGJ3F08Ob1aOU/OHQpxWcj0zhnirWbe2/PLuku8takfYgQxQhN1olP1bEvi5YEiKRY/tiTqryV2uReRIrkRn2mKNRKB2GTIskynz7qAnaZecwwtTE7nOl29jPHDj2rGEFDf4p92N397IIzxXwivAlxf8C2qd2hGSS2zDIwMj0p1IgtOWrTsoHNz4SOJhBRTcHE30bkxXD7JekWwIgQZoEt1DcpiBmaQ5attRzQQILV1UfudHEnmXq3DBDf/gTFP08a0GVdIG+8EzLz2/GFVPrFBp90LhyZzMnUJFdVfIeLvCKSiLq37xmhCPeFoWfui+uBXTFQF7ceTgHVRJqwyDTM7MEIB2JSL2xQlAVbpo4Gq+AFpRtaWh24v4jCB+CVSdDV4WbHP0dLE8XXfX7mlez5wB0u6SZN12iqVLAUwJDdi0NwRdGflr+wawcz0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jul 18, 2023 at 04:44:55PM -0700, Sean Christopherson wrote: > +static int kvm_gmem_release(struct inode *inode, struct file *file) > +{ > + struct kvm_gmem *gmem = file->private_data; > + struct kvm_memory_slot *slot; > + struct kvm *kvm = gmem->kvm; > + unsigned long index; > + > + filemap_invalidate_lock(inode->i_mapping); > + > + /* > + * Prevent concurrent attempts to *unbind* a memslot. This is the last > + * reference to the file and thus no new bindings can be created, but > + * dereferencing the slot for existing bindings needs to be protected > + * against memslot updates, specifically so that unbind doesn't race > + * and free the memslot (kvm_gmem_get_file() will return NULL). > + */ > + mutex_lock(&kvm->slots_lock); > + > + xa_for_each(&gmem->bindings, index, slot) > + rcu_assign_pointer(slot->gmem.file, NULL); > + > + synchronize_rcu(); > + > + /* > + * All in-flight operations are gone and new bindings can be created. > + * Zap all SPTEs pointed at by this file. Do not free the backing > + * memory, as its lifetime is associated with the inode, not the file. > + */ > + kvm_gmem_invalidate_begin(gmem, 0, -1ul); > + kvm_gmem_invalidate_end(gmem, 0, -1ul); > + > + mutex_unlock(&kvm->slots_lock); > + > + list_del(&gmem->entry); > + > + filemap_invalidate_unlock(inode->i_mapping); > + > + xa_destroy(&gmem->bindings); > + kfree(gmem); > + > + kvm_put_kvm(kvm); > + > + return 0; > +} The lockdep complains with the filemapping lock and the kvm slot lock. >From bc45eb084a761f93a87ba1f6d3a9949c17adeb31 Mon Sep 17 00:00:00 2001 Message-Id: From: Isaku Yamahata Date: Thu, 20 Jul 2023 14:16:21 -0700 Subject: [PATCH] KVM/gmem: Fix locking ordering in kvm_gmem_release() The lockdep complains the locking order. Fix kvm_gmem_release() VM destruction: - fput() ... \-kvm_gmem_release() \-filemap_invalidate_lock(inode->i_mapping); lock(&kvm->slots_lock); slot creation: kvm_set_memory_region() mutex_lock(&kvm->slots_lock); __kvm_set_memory_region(kvm, mem); \-kvm_gmem_bind() \-filemap_invalidate_lock(inode->i_mapping); ====================================================== WARNING: possible circular locking dependency detected ------------------------------------------------------ ... the existing dependency chain (in reverse order) is: -> #1 (mapping.invalidate_lock#4){+.+.}-{4:4}: ... down_write+0x40/0xe0 kvm_gmem_bind+0xd9/0x1b0 [kvm] __kvm_set_memory_region.part.0+0x4fc/0x620 [kvm] __kvm_set_memory_region+0x6b/0x90 [kvm] kvm_vm_ioctl+0x350/0xa00 [kvm] __x64_sys_ioctl+0x95/0xd0 do_syscall_64+0x39/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 -> #0 (&kvm->slots_lock){+.+.}-{4:4}: ... mutex_lock_nested+0x1b/0x30 kvm_gmem_release+0x56/0x1b0 [kvm] __fput+0x115/0x2e0 ____fput+0xe/0x20 task_work_run+0x5e/0xb0 do_exit+0x2dd/0x5b0 do_group_exit+0x3b/0xb0 __x64_sys_exit_group+0x18/0x20 do_syscall_64+0x39/0x90 entry_SYSCALL_64_after_hwframe+0x6e/0xd8 other info that might help us debug this: Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(mapping.invalidate_lock#4); lock(&kvm->slots_lock); lock(mapping.invalidate_lock#4); lock(&kvm->slots_lock); Signed-off-by: Isaku Yamahata --- virt/kvm/guest_mem.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/virt/kvm/guest_mem.c b/virt/kvm/guest_mem.c index ab91e972e699..772e4631fcd9 100644 --- a/virt/kvm/guest_mem.c +++ b/virt/kvm/guest_mem.c @@ -274,8 +274,6 @@ static int kvm_gmem_release(struct inode *inode, struct file *file) struct kvm *kvm = gmem->kvm; unsigned long index; - filemap_invalidate_lock(inode->i_mapping); - /* * Prevent concurrent attempts to *unbind* a memslot. This is the last * reference to the file and thus no new bindings can be created, but @@ -285,6 +283,8 @@ static int kvm_gmem_release(struct inode *inode, struct file *file) */ mutex_lock(&kvm->slots_lock); + filemap_invalidate_lock(inode->i_mapping); + xa_for_each(&gmem->bindings, index, slot) rcu_assign_pointer(slot->gmem.file, NULL); @@ -299,12 +299,12 @@ static int kvm_gmem_release(struct inode *inode, struct file *file) kvm_gmem_issue_arch_invalidate(gmem->kvm, file_inode(file), 0, -1ul); kvm_gmem_invalidate_end(gmem, 0, -1ul); - mutex_unlock(&kvm->slots_lock); - list_del(&gmem->entry); filemap_invalidate_unlock(inode->i_mapping); + mutex_unlock(&kvm->slots_lock); + xa_destroy(&gmem->bindings); kfree(gmem); -- 2.25.1 -- Isaku Yamahata