From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1CFFC54ED1 for ; Tue, 27 May 2025 18:03:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D89C6B00A4; Tue, 27 May 2025 14:03:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 88B706B00A6; Tue, 27 May 2025 14:03:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7034D6B00A7; Tue, 27 May 2025 14:03:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 4C55A6B00A4 for ; Tue, 27 May 2025 14:03:17 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 08B73BEE4C for ; Tue, 27 May 2025 18:03:17 +0000 (UTC) X-FDA: 83489459634.29.FD4C2E4 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf02.hostedemail.com (Postfix) with ESMTP id 1D1708000D for ; Tue, 27 May 2025 18:03:14 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=dA3RQdzm; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf02.hostedemail.com: domain of 3Yf41aAUKCH4vcddciqqing.eqonkpwz-oomxcem.qti@flex--tabba.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3Yf41aAUKCH4vcddciqqing.eqonkpwz-oomxcem.qti@flex--tabba.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748368995; a=rsa-sha256; cv=none; b=QlM5i8WFcL9fsaReFKV4fdWTEqxhHZ5BoFX3RO+S88uGx26hGxmsgQwCJKBAWUKGODvjEm 8vn668RdfYyuJorDzd4AO8YLmLmzg9o/RyzApgWHNZJUnIC6ogeWpWi7SogCEaZLDTdt6F 4HhfarHZXCBGrtC5F49Y2hA59u/2Fic= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=dA3RQdzm; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf02.hostedemail.com: domain of 3Yf41aAUKCH4vcddciqqing.eqonkpwz-oomxcem.qti@flex--tabba.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3Yf41aAUKCH4vcddciqqing.eqonkpwz-oomxcem.qti@flex--tabba.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748368995; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ur7OOliet+Wmg5x1tJxiZ+Y9kodvHeTOGK3Qgb8DW2U=; b=M/S1mFGtv0cZfZ4PJ9PXpA31TDNIljuJAd1o45Q8HsxWL5Oe8312zFKh8xKGuw4Vi+V0jj 1QTpnpRMomkkHpnC1N10iCix5DuvtKlgTLuxmLJMUPpLedHKDiKtNnjQEfJ9qwtYZliQWj +I/+gol06xYb6Z8ilmVw9sBlJljx/JA= Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-43ce8f82e66so18815055e9.3 for ; Tue, 27 May 2025 11:03:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1748368994; x=1748973794; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Ur7OOliet+Wmg5x1tJxiZ+Y9kodvHeTOGK3Qgb8DW2U=; b=dA3RQdzmwpBLpFZ2bQnh7NPOarY/DkUu75fM6M1RpEpmEYn6H4id2y1Ks+m2BkvdMb 7WVuW33eTUQKvCbr98iIjLyVg2mT2yi90Zp62FIYInPobq13L8sL/tgMhVR9+FwMRXft ltsA4H3E2gAZXQhLqqogBX2yvpB+C8VDJHDhd+/8NVdF2HnHW0NfeCXxGGOU0xJaeniu e0dgpQ2ovE+EcIbV6iOPAG/agUsZn72ZKyjEvDXOhi82CBavkkFjkTCUpSovmNocnVQ0 ZxQgl+v6V3btbT3wWGddrN1YyP0vTF0//FlZArBNPnjSfkJfKOZjGvZ2rFi0M+flF7rx uV8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748368994; x=1748973794; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Ur7OOliet+Wmg5x1tJxiZ+Y9kodvHeTOGK3Qgb8DW2U=; b=UUtEmAQwUQVc0WXs7sxA8UsQfJc13MQi0CV/pYqPcHw/Bo5DrgblWC7QqnFzQEmnmI gJyObGZAyBFKAL8G8KfVJfe++56T/mqXpday0RCG50ccL78GWcBDNmYRhb+MffpI3WXu i8n/7lLmTk0qinEOs+glLNy0/axlj3hSyAGsmct+5N6YeLX80DSVFYX3Tv72NqVxXdsf 3pDCXGniYnda2WiGQqC4oJbdmOQ+J0cVwGbLvJ4SsG+RiWYLHl7Qp5WWi53QPaPNzp0Z bSO9nlhiSpBKyDkha9F7iLs7vps2QcCHW5qhYlYKR+hQwz4QU19oZ5OELEUh4tzhtuCn 60Rg== X-Forwarded-Encrypted: i=1; AJvYcCWPEOVTHko9w6RN1iLvnjIElMrP8ZTpUwgTcwxnBEuVNznlh1bWxgCfIE+448MFo/YH9a/JHwPRLg==@kvack.org X-Gm-Message-State: AOJu0YxpOCXz4R73HZ3m4YBP5CSZaFcqA3rsHOOTd3rMwXaGxh+MUIEB E8XSi3MJh8ZaKfnzUG2ivOW57dFHQyXM9o49QFqj8QT8IhTGra+cBKytt/1OBEUrZazr6xTzChv IoA== X-Google-Smtp-Source: AGHT+IHCf0bDJ76hTT4kSC4YiVQWOuaVECuOWgSL6WNt6PYLNGGO9OXNopWm7SAaIEhfPW/80Tr8Ykb3uA== X-Received: from wmbed10.prod.google.com ([2002:a05:600c:614a:b0:442:ccfa:102]) (user=tabba job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:512a:b0:44b:2f53:351c with SMTP id 5b1f17b1804b1-44c91dcb6e7mr133582365e9.18.1748368993658; Tue, 27 May 2025 11:03:13 -0700 (PDT) Date: Tue, 27 May 2025 19:02:42 +0100 In-Reply-To: <20250527180245.1413463-1-tabba@google.com> Mime-Version: 1.0 References: <20250527180245.1413463-1-tabba@google.com> X-Mailer: git-send-email 2.49.0.1164.gab81da1b16-goog Message-ID: <20250527180245.1413463-14-tabba@google.com> Subject: [PATCH v10 13/16] KVM: arm64: Handle guest_memfd-backed guest page faults From: Fuad Tabba To: kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-mm@kvack.org Cc: pbonzini@redhat.com, chenhuacai@kernel.org, mpe@ellerman.id.au, anup@brainfault.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, seanjc@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, willy@infradead.org, akpm@linux-foundation.org, xiaoyao.li@intel.com, yilun.xu@intel.com, chao.p.peng@linux.intel.com, jarkko@kernel.org, amoorthy@google.com, dmatlack@google.com, isaku.yamahata@intel.com, mic@digikod.net, vbabka@suse.cz, vannapurve@google.com, ackerleytng@google.com, mail@maciej.szmigiero.name, david@redhat.com, michael.roth@amd.com, wei.w.wang@intel.com, liam.merwick@oracle.com, isaku.yamahata@gmail.com, kirill.shutemov@linux.intel.com, suzuki.poulose@arm.com, steven.price@arm.com, quic_eberman@quicinc.com, quic_mnalajal@quicinc.com, quic_tsoni@quicinc.com, quic_svaddagi@quicinc.com, quic_cvanscha@quicinc.com, quic_pderrin@quicinc.com, quic_pheragu@quicinc.com, catalin.marinas@arm.com, james.morse@arm.com, yuzenghui@huawei.com, oliver.upton@linux.dev, maz@kernel.org, will@kernel.org, qperret@google.com, keirf@google.com, roypat@amazon.co.uk, shuah@kernel.org, hch@infradead.org, jgg@nvidia.com, rientjes@google.com, jhubbard@nvidia.com, fvdl@google.com, hughd@google.com, jthoughton@google.com, peterx@redhat.com, pankaj.gupta@amd.com, ira.weiny@intel.com, tabba@google.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 1D1708000D X-Stat-Signature: wc1ooax4h7mk9bmms19jeue8kof7bxam X-Rspam-User: X-HE-Tag: 1748368994-762301 X-HE-Meta: U2FsdGVkX18iEI1Sor7sIcjGeOD9veEPg8RdcQr6CG+5W9oADAcWMa7fL0MBWa4cNMyyFegSzMcpbvQ29QZattZxSxHAZ29II+FlgR8SqWhnD11sNHWWH1MBIIiUmBdRPkYqlwqgKTsZp8vVV32MrB6K9CQtDw/zUMzy3iHK6ubGNLr5dx0pdoHPklll4kWn9PVBebXh1I0G1q2fjfjUY/XgMjII6A+NjDUQByNNewo8T5iUofccgJ5rYiXLiZ/q9nC3ozWfAzVI9QLO4WiUVsmJSlFkkotxhVId4jiR9tvROCav/8d846TPHc6dalLHMhLOIRNukFi4V66+KsrFUIowsQ56f9WxyXpqOiOD8CqDA3yV+mTKq4wmHlq1H0eIj1/ytuJHfkiYHuSQnl9v4umAAHg+Qm+3pbqLf6NIJE9s9Krd/EfLhtHql/MExFHdDKjA1Mdka8oDqO+u+cQfxv6L7lEbfPUgw9qZsEHYVyb+Bm9xkWK38lJ5R4QAHQeSvwxWyoii1cs3NMapnnBvuBtcXnjPFo32IOsxDs5MLKYiH3H7Qd6mBYaTNMwA3wLMXstNF1sjXhmQ84BO93EyPt4ofrTMS6PrabTjxA1PellSCiN45yabk90igYuby5flqzLn+yMB+iEqp2YARBkOv3GTmQYpf9foV15xmURtqZRsLkZGD5Y5y7LCdk4JUyi9rsg0gx5NblkJQnwLCsICQUGRsdqRC65UqCBUSa3rzeU6h9ZNckACcSM4qAOHIXG39/XAirqxgwuA2+rYW5lcD+AATQF/b1dTkoPdXnTaU4Ofb4LMQkvNulS6rOiVFml5sOh3ypFuEIPxlPXxNXbikRSnSGv4CKp5f4Ia1/ScNd1cSCCnKckm11s0MFA2HtbyQz+vwOrlUESKWW/yzfwigY3mwqNk+dGrzqNQGkFn1Hm9H64ZfKO4NSFKoigK9PS/J3QL0+GE4WTPHFg3zUA 1gcjvlNC ofwEThtM+MHMfllWEu9P868shG7Fsa1/IOmtq9OdjRY1D1ShlR3d7RaVzgFAq8gm1wrnw6d9ndPcNFLbKbra9sun4Utz4Cg2VfW64cT/X3aIOQwDuddGPFcfxhLZ1Htd8cNtXAa2beYTNP5HMp2LhoehWHtiV397LUyXuDVcuxHhIlUHsPrVXlPmp6cQEY3oFoUH4FXdkeDuaZErHdsYV9cbUnhuWmWA9SIVt6evJv7f7wJs2j+V+8ozaZEQNfwgJVPoC5Wgmxpvv5R7+0f/YuhQWlDj5XfoQ8EFfTR9uDzkQ40Wg2q3NqnubHsDPnyR2C9TDGgeGXs+5/gX9ksLPLfLGV9UC9X2AcI8pBt8LIhJ7TSgiT9P2tznz47jYBuAdtAbv/LVli7RTrYAa3w2bA9IGRgx02Ibh0VNaSg0ZXK+B4jvgLcmq3W8tBqIwyVetGFFt5u8Yu4iKqFb8ziJQewf77Fjh/7fLIteYTwXqOUGiw32h3SO3VPBUXs0hWjTnUar0847RBpDCSBnY3bbOH/j3LPsAiBe0yVTkHv7qnPKIKF+8ZpvnR1kucz/uTnKsgl27lVAr8G27AecwX/Z9WuW1yu3jJfe9WWE6Qav6MEl92Ug/Cp4Hs+SuzvS4OnLr0DYTpEWvTIkRglu+6ju7cgJ9SA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add arm64 support for handling guest page faults on guest_memfd backed memslots. Until guest_memfd supports huge pages, the fault granule is restricted to PAGE_SIZE. Signed-off-by: Fuad Tabba --- Note: This patch introduces a new function, gmem_abort() rather than previous attempts at trying to expand user_mem_abort(). This is because there are many differences in how faults are handled when backed by guest_memfd vs regular memslots with anonymous memory, e.g., lack of VMA, and for now, lack of huge page support for guest_memfd. The function user_mem_abort() is already big and unwieldly, adding more complexity to it made things more difficult to understand. Once larger page size support is added to guest_memfd, we could factor out the common code between these two functions. --- arch/arm64/kvm/mmu.c | 89 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 87 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 9865ada04a81..896c56683d88 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1466,6 +1466,87 @@ static bool kvm_vma_mte_allowed(struct vm_area_struct *vma) return vma->vm_flags & VM_MTE_ALLOWED; } +static int gmem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, + struct kvm_memory_slot *memslot, bool is_perm) +{ + enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_HANDLE_FAULT | KVM_PGTABLE_WALK_SHARED; + enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; + bool logging, write_fault, exec_fault, writable; + struct kvm_pgtable *pgt; + struct page *page; + struct kvm *kvm; + void *memcache; + kvm_pfn_t pfn; + gfn_t gfn; + int ret; + + if (!is_perm) { + int min_pages = kvm_mmu_cache_min_pages(vcpu->arch.hw_mmu); + + if (!is_protected_kvm_enabled()) { + memcache = &vcpu->arch.mmu_page_cache; + ret = kvm_mmu_topup_memory_cache(memcache, min_pages); + } else { + memcache = &vcpu->arch.pkvm_memcache; + ret = topup_hyp_memcache(memcache, min_pages); + } + if (ret) + return ret; + } + + kvm = vcpu->kvm; + gfn = fault_ipa >> PAGE_SHIFT; + + logging = memslot_is_logging(memslot); + write_fault = kvm_is_write_fault(vcpu); + exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu); + VM_BUG_ON(write_fault && exec_fault); + + if (is_perm && !write_fault && !exec_fault) { + kvm_err("Unexpected L2 read permission error\n"); + return -EFAULT; + } + + ret = kvm_gmem_get_pfn(vcpu->kvm, memslot, gfn, &pfn, &page, NULL); + if (ret) { + kvm_prepare_memory_fault_exit(vcpu, fault_ipa, PAGE_SIZE, + write_fault, exec_fault, false); + return ret; + } + + writable = !(memslot->flags & KVM_MEM_READONLY) && + (!logging || write_fault); + + if (writable) + prot |= KVM_PGTABLE_PROT_W; + + if (exec_fault || cpus_have_final_cap(ARM64_HAS_CACHE_DIC)) + prot |= KVM_PGTABLE_PROT_X; + + pgt = vcpu->arch.hw_mmu->pgt; + + kvm_fault_lock(kvm); + if (is_perm) { + /* + * Drop the SW bits in favour of those stored in the + * PTE, which will be preserved. + */ + prot &= ~KVM_NV_GUEST_MAP_SZ; + ret = KVM_PGT_FN(kvm_pgtable_stage2_relax_perms)(pgt, fault_ipa, prot, flags); + } else { + ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, fault_ipa, PAGE_SIZE, + __pfn_to_phys(pfn), prot, + memcache, flags); + } + kvm_release_faultin_page(kvm, page, !!ret, writable); + kvm_fault_unlock(kvm); + + if (writable && !ret) + mark_page_dirty_in_slot(kvm, memslot, gfn); + + return ret != -EAGAIN ? ret : 0; +} + static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, struct kvm_s2_trans *nested, struct kvm_memory_slot *memslot, unsigned long hva, @@ -1944,8 +2025,12 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) goto out_unlock; } - ret = user_mem_abort(vcpu, fault_ipa, nested, memslot, hva, - esr_fsc_is_permission_fault(esr)); + if (kvm_slot_has_gmem(memslot)) + ret = gmem_abort(vcpu, fault_ipa, memslot, + esr_fsc_is_permission_fault(esr)); + else + ret = user_mem_abort(vcpu, fault_ipa, nested, memslot, hva, + esr_fsc_is_permission_fault(esr)); if (ret == 0) ret = 1; out: -- 2.49.0.1164.gab81da1b16-goog