From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CAB0C5B543 for ; Thu, 5 Jun 2025 15:38:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7EAFC6B00AF; Thu, 5 Jun 2025 11:38:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 79ABB6B00B0; Thu, 5 Jun 2025 11:38:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68A6A6B00B1; Thu, 5 Jun 2025 11:38:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 4689C6B00AF for ; Thu, 5 Jun 2025 11:38:35 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 03374140DFD for ; Thu, 5 Jun 2025 15:38:34 +0000 (UTC) X-FDA: 83521754190.23.4869F0A Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf05.hostedemail.com (Postfix) with ESMTP id 28E8D100003 for ; Thu, 5 Jun 2025 15:38:32 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=djWKuS30; spf=pass (imf05.hostedemail.com: domain of 397lBaAUKCLoyfggflttlqj.htrqnsz2-rrp0fhp.twl@flex--tabba.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=397lBaAUKCLoyfggflttlqj.htrqnsz2-rrp0fhp.twl@flex--tabba.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1749137913; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NPZBE5PTGW9kWBzPk02s4iKZXmTT2aIVBFlSN4SanJw=; b=OdkP64nRI6D4dozMmZLx4HwbKvDIXIurRxjMdNFjF0Eaf0UpKK6hKkAPp0wQM6qu63AGyF a3owq4K62dWQhyIT715/TMNE7UQM19wz6LyUM6FKbDzwZgAzdt7kPq4yKXOR0oU6FNtOiV Pz2b9mAr5GZczQA5ocPZYovNTQgKNxc= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=djWKuS30; spf=pass (imf05.hostedemail.com: domain of 397lBaAUKCLoyfggflttlqj.htrqnsz2-rrp0fhp.twl@flex--tabba.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=397lBaAUKCLoyfggflttlqj.htrqnsz2-rrp0fhp.twl@flex--tabba.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1749137913; a=rsa-sha256; cv=none; b=ioIeVTwB9p7FEiqz+9D1hw73UBEFb3FUjNYdRNbct3ymQqLzuy9M4Jj+ebhG8slZww3xAJ QSuAGfknKRHdCoxizPzYWdJAV6e3dQ6px4K3V4HEw3JAcgqTMK9KM6XHnjJfzjhcUdHYmG T1ojPyES4Ln6rNIpgV616woWwREMHVQ= Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43eed325461so6794805e9.3 for ; Thu, 05 Jun 2025 08:38:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1749137912; x=1749742712; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=NPZBE5PTGW9kWBzPk02s4iKZXmTT2aIVBFlSN4SanJw=; b=djWKuS30lsRF1DFg89mdn85RvGlBtQrolRarjBfeTtD9eVAbJ5OrEcnsMXJSJKBXV0 FQGAJYmT2U7455ZqXrpLjCMkbFq7kkum3N32YgqB2X4j/KJ/lYPBUo9/baaQUgtbtqk/ +zEnDwNYDnALWeFXjCdS6cIcGj3m7vx2z/LjaaBc2aNZV70JcjSETZDlMJ66VJLncDQu TItMwgnV1Y4GzCQMNxCCETmKLqy7wJpQlH+bLbgGuAWINGtW0mdM6X/ZdnmkIfs61sNa bwHkJmW+LYEvdTRJ7pr+LFSJf2pzkatjpgSnHkErhMf3dOGYYReWr6zXFtq2KFTkdeA+ tpwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749137912; x=1749742712; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NPZBE5PTGW9kWBzPk02s4iKZXmTT2aIVBFlSN4SanJw=; b=N1oeDz1OwayAdw3tb2GQtH63M4AoRVEfrR9n+cLJzZZ6zhDQwFDWyyn06SwmERFCjX mLEx596N4DQlUaQua0vLcWPBqh+6Ug647lOZyVx9ts28RbNELamu0p9ipl+5w+YBDakH Dm+jxcS5mcZlkMRHbFhfskG3AkS5BjdY8ZTuGFgopEe1vTpdvDpviFPuJ5Mqs6xSQLX2 WFyOuRBG6ppxTsSseWygofGT7Mbxjd4Ue0+FJHLLe2XjJZL3j1gECs2IKcvqgaxYw1di RK0rh5Xyk/+v8GXPxA6/JfuQk8WI2WJy+C/CCi5qwx6S6yLpO3bPEwttjwBuGggqpuQD DtqQ== X-Forwarded-Encrypted: i=1; AJvYcCWyNAfVr4/HWFV+rkTImfjQ6RZQSsyyuGj45Kfdl4D9QxZkojZlah5Orxu2Fb1mToWlJlbUFANrPg==@kvack.org X-Gm-Message-State: AOJu0YycxSK5Q7HJ7cU2E/Z9H2FpB2INuYFlZjgpbhnMa+HPHVIbAFox SS+WpgxZzXimPygR67R+sxFUHMsRUrG4K/gWHUkeVpOxub+ygm3tMGgiIutzdIyJF4ZrMq1cRFj rqA== X-Google-Smtp-Source: AGHT+IGMJty4mR+w6Q7W1YmFTUSwU16vv8WAlkYUzsCh1k98WN49+HkHCW9IHOZiu4FjckwQZIFoRzzqew== X-Received: from wmbdt15.prod.google.com ([2002:a05:600c:630f:b0:440:595d:fba9]) (user=tabba job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3595:b0:43d:7588:667b with SMTP id 5b1f17b1804b1-451f0a88e69mr92705785e9.10.1749137911624; Thu, 05 Jun 2025 08:38:31 -0700 (PDT) Date: Thu, 5 Jun 2025 16:37:56 +0100 In-Reply-To: <20250605153800.557144-1-tabba@google.com> Mime-Version: 1.0 References: <20250605153800.557144-1-tabba@google.com> X-Mailer: git-send-email 2.49.0.1266.g31b7d2e469-goog Message-ID: <20250605153800.557144-15-tabba@google.com> Subject: [PATCH v11 14/18] KVM: arm64: Handle guest_memfd-backed guest page faults From: Fuad Tabba To: kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-mm@kvack.org, kvmarm@lists.linux.dev Cc: pbonzini@redhat.com, chenhuacai@kernel.org, mpe@ellerman.id.au, anup@brainfault.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, seanjc@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, willy@infradead.org, akpm@linux-foundation.org, xiaoyao.li@intel.com, yilun.xu@intel.com, chao.p.peng@linux.intel.com, jarkko@kernel.org, amoorthy@google.com, dmatlack@google.com, isaku.yamahata@intel.com, mic@digikod.net, vbabka@suse.cz, vannapurve@google.com, ackerleytng@google.com, mail@maciej.szmigiero.name, david@redhat.com, michael.roth@amd.com, wei.w.wang@intel.com, liam.merwick@oracle.com, isaku.yamahata@gmail.com, kirill.shutemov@linux.intel.com, suzuki.poulose@arm.com, steven.price@arm.com, quic_eberman@quicinc.com, quic_mnalajal@quicinc.com, quic_tsoni@quicinc.com, quic_svaddagi@quicinc.com, quic_cvanscha@quicinc.com, quic_pderrin@quicinc.com, quic_pheragu@quicinc.com, catalin.marinas@arm.com, james.morse@arm.com, yuzenghui@huawei.com, oliver.upton@linux.dev, maz@kernel.org, will@kernel.org, qperret@google.com, keirf@google.com, roypat@amazon.co.uk, shuah@kernel.org, hch@infradead.org, jgg@nvidia.com, rientjes@google.com, jhubbard@nvidia.com, fvdl@google.com, hughd@google.com, jthoughton@google.com, peterx@redhat.com, pankaj.gupta@amd.com, ira.weiny@intel.com, tabba@google.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 28E8D100003 X-Stat-Signature: pgm4ibe5oi7ex3tuut3h8wekegw6f5cp X-Rspam-User: X-HE-Tag: 1749137912-238948 X-HE-Meta: U2FsdGVkX1/E3Lm7sN+qcCD1Lj0zrFPRU8NYYsBIAcPlG6q9ijluJdiIv1+ngSd1e9e1F7zI3OnFHeBOHsLac+Tbd7wSKz+KyKImwzx0CLgJN0UDBq+bwXpdJBXb+/uUF6N1gwSVj7uUbsz+4yL7jj3uK4ERUKEq7K+v58pecAKYJTH01aCZaF8IDtj3krXYdXicQvUeIhYjZ8BZ+JP1/hdweZXKaVfnluDlFebTTM1iSKfdS3gGwhUfyFffa6AhjVXblqC6zrmons/3mJPqZyx+R4IhZsqysw+IPOwBCteWLpSjHntGfvisM9/qrE/y/t+LYtL5qNjamoc5dbMQqBtG8zHMJj5FNTsFucG0Ht5js2u30/E5HaPz/BHrT3hv/9zXYH1H2Q9TSHANLfW2yHy8CXi8ck8SeEOzne3T5f9K7IoO00edmZjPwcB1eF05McTc8JTQ+Yim4zPcOKzIN/JnrpS+zMX/aj3mPmpQnERYySACjsDum7vIhSCrbD0QCpt+Uu39atWftWuWKnP0xA45G48gghVW7oofEqpTDuBxzZ820zZngosjOMfmyoIacOk6VORaZQb4+RE75kQBmYAgYp5Vs9nGtRkB4YEB/KSMIYAari4wfDR9Eseu/pGbr1SAIcr+bp3JFEY4xKrxARVjl0i/Z/FJ3jb97Bs880r95jkoG0huyf6L59O3wv65czuUqiNQD5+qazO4ix6moajerqnI71RDGc9PiqXrbNDgM+ew8IauCz0pgwipbZ6kuzU93O2wMNo7XAHrlYFme18B00ZOu573I/t3viumMsW/kns0LEW+D+K8RoAyAZ2eCKpGutPPnCM/Rmhqpu7NZ/UKLcCzlI0K4TzU6zB7ZbjRqxDEZAEq6UGAMojqACDLX8Pi3yH9HLvE26t9YNXqDHQt3hobiWUyqkGLerZ3Poepp3w36IAsd1/y2v3u2+CQZJC63bfEmAA5HEd9cjK OodMXm8A HUwdZBqY7sn2YFTWK+aCoWtoBf0u3B9vz2xQXMCvMUqrHj6nSmDAacLom9DRcKaFwW36ofcft2yNbKO3J1vMheO3HxPIiUW/ZIXEbAiB5Y14LWjzI87nHfyLOgr6k5ZOGWD7F4Bo+X8LfuOe6Jt0M43WmEq9/D5dLkrbgw72RU4zqZevh077u02+rCu5y48DxjBW6AoJSc/q6QKRNdeBpvLedHCSxCfpNIqhv2qJelsNHDjPwpCzYzrMSQ73WtishkiI66+kpLYDIzp/8DJlY3D9U1saFZ3vy/bXTxTvAWwsbGXD0eKFbNeSsOfmGZwv4YUjGSffSIfFX4qvr8mWHb4leUzi/IZZgxQqjdDe/3JLbEwUqhN1Nyil+7Qsy56kBwUtjq1+5KjQyoM6vCA0ty8B6KssSr0ZKIg6QL0L5fs4fqO7s1YbUGVhZzMm1H7iZTf5pIZrweVkeTxgW+12IuVWxP7DQpDJzMhf9PzWCl4gUP5sAQJWb4D8u4qnu9VVZopkUgRrnvUlFssQC/kbpnKH6pqI1kMQ6nam7UjKXtIlhgjLz/c9SOl2ybzvMjWtPNGixO8xzrE7EVurDpZ1z3VPg3ip7uI+jV+pFyjFhiX0Skb+h/eHGPh/3aAUtUuPCuYSfVQ8IWt2OimxKrQP1uwSp8A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add arm64 support for handling guest page faults on guest_memfd backed memslots. Until guest_memfd supports huge pages, the fault granule is restricted to PAGE_SIZE. Signed-off-by: Fuad Tabba --- arch/arm64/kvm/mmu.c | 93 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 90 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index ce80be116a30..f14925fe6144 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1508,6 +1508,89 @@ static void adjust_nested_fault_perms(struct kvm_s2_trans *nested, *prot |= kvm_encode_nested_level(nested); } +#define KVM_PGTABLE_WALK_MEMABORT_FLAGS (KVM_PGTABLE_WALK_HANDLE_FAULT | KVM_PGTABLE_WALK_SHARED) + +static int gmem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, + struct kvm_s2_trans *nested, + struct kvm_memory_slot *memslot, bool is_perm) +{ + bool logging, write_fault, exec_fault, writable; + enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_MEMABORT_FLAGS; + enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; + struct kvm_pgtable *pgt = vcpu->arch.hw_mmu->pgt; + struct page *page; + struct kvm *kvm = vcpu->kvm; + void *memcache; + kvm_pfn_t pfn; + gfn_t gfn; + int ret; + + ret = prepare_mmu_memcache(vcpu, !is_perm, &memcache); + if (ret) + return ret; + + if (nested) + gfn = kvm_s2_trans_output(nested) >> PAGE_SHIFT; + else + gfn = fault_ipa >> PAGE_SHIFT; + + logging = memslot_is_logging(memslot); + write_fault = kvm_is_write_fault(vcpu); + exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu); + + if (write_fault && exec_fault) { + kvm_err("Simultaneous write and execution fault\n"); + return -EFAULT; + } + + if (is_perm && !write_fault && !exec_fault) { + kvm_err("Unexpected L2 read permission error\n"); + return -EFAULT; + } + + ret = kvm_gmem_get_pfn(kvm, memslot, gfn, &pfn, &page, NULL); + if (ret) { + kvm_prepare_memory_fault_exit(vcpu, fault_ipa, PAGE_SIZE, + write_fault, exec_fault, false); + return ret; + } + + writable = !(memslot->flags & KVM_MEM_READONLY) && + (!logging || write_fault); + + if (nested) + adjust_nested_fault_perms(nested, &prot, &writable); + + if (writable) + prot |= KVM_PGTABLE_PROT_W; + + if (exec_fault || + (cpus_have_final_cap(ARM64_HAS_CACHE_DIC) && + (!nested || kvm_s2_trans_executable(nested)))) + prot |= KVM_PGTABLE_PROT_X; + + kvm_fault_lock(kvm); + if (is_perm) { + /* + * Drop the SW bits in favour of those stored in the + * PTE, which will be preserved. + */ + prot &= ~KVM_NV_GUEST_MAP_SZ; + ret = KVM_PGT_FN(kvm_pgtable_stage2_relax_perms)(pgt, fault_ipa, prot, flags); + } else { + ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, fault_ipa, PAGE_SIZE, + __pfn_to_phys(pfn), prot, + memcache, flags); + } + kvm_release_faultin_page(kvm, page, !!ret, writable); + kvm_fault_unlock(kvm); + + if (writable && !ret) + mark_page_dirty_in_slot(kvm, memslot, gfn); + + return ret != -EAGAIN ? ret : 0; +} + static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, struct kvm_s2_trans *nested, struct kvm_memory_slot *memslot, unsigned long hva, @@ -1532,7 +1615,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; struct kvm_pgtable *pgt; struct page *page; - enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_HANDLE_FAULT | KVM_PGTABLE_WALK_SHARED; + enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_MEMABORT_FLAGS; if (fault_is_perm) fault_granule = kvm_vcpu_trap_get_perm_fault_granule(vcpu); @@ -1959,8 +2042,12 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) goto out_unlock; } - ret = user_mem_abort(vcpu, fault_ipa, nested, memslot, hva, - esr_fsc_is_permission_fault(esr)); + if (kvm_slot_has_gmem(memslot)) + ret = gmem_abort(vcpu, fault_ipa, nested, memslot, + esr_fsc_is_permission_fault(esr)); + else + ret = user_mem_abort(vcpu, fault_ipa, nested, memslot, hva, + esr_fsc_is_permission_fault(esr)); if (ret == 0) ret = 1; out: -- 2.49.0.1266.g31b7d2e469-goog