From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6A69C77B73 for ; Tue, 30 May 2023 21:44:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75DC3280002; Tue, 30 May 2023 17:44:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 70D51280001; Tue, 30 May 2023 17:44:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D4EF280002; Tue, 30 May 2023 17:44:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 4E238280001 for ; Tue, 30 May 2023 17:44:45 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1B5F51C72A6 for ; Tue, 30 May 2023 21:44:45 +0000 (UTC) X-FDA: 80848251330.17.F279079 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) by imf30.hostedemail.com (Postfix) with ESMTP id 5804B8001A for ; Tue, 30 May 2023 21:44:43 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=YB21CRWL; spf=pass (imf30.hostedemail.com: domain of 3Sm52ZAYKCCsZLHUQJNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--seanjc.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3Sm52ZAYKCCsZLHUQJNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--seanjc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685483083; a=rsa-sha256; cv=none; b=gdC9Pt9ZQRU610lcwTKGfuD/ammrpDP4YMnC/ZdWZG1SsvX7qNQ9ykdF0lICI8XR2mzAln XtD9YpWiV8bAmpZYD14BWyzUcyktBBoQiduuX2DqXhS22P8AWvH779kkfE7oNUsG2NgvM2 d7Qrh2geoWj0zceFMkHnn7j8gal73VI= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=YB21CRWL; spf=pass (imf30.hostedemail.com: domain of 3Sm52ZAYKCCsZLHUQJNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--seanjc.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3Sm52ZAYKCCsZLHUQJNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--seanjc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685483083; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gllGKsW6EGZXrfCcY2RdUJbcNCKzFTs597NLojSvrvI=; b=VXIWZ44FJIcueVneJGw2MEyfY0Ew7LoWtPoa5YbQX68I4P249SZtSenIKQhcWbcELZxnqA bXITminzJcWNGJMhrSVckmGe4fsDcqLwiUsSXRuXqE8c74Wtjsr8HbUrGOJgWoggKQOmFI 9qMY+I4lAcFTww7c3OQbSKXJpPajoA0= Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-64d5cc4e53eso5738672b3a.2 for ; Tue, 30 May 2023 14:44:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1685483082; x=1688075082; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=gllGKsW6EGZXrfCcY2RdUJbcNCKzFTs597NLojSvrvI=; b=YB21CRWLPbjctiVhKFr/qkUBgxJuoKPSp9aosRn38iTlwGDonkS0O/Z/efZVwH0eaI gW8phHmjr+GiJQZ8ewOsEijJkAOs9fcAFR0JoSwPuyfZuY6kI1nH3o+WdiRmkotLzm7E VZj947GiWEycCEkfzX1DoTNI5bwouXLx3YLqamJ3Dkj61BAHyfpA/tOra1Ct40WXkm9G jl3qNPsprl6wnx/eXDRLVpYH+IDiUfLK/D7yK7Ap9j8KODZ9WXVjJyx/h7sUYiXzddIq CdjwHdDutN+q6h1txM5nGCYa6MW6xx8HDPXIMKAyDiiuBMFOJ2Jc66k5rVxLkpQpsO// p5Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685483082; x=1688075082; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gllGKsW6EGZXrfCcY2RdUJbcNCKzFTs597NLojSvrvI=; b=LZmSmpTI1CAXNGqZ+spD5p+0Q/mxaePByOm7g39juuDVwar13/i5kA9f8ZLC/NGoT1 yA87eyQY1YIoo0A9jVnhV4L/D73jI3mQgDRg9IxYAKX/dlS+u8wavJUa0C6vY/4zpSl8 dU76IX40eI3NkavTtXHmLrsoi3pJxamIa6LW3D9owXrmzVvwncEEQIubl5cQEkbQsSeL MAd5LEWlXmy+zo15n1ThH7v8Z8AF4J/hRdQjHR29IGkB9OF2qnacFdERrkYgRrFa1ZTL UntjXl2pMQGFOqEBILqZVpy0NBdJf0KswXI2STzx2RTHNWuDwqf5famMP+HCSTszzZW4 85sg== X-Gm-Message-State: AC+VfDx/rvbfRltJ1IykGDhMsLT/dPllDpR6x5rf09Q3Q5dxw0PQYrdV BADUFXrkvU5a0PZiNwicJEvuw+8VnAQ= X-Google-Smtp-Source: ACHHUZ5d+yPelHfD9nH7zTboFYLRXWmOGbgt1EDhXbPu0aPAY8uJr08Pgxaff9F0pxr9tOTbGa5MVHoeIPM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:1399:b0:647:1eaf:9577 with SMTP id t25-20020a056a00139900b006471eaf9577mr1326642pfg.1.1685483082042; Tue, 30 May 2023 14:44:42 -0700 (PDT) Date: Tue, 30 May 2023 14:44:40 -0700 In-Reply-To: Mime-Version: 1.0 References: <3cece716fc09724793aa832e755abfc9d70a8bb3.1684892404.git-series.apopple@nvidia.com> <5d8e1f752051173d2d1b5c3e14b54eb3506ed3ef.1684892404.git-series.apopple@nvidia.com> <87pm6ii6qi.fsf@nvidia.com> <89dba89c-cb49-f917-31e4-3eafd484f4b2@arm.com> Message-ID: Subject: Re: [PATCH 2/2] arm64: Notify on pte permission upgrades From: Sean Christopherson To: Jason Gunthorpe Cc: Robin Murphy , Alistair Popple , Andrew Morton , will@kernel.org, catalin.marinas@arm.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, nicolinc@nvidia.com, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, John Hubbard , zhi.wang.linux@gmail.com Content-Type: text/plain; charset="us-ascii" X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 5804B8001A X-Stat-Signature: egubh3sjofhjcexec6mgggbr9spxpdsq X-Rspam-User: X-HE-Tag: 1685483083-977254 X-HE-Meta: U2FsdGVkX18uS9PJt+wS/huCZ0tw3D+EJhyLhP/YTjgPFPmy26b/OxdlWf+xnKcjdxGIyhnzxgjE9zKvrjm0hu9J8eMP1jqEaGYM5vdBDHaVEG6QLP3Q7fRzMlJ1L0nO8YGpiQ27JQ3AsIAu2OtJNc74Nswz5gDqY/QBsEutRILbokuO72KwiiwL1gj17OuahT1U/72iTwqGHtnIJj0sTt2n4P6yDHiQ0Kwshtb9r6yvOkz734mOZr4tW6wkDqaaORbzFrHt6Naf3/k3UjfxdwSY55Jq8Elb/6dzhnHdOc7nNt5+An4qEM7xtXXa/cSNpHjbHnZJ65X1DtB0B+2Ll9a5mRytDDWIkefubibKXLsI3SkpkrKYgvAkSUsbFsBombHy266vauPp6SULxfPpkBe6ksUXLDyASHzkMxPeRL78fH2aAblcCVL+gsudDAJpEVFCWyuZuwMb9rz5cZ1S9WNqOlA5Uv4jUTK2siETkeHBVEP7b93dyTmcmy4cTd3fPknelYs+u+h4MMTpUvyOb0ZGJvUS/oGBVzOqygGwNT+Oly+XaNQpd6DlOw0ITo1zjX7HnpfJv0Duoqn0Q7vJEFKnLQSglq4bpU4lMiIigxuYNtyegrQPAtPeRhu6afxlZJycbJHfVj4MzAzLkem6VN5jXghL1mygcuCuxmcJj52K2+3sFmK13EGkMCS26R1CqYrF241vl6JIemRwzcMvEPWPrAugvoPdJteJdLI0/DNZjxb1kvTGZ9MEwzZlvbBCBH79MrobOb9Kept/Hnvx09fTVfUTfihhTBcd3ZHhKTmAtWj2siYnpgi6TyV0cD49gSEDe1q2uBsC26/d8JxrwKO0/cce5MG/U0bOqrxSFZ3fyWUdnk1dTXt6En6XMb5lv70AV1Lrd/AKiHbMQV3KulSh+z7Gi5uRHiGA50Y4Y3pNTaTfI7oxUidJhvo7B46AeZQw152cncspm1zQRBI MJJp2da4 QSh8ns0tRD83UbHoNXtycB9V1mTIRtiSc1lSzrVJkliGPk8t7IEKNX4DhP7jxUhlS15BavAaEOAcLamnaolxQI1CQYO1GJVdLL6w2FGbNK9zqg5jUZgvm8nLSYwlD5kco5zzXXtXYQ0qsMIn0PjeYHy+CHG0cMNSDNuFDe1bU5CPyb4sZKaDGP34BCAPoCxGjlcoaBDnsk8Y7Hz/CmoVHFvvxxNPBZk9Af7UHu6M3tSxXE5bE2AhyzDGMl2+xiN8Eh0wjOcCUUPT7of5d2dl7BqxjrqzWKgYfWrFlgLEJglI4NgrxRyHKjggAjZLKsTdLgq9eix2LhsWWEOLBg+x0Cg0Trm2NrYBXKDvToD6+2F2Bq4vjbpRbXyEmoQiRk5Tko548vRcMkLN2ETda2etq6MEowF58ieD857Lbb4AcpLHrsSCo0JzAIEfuj+bNP1WkZCVGzsS904vPySQA/EeY2y+j94PZO56N2afbgQeVQvdxCF+xF38N61hNKcYrl8B61DX83a2REBHlhO1xh0IuK7OQ5zNx3Bkbf6K/Yci0v5pqIsICfsnebTkoh4zkVuxUS9mMHJzaLr/VY/z57oyanERrUuzWntnH/a7zT4IYbtI6dxBm2RFv3zzEuTU0f7k2r8/Frg3LWUGfrx8taymIGL1z1gh+16QqNjKZxLwiSzne7hw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 30, 2023, Jason Gunthorpe wrote: > IMHO I think we messed this up at some point.. > > Joerg added invalidate_range just for the iommu to use, so having it > be arch specific could make some sense. > > However, KVM later co-opted it to do this: > > commit e649b3f0188f8fd34dd0dde8d43fd3312b902fb2 > Author: Eiichi Tsukata > Date: Sat Jun 6 13:26:27 2020 +0900 > > KVM: x86: Fix APIC page invalidation race ... > The fix therefore is to reload the APIC-access page field in the VMCS > from kvm_mmu_notifier_invalidate_range() instead of ..._range_start(). > > Which I think is a hacky fix. Agreed, though as you note below, the invalidate_range() description does make it seem like a somewhat reasonable thing to do. > KVM already has locking for invalidate_start/end - it has to check > mmu_notifier_retry_cache() with the sequence numbers/etc around when > it does does hva_to_pfn() > > The bug is that the kvm_vcpu_reload_apic_access_page() path is > ignoring this locking so it ignores in-progress range > invalidations. It should spin until the invalidation clears like other > places in KVM. > > The comment is kind of misleading because drivers shouldn't be abusing > the iommu centric invalidate_range() thing to fix missing locking in > start/end users. :\ > > So if KVM could be fixed up we could make invalidate_range defined to > be an arch specific callback to synchronize the iommu TLB. And maybe rename invalidate_range() and/or invalidate_range_{start,end}() to make it super obvious that they are intended for two different purposes? E.g. instead of invalidate_range(), something like invalidate_secondary_tlbs(). FWIW, PPC's OpenCAPI support (drivers/misc/ocxl/link.c) also uses invalidate_range(). Though IIUC, the use case is the same as a "traditional" IOMMU, where a device can share the CPU's page tables, so maybe the devices can be considered IOMMUs in practice, if not in name? It allows an accelerator (which could be an FPGA, ASICs, ...) to access the host memory coherently, using virtual addresses. An OpenCAPI device can also host its own memory, that can be accessed from the host. > Sean? It's doable, though definitely not 6.4 material. I have patches coded up. Assuming testing goes well, I'll post them regardless of the OCXL side of things. I've disliked KVM's one-off use of invalidate_range() for a long time, this is a good excuse to get rid of it before KVM gains more usage.