From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 124C8C3DA6E for ; Wed, 20 Dec 2023 14:12:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 695926B0093; Wed, 20 Dec 2023 09:12:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 645EA6B0095; Wed, 20 Dec 2023 09:12:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E62C6B0096; Wed, 20 Dec 2023 09:12:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 392B46B0093 for ; Wed, 20 Dec 2023 09:12:19 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0DA2880B9F for ; Wed, 20 Dec 2023 14:12:19 +0000 (UTC) X-FDA: 81587386398.27.3CB0CDF Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf20.hostedemail.com (Postfix) with ESMTP id 3C9861C002E for ; Wed, 20 Dec 2023 14:12:17 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=kfFn5Dvb; spf=pass (imf20.hostedemail.com: domain of maz@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=maz@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703081537; a=rsa-sha256; cv=none; b=csff11sqTKaSq5NoOBowQLoBIMva11ZgNqoX4XCyddr6yQD6bbyHTHkB0YICAlQfIBP8F/ /vOiAXLY9jFXrsC0iRsexWPLsIJq8yYEkZWl79KlI/7+LVYbaWP+4V2SGXEPhUZiBifOTu XLaPadX92LaPvuxh0T9rOZdsoIFzvV4= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=kfFn5Dvb; spf=pass (imf20.hostedemail.com: domain of maz@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=maz@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703081537; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gxJ1bFhL9KCQeSQVumKw94jpnFncaXwGbKMNhScnvHY=; b=ZLBCc85945gQevMjso5zWWVm4EPTabhp4UW/6fssjqU7nZDVvbAljdljwJT18r8lLdxcOb gTrmXhqOicPCYiGmyrFK5mN+hx9Vb1eayGo7E50p3EGQdLIML/ukAWBMrvhzQvUEetNf0a hgLKxRjS7aiB9PXlIZfBA02pZ03RUes= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 2BD8561652; Wed, 20 Dec 2023 14:12:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BEB7DC433C7; Wed, 20 Dec 2023 14:12:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1703081535; bh=omuGEKL0AnBbS+fvO5aEEjVVdipBqKAYv93nwOJV9Ss=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=kfFn5Dvbbrr6iPRuNItHtdv6Wg3PZAyXrr95xvGleIi39+oOv90e36Fw7E1LPRtrk YIPmh5cnnozM0+VqfHkzu0sxLR1qmEAGMcRahaLSd3fvG/HpUsohvAYXNUHHihIPuq wpz3UEBa/45OpzSRAdrVOgLrwVJAELez9l+J+jyKcY9k6nNPMWYp6Ysoegw8M0X/RP ig2hn1EO9RCfzUcoq0/LlX5UBTbAkRD5mD/6Tm5taU7mVRmI4KkzNm7SCpi1tWIIsI KuYSCRF9jMYHZq7o+z3SQPS3dNYSXGJzzIromkeLLnA+vUAZXDmH4Gg1FChTGR7hyg wUXs6tFD7treQ== Received: from [104.132.45.104] (helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1rFxIz-005haC-Go; Wed, 20 Dec 2023 14:12:13 +0000 Date: Wed, 20 Dec 2023 14:12:12 +0000 Message-ID: <87wmt9t137.wl-maz@kernel.org> From: Marc Zyngier To: Cc: , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: Re: [PATCH v4 1/3] kvm: arm64: introduce new flag for non-cacheable IO memory In-Reply-To: <20231218090719.22250-2-ankita@nvidia.com> References: <20231218090719.22250-1-ankita@nvidia.com> <20231218090719.22250-2-ankita@nvidia.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 104.132.45.104 X-SA-Exim-Rcpt-To: ankita@nvidia.com, jgg@nvidia.com, oliver.upton@linux.dev, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, alex.williamson@redhat.com, kevin.tian@intel.com, yi.l.liu@intel.com, ardb@kernel.org, akpm@linux-foundation.org, gshan@redhat.com, mochs@nvidia.com, lpieralisi@kernel.org, aniketa@nvidia.com, cjia@nvidia.com, kwankhede@nvidia.com, targupta@nvidia.com, vsethi@nvidia.com, acurrid@nvidia.com, apopple@nvidia.com, jhubbard@nvidia.com, danw@nvidia.com, linux-mm@kvack.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 3C9861C002E X-Stat-Signature: yuan3f4drd6saen3ezidkiejka7dnoc9 X-Rspam-User: X-HE-Tag: 1703081537-184125 X-HE-Meta: U2FsdGVkX18FOO0Ba+NmF9uE+HTmB+SZTy60n55aVyHUzDOmHMYNX3tB+6qnIDQcJ3MMzOOOGd5Y+dZyzlsiUMJODO8q+zdSiTBFr1Ax9Hdt845TBAR1X61tFW2XteteKs1oYVVA4Rv2tkPYOBVqZoz+ToVHoASGr8ZF1D4NaVa3rtjqsEpsWi3PtunHOLfJN/OB79vSLZyFarHYc0OLSrBhivjgBd3/CcKgp/JUfx5mhkeoZeUKxHtkxm3AdLApE0bDbKA9Vq5Vu6OATvruVyZSLeDjXw9CXKBhtffsx5sLyrfaDrv3dFA0sM0du2OVhisQfEB4gb4yMCyjiNGbiRdZLcqHSFVFDfZor0jv+OSRjGXDMJAK4ibxCTDsFc8sObE6j2G972K+EdeZOrw2yDueXj/6H6Icp77/CNqZgD3p9BW0XVEqKZvmMPzKOw/Fx7zwNoJt82wfl4W7y0ol9pCUlKAIUxggUYDnLu7RPlmaywr0a3VNYnkeQ2kW8MOoaiYvnlvA4v4rOLeYG/Ey3As6VERGw3VidWWeyJ51De51XjJi7TmtfjspPLDWb1sUf7g8u0leW/4a6Fi5prFp9w6cIajRZo0oIO8eH0y4AtbnnA2iv23qBs7s5N1i+a8d+G3ALvGZPuTuvWR62DB4k8QQ8z0GV2FunFNoc1+BsyNGJjd6euK0qsw27r2UUZpe6ryZqDvrmt/EMQgElW3OrgMbBM3XeTSTXTKy+8NCQCb5bE4FIU9bX3GLvATOj+cUBi/lqq8ZMQuY3le2y8nmbPKEyMiz7ohRw5uqI7+HlZaXBAqYaRcAp1ft3VmBgkr/hn6I/vCcEheWR5EtGAHWiMhpli9QwuVenaxqtCV3E/ecpwio5ANsHwinTRvgQ0r2NGVj5TBRBOM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, 18 Dec 2023 09:07:17 +0000, wrote: > > From: Ankit Agrawal > > For various reasons described in the cover letter, and primarily to Well, the cover letter does not end-up in the git tree, so you must put some actual information here. > allow VM get IO memory with NORMALNC properties, it is desired > to relax the KVM stage 2 device memory attributes from DEVICE_nGnRE > to NormalNC. So set S2 PTE for IO memory as NORMAL_NC. > > A Normal-NC flag is not present today. So add a new kvm_pgtable_prot > (KVM_PGTABLE_PROT_NORMAL_NC) flag for it, along with its > corresponding PTE value 0x5 (0b101) determined from [1]. > > Lastly, adapt the stage2 PTE property setter function > (stage2_set_prot_attr) to handle the NormalNC attribute. > > [1] section D8.5.5 of DDI0487J_a_a-profile_architecture_reference_manual.pdf > > Signed-off-by: Ankit Agrawal > Suggested-by: Jason Gunthorpe > Acked-by: Catalin Marinas > Tested-by: Ankit Agrawal > --- > arch/arm64/include/asm/kvm_pgtable.h | 2 ++ > arch/arm64/include/asm/memory.h | 2 ++ > arch/arm64/kvm/hyp/pgtable.c | 13 +++++++++++-- > 3 files changed, 15 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h > index cfdf40f734b1..19278dfe7978 100644 > --- a/arch/arm64/include/asm/kvm_pgtable.h > +++ b/arch/arm64/include/asm/kvm_pgtable.h > @@ -197,6 +197,7 @@ enum kvm_pgtable_stage2_flags { > * @KVM_PGTABLE_PROT_W: Write permission. > * @KVM_PGTABLE_PROT_R: Read permission. > * @KVM_PGTABLE_PROT_DEVICE: Device attributes. > + * @KVM_PGTABLE_PROT_NORMAL_NC: Normal noncacheable attributes. > * @KVM_PGTABLE_PROT_SW0: Software bit 0. > * @KVM_PGTABLE_PROT_SW1: Software bit 1. > * @KVM_PGTABLE_PROT_SW2: Software bit 2. > @@ -208,6 +209,7 @@ enum kvm_pgtable_prot { > KVM_PGTABLE_PROT_R = BIT(2), > > KVM_PGTABLE_PROT_DEVICE = BIT(3), > + KVM_PGTABLE_PROT_NORMAL_NC = BIT(4), > > KVM_PGTABLE_PROT_SW0 = BIT(55), > KVM_PGTABLE_PROT_SW1 = BIT(56), > diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h > index fde4186cc387..c247e5f29d5a 100644 > --- a/arch/arm64/include/asm/memory.h > +++ b/arch/arm64/include/asm/memory.h > @@ -147,6 +147,7 @@ > * Memory types for Stage-2 translation > */ > #define MT_S2_NORMAL 0xf > +#define MT_S2_NORMAL_NC 0x5 > #define MT_S2_DEVICE_nGnRE 0x1 > > /* > @@ -154,6 +155,7 @@ > * Stage-2 enforces Normal-WB and Device-nGnRE > */ > #define MT_S2_FWB_NORMAL 6 > +#define MT_S2_FWB_NORMAL_NC 5 > #define MT_S2_FWB_DEVICE_nGnRE 1 > > #ifdef CONFIG_ARM64_4K_PAGES > diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c > index c651df904fe3..0fff079a0ef3 100644 > --- a/arch/arm64/kvm/hyp/pgtable.c > +++ b/arch/arm64/kvm/hyp/pgtable.c > @@ -718,10 +718,19 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p > kvm_pte_t *ptep) > { > bool device = prot & KVM_PGTABLE_PROT_DEVICE; > - kvm_pte_t attr = device ? KVM_S2_MEMATTR(pgt, DEVICE_nGnRE) : > - KVM_S2_MEMATTR(pgt, NORMAL); > + bool normal_nc = prot & KVM_PGTABLE_PROT_NORMAL_NC; > + kvm_pte_t attr; > u32 sh = KVM_PTE_LEAF_ATTR_LO_S2_SH_IS; > > + WARN_ON_ONCE(device && normal_nc); > + > + if (device) > + attr = KVM_S2_MEMATTR(pgt, DEVICE_nGnRE); > + else if (normal_nc) > + attr = KVM_S2_MEMATTR(pgt, NORMAL_NC); > + else > + attr = KVM_S2_MEMATTR(pgt, NORMAL); > + This whole thing can be written a bit more elegantly: switch (prot & (KVM_PGTABLE_PROT_DEVICE | KVM_PGTABLE_PROT_NORMAL_NC)) { case 0: attr = KVM_S2_MEMATTR(pgt, NORMAL); break; case KVM_PGTABLE_PROT_DEVICE: if (prot & KVM_PGTABLE_PROT_X) return -EINVAL; attr = KVM_S2_MEMATTR(pgt, DEVICE_nGnRE); break; case KVM_PGTABLE_PROT_NORMAL_NC: attr = KVM_S2_MEMATTR(pgt, NORMAL_NC); break; default: WARN_ON_ONCE(1); } and you can get rid of all of the boolean crud. M. -- Without deviation from the norm, progress is not possible.