From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C028C4167B for ; Tue, 12 Dec 2023 12:17:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA1A36B02C0; Tue, 12 Dec 2023 07:17:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A51696B02C1; Tue, 12 Dec 2023 07:17:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 919456B02C2; Tue, 12 Dec 2023 07:17:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 81E336B02C0 for ; Tue, 12 Dec 2023 07:17:58 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3F598120932 for ; Tue, 12 Dec 2023 12:17:58 +0000 (UTC) X-FDA: 81558067836.14.1BB4D1E Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf22.hostedemail.com (Postfix) with ESMTP id DFFE1C0014 for ; Tue, 12 Dec 2023 12:17:55 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=EKEqXSYr; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf22.hostedemail.com: domain of will@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=will@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702383476; a=rsa-sha256; cv=none; b=JaeEWJNo/V8Lhes7YuwUP4Sm0oMyVi0ZLTemd1wKfS1lXyj4TFldM0eSaTte1q0M8R71p9 Rqk/tR8MfEVxsxMk8N00f33CxGWspL2tWH3rwdDVKHX2jgPcvelMVudsZ+8AzkvrD7h+9/ 55EnOsx5LtbrNMCaWICjFv2uQ2Mo9Hc= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=EKEqXSYr; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf22.hostedemail.com: domain of will@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=will@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702383476; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ITDiqshb77mg7AP/gzbeBxC8JoFxiWj9UfHEO/Isigk=; b=pClti4yx5uQA3NzUhOn0RO015Cw5Pmtgj9wN1zaM7zaGS5zU/JOMTYqF3FwRYI+fDUBv1b Vcj7Jr9ky8c9vei/bmTdiylk3iwKbwFlAIaivaSoTXroODKae81zIR5sVCnvfcnUYulQ17 qdwp2yLJIKTpvZS2kQplqyrgdTyDrpA= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 66D30CE1144; Tue, 12 Dec 2023 12:17:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 90F7AC433C8; Tue, 12 Dec 2023 12:17:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1702383471; bh=PSsletKarNFcUm/hB4a9Sb8O26Kgrcut7nfxUAiV0G0=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=EKEqXSYrh9aAc4uI9mhEFQNclPqu4W8YM1VERL6UvetYXIzQ1IXiNblhjc1CC5QVj YLNWemmKoqQyMBVNlirwrMDfW6bgeybH6ppIEef/bqR6ilbNBPVXF5DpkeOkWMI9zk NfxfciR7kvIckAhNRDssDu1BjCUh2HdEBlyzz8RwQYkbW8I2EBVDyeN1HMxxmoRRXq ZC/dfV7hCUbi02ScIlADNOnJfsgTbTlVDZh5JdyXURRqSJISGzqBXlYCXJ2Wgum+pK peFPOPzAVtnJNOliurIg/B5MTrXOINv+ds72CjhIc9bDa4X1uG52VVsego3Om41jm0 fLGMd5Y1TuBSQ== Date: Tue, 12 Dec 2023 12:17:42 +0000 From: Will Deacon To: ankita@nvidia.com Cc: jgg@nvidia.com, maz@kernel.org, oliver.upton@linux.dev, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, alex.williamson@redhat.com, kevin.tian@intel.com, yi.l.liu@intel.com, ardb@kernel.org, akpm@linux-foundation.org, gshan@redhat.com, linux-mm@kvack.org, lpieralisi@kernel.org, aniketa@nvidia.com, cjia@nvidia.com, kwankhede@nvidia.com, targupta@nvidia.com, vsethi@nvidia.com, acurrid@nvidia.com, apopple@nvidia.com, jhubbard@nvidia.com, danw@nvidia.com, mochs@nvidia.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH v3 1/2] kvm: arm64: introduce new flag for non-cacheable IO memory Message-ID: <20231212121742.GA29205@willie-the-truck> References: <20231208164709.23101-1-ankita@nvidia.com> <20231208164709.23101-2-ankita@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231208164709.23101-2-ankita@nvidia.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: DFFE1C0014 X-Stat-Signature: ems4pqop395st5g4rhg1bxs3jtdcrnuh X-HE-Tag: 1702383475-650217 X-HE-Meta: U2FsdGVkX199oKbHiQJzuPGXMW7t9pSq+npnjNa/CI+we2yn5kC2J3jeFg6hwOtut35yoqKLRAO6otF48Bu5vFTo9XV90lL2wV5uSioDCDbjllUXR7R2kQloM/GHPjwoV3IGbuXPNgUYTe4wj6Tb55b0NXwfDJ6SvZ2hr5+VMVPkAvwEpOb0VFDgI5o4K3GjaZvFDg9pFciLFQXN7T+yZYo3Z4pPZqxRCiarDGtOv03b05p47nKN6CU9bTg9NSMldmpDqRwd7ZqED2+wbSFAAqPxJP8Bgje79Lxy0FE6pK7GIWUrN41M1Du/imaFutKYfSkItux8PtbAldjcDDX4l86G5orfahB26NLir/FX37bHtC7ExuuGeDfiApi0PNPboEXgDg6fms7IJYArxyELLBwQ1XPKrcDgReJmxU3GS7EeLIdZDqtHttxl94w9u4TucrHrEWmPQGPHO28BORZjPXUu3hfbSJKEBe4Em6L/euSDyjftdKYECgsnyTyirJzoLUHPLfw/nG8+gPhvG56oL59Bmb7e7LPXcZw9fRiiPD6I63F0kDdysErVgbPS86hBLOXCbJ/vTFNEwOvSjqPeZtOlnJ6EEr6pWXkGFRNLL805boEtJbD4cT1LUJmiVaWQ3nMr7yprQtaM/JJwhmRhHATl+jIVLU1sorp9VVi8LGb1NjcEu0Wj1fZgTTjKb/C/tzF8tfblyRzoOkJ38oo5b2znuL1z5tQsIeMe73ic5lZvO3rzpqgfxGt8SPpTFZtf1fawY6QAAy2TieUfR6FUr6qFDPH61bLXY6bz6nWP8vX7CSIl0yx2/jFa5pwi2YrcYb4NswVcZcpYFQgqmKJcEJLfUnEvsjGMlUKSJgJYjNPaRaZ0cjvuHuhF8hp9M0KhhoarTVv/yhQUt4QgZ2q6HgZMfr5RElSe1sUyZbdEaWv8P9mDXjcF4y2ar09m9JQN6qtsS50v3H4sIBYwuvd +6GaZAQI BT3RslJcB1t3uBXNH8wluWRIC14NkrsAjnIUG2fgR3zpSD7bdICYkmT5eaO9r1kJf+4wd4wy5FEWvIuFq+NFCddsurjWmbLFqY0fjONTuO4TyfxIyRqQ46wniHklAMs5ILwW6TV1UqvCGU/sOvjXfGJDAk6SWXsq4Qs5JjW71+YTtLYMUcAdnkh8zlJmsRe1WYbCZ4nGgmUgpDVd+TgDgpsbImHAfGXTkafCshGcZU3QoZdmWZIYLzIlfMvC3Q9NJISj+P+N1JXabM1A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Dec 08, 2023 at 10:17:08PM +0530, ankita@nvidia.com wrote: > From: Ankit Agrawal > > For various reasons described in the cover letter, and primarily to > allow VM get IO memory with NORMALNC properties, it is desired > to relax the KVM stage 2 device memory attributes from DEVICE_nGnRE > to NormalNC. So set S2 PTE for IO memory as NORMAL_NC. > > A Normal-NC flag is not present today. So add a new kvm_pgtable_prot > (KVM_PGTABLE_PROT_NORMAL_NC) flag for it, along with its > corresponding PTE value 0x5 (0b101) determined from [1]. > > Lastly, adapt the stage2 PTE property setter function > (stage2_set_prot_attr) to handle the NormalNC attribute. > > [1] section D8.5.5 of DDI0487J_a_a-profile_architecture_reference_manual.pdf > > Signed-off-by: Ankit Agrawal > Suggested-by: Jason Gunthorpe > Acked-by: Catalin Marinas > Tested-by: Ankit Agrawal > --- > arch/arm64/include/asm/kvm_pgtable.h | 2 ++ > arch/arm64/include/asm/memory.h | 2 ++ > arch/arm64/kvm/hyp/pgtable.c | 11 +++++++++-- > 3 files changed, 13 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h > index cfdf40f734b1..19278dfe7978 100644 > --- a/arch/arm64/include/asm/kvm_pgtable.h > +++ b/arch/arm64/include/asm/kvm_pgtable.h > @@ -197,6 +197,7 @@ enum kvm_pgtable_stage2_flags { > * @KVM_PGTABLE_PROT_W: Write permission. > * @KVM_PGTABLE_PROT_R: Read permission. > * @KVM_PGTABLE_PROT_DEVICE: Device attributes. > + * @KVM_PGTABLE_PROT_NORMAL_NC: Normal noncacheable attributes. > * @KVM_PGTABLE_PROT_SW0: Software bit 0. > * @KVM_PGTABLE_PROT_SW1: Software bit 1. > * @KVM_PGTABLE_PROT_SW2: Software bit 2. > @@ -208,6 +209,7 @@ enum kvm_pgtable_prot { > KVM_PGTABLE_PROT_R = BIT(2), > > KVM_PGTABLE_PROT_DEVICE = BIT(3), > + KVM_PGTABLE_PROT_NORMAL_NC = BIT(4), > > KVM_PGTABLE_PROT_SW0 = BIT(55), > KVM_PGTABLE_PROT_SW1 = BIT(56), > diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h > index fde4186cc387..c247e5f29d5a 100644 > --- a/arch/arm64/include/asm/memory.h > +++ b/arch/arm64/include/asm/memory.h > @@ -147,6 +147,7 @@ > * Memory types for Stage-2 translation > */ > #define MT_S2_NORMAL 0xf > +#define MT_S2_NORMAL_NC 0x5 > #define MT_S2_DEVICE_nGnRE 0x1 > > /* > @@ -154,6 +155,7 @@ > * Stage-2 enforces Normal-WB and Device-nGnRE > */ > #define MT_S2_FWB_NORMAL 6 > +#define MT_S2_FWB_NORMAL_NC 5 > #define MT_S2_FWB_DEVICE_nGnRE 1 > > #ifdef CONFIG_ARM64_4K_PAGES > diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c > index c651df904fe3..d4835d553c61 100644 > --- a/arch/arm64/kvm/hyp/pgtable.c > +++ b/arch/arm64/kvm/hyp/pgtable.c > @@ -718,10 +718,17 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p > kvm_pte_t *ptep) > { > bool device = prot & KVM_PGTABLE_PROT_DEVICE; > - kvm_pte_t attr = device ? KVM_S2_MEMATTR(pgt, DEVICE_nGnRE) : > - KVM_S2_MEMATTR(pgt, NORMAL); > + bool normal_nc = prot & KVM_PGTABLE_PROT_NORMAL_NC; > + kvm_pte_t attr; > u32 sh = KVM_PTE_LEAF_ATTR_LO_S2_SH_IS; > > + if (device) > + attr = KVM_S2_MEMATTR(pgt, DEVICE_nGnRE); > + else if (normal_nc) > + attr = KVM_S2_MEMATTR(pgt, NORMAL_NC); > + else > + attr = KVM_S2_MEMATTR(pgt, NORMAL); I think it would be worth rejecting the case where both KVM_PGTABLE_PROT_DEVICE and KVM_PGTABLE_PROT_NORMAL_NC are passed, since that's clearly a bug in the caller and silently going with device is arbitrary and confusing. Will