From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DDF3C38BE1 for ; Mon, 24 Feb 2020 18:25:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CC4B520838 for ; Mon, 24 Feb 2020 18:25:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="DVVCVC36" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC4B520838 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BC2386B0081; Mon, 24 Feb 2020 13:24:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B5A866B0083; Mon, 24 Feb 2020 13:24:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A07A6B0085; Mon, 24 Feb 2020 13:24:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0118.hostedemail.com [216.40.44.118]) by kanga.kvack.org (Postfix) with ESMTP id 768D36B0081 for ; Mon, 24 Feb 2020 13:24:50 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 26E54180AD806 for ; Mon, 24 Feb 2020 18:24:50 +0000 (UTC) X-FDA: 76525846740.21.fifth66_6454674a26041 X-HE-Tag: fifth66_6454674a26041 X-Filterd-Recvd-Size: 11035 Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Mon, 24 Feb 2020 18:24:49 +0000 (UTC) Received: by mail-wr1-f67.google.com with SMTP id p18so7934117wre.9 for ; Mon, 24 Feb 2020 10:24:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xgVOPZ49x+1WrAdlWgbH+8j706qx2NuqDvbYc/uYWqQ=; b=DVVCVC36hOalbx91/U6kWD3ccIyGZj7ZRW5vpODSj3gu99YZ5+LCzw4kQfmBti4iJW I5ScSgKsJDdcwatGi5aA2Qx5/ffbToJmmMzZ0xftW+g2u6cChxL8uMOCYwct5IVk707S 2FuW+tN52SNX4qyRaGUM2rAAJxuwMk1Fq+u9P0FhmYP9BIpo4PCfwht7+KJOtLqVYX09 zXN9rgCcbRujvfDXfatzgxkjUn76Zuiq06G0lQzeP6mQUtufL30DcY2NyWUj7SjEqnJY IYeeZ+Y5NWWRqXvsEAL/j2Ic6zO4+PGLo4Wxn3yw4PROGnWprozGZFdCjKu3vnhPwHKa sP2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xgVOPZ49x+1WrAdlWgbH+8j706qx2NuqDvbYc/uYWqQ=; b=Y8+j3w7rgiHXBySj372J3wuNZ4lT0cB3xa6Dtht5JkH7GRYWh2lf3dASqw02gIicRT Jpu9VyUj+woZSnq0f6/xw+WVYwwfSZ8aG41r+TsCnlO8+tHRYcPgsKuOjkqXv1Xwl9lM 96GEseMjOMpFyxSob4qEM6QSRFoLYCBi3jIS37PYsCxhe7mNA+QDBG/WdM42jAjJXU/6 ru+JXmFhilhBazYEeAMjRwWAsCOap1xj4/dRby+NkUp8jwerqmUy1UnxyEu1sXuX7fAF 9HgaiGkKpXUCKeXGbX2vbygGQYy8gYY1q9pJX5mE+qdMwfoXJK3XOWTjWGoP/w8+6rjn 7mGA== X-Gm-Message-State: APjAAAUu3l42BImwEzCRjHdiN9PUTPOEwJImxt+wxj+A99pDW9Acp7u8 g2mmkt2O/RLcsH/7p+5TgSgj+Q== X-Google-Smtp-Source: APXvYqySTJeCzOmWaA354F9WnN4bUCNey753ZHQTjm+04zgAPahujuB+OVrEUHRHVNITM4h9f5/NJw== X-Received: by 2002:a5d:4847:: with SMTP id n7mr67178382wrs.30.1582568688179; Mon, 24 Feb 2020 10:24:48 -0800 (PST) Received: from localhost.localdomain ([2001:171b:c9a8:fbc0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id n3sm304255wmc.27.2020.02.24.10.24.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2020 10:24:47 -0800 (PST) From: Jean-Philippe Brucker To: iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, linux-mm@kvack.org Cc: joro@8bytes.org, robh+dt@kernel.org, mark.rutland@arm.com, catalin.marinas@arm.com, will@kernel.org, robin.murphy@arm.com, kevin.tian@intel.com, baolu.lu@linux.intel.com, Jonathan.Cameron@huawei.com, jacob.jun.pan@linux.intel.com, christian.koenig@amd.com, yi.l.liu@intel.com, zhangfei.gao@linaro.org, Jean-Philippe Brucker Subject: [PATCH v4 17/26] iommu/arm-smmu-v3: Implement mm operations Date: Mon, 24 Feb 2020 19:23:52 +0100 Message-Id: <20200224182401.353359-18-jean-philippe@linaro.org> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200224182401.353359-1-jean-philippe@linaro.org> References: <20200224182401.353359-1-jean-philippe@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jean-Philippe Brucker Hook SVA operations to support sharing page tables with the SMMUv3: * dev_enable/disable/has_feature for device drivers to modify the SVA state. * sva_bind/unbind and sva_get_pasid to bind device and address spaces. * The mm_attach/detach/invalidate/free callbacks from iommu-sva Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/Kconfig | 1 + drivers/iommu/arm-smmu-v3.c | 176 +++++++++++++++++++++++++++++++++++- 2 files changed, 175 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index 211684e785ea..05341155d34b 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -434,6 +434,7 @@ config ARM_SMMU_V3 tristate "ARM Ltd. System MMU Version 3 (SMMUv3) Support" depends on ARM64 select IOMMU_API + select IOMMU_SVA select IOMMU_IO_PGTABLE_LPAE select GENERIC_MSI_IRQ_DOMAIN help diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 54bd6913d648..3973f7222864 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -36,6 +36,7 @@ #include =20 #include "io-pgtable-arm.h" +#include "iommu-sva.h" =20 /* MMIO registers */ #define ARM_SMMU_IDR0 0x0 @@ -1872,7 +1873,6 @@ static struct arm_smmu_ctx_desc *arm_smmu_share_asi= d(u16 asid) return NULL; } =20 -__maybe_unused static struct arm_smmu_ctx_desc *arm_smmu_alloc_shared_cd(struct mm_stru= ct *mm) { u16 asid; @@ -1969,7 +1969,6 @@ static struct arm_smmu_ctx_desc *arm_smmu_alloc_sha= red_cd(struct mm_struct *mm) return ERR_PTR(ret); } =20 -__maybe_unused static void arm_smmu_free_shared_cd(struct arm_smmu_ctx_desc *cd) { if (arm_smmu_free_asid(cd)) { @@ -2958,6 +2957,12 @@ static int arm_smmu_attach_dev(struct iommu_domain= *domain, struct device *dev) =20 smmu =3D master->smmu; =20 + if (iommu_sva_enabled(dev)) { + /* Did the previous driver forget to release SVA handles? */ + dev_err(dev, "cannot attach - SVA enabled\n"); + return -EBUSY; + } + arm_smmu_detach_dev(master); =20 mutex_lock(&smmu_domain->init_mutex); @@ -3057,6 +3062,81 @@ arm_smmu_iova_to_phys(struct iommu_domain *domain,= dma_addr_t iova) return ops->iova_to_phys(ops, iova); } =20 +static void arm_smmu_mm_invalidate(struct device *dev, int pasid, void *= entry, + unsigned long iova, size_t size) +{ + /* TODO: Invalidate ATC */ +} + +static int arm_smmu_mm_attach(struct device *dev, int pasid, void *entry= , + bool attach_domain) +{ + struct arm_smmu_ctx_desc *cd =3D entry; + struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); + struct arm_smmu_domain *smmu_domain =3D to_smmu_domain(domain); + + /* + * If another device in the domain has already been attached, the + * context descriptor is already valid. + */ + if (!attach_domain) + return 0; + + return arm_smmu_write_ctx_desc(smmu_domain, pasid, cd); +} + +static void arm_smmu_mm_detach(struct device *dev, int pasid, void *entr= y, + bool detach_domain) +{ + struct arm_smmu_ctx_desc *cd =3D entry; + struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); + struct arm_smmu_domain *smmu_domain =3D to_smmu_domain(domain); + + if (detach_domain) { + arm_smmu_write_ctx_desc(smmu_domain, pasid, NULL); + + /* + * The ASID allocator won't broadcast the final TLB + * invalidations for this ASID, so we need to do it manually. + * For private contexts, freeing io-pgtable ops performs the + * invalidation. + */ + arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); + } + + /* TODO: invalidate ATC */ +} + +static void *arm_smmu_mm_alloc(struct mm_struct *mm) +{ + return arm_smmu_alloc_shared_cd(mm); +} + +static void arm_smmu_mm_free(void *entry) +{ + arm_smmu_free_shared_cd(entry); +} + +static struct io_mm_ops arm_smmu_mm_ops =3D { + .alloc =3D arm_smmu_mm_alloc, + .invalidate =3D arm_smmu_mm_invalidate, + .attach =3D arm_smmu_mm_attach, + .detach =3D arm_smmu_mm_detach, + .release =3D arm_smmu_mm_free, +}; + +static struct iommu_sva * +arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdat= a) +{ + struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); + struct arm_smmu_domain *smmu_domain =3D to_smmu_domain(domain); + + if (smmu_domain->stage !=3D ARM_SMMU_DOMAIN_S1) + return ERR_PTR(-EINVAL); + + return iommu_sva_bind_generic(dev, mm, &arm_smmu_mm_ops, drvdata); +} + static struct platform_driver arm_smmu_driver; =20 static @@ -3175,6 +3255,7 @@ static void arm_smmu_remove_device(struct device *d= ev) =20 master =3D fwspec->iommu_priv; smmu =3D master->smmu; + iommu_sva_disable(dev); arm_smmu_detach_dev(master); iommu_group_remove_device(dev); iommu_device_unlink(&smmu->iommu, dev); @@ -3294,6 +3375,90 @@ static void arm_smmu_get_resv_regions(struct devic= e *dev, iommu_dma_get_resv_regions(dev, head); } =20 +static bool arm_smmu_iopf_supported(struct arm_smmu_master *master) +{ + return false; +} + +static bool arm_smmu_dev_has_feature(struct device *dev, + enum iommu_dev_features feat) +{ + struct arm_smmu_master *master =3D dev_to_master(dev); + + if (!master) + return false; + + switch (feat) { + case IOMMU_DEV_FEAT_SVA: + if (!(master->smmu->features & ARM_SMMU_FEAT_SVA)) + return false; + + /* SSID and IOPF support are mandatory for the moment */ + return master->ssid_bits && arm_smmu_iopf_supported(master); + default: + return false; + } +} + +static bool arm_smmu_dev_feature_enabled(struct device *dev, + enum iommu_dev_features feat) +{ + struct arm_smmu_master *master =3D dev_to_master(dev); + + if (!master) + return false; + + switch (feat) { + case IOMMU_DEV_FEAT_SVA: + return iommu_sva_enabled(dev); + default: + return false; + } +} + +static int arm_smmu_dev_enable_sva(struct device *dev) +{ + struct arm_smmu_master *master =3D dev_to_master(dev); + struct iommu_sva_param param =3D { + .min_pasid =3D 1, + .max_pasid =3D 0xfffffU, + }; + + param.max_pasid =3D min(param.max_pasid, (1U << master->ssid_bits) - 1)= ; + return iommu_sva_enable(dev, ¶m); +} + +static int arm_smmu_dev_enable_feature(struct device *dev, + enum iommu_dev_features feat) +{ + if (!arm_smmu_dev_has_feature(dev, feat)) + return -ENODEV; + + if (arm_smmu_dev_feature_enabled(dev, feat)) + return -EBUSY; + + switch (feat) { + case IOMMU_DEV_FEAT_SVA: + return arm_smmu_dev_enable_sva(dev); + default: + return -EINVAL; + } +} + +static int arm_smmu_dev_disable_feature(struct device *dev, + enum iommu_dev_features feat) +{ + if (!arm_smmu_dev_feature_enabled(dev, feat)) + return -EINVAL; + + switch (feat) { + case IOMMU_DEV_FEAT_SVA: + return iommu_sva_disable(dev); + default: + return -EINVAL; + } +} + static struct iommu_ops arm_smmu_ops =3D { .capable =3D arm_smmu_capable, .domain_alloc =3D arm_smmu_domain_alloc, @@ -3312,6 +3477,13 @@ static struct iommu_ops arm_smmu_ops =3D { .of_xlate =3D arm_smmu_of_xlate, .get_resv_regions =3D arm_smmu_get_resv_regions, .put_resv_regions =3D generic_iommu_put_resv_regions, + .dev_has_feat =3D arm_smmu_dev_has_feature, + .dev_feat_enabled =3D arm_smmu_dev_feature_enabled, + .dev_enable_feat =3D arm_smmu_dev_enable_feature, + .dev_disable_feat =3D arm_smmu_dev_disable_feature, + .sva_bind =3D arm_smmu_sva_bind, + .sva_unbind =3D iommu_sva_unbind_generic, + .sva_get_pasid =3D iommu_sva_get_pasid_generic, .pgsize_bitmap =3D -1UL, /* Restricted during device attach */ }; =20 --=20 2.25.0