From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACD21C43611 for ; Wed, 12 May 2021 16:01:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 24AF861E20 for ; Wed, 12 May 2021 16:01:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 24AF861E20 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 33A856B006C; Wed, 12 May 2021 12:01:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 310796B006E; Wed, 12 May 2021 12:01:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 189D06B0070; Wed, 12 May 2021 12:01:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0079.hostedemail.com [216.40.44.79]) by kanga.kvack.org (Postfix) with ESMTP id DC5FD6B006C for ; Wed, 12 May 2021 12:01:16 -0400 (EDT) Received: from smtpin35.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 680C61815A833 for ; Wed, 12 May 2021 16:01:16 +0000 (UTC) X-FDA: 78133043352.35.AA17AF0 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf07.hostedemail.com (Postfix) with ESMTP id DE860A00908D for ; Wed, 12 May 2021 16:01:11 +0000 (UTC) Received: by mail-pj1-f52.google.com with SMTP id v11-20020a17090a6b0bb029015cba7c6bdeso3298789pjj.0 for ; Wed, 12 May 2021 09:01:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=5/MplZ6oYXtQw2JFwTKJsqNpcX+apjnkiw44w1Cqcuo=; b=EQwntmF6pP3DBJ88bmhariOI9t4IKKtCPxB2phWGOC74jrMotb3ZCiTXXW16g67V0h 8HVeJ0GheMO0pRAFNX4jKbmzmE+Qf8EKG3s79mh2+TjoJ77eNPmt01t4bFBgCEpZMQYS LtqGvaxjzI7bnbkr39ucdoF9COsZV+kpDzrwJ3LopkNaVGle3wN8WsASeqDSWcrxUXNq l5Oqo36f1zOS36IgBY0vrE19xWxnxCnuQZW0ACKwM1vfnoNVDWJ5KT6c8CBkMQmT0Avv j0Aoqpt+TzspMyc0v3GAvNXVY/RSbD7UNpll9CI5ElGs5Xyk4zXN78+QI0vIQtbqXWFj 2nGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=5/MplZ6oYXtQw2JFwTKJsqNpcX+apjnkiw44w1Cqcuo=; b=ksAsS6C0HWYmcX8w8HCyzLvL/jbXF4ZFZ1KsSMte6Am2VC/vNyLy1vWXSioXBxKh7V hIIaEA5hivJac6mFyZeBqZjVf3q13FYgW5IoUC2OITh+V2jRhyUauOyOC1I24m8duB6s 5CkJqpn0QDjC0guAWq/A9WYamDy+5lHkes00zRVo+nt5fmxvp82RZ+b+wvXK6OWvtVub wfSpAoOBs0Xd4cSTavQ1B+LuuxlgHnkAN9BjhI6XPQgdR0koMd65ExSEgNOkLbJ8nuZz ChsXrrpRVPaDi4MmylyFJhOCBGFvCAPyNBAAZG41lOoPuesNUs28UIll1Z584QVHqI+A l8QA== X-Gm-Message-State: AOAM533dFDJSKBYQshSy7MpBzYu9GxZpOoVe/+U8Rb8QSraQUDfUPIzs tfrpeoz9T18c4GJ/TmQw1gk= X-Google-Smtp-Source: ABdhPJxtWg7IxeHd7M+hWJJqKPqAcUV/yJEzGw/b/vFIZdhJW26Hi6Inh2Unritq+GbozDcH9ke1gw== X-Received: by 2002:a17:90a:7442:: with SMTP id o2mr2519021pjk.44.1620835273807; Wed, 12 May 2021 09:01:13 -0700 (PDT) Received: from ?IPv6:2404:f801:0:5:8000::4b1? ([2404:f801:9000:1a:efea::4b1]) by smtp.gmail.com with ESMTPSA id i24sm238402pfd.35.2021.05.12.09.01.02 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 12 May 2021 09:01:13 -0700 (PDT) Subject: Re: [Resend RFC PATCH V2 10/12] HV/IOMMU: Add Hyper-V dma ops support To: Christoph Hellwig , konrad.wilk@oracle.com Cc: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, wei.liu@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, akpm@linux-foundation.org, gregkh@linuxfoundation.org, konrad.wilk@oracle.com, m.szyprowski@samsung.com, robin.murphy@arm.com, joro@8bytes.org, will@kernel.org, davem@davemloft.net, kuba@kernel.org, jejb@linux.ibm.com, martin.petersen@oracle.com, Tianyu Lan , iommu@lists.linux-foundation.org, linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-scsi@vger.kernel.org, netdev@vger.kernel.org, vkuznets@redhat.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, sunilmut@microsoft.com References: <20210414144945.3460554-1-ltykernel@gmail.com> <20210414144945.3460554-11-ltykernel@gmail.com> <20210414154729.GD32045@lst.de> From: Tianyu Lan Message-ID: Date: Thu, 13 May 2021 00:01:01 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: <20210414154729.GD32045@lst.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=EQwntmF6; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf07.hostedemail.com: domain of ltykernel@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=ltykernel@gmail.com X-Stat-Signature: nnomr9p7wqtg68zj6omboabyjsqiprw4 X-Rspamd-Queue-Id: DE860A00908D X-Rspamd-Server: rspam02 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail-pj1-f52.google.com; client-ip=209.85.216.52 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1620835271-312784 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Christoph and Konrad: Current Swiotlb bounce buffer uses a pool for all devices. There is a high overhead to get or free bounce buffer during performance test. Swiotlb code now use a global spin lock to protect bounce buffer data. Several device queues try to acquire the spin lock and this introduce additional overhead. For performance and security perspective, each devices should have a separate swiotlb bounce buffer pool and so this part needs to rework. I want to check this is right way to resolve performance issues with swiotlb bounce buffer. If you have some other suggestions,welcome. Thanks. On 4/14/2021 11:47 PM, Christoph Hellwig wrote: >> +static dma_addr_t hyperv_map_page(struct device *dev, struct page *page, >> + unsigned long offset, size_t size, >> + enum dma_data_direction dir, >> + unsigned long attrs) >> +{ >> + phys_addr_t map, phys = (page_to_pfn(page) << PAGE_SHIFT) + offset; >> + >> + if (!hv_is_isolation_supported()) >> + return phys; >> + >> + map = swiotlb_tbl_map_single(dev, phys, size, HV_HYP_PAGE_SIZE, dir, >> + attrs); >> + if (map == (phys_addr_t)DMA_MAPPING_ERROR) >> + return DMA_MAPPING_ERROR; >> + >> + return map; >> +} > > This largerly duplicates what dma-direct + swiotlb does. Please use > force_dma_unencrypted to force bounce buffering and just use the generic > code. > >> + if (hv_isolation_type_snp()) { >> + ret = hv_set_mem_host_visibility( >> + phys_to_virt(hyperv_io_tlb_start), >> + hyperv_io_tlb_size, >> + VMBUS_PAGE_VISIBLE_READ_WRITE); >> + if (ret) >> + panic("%s: Fail to mark Hyper-v swiotlb buffer visible to host. err=%d\n", >> + __func__, ret); >> + >> + hyperv_io_tlb_remap = ioremap_cache(hyperv_io_tlb_start >> + + ms_hyperv.shared_gpa_boundary, >> + hyperv_io_tlb_size); >> + if (!hyperv_io_tlb_remap) >> + panic("%s: Fail to remap io tlb.\n", __func__); >> + >> + memset(hyperv_io_tlb_remap, 0x00, hyperv_io_tlb_size); >> + swiotlb_set_bounce_remap(hyperv_io_tlb_remap); > > And this really needs to go into a common hook where we currently just > call set_memory_decrypted so that all the different schemes for these > trusted VMs (we have about half a dozen now) can share code rather than > reinventing it. >