From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4489BC5B543 for ; Fri, 30 May 2025 23:07:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9DF686B0179; Fri, 30 May 2025 19:07:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 98F776B017A; Fri, 30 May 2025 19:07:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8819F6B017B; Fri, 30 May 2025 19:07:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6721C6B0179 for ; Fri, 30 May 2025 19:07:07 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 040988040D for ; Fri, 30 May 2025 23:07:06 +0000 (UTC) X-FDA: 83501111694.18.1C1AC52 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf29.hostedemail.com (Postfix) with ESMTP id DA6ED120004 for ; Fri, 30 May 2025 23:07:04 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b="imE/ZAWy"; dmarc=pass (policy=reject) header.from=fb.com; spf=pass (imf29.hostedemail.com: domain of "prvs=2245ae2f89=amastro@meta.com" designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=2245ae2f89=amastro@meta.com" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748646425; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QmEUoEof0XuWSY9NCPw+hCQm0lsio92LEaLT6yMorqU=; b=eVm7A/T2reJnPay5U0gNiyEzkNz0QkkI6tvN7OcjFae4opegliuRplK8qfoMQ5l5qbeumN yD3w/dXtm/86E+0i/cmqgFWXIU7ETnSLKaQWQKZrhIc4g9nsjKBlEa+XomnkewuHCvCF+e Bfneoq3rIxxWlqU/pH/R0eHC+jYk4jM= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b="imE/ZAWy"; dmarc=pass (policy=reject) header.from=fb.com; spf=pass (imf29.hostedemail.com: domain of "prvs=2245ae2f89=amastro@meta.com" designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=2245ae2f89=amastro@meta.com" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748646425; a=rsa-sha256; cv=none; b=wTEsjxP4s0hjtYr08D1rC3gPWZOa7UcbBDfZa+MZz3zlmZZ0CRcLqWDVswChBFpvs/lGDz U/d5xCalRT9N/nDwv6Fdd51grklHWT88V71vGs4vdsVrAIZVVn4Nb1ly5Jk3TcdYtBXH+g +Xu+EFuR4kD9UcabqXQlrvgz4UonJ2k= Received: from pps.filterd (m0109331.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 54UL6452003495 for ; Fri, 30 May 2025 16:07:03 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=facebook; bh=Q mEUoEof0XuWSY9NCPw+hCQm0lsio92LEaLT6yMorqU=; b=imE/ZAWynTjwlpBYh PCW2pGUFiXZl3Bvejc9tti4hCz8QIv8KeJ0/jfLsTsVV6W+Knk++GgyogmMlhJx3 iYjRaL6Ac4BMBDOhK3qi79DGZFtq4q5S3ihQIMIG1uvAD3QaxsasWFWbsYaxWExJ 3vGCZ3f3jcXaGG5+1YWEx73Qz4= Received: from maileast.thefacebook.com ([163.114.135.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 46y7uw5mh9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 30 May 2025 16:07:03 -0700 (PDT) Received: from twshared37834.15.frc2.facebook.com (2620:10d:c0a8:1b::2d) by mail.thefacebook.com (2620:10d:c0a9:6f::8fd4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1748.24; Fri, 30 May 2025 23:07:02 +0000 Received: by devgpu004.nha5.facebook.com (Postfix, from userid 199522) id AC0F14B34DC; Fri, 30 May 2025 16:06:58 -0700 (PDT) From: Alex Mastro To: CC: , , , , Subject: Re: [BUG?] vfio/pci: VA alignment sensitivity of VFIO_IOMMU_MAP_DMA which target MMIO Date: Fri, 30 May 2025 16:05:10 -0700 Message-ID: <20250530230510.2680688-1-amastro@fb.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-FB-Internal: Safe Content-Type: text/plain X-Authority-Analysis: v=2.4 cv=OMQn3TaB c=1 sm=1 tr=0 ts=683a3a17 cx=c_pps a=MfjaFnPeirRr97d5FC5oHw==:117 a=MfjaFnPeirRr97d5FC5oHw==:17 a=dt9VzEwgFbYA:10 a=20KFwNOVAAAA:8 a=BpDLBPb1mtdKZWO58xwA:9 X-Proofpoint-ORIG-GUID: uO1-lYt9E1he-a-aBkKO6veIMmH2Ypom X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNTMwMDIwNyBTYWx0ZWRfXwRkcU1zC2izq Btg2W7qNilE37iv1js+sYizoBdEdqwkn4sBO3gCJE1RHGBmsYqy+i+nmVL2WbuaH5iXW38xtEcX yZa70V7/9VvwkuUTzWq0Yg4F92ixNF55La7ORse+91g+ssBPnWMXGiE6Scy5t3De6KyRWsm3UIb nkl/RIfbluSVDZRso8m5H+jLY9b8Pj+ixmjGIG6b51xA5kdMVdnd475KhehUZYyFOOqQUqTHsZ3 hT1UVXq9kxLI7flfvJIXYe2vTyaJLb0OAqn4vgpedbwJ4mi/C62N8CILnpX7EhowD6ka/aTS/hv wa2D77oJpPB4ZCOA4gL6CRVZSyieatrQpdJokuVm9ZL7ozFaVGPEKn8G7x1J4YmCZyguGjPj6r0 KXXXoEaURkLpYbG11z8GkKj2NQBLne38Cum1/m+NEcxGi6iNvOyxMvKKNsxE9dBjeFZzQUTD X-Proofpoint-GUID: uO1-lYt9E1he-a-aBkKO6veIMmH2Ypom X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-05-30_10,2025-05-30_01,2025-03-28_01 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: DA6ED120004 X-Stat-Signature: 9zt6j5mtisb3pcht77ya1ebo656k9j3c X-Rspam-User: X-HE-Tag: 1748646424-394153 X-HE-Meta: U2FsdGVkX18PO/pkBWrBuSb7ZHdGxB5cfiXqUj5iD3KpNhwHMTvU5jHR51a48GE3nk5LB7EqcjLliPy15HuqNXjbB2mTGSouue/Tcz3ZQoPEe+2g9WIxmGZbpkzEkhZCiFploYgeJzqjTuwpBtCP4cqLWZ8qa7RCcnjHGOg1cNmYEyk33vR2bazjWrMTkFEm6XiqBN5hOzEzc8hBRLGlQEAfL8/wH6++rd9EpEGLr0C/PgbczpJINS+aN3f24WdwPaVVc4OkYaMADinwKhF4npMBMitFRCCbYIlzQh+/feXrMEfdpDYxFk1tdp6nz4xdOtBkYhk/SCJs2+0kG5FWSWt7xgH/+vvDGqjZ61UNboloaLoCe5Phj5wqvPU9epsOQn+PNaA6JFnkHjHmmf00XCQhZ1PuQB9j9g0OgunrfgbBfGH4FMeWWZq8hYYjfz59qwf8fYEoUA/a2H7kv4YlyGre/08j8LiG+2vKV88VwHOuhtrlOeYqLSmzxk/0e7bMgEAqjOlSPQ5hXw8gZEQh0bbkNbMjh4zhuz3T1T3A68nIXjjJZvQcv5xgOH2W9CeVnR6A+HriLHp1Q2aPcjLsknoTLYi/NYS/IFUzViBEHCNYQXsbxtcq/n+0nhnbW2ieOuqWbF59QJr4Y9xAqayuUNezqJmzK+YgI/w9JEn/PP5HtFyH/7knpdTOkd7N95QjGQy2BaQIcFEc3s5i4qVUX6Z5hp1X9XDWN8s0sD+U5WSqcEtTT2x1sxUZeXNOWZe+a9WGBoGqArOGdItpUx9ImEzuzr+HWJBj9bRn803arhuMhcAIb0B/q0JJEXasQ4BdZmOmChsx+Krp1T2iT/oDYrkPneiL0xVhv8HLBmPHwIjpW546hYQmsLM6glJ43tMHzFCP5fQhZRhZb0ZF/wpEY+rYiRhipJbF/yLmLNBoPRoR+gQnz0F7o3byzRSabsVeSBqFEG0icotTWGTE+Nc KA/tgyTK s7edJf5lDgAgAGCAT0d+Y48aidHXUL7zyT8aov0p0NviKYgm+YzwzBkro7KthGzikL79Q+RDTgYGkSMWTF4DGcHwsThGMQ192F2R3hcxnEKqmIr4jQ2V6WutN7M6Qw77Z8Jogfr+r+HdM8z6fIed7Vg2GBHOTz+8/XBURZjxcsJEYD8PsREqkY+zWq5f5X1efFvrO40KNuLoJO+BaLmR3ZUf9jeiYdvcxkv3MV7Ozyssog5iG9nBgw94yMxMIRwBla42lgfDix3XUH+PtZy6iUY0Y+rvE7EEntloHqOYyYTuTpDSqZIHI/CUNAR2FoLWkT6QgFUDkSUBZ6Owre68pEGHXMbRZdVobqfTPS30+OK+2yzBamfW2aOhq1+XxmXADexgyTSXNmhaJXPglqDaAEUA05jiEIjUmeAIYE+pUqSxHtePBrZl9z5UzhM9kLNYK7ex8U1CcYFjBwo93M1dBzHbU35p51/IV8gGrl3cafgGr8JqiuUqc4XHOReE7KcikP1b1deEbizCskHQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, 30 May 2025 10:25:01 -0400 Peter Xu wrote: > On Fri, May 30, 2025 at 10:10:50AM -0300, Jason Gunthorpe wrote: > Probably due to aac6db75a9fc vfio/pci: Use unmap_mapping_range(). Ack. > > I think this is something we have missed. VFIO should automatically > > align the VMA's address if not MAP_FIXED, otherwise it can't use the > > efficient huge page sizes anymore. qemu uses MAP_FIXED so we've left > > out the non-qemu users from this performance optimization. Thanks for confirming. > Good point! I overlooked the VA hints when QEMU doesn't need it. I ca= n > have a closer look if nobody else will. This would be appreciated -- thank you! > > I think if you are mmaping a huge huge BAR it is not surprising that > > it will take a huge amount of time to write out all of the 4K > > PTEs.=20 Agreed. This matches what we observed. > I think if your trace shows correct huge faults when you did correct > alignment, it should mean it doesn't affect your case (likely your app > sequentially fault in the bar region. Yes, this is the faulting triggered by the call stack below, downstream f= rom VFIO_IOMMU_MAP_DMA, which faults in the entire VA range to be mapped. vfio_pci_mmap_huge_fault+0xf5/0x1b0 [vfio_pci_core] __do_fault+0x3f/0x130 do_pte_missing+0x363/0xf40 handle_mm_fault+0x6d2/0x1200 fixup_user_fault+0x121/0x280 vaddr_get_pfns+0x185/0x3c0 [vfio_iommu_type1] vfio_pin_pages_remote+0x1a1/0x590 [vfio_iommu_type1] vfio_pin_map_dma+0xe6/0x2c0 [vfio_iommu_type1] vfio_iommu_type1_ioctl+0xd32/0xea0 [vfio_iommu_type1] I also confirmed that cherry picking "vfio/pci: Align huge faults to orde= r" does not affect our usage of this path (manual mmap alignment is still required). Thanks, Alex Mastro