From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73C72C0219B for ; Fri, 7 Feb 2025 01:39:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E07FC280002; Thu, 6 Feb 2025 20:39:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DB789280001; Thu, 6 Feb 2025 20:39:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C624A280002; Thu, 6 Feb 2025 20:39:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A374F280001 for ; Thu, 6 Feb 2025 20:39:19 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 1AC771A07F1 for ; Fri, 7 Feb 2025 01:39:19 +0000 (UTC) X-FDA: 83091440838.24.804581F Received: from smtp-relay-internal-1.canonical.com (smtp-relay-internal-1.canonical.com [185.125.188.123]) by imf29.hostedemail.com (Postfix) with ESMTP id E7A5F120013 for ; Fri, 7 Feb 2025 01:39:16 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=canonical.com header.s=20210705 header.b=jWTOyRgf; dmarc=pass (policy=none) header.from=canonical.com; spf=pass (imf29.hostedemail.com: domain of mitchell.augustin@canonical.com designates 185.125.188.123 as permitted sender) smtp.mailfrom=mitchell.augustin@canonical.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738892357; a=rsa-sha256; cv=none; b=N7cHjBzAm3EycCxsYTg8gIqCyRn0SJu8CaOP6vdjATje111AyAisd6udajqxjGQxWQUuXM qMijDgP+y68T3r0zGMvJpPklMHzCAHuc9nZ3KFL73gyVeevQFyPu+vyKX6zDPtwac+asgj xd8djjrYWtbCq4ST+6i4oSTqCrCX+yc= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=canonical.com header.s=20210705 header.b=jWTOyRgf; dmarc=pass (policy=none) header.from=canonical.com; spf=pass (imf29.hostedemail.com: domain of mitchell.augustin@canonical.com designates 185.125.188.123 as permitted sender) smtp.mailfrom=mitchell.augustin@canonical.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738892357; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fNB47AZskNaVXFROFOsVtdHZC/VEBES+Tul1gkugFo8=; b=LVTv9iQq8SZAnhc0VWbyyjZl0/8rnJyV99puaM6Hg92aI/f5uaGb3vckxOI/bQCL2gJwFH dcDyf8t+OlvghJRkUN1TV3IZeUdUiahMe7qEP6zN045+fJGMFdtMe5KJDWFsMtQsryNlty LUKiUKsncENiNlexZJpWL/+dcEv4Uxc= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 682CE3F212 for ; Fri, 7 Feb 2025 01:39:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1738892353; bh=fNB47AZskNaVXFROFOsVtdHZC/VEBES+Tul1gkugFo8=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=jWTOyRgf5z8NuhgTgkIEj8/tkREgNduQKQlDoROR6i8xWfz9W4S0pDDyHQ+C3VKF9 9e8YJm50eTACuJN4/8uTpP3qx8duHEphbticQqxo0rCfWk6k3E7pJDn6D27AZq42oX hBVPX+tacvD8KzX8kA3A5HJlPkQaIMsh/jtqpdolyBUoQ8Z7ZYUie7w30ZorCWXOZw /ic3NbtOLpK9+DVA1/5OJ6UnYQe4Xk8B8Bi6PfOOIWWt2/EeTePPEQ/KAYppq2VJOD M/FhsCQ04G9ztGgJzD/Co0ciVhsviD16w1ZtvqFOsz5GoSc3tohNSYuujpVhkDj/9N 3enKDDekEQM8Q== Received: by mail-ed1-f72.google.com with SMTP id 4fb4d7f45d1cf-5de4a465673so111734a12.2 for ; Thu, 06 Feb 2025 17:39:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738892352; x=1739497152; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fNB47AZskNaVXFROFOsVtdHZC/VEBES+Tul1gkugFo8=; b=FkbTjAjWSfYTGP1dx4MBUvIY7t9QMXIK6OzENeoTnb/zHTZ+TChxXEawGC5ELF3sBI hDEcG0XyqnbJNs4gTyubj0fsU8ph+0zEiYuGlKN0TyyjKNRkoI6MHoi88zbYDpN0SKn7 lsY/kdS4L0xepOOQxO5CtFraJXDauJcP9VamhH0b/LCs6aCfWb0RaAZNTCu828bYJZD9 LT84ua87Yg1KGhS889lpfpX6Z6G6/WqbrAGPUR1aJ6kj8GpGLai3Tty9REbD2iMZEcTC s7E/EqT9UWdAdCUJrizrHrL0ElC+CuJnlFu9txm8WiDNM1X5Z40bs8CyF5XnbLGQ94G8 DXVA== X-Forwarded-Encrypted: i=1; AJvYcCWhkPYxfGz6E5sVve6XI3ogpusAPVSW7PJKMuUFMQh4CruhNJ/2MfTnsYgR5VfnS6xNVF8oYwRAoQ==@kvack.org X-Gm-Message-State: AOJu0YxVzRY9AIOCHgsjL7h0YRQQ4gccqCsFc6daeFUKPm9MrNb7yU9g UgdqWnbPQIgXXH5/OISfV2OVCkWYCmUe+Szuz84td/sJLT9A7A3buHIC+yI5GaNkzw/Uub3cDys sQVLtQzyvDuYEyAmXoYZ9Y54KuZqIION5MqzG2vT9TIbhyHgQt091j9C8SDIJfPT5Q9T85LH3h4 VBaUc4dWO4gUQt1U4jWpoMLclbZUE8c1V2TRFoYfg= X-Gm-Gg: ASbGnctK3IOgZK07LBJdJh5OI5TvfREfZsdFSh949hNHplTb28NUpnU3ynCFXxAYO7h xybKrrMgV2JHoaYdoZBzTezCDD27PirocUB6NEzFPBnbLZpp5WKo4EaQWXTKW X-Received: by 2002:a05:6402:1ecf:b0:5dc:1395:1d3a with SMTP id 4fb4d7f45d1cf-5de45040136mr1224807a12.1.1738892352092; Thu, 06 Feb 2025 17:39:12 -0800 (PST) X-Google-Smtp-Source: AGHT+IF/hfeNzW0PLF4wIrEEYadVAd/ZpRfylvoon43qb7nTnRX2cUuBVjBJ/j4A/WtVm/FBP3qMLamc0NsqBBkMXCE= X-Received: by 2002:a05:6402:1ecf:b0:5dc:1395:1d3a with SMTP id 4fb4d7f45d1cf-5de45040136mr1224796a12.1.1738892351707; Thu, 06 Feb 2025 17:39:11 -0800 (PST) MIME-Version: 1.0 References: <20250205231728.2527186-1-alex.williamson@redhat.com> <20250205231728.2527186-6-alex.williamson@redhat.com> In-Reply-To: <20250205231728.2527186-6-alex.williamson@redhat.com> From: Mitchell Augustin Date: Thu, 6 Feb 2025 19:39:00 -0600 X-Gm-Features: AWEUYZm-8N3C_RogZQE-LMB_5GAV_JU3d6aW2EE3P7c5fBbLXqIJWAR7Ls5C54k Message-ID: Subject: Re: [PATCH 5/5] vfio/type1: Use mapping page mask for pfnmaps To: Alex Williamson Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, peterx@redhat.com, clg@redhat.com, akpm@linux-foundation.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: E7A5F120013 X-Stat-Signature: qnaxmbak55fepwobdg3qawk83ftrsbc8 X-Rspam-User: X-HE-Tag: 1738892356-930234 X-HE-Meta: U2FsdGVkX18D6TuP3WFDwYmRTgQ1WhdxSplCHAk8fPlzQbZD1NmUEyMs7LwCxoNYa1Mx/D+LmTC3+OQnZ70tMWBE89SDwpbnKG3iJTh+KOt4rYj9nQ70/OmeHxMBJexdru8Fi/63l1vdMnA/5PTmotk9Qdypyka7cpCMkAb74j777vyAlu9OnUYRVcYLyBaZ25MBUhCQITcmOuQgKD5mvI038HXJu9upJl5EET64/3DzEO3NNWlG2FWpVQYgr7Ss5Qn7BvpmJh0dg8wotWPD74l7TQZNBAkxrQqIFr+lGdk1oGroQ+zmFpQ4ORa1t56Ve5X+VMykr1n2MyRwtjF9yop59OIzAKNdbdtbr6zM36ajlf3VBwDcmR5p2RUY/B0IP/q7yZrjoG5mOqhDgsylw9gdqN2+VcCjzyJTvVqldw0vtqDRgMEMAxAg4LDI5Im59luE1yLOB1eHjez0oeWVHF09hX4yMXD+Sfpe1fIBD0Et2BSgJR0uLzuQhj+B083sl8pmwUh6gzh0weKMmqgLQD4Ql2AF8NzOOtfY6KJ3Yc/2jGwphW8l3AQfqbEeX9Os0Uh/cEdqkY9q7s4qxOTnndyh4NxVXB24hW8vyH3E/8ny5QCjj58GRWeywr2GvR2vyDNprUtDmx5wkZqu/rL6u+u3WAHEHZ95AvxVBkAQVmpuvV8STl8VqCQG/wrl3OY+lLki3qESefBYS2pFv3E2LtAPLBMCdxGi7nJr1BObTmTpdV2+H3v+DB36t03gJUlR15j3IXleCTys1JRZvRYxJEp3f/oXbF7sVr+J2m276zoWw31lR+2T5dN20Z3mKQUmglm6jMHiNQRu8YnSLkNMwJNBzVoBc3j8T5INXEC42i+b98eN86D5Hn2rAUSAii0NQZ30o8GNqZ2vJdec1UA3TU6KOfjK81R2Cj7kRUMvaJSu4eH/UgDJmhEN2FrQtQ2ms7xeVabjFGWXgqqqRdc w/r91Z8s 8Gccu2R2EWYCSy6PXdP6pGSWXZwIW56WB0VVdpvdyW+8ZeZQpT8QO1cRe8Iz/dJWqidcWlq56OSxLgUANfN3bxPLnN87NPRpRq27VQK4NOOiK88uQjhrfstGt1BC1NFiFlgzH8weECE8zA7VKsPDNKXjqiqbRdjXEhxS3KjBfexTPpJFwt2brJWy1//OIZvLtlnvsdaB186YVYT6Qkk74U6cqwA+a4DtFmRqTAkmwLyqyvDsfhG9vdlEp9tUIZ9gC99GXuQalXZtSqyCBkfWeAlatAgLC5RXV/gIlbIyYlg/Jkz74ZjnkzuatZjNJ7jNpIfqn4ZRF+Tx9QwQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000085, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: LGTM and completely eliminates guest VM PCI initialization slowdowns on H100 and A100. Also not seeing any obvious regressions on my side. Reported-by: "Mitchell Augustin" Reviewed-by: "Mitchell Augustin" Tested-by: "Mitchell Augustin" On Wed, Feb 5, 2025 at 5:18=E2=80=AFPM Alex Williamson wrote: > > vfio-pci supports huge_fault for PCI MMIO BARs and will insert pud and > pmd mappings for well aligned mappings. follow_pfnmap_start() walks the > page table and therefore knows the page mask of the level where the > address is found and returns this through follow_pfnmap_args.pgmask. > Subsequent pfns from this address until the end of the mapping page are > necessarily consecutive. Use this information to retrieve a range of > pfnmap pfns in a single pass. > > With optimal mappings and alignment on systems with 1GB pud and 4KB > page size, this reduces iterations for DMA mapping PCI BARs by a > factor of 256K. In real world testing, the overhead of iterating > pfns for a VM DMA mapping a 32GB PCI BAR is reduced from ~1s to > sub-millisecond overhead. > > Signed-off-by: Alex Williamson > --- > drivers/vfio/vfio_iommu_type1.c | 24 +++++++++++++++++------- > 1 file changed, 17 insertions(+), 7 deletions(-) > > diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_ty= pe1.c > index 939920454da7..6f3e8d981311 100644 > --- a/drivers/vfio/vfio_iommu_type1.c > +++ b/drivers/vfio/vfio_iommu_type1.c > @@ -520,7 +520,7 @@ static void vfio_batch_fini(struct vfio_batch *batch) > > static int follow_fault_pfn(struct vm_area_struct *vma, struct mm_struct= *mm, > unsigned long vaddr, unsigned long *pfn, > - bool write_fault) > + unsigned long *pgmask, bool write_fault) > { > struct follow_pfnmap_args args =3D { .vma =3D vma, .address =3D v= addr }; > int ret; > @@ -544,10 +544,12 @@ static int follow_fault_pfn(struct vm_area_struct *= vma, struct mm_struct *mm, > return ret; > } > > - if (write_fault && !args.writable) > + if (write_fault && !args.writable) { > ret =3D -EFAULT; > - else > + } else { > *pfn =3D args.pfn; > + *pgmask =3D args.pgmask; > + } > > follow_pfnmap_end(&args); > return ret; > @@ -590,15 +592,23 @@ static int vaddr_get_pfns(struct mm_struct *mm, uns= igned long vaddr, > vma =3D vma_lookup(mm, vaddr); > > if (vma && vma->vm_flags & VM_PFNMAP) { > - ret =3D follow_fault_pfn(vma, mm, vaddr, pfn, prot & IOMM= U_WRITE); > + unsigned long pgmask; > + > + ret =3D follow_fault_pfn(vma, mm, vaddr, pfn, &pgmask, > + prot & IOMMU_WRITE); > if (ret =3D=3D -EAGAIN) > goto retry; > > if (!ret) { > - if (is_invalid_reserved_pfn(*pfn)) > - ret =3D 1; > - else > + if (is_invalid_reserved_pfn(*pfn)) { > + unsigned long epfn; > + > + epfn =3D (((*pfn << PAGE_SHIFT) + ~pgmask= + 1) > + & pgmask) >> PAGE_SHIFT; > + ret =3D min_t(int, npages, epfn - *pfn); > + } else { > ret =3D -EFAULT; > + } > } > } > done: > -- > 2.47.1 > -- Mitchell Augustin Software Engineer - Ubuntu Partner Engineering