From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 628BFC02192 for ; Wed, 5 Feb 2025 23:18:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F32C76B0095; Wed, 5 Feb 2025 18:18:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EE41F6B0096; Wed, 5 Feb 2025 18:18:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DAAD76B0099; Wed, 5 Feb 2025 18:18:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B9DB26B0095 for ; Wed, 5 Feb 2025 18:18:23 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 61C8180D29 for ; Wed, 5 Feb 2025 23:18:23 +0000 (UTC) X-FDA: 83087456886.17.F431A61 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf12.hostedemail.com (Postfix) with ESMTP id AA81E40003 for ; Wed, 5 Feb 2025 23:18:21 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KhGSm9ji; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf12.hostedemail.com: domain of alex.williamson@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=alex.williamson@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738797501; a=rsa-sha256; cv=none; b=pIdA2EK+N3vjRzAzNaSWTUFjYZdIkUiqGU4oXVItvkytpWCL2o10Mlo43fHRifCrN3PsJg KoiquZs2zRIi9mYgUfWd1mgGi8mHE5/UTN3yfuP1qwrkyGVhS1aHldX4Z0JosCd7GPSs8K +snV8s7EzVG/fsPv2Uf8/Jrs9HccHUM= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KhGSm9ji; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf12.hostedemail.com: domain of alex.williamson@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=alex.williamson@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738797501; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8W4JP+Fju0ACXLthBx+3G3GONe8TLTlsBTTA1Q3RDZE=; b=qYC2XLUnL85tunL5/2w/tkl5NYwbqY3sJyxIqxjzYACrop0ZqHIhKw/aj5zQez0NTE9iCP Yo1cu/yNtGgX6l/7QUeDqVnYmsWyUXumgMLABCwbqCOBeCn2kjomHzC8OtUOAoO255hoiG eDwWyPVPboACg6pmAU9gxY0bzTtnoEg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738797501; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8W4JP+Fju0ACXLthBx+3G3GONe8TLTlsBTTA1Q3RDZE=; b=KhGSm9jiAYbc9klwd8G76Z8RtmqqTI4FiDwnPisBWqlosWWihrX62nQQXKBdvtxUxQQgO6 MQkUMeYkh8ufp4csvAcYPnOdwV9x1DcUGun+zo8QBnM7btsPHb4qLaZLrPAPXF00csBFmU MSUB5wFdZk2/Hy5IGYywyN9Ht1BTWVE= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-561-FkNDRqmMM9yE_UzeBVYLPQ-1; Wed, 05 Feb 2025 18:18:17 -0500 X-MC-Unique: FkNDRqmMM9yE_UzeBVYLPQ-1 X-Mimecast-MFC-AGG-ID: FkNDRqmMM9yE_UzeBVYLPQ Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 871FF1800985; Wed, 5 Feb 2025 23:18:16 +0000 (UTC) Received: from omen.home.shazbot.org (unknown [10.22.81.141]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9D68A1800570; Wed, 5 Feb 2025 23:18:14 +0000 (UTC) From: Alex Williamson To: alex.williamson@redhat.com Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, peterx@redhat.com, mitchell.augustin@canonical.com, clg@redhat.com, akpm@linux-foundation.org, linux-mm@kvack.org Subject: [PATCH 4/5] mm: Provide page mask in struct follow_pfnmap_args Date: Wed, 5 Feb 2025 16:17:20 -0700 Message-ID: <20250205231728.2527186-5-alex.williamson@redhat.com> In-Reply-To: <20250205231728.2527186-1-alex.williamson@redhat.com> References: <20250205231728.2527186-1-alex.williamson@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: AA81E40003 X-Stat-Signature: ikxomcjdnwftds931woad6uayxe3szer X-HE-Tag: 1738797501-291336 X-HE-Meta: U2FsdGVkX1/xLcZkJ8+LGr16maYBmX5TOvcQ0JjVs0I0RmK7+ep0aZdGAAgdrfgH3IbuQsbP/1zyMlgJOP2p0ZdicyE06xavhbpn8GamedSPpk2o/1O2f0OsVnDqa4sv81yDJkjnYZXbO2Zsa9EokA+C8og3C6w0dK1eGbSS6ToAIr72a2R9h9xYKhXw+jIyeGtuzPMg/+jTHzDaWzsmkjhCWFLQUQLlRyMvUi92j55Fy5oFSoZS2rgbvjJ7OcqMbYuVK4bO0hvyHZpRORc3GUeI93MKuz3I+6cpYYqSGaLvAWPluAUdptkGQwJOIpcOeaNrKhXNUAjihtcutwc2B3lJVgH7gImR2EtV8EqnqvgofOvue/VB9vRoc+YDE2sK/ADVzvpWfEAfkhce35xqKcLZbY1S1pCnV9xfUc4bngP/9f1ilCD179P3e17Cq8Y2RYWQGpEUhVbMaEkPia8VoyXZgrEp5NVvim9kDUA/w3emlxbecsIRVI/brOsy1ffIxiesPdovIvXGgT2ljjWHxEl1taenw0C1GEmPwIh3ezD7CzeYHmK8SiuuTxR0JtRcyL5olz921vM/Vwvzu5eHBXXWL/NrvyEcroV7y72X0d9KSIDYF0rXuhnuEoqIDKkXool+zhzYP9f2XSCiqr5dqmAbXoEfEQnpz/DCpLEg5CIvMC4WGdbcfjYqQsecTwD1y5bCxu4bhZLD9D5lejHgmHrTaoNy2M1uhQhl6XD08Z1A+0eHw7I+Q5wC3FrsZTi/+7/cBOmtMehS03Rslum6+HFIENF/BAcyr5cPiQ8cPxB1MIkrkFwr3vA+BTivpUWFxwBrQBDiBHI7IZjahEi4p3Xr4S86J1spCOzY03oZoaPJyqMbcfiyF8rgmHjWELM80JAn5ooR9PsPLQtpb4xiL9pGXSXDShKfhIVv/bEXdm/g3F4fygTRoYICLv7je3aR8TmpW3/AiQE1eJ3Tyht HjPXyMeF kW4fI/DkiTyzxRoXe80YJVCfTyKAQL1mfNS8CbS7uSVTSTPyY6qy3vXUjpIRaVZkjuirON1+A1ApXGivUpDbt1Gig4QChiCfh9F8R+hj13asJBeLWgRxotD2rRR82CZtUdht3oHMSR0MiqDPI7ZvBbyERJRcQrdi9aoMuZZG4QEkwReJ2xEVgQ/DC9BzOA7AG/qTgi+LB1XbvdvUid0J+u/W3+si0ZPDlh10XvN+ktyZKXbrNBgwXe3bBWNB3bF2D9PHtEAWB7gSMHkAfNNoFxNHHjsKmUUdt2lK1AnP8TUBw/B8/egXRDksAYkQ+lWhelgCx4ueadMDxZH2FoXOhHtf70AYXsRJsSdPekr1IvK+zX0hg9EbGVMHFcoMGFpJPXRbD5TySPpaMaGNK+avkVX8rXb0VFuhZOXC4czvRnxQpr6SBGLZGpl5QaXRucX2ddlCk92KgoO4ACqXEU1VxupjbllBkaxZD7k2hLfeXjK5MmaE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000373, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: follow_pfnmap_start() walks the page table for a given address and fills out the struct follow_pfnmap_args in pfnmap_args_setup(). The page mask of the page table level is already provided to this latter function for calculating the pfn. This page mask can also be useful for the caller to determine the extent of the contiguous mapping. For example, vfio-pci now supports huge_fault for pfnmaps and is able to insert pud and pmd mappings. When we DMA map these pfnmaps, ex. PCI MMIO BARs, we iterate follow_pfnmap_start() to get each pfn to test for a contiguous pfn range. Providing the mapping page mask allows us to skip the extent of the mapping level. Assuming a 1GB pud level and 4KB page size, iterations are reduced by a factor of 256K. In wall clock time, mapping a 32GB PCI BAR is reduced from ~1s to <1ms. Cc: Andrew Morton Cc: linux-mm@kvack.org Signed-off-by: Alex Williamson --- include/linux/mm.h | 2 ++ mm/memory.c | 1 + 2 files changed, 3 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index b1c3db9cf355..0ef7e7a0b4eb 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2416,11 +2416,13 @@ struct follow_pfnmap_args { * Outputs: * * @pfn: the PFN of the address + * @pgmask: page mask covering pfn * @pgprot: the pgprot_t of the mapping * @writable: whether the mapping is writable * @special: whether the mapping is a special mapping (real PFN maps) */ unsigned long pfn; + unsigned long pgmask; pgprot_t pgprot; bool writable; bool special; diff --git a/mm/memory.c b/mm/memory.c index 398c031be9ba..97ccd43761b2 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6388,6 +6388,7 @@ static inline void pfnmap_args_setup(struct follow_pfnmap_args *args, args->lock = lock; args->ptep = ptep; args->pfn = pfn_base + ((args->address & ~addr_mask) >> PAGE_SHIFT); + args->pgmask = addr_mask; args->pgprot = pgprot; args->writable = writable; args->special = special; -- 2.47.1