From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6385FC71136 for ; Mon, 16 Jun 2025 22:06:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D15B6B0089; Mon, 16 Jun 2025 18:06:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 681F46B008A; Mon, 16 Jun 2025 18:06:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5709F6B0092; Mon, 16 Jun 2025 18:06:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 418A46B0089 for ; Mon, 16 Jun 2025 18:06:36 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C1EA1120FEA for ; Mon, 16 Jun 2025 22:06:35 +0000 (UTC) X-FDA: 83562648750.13.52D81B6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf13.hostedemail.com (Postfix) with ESMTP id 6DC582000E for ; Mon, 16 Jun 2025 22:06:32 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="QcXu/Ta5"; spf=pass (imf13.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750111592; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dFHYfgXcZKBPfdb0AbYsNeZbCn3z17biLchrL/Mt8bQ=; b=HxCq3dr5wpQon0N4O1qOv7sCPmNVJqQwSQYJjaj8plwskvoT9F6r7SV+u3BKd3nUJqUT0K KvuqdDmJv7HM7X0b1a5aoPu2cL/Jle9i18zLZ2b0zkrHjSLOJ/ToNNvfOAcrkOB9QAYqhi 96BUrxg0ej/U7seIcw5x4eqsTYSw1bE= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="QcXu/Ta5"; spf=pass (imf13.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750111592; a=rsa-sha256; cv=none; b=C+1f9jp+59Qv77l9qUUM8cz4jlMQgBSyawCMgZ9TIz2I1RtSDs0WvZSK0EDdzNn9q2vTPX zR8Q0r5xdUoQ8o4enF2gF5kFT6lSces4G60NCtu5hC7XqFRjTDE2r1e8xRZCl3FblSeohe MVPftY7qOhNvnkJz1VPLYSwRlosdLiE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750111591; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=dFHYfgXcZKBPfdb0AbYsNeZbCn3z17biLchrL/Mt8bQ=; b=QcXu/Ta56+A+QpZp2Q8/ficAOzDy7xatfJqfsVdk5phfTkAatYnA1gSVLq6R+xTbAldVpy gnrpyg8b0mXx3cKyQRwm+ucDk+NwfJNKuRVNPUIc6jS8QJxF7FLo0JEOwgRuC09W9pumTw QT7X2VunOSCjJyX0E4ANy77qQfKl0oA= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-639-wr6BvRvEOliX86vmhiz0Pw-1; Mon, 16 Jun 2025 18:06:30 -0400 X-MC-Unique: wr6BvRvEOliX86vmhiz0Pw-1 X-Mimecast-MFC-AGG-ID: wr6BvRvEOliX86vmhiz0Pw_1750111590 Received: by mail-qk1-f198.google.com with SMTP id af79cd13be357-7d09bc05b77so826171885a.1 for ; Mon, 16 Jun 2025 15:06:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750111590; x=1750716390; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=dFHYfgXcZKBPfdb0AbYsNeZbCn3z17biLchrL/Mt8bQ=; b=aHbsjsv0EyfbV8UDXJBNicrQzNoFJk6ZCeltVkbKk3l6bYQT01r9TjNYPUNjTVVDl+ 5nhoWGXVjf053C4OP3rce+GyPqCG4nXMoth2C8584pSgm23EZtISsiYvh4X7O63opzFw fuNgafEATrL5vSZho1z0GV9E1uVJUnxkLuAtKCq9gEpVci2O87XdCRi0IlK+z7k/KwMj DfJt52TQLYG3q6BGTWqKBi94LvgIAzvDFqaeR7UEFMAyqC7+GAAk+hb0ED0Tvvbx2YeW zwYtQhVNgoNZ6Uf4IL74Gt5iKX8GyaprhV3Xp5U2Lycq8I/8dd+YruGgJ3J97caw6cqX QkCw== X-Forwarded-Encrypted: i=1; AJvYcCWWPhmKiC1W+04J3+yEkd43id/i3MF505+LT3DK08jwFCC931p2BlOgkNMsysqzyn3YyYwbNqRITA==@kvack.org X-Gm-Message-State: AOJu0YxhS8SMywryyEFNjeJOrnVZyvNxa38nWvTWJU2ZLE1GgvUi2RmF 4nJMXk/30QuHOQ409TNPQxAbDrdMhU/dCw1MefTIn66FFXxUbd+LztYWAeI5ZT8Gw+yd8LaKSpy E0TNKtAY3IF7VW65LHNABbcv3P/bz+gOW159+sER/ahmCJz1gA7mv X-Gm-Gg: ASbGnct/Drw+5BHE7P8+dUm2DCBzoAcjCV2F0n3ENadfP04qHDcIvru5wjYvfjzkidb 0BWDTBDczNJTR+56tkR9VmX0dmxAifw3v3v4WeLGBB8Mt+FQ/BnJdRF5fGTkiypfUOAKocBrQMo +949bIngBkDV1pYPU2S/lPuk22XI7DGBnztURXcSw690yyOtks9VmTzwX0XidbIe/kKhriEr48E wDBeXEo2tZaptpMGeyiuqxdMt/LfOFBLcRAG5LN4Yz2BdsYGbwsSQchefXRIFUbIeiOk9QeNFeu V3It+Aok6oluDg== X-Received: by 2002:a05:620a:2a0f:b0:7d0:9f1e:40dc with SMTP id af79cd13be357-7d3c6d0caf4mr1880930585a.56.1750111589982; Mon, 16 Jun 2025 15:06:29 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHb2JRUDP2T8baEWYp/vOc4digV3ytuAMVFRb0HHQswTS/EU4/mjLVkZD74bYg3SYHshW7vyQ== X-Received: by 2002:a05:620a:2a0f:b0:7d0:9f1e:40dc with SMTP id af79cd13be357-7d3c6d0caf4mr1880926685a.56.1750111589617; Mon, 16 Jun 2025 15:06:29 -0700 (PDT) Received: from x1.local ([85.131.185.92]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7d3b8dfe347sm576951985a.36.2025.06.16.15.06.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Jun 2025 15:06:28 -0700 (PDT) Date: Mon, 16 Jun 2025 18:06:23 -0400 From: Peter Xu To: Jason Gunthorpe Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kvm@vger.kernel.org, Andrew Morton , Alex Williamson , Zi Yan , Alex Mastro , David Hildenbrand , Nico Pache Subject: Re: [PATCH 5/5] vfio-pci: Best-effort huge pfnmaps with !MAP_FIXED mappings Message-ID: References: <20250613134111.469884-1-peterx@redhat.com> <20250613134111.469884-6-peterx@redhat.com> <20250613142903.GL1174925@nvidia.com> <20250613160956.GN1174925@nvidia.com> <20250613231657.GO1174925@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20250613231657.GO1174925@nvidia.com> X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: ZCgqTTCA5tNkUunSGUMeKB7ut2pmotQjsO5GUlT9K_s_1750111590 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6DC582000E X-Stat-Signature: xizy1xsp8mo34tupap77xxzbt5obht4k X-Rspam-User: X-HE-Tag: 1750111592-212309 X-HE-Meta: U2FsdGVkX19KbhtJELsTitfliwRbRSjHLhXPicjP7sD8Z4aRy1KTy4uQaVET7ibWmeFyrpDz6zt9iirTIjdvzVNhZl4XXUAVNNHuzFhn5Jt1a1K/5h5ObYfxGOhX4FIH6nlo0xMeSY6nrl0qXLYsb/gYnGSN4oRafpdoDOSTLvm5oSV9dhiGVKDITMhzsp/1HI8B04dR3QIM1nIOHELWiDMoqX+hVpmV8lLy+ZlcaBIeVCJlpq2IkNl8PfGbBXAUUUl4tTEjVQFrM+goSguAQ8J76GzkRxBueJwwoVC8sgdIJ5n8uw2O+Vw3RejDmKBTM0pqgEKHq0kVlR4994DRqZsjh1h18swmZLhcGZk9TtrEgSBJWHVrdzT3H0Nn7+yfobEXSCd19XRAvhkB7Me2ZNEXxYDqWY6aQyG79Av42vDPs+LnT8tXAfC/TTdpBs+ZGp6H5O1PGypmqVLcoBj0i3akZqDRPtcqBfh6WXjigv/7NCmU/8KvtXML2C9bLLR7mWcQyjoL4ShslkdsqbWhaX7jmnmn2wxISNlaVu+d0xliwJztdKw0AJ9OkwVmBogUiyOpMQ1+pBh0OPnCY1cbe/JgBhnHsTPzH/3bXTdwUpOOv/mpzZa4sjH9IumfUiiZRw/oL2Z/ZzwVUkEK/EtU7tNQM/04yydE/yBfkD0wnHHi+/QCw9Rgjfp6Rp2E+H/cHXR1hWAlDL3x+jh7wPE68tkV6U4KoDvQEc693gqxW12qg0hr3hgKNQfbJJB7kxu/K4QEv8PZu1O3QykdDIQw/YDb5yHCLF13f/cTxTF03uUuta8kyKG7MfMeanHjF283tF1k8yjeQ/AmAnnri55b9L4/vVm83SVO+4OBo5ExlETawIWyBmb40uP4qOyE6jU99SXpVGi84LifeAazEwDIkpVPICWUM3jcwje2yH/mUzcK+d4ALlwDqNk87CazA+YPpTMsgKBbwKXzSPMb9QJ hAdGFdCX WZrce1qqT3qgwNUoYl1ncWQXajuE1H9qbEB2kZqeKbC80AuSfoxY3G3fD2x8wmzuej2tVbxeJAYZgnaT8JNgVrVY7dQwNgoqVx6waEEQHIyWkXxwJKnQfDavgQiZ3r+xUKt/+IdpsiXyqMa02JEisDxVGvDx/J/bBGZTkxbxoXJIwToJ7oNd5k5CfPvcO/ln6XztSstLv6uAMqTNGIw8i4NbgSogkLIzgeXTtqpUjHuvuciM/IqWY50TuxuKFAjBBIbYSpwDxNd1MdvHpWWvro83/MIZLf6s+qY9FBOsAsLey4rP+d72tHl2yKG3L+pO05DTIj0azZWEbeHj5v0REkY/IsOhmoyIlN8oR+tqzr+zHFJcNLgfPUQAYA3NNlq5id5+fvStQV9+vHiWT/FuuDPHOXGf0YGevZRg4AqpgetbPQ+U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jun 13, 2025 at 08:16:57PM -0300, Jason Gunthorpe wrote: > On Fri, Jun 13, 2025 at 03:15:19PM -0400, Peter Xu wrote: > > > > > > + if (phys_len >= PMD_SIZE) { > > > > > > + ret = mm_get_unmapped_area_aligned(file, addr, len, phys_addr, > > > > > > + flags, PMD_SIZE, 0); > > > > > > + if (ret) > > > > > > + return ret; > > > > > > + } > > > > > > > > > > Hurm, we have contiguous pages now, so PMD_SIZE is not so great, eg on > > > > > 4k ARM with we can have a 16*2M=32MB contiguity, and 16k ARM uses > > > > > contiguity to get a 32*16k=1GB option. > > > > > > > > > > Forcing to only align to the PMD or PUD seems suboptimal.. > > > > > > > > Right, however the cont-pte / cont-pmd are still not supported in huge > > > > pfnmaps in general? It'll definitely be nice if someone could look at that > > > > from ARM perspective, then provide support of both in one shot. > > > > > > Maybe leave behind a comment about this. I've been poking around if > > > somone would do the ARM PFNMAP support but can't report any commitment. > > > > I didn't know what's the best part to take a note for the whole pfnmap > > effort, but I added a note into the commit message on this patch: > > > > Note 2: Currently continuous pgtable entries (for example, cont-pte) is not > > yet supported for huge pfnmaps in general. It also is not considered in > > this patch so far. Separate work will be needed to enable continuous > > pgtable entries on archs that support it. > > > > > > > > > > > +fallback: > > > > > > + return mm_get_unmapped_area(current->mm, file, addr, len, pgoff, flags); > > > > > > > > > > Why not put this into mm_get_unmapped_area_vmflags() and get rid of > > > > > thp_get_unmapped_area_vmflags() too? > > > > > > > > > > Is there any reason the caller should have to do a retry? > > > > > > > > We would still need thp_get_unmapped_area_vmflags() because that encodes > > > > PMD_SIZE for THPs; we need the flexibility of providing any size alignment > > > > as a generic helper. > > > > > > There is only one caller for thp_get_unmapped_area_vmflags(), just > > > open code PMD_SIZE there and thin this whole thing out. It reads > > > better like that anyhow: > > > > > > } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && !file > > > && !addr /* no hint */ > > > && IS_ALIGNED(len, PMD_SIZE)) { > > > /* Ensures that larger anonymous mappings are THP aligned. */ > > > addr = mm_get_unmapped_area_aligned(file, 0, len, pgoff, > > > flags, vm_flags, PMD_SIZE); > > > > > > > That was ok, however that loses some flexibility when the caller wants to > > > > try with different alignments, exactly like above: currently, it was trying > > > > to do a first attempt of PUD mapping then fallback to PMD if that fails. > > > > > > Oh, that's a good point, I didn't notice that subtle bit. > > > > > > But then maybe that is showing the API is just wrong and the core code > > > should be trying to find the best alignment not the caller. Like we > > > can have those PUD/PMD size ifdefs inside the mm instead of in VFIO? > > > > > > VFIO would just pass the BAR size, implying the best alignment, and > > > the core implementation will try to get the largest VMA alignment that > > > snaps to an arch supported page contiguity, testing each of the arches > > > page size possibilities in turn. > > > > > > That sounds like a much better API than pushing this into drivers?? > > > > Yes it would be nice if the core mm can evolve to make supporting such > > easier. Though the question is how to pass information over to core mm. > > I was just thinking something simple, change how your new > mm_get_unmapped_area_aligned() works so that the caller is expected to > pass in the size of the biggest folio/pfn page in as > align. > > The mm_get_unmapped_area_aligned() returns a vm address that > will result in large mappings. > > pgoff works the same way, the assumption is the biggest folio is at > pgoff 0 and followed by another biggest folio so the pgoff logic tries > to make the second folio map fully. > > ie what a hugetlb fd or thp memfd would like. > > Then you still hook the file operations and still figure out what BAR > and so on to call mm_get_unmapped_area_aligned() with the correct > aligned parameter. > > mm_get_unmapped_area_aligned() goes through the supported page sizes > of the arch and selects the best one for the indicated biggest folio > > If we were happy writing this in vfio then it can work just as well in > the core mm side. So far, the new vfio_pci_core_get_unmapped_area() almost does VFIO's own stuff, except that it does retry with different sizes. Can I understand it as a suggestion to pass in a bitmask into the core mm API (e.g. keep the name of mm_get_unmapped_area_aligned()), instead of a constant "align", so that core mm would try to allocate from the largest size to smaller until it finds some working VA to use? > > > It's similar to many other use cases of get_unmapped_area() users. For > > example, see v4l2_m2m_get_unmapped_area() which has similar treatment on at > > least knowing which part of the file was being mapped: > > > > if (offset < DST_QUEUE_OFF_BASE) { > > vq = v4l2_m2m_get_src_vq(fh->m2m_ctx); > > } else { > > vq = v4l2_m2m_get_dst_vq(fh->m2m_ctx); > > pgoff -= (DST_QUEUE_OFF_BASE >> PAGE_SHIFT); > > } > > Careful thats only use for nommu :) My fault, please ignore it.. :) I'm also surprised it is even available for !MMU.. but I decided to not dig anymore today on that. Thanks, -- Peter Xu