From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70314C47073 for ; Thu, 4 Jan 2024 07:48:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EBC7D6B03AC; Thu, 4 Jan 2024 02:48:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E446C6B03AD; Thu, 4 Jan 2024 02:48:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CBE816B03AE; Thu, 4 Jan 2024 02:48:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B6A7A6B03AC for ; Thu, 4 Jan 2024 02:48:55 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 736E9160835 for ; Thu, 4 Jan 2024 07:48:55 +0000 (UTC) X-FDA: 81640852230.16.4414721 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.43]) by imf12.hostedemail.com (Postfix) with ESMTP id 0CADF40002 for ; Thu, 4 Jan 2024 07:48:51 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=KZkDh4dE; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf12.hostedemail.com: domain of vivek.kasireddy@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704354533; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=W7Rle4tMEwr/oqkgPy75tWLWxWnAIjBU2h1EbBHzr8I=; b=ZtfbTOatYExSCdw1rH7OiBGpFa8Khn2P8SeMp4n2h/bmYKVn8BYKuSeM51AmbwSA31+n0N 59cSdV8huFCVYtdg+GRI6v8/tQ+kRse2qjvQbqY03ts7M6WI2UkIv3IC+glYQYN+wVlPBY 9wXMCMtenFhq96YIkrDP/ZIeqC4wN4M= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=KZkDh4dE; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf12.hostedemail.com: domain of vivek.kasireddy@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704354533; a=rsa-sha256; cv=none; b=icpDefry96NF/NuXiHqn/rV804k3zRZNnskCBnTjuDjRDepVCVEe3Ri7O3JZiaXPaw13sU S9vzZGO6+/+yx9m2C2EXA9wf44sR6RnqW+Z3TXgDIgcM9hvFQFwFiJuWPqlwML2Q9M8TyE nY4LwhO6p8l70aYzTj+f3KcESzj1fow= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704354532; x=1735890532; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=OBqHFWuwBro6StZdTiXbE/yuwHuYvPjTNtIMg9+Kngo=; b=KZkDh4dETIg4uv1rr/Onc6l0xRdD4WnDA3Nb2N0eIRfc7+Op+zISLzSR BYx2H9LMRF/0NYFcJuDT4lvCFengohMsuT3iBOptCDja13fxxn8b36Fmr BrpNUUcRWc9yyLm59v3jiY9O9Z3pkxuABYA146tJ42RVDAjj3JCxQ3oGm xEXJ7l3Uue6bG0bYooWP4erdzRxblWIzR8yrjlteNWMkMOu4GPkjFQxiS ciL3Vow4v0GV9Fg72JLzYNS7ohMRTEq/len5dOeQQ0OLE9+GzRSyiph8y fd+yVP+Izf33vwM6/1vpbVSTg6PV97bN0v5aYwkn1PzfvTFFRnP0qrwt7 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10942"; a="483331103" X-IronPort-AV: E=Sophos;i="6.04,330,1695711600"; d="scan'208";a="483331103" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jan 2024 23:48:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,330,1695711600"; d="scan'208";a="14776716" Received: from vkasired-desk2.fm.intel.com ([10.105.128.132]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jan 2024 23:48:47 -0800 From: Vivek Kasireddy To: dri-devel@lists.freedesktop.org, linux-mm@kvack.org Cc: Vivek Kasireddy , David Hildenbrand , Matthew Wilcox , Christoph Hellwig , Daniel Vetter , Mike Kravetz , Hugh Dickins , Peter Xu , Jason Gunthorpe , Gerd Hoffmann , Dongwon Kim , Junxiao Chang Subject: [PATCH v10 0/8] mm/gup: Introduce memfd_pin_folios() for pinning memfd folios (v10) Date: Wed, 3 Jan 2024 23:25:10 -0800 Message-Id: <20240104072518.807950-1-vivek.kasireddy@intel.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0CADF40002 X-Stat-Signature: hha4sy8uxwjdfany7xen9a3mqro7enwd X-Rspam-User: X-HE-Tag: 1704354531-853537 X-HE-Meta: U2FsdGVkX1+H4P46JHevo/Fmkn+o0rQRmrt9f/HyIgNyjpA7+5GiNHCUs6ItigVyCOhCzxl9K3kmMoAxtSnhAL/YJdGnnyo4FfvvTqPfnyblyOQ9IJLC1HmwaJtD/hHd/tmNAjz7TtJFvdayPEUzZk8pWZ+d9Cw1osLQLnlnxiSOOBRvURl8QUmfKHrSBBnODek7FwOZqdJNjb5lS7qHivzCrU5mFgF82Rvp5FCv4kIXbaMOLG8Bq8G0rkTuJtpLskJvSXEsewQeuQEPOuAoWxu/oCJEN8CTEAC8qHH1LLr1rpnUMAaBbWrvirO81wGPDFgPE3ET6paSS79LtAEvKj1huKbBDz0Vr3TYyB/5U1d1rWxrpa0KHiHyEo0mx4ORlvVCy3UmDGd/th0mGRQoKhyeU/dxAbghZsHbTsSpZjO0LKTaBG4E5KITLZH3zOtop+acghLl2y8N2EgWqNSSmmKRGmOKOBTSa1x7AfhrCte9ZuDsoY0TFtfVeF0DOPnquaEMeWIoU9r19Pt5zzQemaKrJVIINFqK+iniq/iElB/+0VS5QEKQMuyG9ObPGzcS8ieE4S7ATICWlqvajQIimDIVIoJGX2Ujodn9qDurL0lEZgANp5BtJ+TK0Bz2G32I8GSLrl7zuNU+q6Swq4owmGjmZHbWMp9in0ToVJ6UT+GBXW21E/Ym5ksn1v10peu5toKhbevmZW1c2Wum6wrFH+QhHJ3c5QApzWAlbjcYjp0JU2Xyd1OmFkHF4pFve9N6Zr2ZFvEq+vTLPZxbFo9ndzfTqPa/CGsp0lVhqTamrxaURBjrM6/CtNf1y6zaYjujmxbyF/4/W0klFaOD2xs9UETJbnk+Xc0UTPutz5jaDbOt+IJ/eRgvC+0V5BY+TITAqmw+ktSC1FvJ9DnlKBsTwbdWRTpUe+oCxZQEQ7JSVhmyS+cbYJCPrNbsPQRbRTf9RLHENfstdW1Dyyphe8h AC3QR+4q T/N2ahNbrRtjupUPwo5Xry+L8r1dNTHXV7Cu/4y7ZQyKi1QgaGdKK6iWZAe6p1nGIfpCmMTqzXTpR0YsR1sucuWNGRN8V+l/fSx81SiJEcvWnp4OaP2NHAczM2lPbUeV/RNDwy8Mc+3tt9pxgOxeQt70VsZXmZzeeijhsEpHtTNWZo3JeKvbfKTUtWWqrbqRXMqfxNuBrexpY0k7ak/PFH/MXHGpzlZdDMCFiB+P5hgFUv3WB3KT2G7hJE/PSI9BuzgwdB4keey4FWug8YKNxwD5Xv76rR3upeqrznsNXqY1AqUJ5A91+pIgPCjSQ6ooYSe+WN7UlnxWtlvAu6KaNqe4akkNdUeWJ9Gih8RodHaXE2C4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The first two patches were previously reviewed but not yet merged. These ones need to be merged first as the fourth patch depends on the changes introduced in them and they also fix bugs seen in very specific scenarios (running Qemu with hugetlb=on, blob=true and rebooting guest VM). Patches 3 and 4 add helpers that'd be used in subsequent patches. The 5th patch introduces memfd_pin_folios() API and the 6th one converts udmabuf driver to use folios. The 7th patch shows how the udmabuf driver can make use of the new API to longterm-pin the folios. And, the last patch adds two new udmabuf selftests to verify data coherency after potential page migration. v2: - Updated the first patch to include review feedback from David and Jason. The main change in this series is the allocation of page in the case of hugetlbfs if it is not found in the page cache. v3: - Made changes to include review feedback from David to improve the comments and readability of code - Enclosed the hugepage alloc code with #ifdef CONFIG_HUGETLB_PAGE v4: - Augmented the commit message of the udmabuf patch that uses pin_user_pages_fd() - Added previously reviewed but unmerged udmabuf patches to this series v5: - Updated the patch that adds pin_user_pages_fd() to include feedback from David to handle simultaneous users trying to add a huge page to the mapping - Replaced find_get_page_flags() with __filemap_get_folio() in the second and third patches to ensure that we only obtain head pages from the mapping v6: (Christoph) - Renamed the new API to memfd_pin_user_pages() - Improved the page cache lookup efficiency by using filemap_get_folios_contig() which uses batches v7: - Rename the new API to memfd_pin_folios() and make it return folios and offsets (David) - Added a new preparatory patch to this series to convert udmabuf driver to use folios v8: - Addressed review comments from Matthew in patches 4 and 5 - Included David's suggestions to have the caller of memfd_pin_folios() pass a range [stard, end], max_folios instead of start, nr_pages - Ensured that a folio is pinned and unpinned only once (David) v9: - Drop the extern and fix the return type in the declaration of memfd_alloc_folio() (Matthew) - Use a list to track the folios that need to be unpinned (patch 5) v10: - Introduced and used unpin_folio(), unpin_folios() and check_and_migrate_movable_folios() helpers This series is tested using following methods: - Run the subtests added in the fifth patch - Run Qemu (master) with the following options and a few additional patches to Spice: qemu-system-x86_64 -m 4096m.... -device virtio-gpu-pci,max_outputs=1,blob=true,xres=1920,yres=1080 -spice port=3001,gl=on,disable-ticketing=on,preferred-codec=gstreamer:h264 -object memory-backend-memfd,hugetlb=on,id=mem1,size=4096M -machine memory-backend=mem1 Cc: David Hildenbrand Cc: Matthew Wilcox (Oracle) Cc: Christoph Hellwig Cc: Daniel Vetter Cc: Mike Kravetz Cc: Hugh Dickins Cc: Peter Xu Cc: Jason Gunthorpe Cc: Gerd Hoffmann Cc: Dongwon Kim Cc: Junxiao Chang Vivek Kasireddy (8): udmabuf: Use vmf_insert_pfn and VM_PFNMAP for handling mmap udmabuf: Add back support for mapping hugetlb pages (v6) mm/gup: Introduce unpin_folio/unpin_folios helpers mm/gup: Introduce check_and_migrate_movable_folios() mm/gup: Introduce memfd_pin_folios() for pinning memfd folios (v10) udmabuf: Convert udmabuf driver to use folios (v2) udmabuf: Pin the pages using memfd_pin_folios() API (v8) selftests/dma-buf/udmabuf: Add tests to verify data after page migration drivers/dma-buf/udmabuf.c | 231 +++++++++--- include/linux/memfd.h | 5 + include/linux/mm.h | 5 + mm/gup.c | 346 +++++++++++++++--- mm/memfd.c | 34 ++ .../selftests/drivers/dma-buf/udmabuf.c | 151 +++++++- 6 files changed, 662 insertions(+), 110 deletions(-) -- 2.39.2