From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11764D5CC9F for ; Wed, 30 Oct 2024 12:00:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 94C908D0002; Wed, 30 Oct 2024 07:59:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8FD0C8D0001; Wed, 30 Oct 2024 07:59:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77A738D0002; Wed, 30 Oct 2024 07:59:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 56C298D0001 for ; Wed, 30 Oct 2024 07:59:59 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C87BF160AEF for ; Wed, 30 Oct 2024 11:59:58 +0000 (UTC) X-FDA: 82730123742.19.FD22765 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2073.outbound.protection.outlook.com [40.107.92.73]) by imf20.hostedemail.com (Postfix) with ESMTP id 1CAA11C0012 for ; Wed, 30 Oct 2024 11:59:23 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=HPfHsbvd; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf20.hostedemail.com: domain of jgg@nvidia.com designates 40.107.92.73 as permitted sender) smtp.mailfrom=jgg@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730289435; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=X82E8ZowwCzD28MJW286kPPg8OevZbML4Ym9yu7EzhA=; b=tIqKiPuCXnUGpJi4ULe2O4FuoTMhhfVz9B4msghjMjQviSb77TLuvSVx5BdscH6sYjG9dH EP/CELM894385ucQkZQjdHWgzHUa1I8Hn6xEthMj9+GFkCWjGmoIeN537i79AjT1nOwcL6 GYTA0iwiYDpfhXxBGbTJtBwPM4JyOvc= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1730289435; a=rsa-sha256; cv=pass; b=kG8w0hwSmhgHIC2mSo+jjrttI97W9f0qk16ysT52XiaAOm+9rorEs+4lgRW1B3nnydGuWH 7n5PnECokaDqkzd8G4Am7nP4t0mQ2Zk8rk65qsGMTZppRHcQLwIZkFxHzpfoclarsW8OA7 ATgcYfS9jShzPGFuImEH/aSmHOsmXRM= ARC-Authentication-Results: i=2; imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=HPfHsbvd; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf20.hostedemail.com: domain of jgg@nvidia.com designates 40.107.92.73 as permitted sender) smtp.mailfrom=jgg@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=YvIg6QLMSe9S+y+JjxsQH1/+d3sSjI8r++XKM6D/PYkbvaCjA91gP4pCxMqffVSj8BuqjCUkP95zUva+FD9bkiHtz0VnTXJ6lst2uW+PRcp1HU53rzrp2tlwOkUNxqIfzzxms4BLo7R6JUd4zI0iDiipI0JsmXOBjA3G2f5cB6z1cA/9svDiU9sklfYzXSd1PTMDQH3MsKM/C1mMej+DA71nR6Go/8+OzRKTP09pFfZFV5o01xo13fkngGdTWchzc5wM7g2Ytr7LToNEzY1XzXfmQ/HCZc3CU9q6O1rx9QGI2Y8l3Cr2SKYXQsjMvbvcnaqGoZU6HQXqqPwpRVkxig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=X82E8ZowwCzD28MJW286kPPg8OevZbML4Ym9yu7EzhA=; b=wuyCzx05KLy0EqhWKlgj5mEl55MBs5iNQnHq2Z1hNE1xLZNGl3jziwRY9YWlaEX1sjER+SY5xMx0tNSrh+8Wpao74YGWJVnYF292QL5eWKtJYIJcUt6ug1CuLeEztTsUbSEdJLXJgLlqleakR8PP/wOeiRI7Z5lWXYIH6J+V4wEbW70LKr6p0TIxuC20SI8bhJgGDpK5aiCB7mewct4MPH726m9/+B1RlwMc/GA3jaXJFgsP5YyJO1OkB+V8MRTLTgtEc+KP44rC4dVWKIo9Ip9YaAqu25M8s4ZixWw2wQuim4ddMplYY7O68+MM6bNVkX07JZXXKTuRvzZ1W2crKQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=X82E8ZowwCzD28MJW286kPPg8OevZbML4Ym9yu7EzhA=; b=HPfHsbvdrv24rWcUyxEr1d2bn9H7UdJBB63Ft3z9bUV+Z8Ml8kin0By8s/TcBXEuexT77XW2d/Cb41I6xHOa//MHUP7V5SDZOHjTYNl0+OWVrXvQijY/+sGfsAsRb2r5+sepCD++NBeUW46NT6GKrciKCra9aBiTc+aZ0bwkV2g7mr+LlPGTRn1ur144WxC87RURwW/kqFStTUnT+sxHHmzovRXc158qGc9oPQ4yCEjYGLWq7WNlFwjY6mP/aCzXT18SEFvNwJknwJLrvVWdnVC5kU7fdDfFT8asVlTKCprbNeKAhxwle1SPwbrTLNZjDKSc6pSyTdyl8cId82QUxg== Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by DM4PR12MB6232.namprd12.prod.outlook.com (2603:10b6:8:a5::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.25; Wed, 30 Oct 2024 11:59:52 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8093.018; Wed, 30 Oct 2024 11:59:52 +0000 Date: Wed, 30 Oct 2024 08:59:51 -0300 From: Jason Gunthorpe To: Christoph Hellwig Cc: John Hubbard , Andrew Morton , LKML , linux-mm@kvack.org, linux-stable@vger.kernel.org, Vivek Kasireddy , David Hildenbrand , Dave Airlie , Gerd Hoffmann , Matthew Wilcox , Peter Xu , Arnd Bergmann , Daniel Vetter , Dongwon Kim , Hugh Dickins , Junxiao Chang , Mike Kravetz , Oscar Salvador Subject: Re: [PATCH] mm/gup: restore the ability to pin more than 2GB at a time Message-ID: <20241030115951.GB6956@nvidia.com> References: <20241030030116.670307-1-jhubbard@nvidia.com> <249d2614-0bcc-4ca8-b24e-7c0578a81dce@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BL0PR02CA0144.namprd02.prod.outlook.com (2603:10b6:208:35::49) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|DM4PR12MB6232:EE_ X-MS-Office365-Filtering-Correlation-Id: 5ac83662-d791-4340-d68c-08dcf8da5cbb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?k03ssPyxIqVSNgRiGowKgrNeDzrjA55mClJAkbhtSk3fbqYRK4Bstw1p3rbf?= =?us-ascii?Q?wDzqhBYlJWXBrii4UVHhnc2opKw9FOlYB+5F3vKO/2vC1WY8e5ZqqL6L1i3c?= =?us-ascii?Q?bcOs/s5SkcNJfpxGRQ1t6KkN0TdUM87pbmZqQi02+L2Tu0MMYeGBelAcGMHU?= =?us-ascii?Q?1IWwIHxRFjO4DIrUBPSa2voxx4AC1nviZPEkLIQ3snnWl8r9OyfXxSwiKYoQ?= =?us-ascii?Q?TxHpLTQEj8oqEV1YcRXXEwyprQ24GWVncKcAKI/3Y5x4cvn+YFRUdYgswZRN?= =?us-ascii?Q?XJhz7hqfJXWvhrylCDwyahkj+gVoeUJC4Hid+cnDjtcx6gIWIseso9TM2umW?= =?us-ascii?Q?7fOpqi1vC5WUFetas0xOeh+n08DolTjSNxRYsc6aP2bH0KzLda9qThzCJbHI?= =?us-ascii?Q?KX2UR++yIDV7WrK1kEtRZmQdZnsXw8PfV8xml9b4l7sgn34yajDkVh9EhDT0?= =?us-ascii?Q?bx/4bD5f0h+IQvfeSr2Lx3g6vKPlv9IGyz6MNtR1/4f0pRlsCb+d8xihGZ4S?= =?us-ascii?Q?QCW+wtNQfS8bm9KAHSTqo0NZBHZcx74DTwuWxTphbTK7YpqmAhbU1y+OcBtY?= =?us-ascii?Q?Wtl3uUXD3IQ/6JvFVu1jgRRnFaYBcjVtVh3sVzEz/vdvuz4zFv0sKoWZgmPR?= =?us-ascii?Q?DK/wUxCN0eoub9fLRG2N4T12SuvMYrgsf8WHq+Fqm09lnyzLLTE7KvYG6Lt/?= =?us-ascii?Q?LiA/H7pvaO+XyoMxCq48OkWix3trc6PkCk4+BHac+sVtacoAFWsNN5GAmGWi?= =?us-ascii?Q?xpPSVWaGVaEmUnEJAcK1ZzChY+Kwmq+iPEZfqNql+1n23LNalCy//Msa2sRR?= =?us-ascii?Q?+a3pb1Lq0pO1phWvL+q9Li07h9rLJAL7yUixqTY8UB+rOuuBl0LXuNFbM5q6?= =?us-ascii?Q?MvTipeufpjqf01YEJGiuGpD5LO2see9jg6gXEIjctxAEafZYdN33qSFSg+IK?= =?us-ascii?Q?ff9a22CWvIyVKmaD7GhnWoRJ7J7gTl3BLlPRJeT607z9Y5sCA6NmDoNKu/iG?= =?us-ascii?Q?b1YH5d2BSuPJkdFj8JoLliDxtfD0n1dVXBAzcU5g7nLdiQ3sjNqzSTLL6w7q?= =?us-ascii?Q?6l14/2eHJZZp2CnRmqOrg8VdHNs38P+bs4ZDj5d2FGUDj0xinvf0Nyl3v386?= =?us-ascii?Q?W9ZjcqZxIZkPN2VZhZn85pxMhRkNejngjMM3mUXR+RdTVd6zjF7Bo5ALDRMk?= =?us-ascii?Q?VQA4UFw7nhWxL0HzxEY9Wp3IJh+i7MSB7nCeBrsYgAbmzikD2Jg1MTBKHNsw?= =?us-ascii?Q?UievULpgrTg6vkg+/AtSEjRuyuY9VaaO5uwiB1qbk9U1G8q9r1VVEWcISb7f?= =?us-ascii?Q?FkE3tDL25EsPrglXJM6l3Acn?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB8659.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Z0d79jCa61D0k3RtAqznIehQrzuSoOGtLjEXgIb8dTRyNKvfu0bu8Yq7fahU?= =?us-ascii?Q?YTWOgMkKwRYG1tWeJl4/Q48Lt4JIdy4RM8NR0V69ahkGK0SHW8hn+bOkihS4?= =?us-ascii?Q?0cgPahLs9xnLCOvIOldFQMbAVe2S4DuuIGD9i+TWAMVFztffZIGkLc3Yc7RZ?= =?us-ascii?Q?Y9A2MS/Vb/j85E4eswP5bWt0XPoMYbTomLGKmnWK+CjRsz0y0iVdUleh5UVS?= =?us-ascii?Q?19MG7Srfqzy5tpiwd9e3ckmKoB6kludaQpKjfEKsnig/MWp35/UEQFpm/xJa?= =?us-ascii?Q?mCt/TIx8EfHC03HpUA90cvzNsu/sdvS+eFsWPUzQl4nTeQ9Lc3RSVXXXOWf8?= =?us-ascii?Q?mOEuRX00rXBSHJME/3Ax2KbqZYRBl3RUuohj2CrRPm/khjloCIj7T9Z9fApB?= =?us-ascii?Q?VkiSjFcKEcCHz0qDOofn/n9QRGalnYLyT005CJvd4eJqRIHbkAmp8vKB45L7?= =?us-ascii?Q?DeNIfozqXFuCf9gpQ6DT3fVz0sy/maVH9XhlacUEx781cewDolqPu7J0Jayy?= =?us-ascii?Q?sQ6tXBoVK7Ki126RfBdNJB/0GDrBxDdVBc9Xs1jkdpJWysuztlr3gQwBo7hU?= =?us-ascii?Q?1faCKneWP79SovPTgptQqkX9dCoJN/V7qVAYdqe7wobrcjJJ4+vDBOa+A+PF?= =?us-ascii?Q?L2MKCSGax6WG6PB5NyzjccGddM4SNQsQYJs9eUONDXDMPGhVRB6OWQW7eOyz?= =?us-ascii?Q?2WG2xNt0hpQDuemeuuASGOGpirp+dYq3ui/lubfjqMJzXjFH+/zOuZQa942U?= =?us-ascii?Q?ICRDj0itHnw1PoBUScksTn/qPqCmL76fjQF8SaK46+sLYkK3C9r3KncDGqly?= =?us-ascii?Q?Iyx/xzsJ5kX0h4R2iMqHMy8jID7LQVU+bRrS+OhR3YmenUfoUt7y3DljSpcM?= =?us-ascii?Q?5hrr9M2nsY6MjIZ1FtJusxOl1fSurFaHKfVF/Sb5UBQF9H5RGaQLu1DcdiDH?= =?us-ascii?Q?lJoVj2O4WAM8Psy2CsERowoESdENE+/1xrYQRqarE1Lt8Za/GDsWNJ9F75L2?= =?us-ascii?Q?etyqNqTaMUAR5aYGHJQDl1XMrl6iDEkxHtRr9prD0OxJwkkZrxq1OASDNxbm?= =?us-ascii?Q?gUSWpeCbmNZ/+4B2JiZNkhXjryTGhvExEgnO5BIwqPhlk+KUJpNWEzSfLDqU?= =?us-ascii?Q?6SlaI8gC820rDMIW3R1583quJQqWoAdgjSb2sG3TxhcrqzjnjjbN1rVIMR3B?= =?us-ascii?Q?A4wRNn7UCmyTiK1tIQYxa3qVS3qNc6F4Nd1m8Dir2Jv6XyU8N6oPt0tdKhJM?= =?us-ascii?Q?vHkOlgiwxOTVasOVrtXguCs/CIzdlh4skPmb/vwHFUn0eNaJ0R6N95gZ6AAR?= =?us-ascii?Q?jm3gFbPb2yJegOIsgQMS+drcVNwyXa91B/JwB+B+eK9ajRFLr0Fy1A9d2sEU?= =?us-ascii?Q?aEIUO1TLm8S/u/1Homf2sUqMHkd2n9gnBqrqfJMY/HdajR5FVU93nYgoG3sF?= =?us-ascii?Q?2k4h9ghSjoYyrowWa2DKKDeFy6pWSRyVi9dgywO/nzo5U/dN0Z/fDvmndcpX?= =?us-ascii?Q?SinxAQs5Ybu1/JVEqeYr76BPfcvXtFLq0JogWnb0zzsnOqSy+U/V1dFvrzUs?= =?us-ascii?Q?kxsm34yIKPJH0IC9v2w=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5ac83662-d791-4340-d68c-08dcf8da5cbb X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 11:59:52.4074 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0pOO/UCPFTvxGgyDx0/lT7Drd/5tMYEAea9Vicb435ZXskh+VhOHxPZ+o9pP+rO6 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6232 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1CAA11C0012 X-Stat-Signature: gj3xe7ph8tkqp3rn5pym5u3bgjdj3b4w X-HE-Tag: 1730289563-806998 X-HE-Meta: U2FsdGVkX1+lnkN1vZKbCerWz8lkQ7tTAdjDc+z8fDx8GLpdqceQn9IRyJkXYy+B0nUNkETBaIK88qH47mPZLEPmqvkv/k0GTr/a3oYadJw2C23eBpgHdX7bqXCE1V0YZg8FtBtbeSsWFNj8g896WJvJWhYglY+epmryj4IDuHFr0mfsjSTwizJzjSs3Oqv9iWM6JazQNtW1FgfG8KMJgOY0J4dzFDO49jySv27kiNe6UJ9FouVH5I9dGfqY9fb0whNTcIXk3iEl68Gtdr3QNCr09fH1ysZskxJARAHirnS2JYQPPA1kVbJ+957rm+AQdNl+v/URRHNJW4TME6evlju9Xc6wbTt5bSrSjCmD07tckcOK3w4badgrlQ7+bMMsh0hIuXy2XHFfH4/IigD6SioATGSbUVTL96hQEXBhQqf0mriH48gg2lnjhMQJUxM0gk5j/SEv7O1cuGMbx8UwC9jEHSZ2/5drmWuBrsfU/QDX5yrLAdgEfEgSka3kruI6ckH6Hmx+XMewKOLiXH//a5WGCFzfWeiiwBqALYvtxZwc9sx3eRW6ICFZBEDLA7qC4dQp8CZuU2heCGzAy3AizkhbN7bmhkQ7mJGTCE5ePYLYbQbAyW1a2pdKFtm8zz8JhXJh751jMntCJw2t/AzoNcQ/6fk144yXmZ7yT3M5MDvipuZODSOUOLu7SUYvTmDthj0TLwxZiLtNnFUgJNIbghJAzTP91bZsKOS0RVXJfFKt0PEvYNBds0rDXRp+ysJZUhO3KWI3qvRXOorx6GN6jejAtULPmoN1++X6VSJsNPAULLTX2jTASHR0CEf/uGGhIQm2PCfKRmngdjSQqukIqmRLtvge3yvmE4/YX3MWRQBg1rx5J5Yxb6EzocMQp9jDqdfp0bxVbwXDYFUxy8LSMRnwCU5Jrj6oCTNFgU0Nnydjh6rjygJXOT4svZucond8xfNQh+IZQabLAfZBvs4 sYbzpgrV gGA9Sqxlr1oHs8kLl9K6JPkDxcZ2kjLfzYCKd1ezr0szAgLF1a6xNsEKifQgHDeSQsCJ9uiK679oVmTueBPksBdpf2ENRB0U4SFpd0Y7HYewR9wRQ2VjYfs/gKzKwnyBweP7vDjmI8MCqXRy2oJT09msuSyDOoh2dSnpy9/8tymlvvCihqL2MeeGry6M/YT4L63OcKIGsR5svQovHsD8MozYtpBXRlEqQWiOfPccXV8X+zjFQ0RG7QV3vi1qmlASUqVU+WMCOmZnxLY4GI7Qe31SKWAdI2+zHzCJzB6vLOVTqmD7lelXQ8Zi0faXPBTeDE4Q0K/DgPDpH2LHVGGFxS9GUDHgnaO8Sl7QP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Oct 29, 2024 at 09:42:10PM -0700, Christoph Hellwig wrote: > On Tue, Oct 29, 2024 at 09:39:15PM -0700, John Hubbard wrote: > > I expect I could piece together something with Nouveau, given enough > > time and help from Ben Skeggs and Danillo and all... > > > > Yes, this originated with the out of tree driver. But it never occurred > > to me that upstream be uninterested in an obvious fix to an obvious > > regression. > > Because pinning down these amounts of memoryt is completely insane. > I don't mind the switch to kvmalloc, but we need to put in an upper > bound of what can be pinned. I have RDMA customers that pin basically the entire machine's memory, which is 100Gb's of RAM. It is a dedicated server doing some HPC or Database job and it has been carefully setup to do that. Many VFIO VM deployments will pin almost all the memory too - just a little left over for the hypervisor. Again the machine was carefully setup to have huge pages that cover almost all system ram and are dedicated to backing memory for VMs. Pinning almost all memory is a legitimate use case. Jason