From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8794CC48260 for ; Fri, 16 Feb 2024 04:59:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CFF666B0082; Thu, 15 Feb 2024 23:59:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CAEBE6B0083; Thu, 15 Feb 2024 23:59:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE3146B0088; Thu, 15 Feb 2024 23:59:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9B4806B0082 for ; Thu, 15 Feb 2024 23:59:31 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id DB598AAD17 for ; Fri, 16 Feb 2024 02:22:00 +0000 (UTC) X-FDA: 81796066800.13.E07D893 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2068.outbound.protection.outlook.com [40.107.93.68]) by imf14.hostedemail.com (Postfix) with ESMTP id 0DFF610001A for ; Fri, 16 Feb 2024 02:21:57 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=GAvYfq+E; spf=pass (imf14.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.68 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708050118; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=w102R3dpBXORyrXpJLkG1YgRdO269Z0eC2q9dog2/eE=; b=QHdiY8u4+bVhocf0n9yh1GNoVFw0QMfWjlY3PGDW6t0s8sd04Qtfmlle0bjFfYkUwpnSUf ZLrMrSHUKg2TXHXFJtCBaytJAkQv3gs3i3To8l6iGmvU/7Keaus0VgUKolQgQMAri4r05t N3jybt5yrmXDthhKEAmO3+tgUpQiS1k= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1708050118; a=rsa-sha256; cv=pass; b=cTCC8RNLAPsMBpaGDEU2jD+x9EeN8SeGw/sq5GxRcA7yqURYuElRr6m1mV+T+6nUvyps8j tEMdFHm7If9iQZjERPPYsCZSJXhoVKoYu/BXwHRt41I+DKEIHfWCPoo3UEDjsDCvsxnGtE r4X+dIcwI2HcjgI4oknYHezmNVGzPC4= ARC-Authentication-Results: i=2; imf14.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=GAvYfq+E; spf=pass (imf14.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.68 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iMOyINZE4e8qYcmmoOUQdEYPuLNJImQOeP0uI+66IRbFVfS+dxif8b7w1RGS668UAkTxeOiH/PdmmR62wUYMPXKW8jiN7FOKQ4Fev9H/sh4Hw0dVXKiVEpTxkJ8gNkvCvR+YlS1W3NRwEZ/VP61sOBvJpx7W7cIjyjKb93tldbtNapgPWm7Yixg2nzyA2fnSG/HbtxRulMlIrJh3ZSAaRTNOzRyVwpi4wk1Pn3xR0hMPJoWYvD7dVqEk9q+pNW/tuwg9RSo96HeYuaeA93SNVL4RzAaY58G8VGan+pROEop3QdiuBQ+4iJcynDn99oXBIItv0kYSXAtrENqp+QZdLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w102R3dpBXORyrXpJLkG1YgRdO269Z0eC2q9dog2/eE=; b=oZt6FJZnWJwSKffZkiWRETWG80Iy3krksOu1vOxxG4j5DSUdM+apicAMwekwZRrCR4edUoHmvckeq9+QjK7LAmlErDPmUqdAxGhJcM3PEOPI1shyeuwIY7owO045RTMmrDbquDLrNAZa75lnzKSp/bi3D8MNbsaRL/3bqObHw3Yn8ud1TpfXTJSc9zbybjlIAU8hhS/E1GL/PH/wsfLJXVNd7o2UoTNMxjHA5OxjkRQDIf88X9Pxl99ExqoscrnJ0w0rw12wjcdEuyWEtbdygw50T7+HK5GeZp8jAMGfypohkCzMdRvxB1+Wl+bu1wAgs8xBUjCVJe0vaEAAxcHf7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w102R3dpBXORyrXpJLkG1YgRdO269Z0eC2q9dog2/eE=; b=GAvYfq+EnqBIbf6ufC3LUnJi8J1CVx9IkyDzr0ZHtuKSNSxKPkOhRWDnIqS1RxRxjwLZlYmGssGy7M+V3evgjkRSnT91j2q4bMfgd9u1/abNq6dnv7LT5inM7GE0lN/svWx6UdXhnLAHfZsfnGjJqNvS5vHOlTqlP3xaqU3xA2hbqcWouimwsqTQGpPW75SJHunZ23JdyOQWQ8xaP+rNSU1j3tK+zJILWOLLhyAcN25t/Ki8/Wf21dmglIklBZC41Zk7X8sdFTSNncT3gIGr2sa7MNvLHO6ByewdDrNOSXRBGbLbNrgXGbu2B5U3YtdNcUdwVthZZ4N34+0rjkaDDg== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by DS0PR12MB8503.namprd12.prod.outlook.com (2603:10b6:8:15a::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.28; Fri, 16 Feb 2024 02:21:55 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::a892:3047:6c68:5adb]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::a892:3047:6c68:5adb%4]) with mapi id 15.20.7292.022; Fri, 16 Feb 2024 02:21:55 +0000 References: <20240214202055.77776-1-sidhartha.kumar@oracle.com> <877cj67jn3.fsf@nvdebian.thelocal> User-agent: mu4e 1.10.8; emacs 29.1 From: Alistair Popple To: Matthew Wilcox Cc: Sidhartha Kumar , linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org Subject: Re: [PATCH 1/2] mm/migrate_device: further convert migrate_device_unmap() to folios Date: Fri, 16 Feb 2024 13:21:26 +1100 In-reply-to: Message-ID: <877cj5du80.fsf@nvdebian.thelocal> Content-Type: text/plain X-ClientProxiedBy: SY5P300CA0001.AUSP300.PROD.OUTLOOK.COM (2603:10c6:10:1fb::13) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|DS0PR12MB8503:EE_ X-MS-Office365-Filtering-Correlation-Id: 17f46a72-ab14-44cf-1148-08dc2e960b73 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TGcnEoYtA0W0WdY+6hYIleRFK2RkaUx9+i1qEv6UUBmKSBcHkCTt2gBsp3PTdyjCSuNe0kevFdh4iJe987A2uIMGgl/mgv7B61fY2Ca9+pItCLfaXEaWiFId/B4aN8BBU3PtYoRxwD/ruS1gqvl7G4x/hrIBdBg5sZWlWxAT1p3n8V//P+hWRjVUQqLWmbrVmaI2SI66gnn298GkirvRPVoomMS2lwwdlek2AzJDthZEiqG+kRUbnRqeO0a+Du+edBC9nU2c3e2LdDv7iW2IqEXY67e0iZ3KSGnT0h2G5Dxh5mS1wsapgLWpFnSNlOYZAZWBp/uKPCCROCyIGdrQTEIvrxAvTIgF4M8TLLw+HF/OeCtcECfg7W+9el+ADA/iWKW7cI/F/1aw6+POThDll44tsOdWj+L/Rkv1sHzec3RSPyb0AueOTQbT6H5ZHy7RQ+f/wjSq/PHL83+p6lZN8OKNVQXkO6oIkOjJb+iKAPhu6nWRxxZ/GapiE4Nxn9kcA+NmuUiUvD0q6GpXaNtPpt2wmgy4VHsVhOvB9jHq1m0xCzN+ucS53upG/sU29uzU X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(346002)(376002)(366004)(39860400002)(230922051799003)(186009)(64100799003)(1800799012)(451199024)(6666004)(83380400001)(86362001)(38100700002)(5660300002)(2906002)(9686003)(8936002)(66556008)(66946007)(316002)(4326008)(6916009)(8676002)(66476007)(478600001)(6506007)(6512007)(6486002)(26005)(41300700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?bwQogqoiCZWznfYfEfyhH0xFw3Oi6Gq8ccqG8TabZCpqGusNEMp3w03UhUEX?= =?us-ascii?Q?1Zpz4zNuWwVzm7V+rVwQVcmfZopSmeDHrvo21azgLEadXM9x1NzRPBnOvm5y?= =?us-ascii?Q?ODmfaAm5yRF+8Nng3oHhHR9o3T2xG2CvNeYZwMg2fO77DLGtyVHxY9xuoQX6?= =?us-ascii?Q?YmHbGUixVbK4QqpQNEyYyfLIBTRFzr51MWqKZAou5VSC7//BA4e6U2QF/mYQ?= =?us-ascii?Q?kAsxc6YA3nu88+ICHPWg7ssqNkxIdsrjRPpLa3mIpSCEtvJa1Pbm//9IR3T3?= =?us-ascii?Q?Ys2viH296VhObslAAUUO7uy2vCKtAhTwSL1m1cnIyu4fY8WMQpDo3Jo+1xa3?= =?us-ascii?Q?57g0eSrGy/Y2NHMC98m3p5t8V6zzwefmYqtR5p6i4GTXwj8tN23ZNWU3YlDz?= =?us-ascii?Q?m3GRPjQqTXT4zPwgwbqJLQsERZ7e174ZslLPjF+S/cRAqugdjSa8i28EXS9e?= =?us-ascii?Q?lRvmzZdkf1Lr2MN0+zdSAJ3yY9VKLooM3bvzZbDiWFQhqv9+LXCaRMGRyb6I?= =?us-ascii?Q?6Z+ruYxFM9+NoUp517EeEcR8P7ENvhtmrdQBOTa+OePGxs6kCsFOYUeU36Mn?= =?us-ascii?Q?E5alRjEgmbsbeKyGvTVl96Po3laTe2swIns8CVPIIpdeU0vioeYwD5VbnD7S?= =?us-ascii?Q?4ct8eUfw6/WVs5scBNlg6MNX5WLUYpDJH18y7wBL3kE9ZLeGZyXGuKU6zyTh?= =?us-ascii?Q?jzHbXegd9ajSp1RSMELEjj9/15czBA7bI77dZs6o7MTKkZqWWRkKiLfbiBfR?= =?us-ascii?Q?xuVHgJhx9WO4ozmfSK8feLe2f+jCi/IZz9Cl7spY14ByrEyFe71WovOO5UTf?= =?us-ascii?Q?k5uXVUNjCGDylfCYWrn+ay6JmyQIFPUDAtiyxiK9FMFDqDFLYsqs4JLxANii?= =?us-ascii?Q?BIrON/2nOTCvBbAOqehUFrLa+jIaRP2X6Bi+mYeN5OLvKMBrah/hQ92+6/rs?= =?us-ascii?Q?2bMm1UxoEIgOcdRMXwLAwvpHwqiyQnX6Jvspw7z6fCZcxq83wIYgtr3kx6WG?= =?us-ascii?Q?QEdAKtv3vJdkdlf1yGgxf40Thh7J3YRO5TDHsW8yy/VuSm9mvdzWVcneMnTR?= =?us-ascii?Q?bKtjBcxfiYpo0oHgZ7rQZEkhtmIexfwqn8WGL0mabWGk4CVhoc4gd/D6aZvR?= =?us-ascii?Q?aVItd5qogMDgALvwu+7waZmOap6Pdpl34VQxxGKizsykioZKvN0VRmSnJBwB?= =?us-ascii?Q?DbwSWwxKZ9Toje8BkQ0VioLmRykG7kYsyICKfj4AGMCsit68wokaAujDhX6/?= =?us-ascii?Q?xVli6k1zm+jJKzNMMem9LpYezoexGga9D4qpQt4/U6PI7xmxZ3C+PS/ODKfC?= =?us-ascii?Q?2YbodgetTV+a7vr8neSBJcg445PYij0vXUtDxY/27WjOpGta+60epFQ7c1f9?= =?us-ascii?Q?d0yc7c9O7a1gYuOhqH/CYPgcmbitcLCaO/0GiOzbR5LVWmVPvvXa9Wh1qig1?= =?us-ascii?Q?oibL3UHh69S4V3G+TFuwYZ72ET4QXfv1Jm/xjxqbWzsyzWAMaI/M/Nof+gus?= =?us-ascii?Q?KWlu1VWtZGCwJezTpPyjFMfHHzBv98zKzPZakqtfD5bjL5V/z4P7IVfw630v?= =?us-ascii?Q?snGKoW10DgzeillMZiO2uc6vjFeaJpG7khG8BPLU?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 17f46a72-ab14-44cf-1148-08dc2e960b73 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Feb 2024 02:21:55.3567 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: C0p8L450afpBLCh874WsYWBAt7kkrN/+xdY00O8tEi5ur9C8CuC4udetfRQlJeRjXjavGJh46SE3eHKi7tlI/g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8503 X-Rspamd-Queue-Id: 0DFF610001A X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: d594e9jkb338ppx49fg7udo8cdwojwfs X-HE-Tag: 1708050117-900649 X-HE-Meta: U2FsdGVkX18+PdG4VXqTMXIEBykr73TxiBP0wPvKrBaMTRDxF+RincGKk1reNMy4q2yOG7HMtr6+YFxU93JnimJeHbb2jDDTP1sicKD42YpcNl27IjzhFVqVnuok7tu8Mi3OJZT6gvCvGqCXIdvt9ee8YOeiEn5lxRDIE+9LQyqoFhnCFhjDx1WMUfyEz3NKANF4imtA08RsX2WWbf33FpruDWVnSZwaoewRic5loCU8UF4IHdqYd+7sNoapLIYDFM3OEygYJUkV4GXoMce7nTLE8g2SNKp2DY4DjqU/IYlDgvfYQk9pMf+PoiphoaeMhafGZGV3kFeAvv0tiqfFGGhhX/WZzajS6K85mxidGq9X/CqCMKoizdPoGGA2UblHk2tRiLhOQQFIn/kKvWgOcy8x5OodryCXlnk6YQIuLxXX6j4ruPeUCCKJEeXAZw84NPWEf3Y5W6AlyLL6QSaHcWGWzrdM9DBqDu6q2ga85Qd4hAcqAcqk//4BxTr1jeZJQdPy/A3IdiTxT/RReo/HQwgekMif/rEm0CCRy6Jr4lNH296UrjoS5cdyRnpLTAUSJgnTuR3ExcLDn3Y3Yts8c/08OTUWf+z+rpqPDqUz7Njj7wVhm5p2n8n82QMUiguGR5q0vY/kwCw0c/Ui5nfHXZHk0Pm0ZvFJiD7HuCwV7/44Lii16Gk8D1LWWs8vAJbX8vP3B4DlcFCjOSC0n84XIEBLTn4sF7OLGRjmM1+AteU9oNeX0MsbQf8Fbty9z4zAxv/eJLPMAJCRZRKmYG4IBxBsKq2ayaha7BMqpDth5o+eqvBNBCEK2qwJv11SHxNOCitHVXFzoz+VehZuOQw+TSk4KhDUZ2lNGDM0KexzY6aSIdHvU4R2VdO+cqBPVGFXD4uy1y+rQ+ItdRwLgS/kAQG1o0MLnDbRy54dImt18779CL1od4SeWB6xhi9+AzFGCp6hxsCz0g23sJwkgvJ DWLPNM2M /3C2IzNl0E24JxeWXrB9j9jeCgW2gEIxv/yUi8aWDKq5u/NP1SOgU2ZSG52DXp9x+YnUW5vYz4xeJBMAL0t00EHQ9m3aoRjBjagl2Ke0sl1RCKQJDIA4iXnBZ0DdiFpQq2Vdz4Is1AGim+0gAeSd4ald1UEMyaGJr5TVDnaRyfnAGybzzI6INlH5y1Zt/LdtlXnNBZUUs+ZvJDJgL8b8+LfDZaWqT8eVbHNSf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Matthew Wilcox writes: > On Thu, Feb 15, 2024 at 09:38:42AM +1100, Alistair Popple wrote: >> > +++ b/mm/migrate_device.c >> > @@ -377,33 +377,33 @@ static unsigned long migrate_device_unmap(unsigned long *src_pfns, >> > continue; >> > } >> > >> > + folio = page_folio(page); >> >> Instead of open coding the migrate pfn to folio conversion I think we >> should define a migrate_pfn_to_folio() and get rid of the intermediate >> local variable. This would also allow a minor clean up to the final for >> loop in migrate_device_unmap(). > > I think we should stop passing pfns into migrate_device_unmap(). > Passing an array of folios would make more sense to every function > involved, afaict. Maybe I overlooked something ... Note these are migration pfns. The main reason we do this is we need to track and possibly modify some per-pfn state around between all these functions during the migration process. > Also, have you had any thoughts on whether device memory is a type of > folio like anon/file memory, or is it its own type? I don't quite follow what the precise distinction there is but I think of them as normal pages/folios like anon/file memory folios because we rely on the same kernel paths and rules to manage them (ie. they get refcounted the same as normal pages, CoWed, etc.). Currently we only allow these to be mapped into private/anon VMAs but I have an experiemental series to allow them to be mapped into shared or filebacked VMAs which basically involves putting them into the page-cache. Most drivers also have a 1:1 mapping of struct page to a physical page of device memory and due to all the folio work it's fairly easy to extend this to support higher order folios. I will try and post the first half of my changes that convert all the page based handling to folios. I got caught up trying figuring out a sane API for splitting/merging during migration but maybe I should just post the folio conversion as a simpler first step.