From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C69A9C433F5 for ; Thu, 23 Sep 2021 16:51:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5201361211 for ; Thu, 23 Sep 2021 16:51:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5201361211 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 62C446B006C; Thu, 23 Sep 2021 12:51:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5DADE6B0071; Thu, 23 Sep 2021 12:51:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 455B2900003; Thu, 23 Sep 2021 12:51:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0178.hostedemail.com [216.40.44.178]) by kanga.kvack.org (Postfix) with ESMTP id 365526B006C for ; Thu, 23 Sep 2021 12:51:48 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id E06D11813ABA5 for ; Thu, 23 Sep 2021 16:51:47 +0000 (UTC) X-FDA: 78619429854.21.7C8B936 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf26.hostedemail.com (Postfix) with ESMTP id 4A7D420019D7 for ; Thu, 23 Sep 2021 16:51:47 +0000 (UTC) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18NG0CYc010863; Thu, 23 Sep 2021 16:51:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=DxwB2DvyAYdqth4oW7OJj0hXB32ZuSeEhuZmD7tWdUc=; b=ePE7ytUm2VaGe8g0htaqzQmCtJvlOuHpq+1zcX5QSmeBou9Dt4HHPHPpJKyVpJv3ECL7 bfgloXQH9JbAasHXKjUqL7swbEKI8cKiR+HAXjr6s29SBS7LYY/DJsgYZYXCz6ntAygg WdHqL/MQhbGKH+C3r/PN3TRIWDelEm37d+VJUrDm9e7KZ6WymIBPFwTZHmAeMirTX8Cn qZOh3ehVCceTLgn9VrFvqd6VwiV4SjZK6AFEG9R1yOc0bmwaF9UtGpReNWDki7NixUnW xOD1KywEMKprBd5fUnE779QcFBtrQWNUR2/U0xbzXBCCCRoXJUeVy14tUE8eJTbM3utR Cw== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3b8mdbkvgs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 23 Sep 2021 16:51:21 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18NGitYx157282; Thu, 23 Sep 2021 16:51:15 GMT Received: from nam10-bn7-obe.outbound.protection.outlook.com (mail-bn7nam10lp2108.outbound.protection.outlook.com [104.47.70.108]) by aserp3020.oracle.com with ESMTP id 3b7q5cgvf7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 23 Sep 2021 16:51:15 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BpEOE4i9TxxoQ2awteBaq/ePHKE+KZ6gWsc/a7kxI22es1pBD6Ska7WHWGJP0pS1fq84FNPy/xqrRCWCpqY3r/bkgiRoCqPA4ViVNY23WXS755AFhgB8Cwf/4fxwOD8xA2jYLRnSFVQ0hYisrDATnxwK5lWk8SYpMJKzClF0L+ojIpg7Geg2hYXdvlBd51M6te6/SBd2C1GF1fWgjJosyj5R6ZbLbvjolwtMgrqq0reH2NwFwWf4Uwz1nBHWmfWZa67NudxX58U38Big2ajMHokdFKTA16/V++JfgnfEqvCX4Zio1thvx18w/0c8dknViQRjSAS4aJIgrjd0QCaFRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=DxwB2DvyAYdqth4oW7OJj0hXB32ZuSeEhuZmD7tWdUc=; b=MkOKMIf+8UZb7C1PPqgvLy6NWEcSoJleyfOI7NBVAm/47f3BkGYdfrpMzYRVD1we0M532KoK+Vbxl0kSmV2YTDwXkA8eJRnhIKih9p2ZDtKP+Civ819YWBtUJBvQyK1YkOTpz0KOJZVqfiHerGChzam0EKGRwUIoL2rL7cqgnRRBDUbWtMyyt9Kaa+iCm1/efpPgHihZnp6jF7Um9UVR093EfVnE50aZjbpwWOGYLjG9pFRQQY83M8tnC0lsyGzZkZymQccjbI+h1cy0FzfRALq1ROYfefEaczWPaqvVsNzbCJj6GIzT8R6/H1vGHFKnSN3+vjRr01JisDJMO9nytQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DxwB2DvyAYdqth4oW7OJj0hXB32ZuSeEhuZmD7tWdUc=; b=jPQBitykUroy1VkfOeWgUZN8z4ggX2s0Kphm5fp5sDB1t86ogkYyWlpUsSe1GGzGLRk+o+n9TRnZhHocfd1vWvHZxFq6AVK3CwYgP2npPoDSP+jJEVB/V51DYhSTlDsvLMg2KXR0wCFKGdJ9HdTeRADG0S2JfCOMuAnqJVy7hP0= Received: from BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) by MN2PR10MB4368.namprd10.prod.outlook.com (2603:10b6:208:1d6::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep 2021 16:51:13 +0000 Received: from BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::bd25:67f1:655a:a615]) by BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::bd25:67f1:655a:a615%9]) with mapi id 15.20.4544.015; Thu, 23 Sep 2021 16:51:13 +0000 Subject: Re: [PATCH v4 08/14] mm/gup: grab head page refcount once for group of subpages To: Jason Gunthorpe , Dan Williams Cc: linux-mm@kvack.org, Vishal Verma , Dave Jiang , Naoya Horiguchi , Matthew Wilcox , John Hubbard , Jane Chu , Muchun Song , Mike Kravetz , Andrew Morton , Jonathan Corbet , Christoph Hellwig , nvdimm@lists.linux.dev, linux-doc@vger.kernel.org References: <20210827145819.16471-1-joao.m.martins@oracle.com> <20210827145819.16471-9-joao.m.martins@oracle.com> <20210827162552.GK1200268@ziepe.ca> <20210830130741.GO1200268@ziepe.ca> <20210831170526.GP1200268@ziepe.ca> From: Joao Martins Message-ID: <8c23586a-eb3b-11a6-e72a-dcc3faad4e96@oracle.com> Date: Thu, 23 Sep 2021 17:51:04 +0100 In-Reply-To: <20210831170526.GP1200268@ziepe.ca> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO2P265CA0271.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a1::19) To BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) MIME-Version: 1.0 Received: from [10.175.173.247] (138.3.204.55) by LO2P265CA0271.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a1::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend Transport; Thu, 23 Sep 2021 16:51:09 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6d177857-bdc0-4802-98e4-08d97eb25a03 X-MS-TrafficTypeDiagnostic: MN2PR10MB4368: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: gw5o/I/KNyvVxP7XvPiBwlCN6vYMH82BdJCv6Lkw8QWafrB55fz9qM4EUfofN62Uoqqz2BBc7tVt9KhyuT9dzozHL2iXCre7/5C0x/l/+j9IB48htDAGIf2KSOp4V8c1JDBTnu+xDdONjFCY6Y4CVMBZmLvwQoAcqCc2QTelFTuk6v23d8JeBeq6ElYvbvAOb4tMHQ1CKRstCgb7kKzVq1ACDoyRa5G1ZRW57BuKJOmp5iJSMUfaouot7ov3x9ue8E3xjF3BYFWR1zpzhqfKdjJjBrXFs1kOz0SqJ2zvdlMgwRjgQQMPteMtzqCy04KdKy67s+Wq2O+iqeoCIhTBmSeIA/Ewl+8XeXFRKTQ4wkk5hTX5irguao7vzPilz3oc46dCHZL6HNgn/4zPVafcA2CK0usaIE00kHNEIzncLFDzKonmijp3atr5aq/cQ8QLFqu6Ye+55e/amdOUr/PyDcB6aPVk/IogD8tmultmJJuAweKg0txzOscYpgG4SXgaMdxsGMtQCpZm4ihaj0Is0RCgdOQg1cLX4xzuE8ovlmWQVcte8E0BmwuVPGYY1nybI/dj/5q1oaMGyhNKIVDoIlMzTpNbMMUyTPp1lsZZmXQPJgUYmWW0U0NZxGaULUGKJ26G7pUmmdU0BvxjCMXAzdO9hQV0IMU8PgqMNjVtzeH2i7SSntiRHC/VxSimS1az07cgNxQ2iahn4Q0wkfBV2yR6LjDQ4qbNzRHZ5SZIUcpftFmql94FM8DtUXEFKXbxKwW6UCZAYVKGkXQWFBXJ3g== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB4835.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(16576012)(7416002)(4326008)(2906002)(316002)(966005)(31696002)(66556008)(26005)(83380400001)(508600001)(6666004)(66476007)(8676002)(66946007)(6486002)(36756003)(8936002)(956004)(2616005)(110136005)(31686004)(186003)(53546011)(38100700002)(54906003)(5660300002)(86362001)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SThmc05qSzIvN0hEblNvd1NaUFhtbFhSQjJ4d2lCV2lVeThrdytvSTVOdEYr?= =?utf-8?B?dWxMeis4SmQ2Y1l4TWU0ZGkzSVdvdjhiVms1SDNrK2FZdjZ1bjZZNkhURHRG?= =?utf-8?B?MjZOMy9kT0psVkdUUE8rY3pscVd3YVB0SGZtcnBGaU9pTEt5bGNrME9LdjdJ?= =?utf-8?B?SUJxS25BOTFvR1ZJWDVOcmFreDZlb2c2b0sxRVhSK3QyZkpnRWpNaGdyU1Ez?= =?utf-8?B?THg1NitoOWpYN2FoMnllYjl4YVE2ZFMrUUQ1NTRpUngxZ2RoWUFhRzBMV0Qw?= =?utf-8?B?RkUyaUZZZzhPQXc3OStvNXFiVW5MMzB2MU85aWEwazhad2drYTlDT0dCRkdS?= =?utf-8?B?a0FTRkk5UTIvMFhPdDBlQlorbnUrcWRaWlhqeTJwWlRQdGIvM0VVVXNCNkZS?= =?utf-8?B?UkppYmNxWDhSV202S1lBeFd0QVoyV0lqUE5tcFJLdnN4ZUVYMUxWYldOODR3?= =?utf-8?B?dFQybjBiWW16RzFud2lubkkvWXdWNW1KR3pnRlg0WllZemw2dlZDeEs1aWQ5?= =?utf-8?B?aUlYS1JsWU8zZ2l5WnNwUDhjdDcwTjREenhPWHBhQ2NYZS9DbUZqczd5b3M4?= =?utf-8?B?RXN1aGVpVW1NQURURVVaY3hNZWZSM1o0NnZRQXVVVGZZOEJuYWhJT1FpbDVa?= =?utf-8?B?MFNVZU9sSENkZEdPd01sU3c3SW81bVo1ajhtc3c2Uk5GZUNwazhhNHVzT2pz?= =?utf-8?B?UHR0TTFXdjg0UTBhS3d4VGJJcUs5Sk1kNURFNThNRU9NdzFrWFJkektPTGgy?= =?utf-8?B?cTdYbC9JUDdzRGZ5ZkFPeUlmMVZKS3RYVVN6U0JzLzhLU0o2SkpnQW9mbGRM?= =?utf-8?B?UW1wMHRCb1FTQWZmZDB4M2pHaUl5SWtsbTd3ZVIxbHM2TVlBbzM4bzk4cnNV?= =?utf-8?B?enFzL0FmTVJ2SkcyZVg4Q1h5TGQ0eXdnSjJQc0sxNEJXSWFLeXpwSE9JVVVz?= =?utf-8?B?RzUrM1Z2S2t5ZS9DcUxndkdWTXg3V1ZZbjZXblZLTnVyL2NuUnA1U0xyWlNt?= =?utf-8?B?czE1NExEUjA0YmRiN3JjdEc0eWxTenAxcEFZMXBCRkpCbWpBbzVQZ3h0VHVL?= =?utf-8?B?YXNXTC9UczBSaldISlNIR1IwMVVkUDEwaURqSVdpL3I5cnRTZ2Z0bXJSL2JM?= =?utf-8?B?bkRDQ2VhczRjM2hPVHN4R1dSeExuSFZWRThuK2lmSTcxRS9hb2lNTDQzVks2?= =?utf-8?B?TEE0aTR3TVE0VTlBWkRHT3MwQkRTOXJYSkJEL3p5OHpUQnRib2NiQVJiUmlX?= =?utf-8?B?VFllSWdVMi9nMzE4NFlQVzY1d0UvZERxTHVwWnBKYzdjRlJqbXNTajJaOVFW?= =?utf-8?B?QUVTMXB2QnUyWVNKdDBMaDJJanczSDRwUTBOck9VVG1TYlVocDJZSWNqWDVV?= =?utf-8?B?aWk1R2FVUy9GTDY0RjBOeEc3bmJYT0ZXQ2cwWVpuMXhRa05Wd2txTFVSbiti?= =?utf-8?B?UTFXc00rcDlpSFIrZW5QSnVtUlcyVXZZUnpJMGQwRDFWNlBwelZ0bWtYWFlH?= =?utf-8?B?TTIyV2hYeDV1OUI3bit2Y0tGVExTZHpxQTN5b2hpZWZVcEkxYXVhbC91bEZz?= =?utf-8?B?MHNnRDZTZGs4Ykw4Vm9CUlZNdmdiK1IvNUM3ME80NXg3N1pJNFJXQm4rakZF?= =?utf-8?B?bS81RjRxMmlURTl6NUFLdmtXT1duSFhSQVRQUWJFcDZ0Z1A2a3lTV0lmT29E?= =?utf-8?B?aDViRzNaVzcrOHFSRVVxZjdzZEJyNkR6Vkh0SVRNYllpbnkrZ2VMbnBKRlJG?= =?utf-8?Q?ijsemSowqqp3gb2cjyQYD87xDbH3Kvvy1S3hno+?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6d177857-bdc0-4802-98e4-08d97eb25a03 X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB4835.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 16:51:13.0553 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dd7Ffst1MkgQTg9P02Xx0bgN0Lo0a9hVvZf99/wekkiAKjDQfVziIe6AeGCPMh7izmcpC3ecGJcLWNhMicFgMS1OM8yJpWE79MgcI6AHahg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4368 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10116 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 suspectscore=0 phishscore=0 mlxlogscore=999 bulkscore=0 mlxscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109200000 definitions=main-2109230103 X-Proofpoint-ORIG-GUID: oPl6L-HUGR9yOnikpT04dp9ZVrzE--qJ X-Proofpoint-GUID: oPl6L-HUGR9yOnikpT04dp9ZVrzE--qJ X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 4A7D420019D7 X-Stat-Signature: dueecwap8h5qao1o88xdj7wuhibxoyio Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2021-07-09 header.b=ePE7ytUm; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=jPQBityk; dmarc=pass (policy=none) header.from=oracle.com; spf=none (imf26.hostedemail.com: domain of joao.m.martins@oracle.com has no SPF policy when checking 205.220.165.32) smtp.mailfrom=joao.m.martins@oracle.com X-HE-Tag: 1632415907-465832 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 8/31/21 6:05 PM, Jason Gunthorpe wrote: > On Tue, Aug 31, 2021 at 01:34:04PM +0100, Joao Martins wrote: >> On 8/30/21 2:07 PM, Jason Gunthorpe wrote: >>> On Fri, Aug 27, 2021 at 07:34:54PM +0100, Joao Martins wrote: >>>> On 8/27/21 5:25 PM, Jason Gunthorpe wrote: >>>>> On Fri, Aug 27, 2021 at 03:58:13PM +0100, Joao Martins wrote: >>>>> >>>>>> #if defined(CONFIG_ARCH_HAS_PTE_DEVMAP) && defined(CONFIG_TRANSPARENT_HUGEPAGE) >>>>>> static int __gup_device_huge(unsigned long pfn, unsigned long addr, >>>>>> unsigned long end, unsigned int flags, >>>>>> struct page **pages, int *nr) >>>>>> { >>>>>> - int nr_start = *nr; >>>>>> + int refs, nr_start = *nr; >>>>>> struct dev_pagemap *pgmap = NULL; >>>>>> int ret = 1; >>>>>> >>>>>> do { >>>>>> - struct page *page = pfn_to_page(pfn); >>>>>> + struct page *head, *page = pfn_to_page(pfn); >>>>>> + unsigned long next = addr + PAGE_SIZE; >>>>>> >>>>>> pgmap = get_dev_pagemap(pfn, pgmap); >>>>>> if (unlikely(!pgmap)) { >>>>>> @@ -2252,16 +2265,25 @@ static int __gup_device_huge(unsigned long pfn, unsigned long addr, >>>>>> ret = 0; >>>>>> break; >>>>>> } >>>>>> - SetPageReferenced(page); >>>>>> - pages[*nr] = page; >>>>>> - if (unlikely(!try_grab_page(page, flags))) { >>>>>> - undo_dev_pagemap(nr, nr_start, flags, pages); >>>>>> + >>>>>> + head = compound_head(page); >>>>>> + /* @end is assumed to be limited at most one compound page */ >>>>>> + if (PageHead(head)) >>>>>> + next = end; >>>>>> + refs = record_subpages(page, addr, next, pages + *nr); >>>>>> + >>>>>> + SetPageReferenced(head); >>>>>> + if (unlikely(!try_grab_compound_head(head, refs, flags))) { >>>>>> + if (PageHead(head)) >>>>>> + ClearPageReferenced(head); >>>>>> + else >>>>>> + undo_dev_pagemap(nr, nr_start, flags, pages); >>>>>> ret = 0; >>>>>> break; >>>>> >>>>> Why is this special cased for devmap? >>>>> >>>>> Shouldn't everything processing pud/pmds/etc use the same basic loop >>>>> that is similar in idea to the 'for_each_compound_head' scheme in >>>>> unpin_user_pages_dirty_lock()? >>>>> >>>>> Doesn't that work for all the special page type cases here? >>>> >>>> We are iterating over PFNs to create an array of base pages (regardless of page table >>>> type), rather than iterating over an array of pages to work on. >>> >>> That is part of it, yes, but the slow bit here is to minimally find >>> the head pages and do the atomics on them, much like the >>> unpin_user_pages_dirty_lock() >>> >>> I would think this should be designed similar to how things work on >>> the unpin side. >>> >> I don't think it's the same thing. The bit you say 'minimally find the >> head pages' carries a visible overhead in unpin_user_pages() as we are >> checking each of the pages belongs to the same head page -- because you >> can pass an arbritary set of pages. This does have a cost which is not >> in gup-fast right now AIUI. Whereas in our gup-fast 'handler' you >> already know that you are processing a contiguous chunk of pages. >> If anything, we are closer to unpin_user_page_range*() >> than unpin_user_pages(). > > Yes, that is what I mean, it is very similar to the range case as we > don't even know that a single compound spans a pud/pmd. So you end up > doing the same loop to find the compound boundaries. > > Under GUP slow we can also aggregate multiple page table entires, eg a > split huge page could be procesed as a single 2M range operation even > if it is broken to 4K PTEs. /me nods FWIW, I have a follow-up patch pursuing similar optimization (to fix gup-slow case) that I need to put in better shape -- I probably won't wait until this series is done contrary to what the cover letter says. >> Switching to similar iteration logic to unpin would look something like >> this (still untested): >> >> for_each_compound_range(index, &page, npages, head, refs) { >> pgmap = get_dev_pagemap(pfn + *nr, pgmap); > > I recall talking to DanW about this and we agreed it was unnecessary > here to hold the pgmap and should be deleted. Yeap, I remember that conversation[0]. It was a long time ago, and I am not sure what progress was made there since the last posting? Dan, any thoughts there? [0] https://lore.kernel.org/linux-mm/161604050866.1463742.7759521510383551055.stgit@dwillia2-desk3.amr.corp.intel.com/ So ... if pgmap accounting was removed from gup-fast then this patch would be a lot simpler and we could perhaps just fallback to the regular hugepage case (THP, HugeTLB) like your suggestion at the top. See at the end below scissors mark as the ballpark of changes. So far my options seem to be: 1) this patch which leverages the existing iteration logic or 2) switching to for_each_compound_range() -- see my previous reply 3) waiting for Dan to remove @pgmap accounting in gup-fast and use something similar to below scissors mark. What do you think would be the best course of action? --->8--- ++static int __gup_device_compound(unsigned long addr, unsigned long pfn, ++ unsigned long mask) ++{ ++ pfn += ((addr & ~mask) >> PAGE_SHIFT); ++ ++ return PageCompound(pfn_to_page(pfn)); ++} ++ static int __gup_device_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr, unsigned long end, unsigned int flags, struct page **pages, int *nr) @@@ -2428,8 -2428,8 +2433,10 @@@ static int gup_huge_pmd(pmd_t orig, pmd if (pmd_devmap(orig)) { if (unlikely(flags & FOLL_LONGTERM)) return 0; -- return __gup_device_huge_pmd(orig, pmdp, addr, end, flags, -- pages, nr); ++ ++ if (!__gup_device_compound(addr, pmd_pfn(orig), PMD_MASK)) ++ return __gup_device_huge_pmd(orig, pmdp, addr, end, ++ flags, pages, nr); } page = pmd_page(orig) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); @@@ -2462,8 -2462,8 +2469,10 @@@ static int gup_huge_pud(pud_t orig, pud if (pud_devmap(orig)) { if (unlikely(flags & FOLL_LONGTERM)) return 0; -- return __gup_device_huge_pud(orig, pudp, addr, end, flags, -- pages, nr); ++ ++ if (!__gup_device_compound(addr, pud_pfn(orig), PUD_MASK)) ++ return __gup_device_huge_pud(orig, pudp, addr, end, ++ flags, pages, nr); } page = pud_page(orig) + ((addr & ~PUD_MASK) >> PAGE_SHIFT);