From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FD80C4167B for ; Fri, 8 Dec 2023 01:31:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A30196B007B; Thu, 7 Dec 2023 20:31:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E02C6B0080; Thu, 7 Dec 2023 20:31:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8824A6B0081; Thu, 7 Dec 2023 20:31:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 783C46B007B for ; Thu, 7 Dec 2023 20:31:06 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 4DC561A036D for ; Fri, 8 Dec 2023 01:31:06 +0000 (UTC) X-FDA: 81541922532.21.D4EB975 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2067.outbound.protection.outlook.com [40.107.102.67]) by imf02.hostedemail.com (Postfix) with ESMTP id 7373E80008 for ; Fri, 8 Dec 2023 01:31:03 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=ubxZXiHq; spf=pass (imf02.hostedemail.com: domain of apopple@nvidia.com designates 40.107.102.67 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701999063; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=V2Fkv8huytLjMsn7KFu0Hu/6eMhU4MYqMVQWUbQsWr8=; b=U0YFxyIcfirWWSpvzaXi458cuh6+X/ohAXR6vrecH4OiWbFwJ2Lxj2YrPjYNHxrebfJQiu 4qQn2wRWWH9MqHuRUscKipnkV9l2z0ITpicXdYD8RKNxBU1aLgM5iXiWwDWCz3qd85OuPJ gAFs0NLtg4AOIMoW1FASg2r0zvKxT5E= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1701999063; a=rsa-sha256; cv=pass; b=ZK6fbKcLoAXt2XZ1pm5CSQn8LH7nWYjxBFgQRh1NUD4Fse4LFmoVpazbNAY1QP4R5j7IjT tQ+ipvcpogHqLQqw1c7K03O6kgzCJl+NeGQS+JdfPYwYhX/hTBvJkJpZH4VE5c37qzXNZY 4Sszi3zk1JO8NHeMIjv6rt8Z0tQLm6c= ARC-Authentication-Results: i=2; imf02.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=ubxZXiHq; spf=pass (imf02.hostedemail.com: domain of apopple@nvidia.com designates 40.107.102.67 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=j0W8tOX9e2ZOLjts2ybo06bwvnorvRQsHVmI1fm4rM1Lc1OmNjzhmmbZwwHc5RF+xbG2Q8/jpG7UWbCJEnrsQ1fJrYr75wByUuux5f4mMLgvDduXLxr8MQLg7WY3txhOpJkV1GOsxWwiK2/hmWjO9V7N/YDen2eybo5TPi3SMh/UG33vFFu0WzkZp+vQhsrw+JbH1o0Pz3uP0J9FP+eqyoUVvf1QqxKBL79SYk4//2lWqvFpcOTTOuZFWVBGPSJU36A8aOA5VSMkSc2uqBq4/TSeUqSLE2Q1++/OIIu503e7d2WDZVoSBiqsfKeXipLP0b5XwZX2aL3eYTv8kfpjWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=V2Fkv8huytLjMsn7KFu0Hu/6eMhU4MYqMVQWUbQsWr8=; b=cLlhk4t1ZyMIafZLeEPd7n7D6NdLTJgLgHX9qKoviLVgskWqcwqbO21MRcOLbgTSPg4lX7eOZnprdUPp8p3eejsz6PMgvsuuhPjJID1PYx0ujbAqLOtDXJWtw0oPnQ1M2UMEUwO/Byj4ugzbuIDkkkXvwFD6ZKWrKdT9xkeex2oO/5gMeTA/aHYE9ivqJJF/8oTO/Z+JXcSYGa1yTJLy0lr9YbmiZWyRCvTAy6TEE4IWB8wJxk3iwXw1AJ2k0mIPypKgyVCC4CO2d5GsWSpH5wxq/2CvbQZnleGIasFOegcp+KbODHJf4BunN61dAKqjQ5PzBbjAU2jqAFGAFGIAGg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V2Fkv8huytLjMsn7KFu0Hu/6eMhU4MYqMVQWUbQsWr8=; b=ubxZXiHqJspjXy3Y07aFZvM0DCFhD2sQ/XPPbsdt52qr+iWKjtBSis6Ij/IO33jHkgiu7Oe66/AKUluvcWtOkHq7UhcY9e82rxubocGHy1943nKtzWzrB/poUiXvnpex/RuGwjtKvYBTUkI0pmoPQYmFOmU9lEEW8ta1vXy9KP/FhTXzeI10G6ZijHvJpuZ16AiwgAumo1vvW1YDQCBN656mwqIq+5n6BcpzzlCy3YW9tpyAYdlOjFDOYFW8kVKCQaqy6bkKhjnj+CnZ7ScMjpIduMNfDb+DltIv+dy2848Z1hdJSKOFaVhyYVXRHJHKt/dUaY6BFL6iPRKQ/klTdQ== Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by BL1PR12MB5828.namprd12.prod.outlook.com (2603:10b6:208:397::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7068.25; Fri, 8 Dec 2023 01:31:00 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::b8a:1b58:1edf:90e6]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::b8a:1b58:1edf:90e6%7]) with mapi id 15.20.7068.027; Fri, 8 Dec 2023 01:30:59 +0000 References: <20231204105440.61448-1-ryan.roberts@arm.com> <20231204105440.61448-3-ryan.roberts@arm.com> User-agent: mu4e 1.8.13; emacs 29.1 From: Alistair Popple To: Ryan Roberts Cc: Catalin Marinas , Will Deacon , Ard Biesheuvel , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Andrew Morton , Anshuman Khandual , Matthew Wilcox , Yu Zhao , Mark Rutland , David Hildenbrand , Kefeng Wang , John Hubbard , Zi Yan , Barry Song <21cnbao@gmail.com>, Yang Shi , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 02/15] mm: Batch-clear PTE ranges during zap_pte_range() Date: Fri, 08 Dec 2023 12:30:40 +1100 In-reply-to: <20231204105440.61448-3-ryan.roberts@arm.com> Message-ID: <87h6kta3ap.fsf@nvdebian.thelocal> Content-Type: text/plain X-ClientProxiedBy: SY5P282CA0125.AUSP282.PROD.OUTLOOK.COM (2603:10c6:10:209::14) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR12MB3176:EE_|BL1PR12MB5828:EE_ X-MS-Office365-Filtering-Correlation-Id: 988e53ad-4cbc-43a3-bd50-08dbf78d54f7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YzNxfdVaucsHU86tODw128/iKp/IYgd3HFvSKEK6EuW/v/rSqWBskzFJnpYS6yjRNQRBu6/sRXIAR6nqKQyG80AItcIXy0r2y72aX/fz5QTEcSoKleSHDR+XtLdz7KdMPHG6rc58oeGIvuSFqL2mh4GHgLGs9V6V51kTIISIkHpAaWmtJGCWvwPE7vcIFov/cYNWTXZOhaPhCXquU40GP4/pQbbFHgJl7Zykrs+0tOHh8ygK7Yj3HY5j/00rk2ylQeIV+hF+kfLSpWA7hLf3lx9jRVv9qD8L/OZ0i/uQqcBXIIA0Z8ulwUM/509o03mNalrV0QNGEGRLNtE6ssIFcixVk5LkfbMffXEsX9XvCoRcmepe/8IzEG1SPT5OjAQ4zhpmkqGWFv8wMBy8oQ8fdAKMtH+lfHQ1jJFcjgTfawdZWdxAlVoD3vjehyVmQMJZlhrJgx2xcbE+hGZfJkDdl69M6yd6AKZdQOPubUMDaHlAIGoIJ5l6CbV5w+GxYAMRxuAmls3cm0g/PPyEJrFpO5v1tNfjy8s5h2L7r6UQJhbbaVrCRM8R6KNbwk4tVHVd X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(366004)(396003)(136003)(39860400002)(376002)(230922051799003)(451199024)(186009)(64100799003)(1800799012)(8676002)(8936002)(7416002)(4326008)(6486002)(6666004)(6512007)(9686003)(6506007)(478600001)(5660300002)(316002)(6916009)(66476007)(54906003)(66556008)(66946007)(41300700001)(2906002)(38100700002)(26005)(86362001)(83380400001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?BW91A+iigNx64R+C7naPWZrUGn6YaWMe1I59dDd9rcWVqNz/6Jrrl9XKjfFy?= =?us-ascii?Q?yRKb+33HfsnykGn/PPFVNtQDdihg8NlJTEE8lpifdaNIgpNVAc6dqzFrm+7M?= =?us-ascii?Q?5+f9nI0wiZ+mBV1fgeUIDmAVsPEUc8hXlnppRXuUDCmt7RJWx8H5IP/Agw/B?= =?us-ascii?Q?N3eXmNNumptUoxYcvmRaNsw6wkCmhcIVAec1v+82h2hF8DjYKvlavT3ABL6Q?= =?us-ascii?Q?E9ZWt/AKcQCBhMCZncIxYem4Moy1YJqvs+LEhXfFbdqM2hWTNbbBNLQZbwjO?= =?us-ascii?Q?xvKFxhbQeRMjmV5WLUPgrYd9OR0lPg3VvPuz6+2ZIkUZ4sI/N7HZJS9bMerq?= =?us-ascii?Q?aY42zu37yLHf1VN2x/T4hI1c2a+/CyhVtoxP7/PVzyEnLS/DWu7glOIkYKKx?= =?us-ascii?Q?uPZJljPr9SvaoTBh9q8rGIijgTeADy5JXYRChVhk0XYi5KKk4w0+HYb3fKKK?= =?us-ascii?Q?kY3J/lYKhOUAWD0iJ+QLtjuK+jXDEuaeWaPXFd3jtJK8cMeM5MmcaR8bRw5M?= =?us-ascii?Q?dCd5lWa1pBp37JROqZjr2WlpL8mhhZnQF7TI0r5Y/1SdbgUqt5FRU/8sPK6U?= =?us-ascii?Q?Mvqzp4QxB6KdidhPIBzSGqfgbaxcD1D5yNn7f9WgBp3xECNfXIQ6oW8fGcHk?= =?us-ascii?Q?t28uFodr60McPLm+3XZy7FZl238ce0Wp0KjO6td26YHDX8sOZCiEVjwpGc7O?= =?us-ascii?Q?buKeS1pbu0vJay0v9UP53yKJtHo66pM7wFRRS1JhCUyvFCpPoWEnAngqSwNd?= =?us-ascii?Q?DvzwSDaPp4AR+B9BVyH+tHSlrG6+b/rtBSKnzqKRByRZrA/L9Lhww1QosxPp?= =?us-ascii?Q?nCP/l8LPD5d1Unj0fOw4AlpEald3Hay83YL81QvhhdWGDwfX8uAUO8waLQEt?= =?us-ascii?Q?+QhcFriwg8/T8CZ8Pqx4UjzwQbiqQRsy8fXIlIGu2Pa5Nnt2funk3DCpvc1+?= =?us-ascii?Q?RzAUMg18ilcN4d4VsfeL/FJmGTfIpvbSpGM4mXfN8iUYP6Cvvms7Ab3DGjBy?= =?us-ascii?Q?DprjSypvzrmK4lzhD42yBTY8nBX0i6apmn0u0vULs1lcxJaxrju17lXpMkel?= =?us-ascii?Q?p7534dfjjMv8VSLNnWTaw5Xfa99wbkwM92wfu9CqluyZ/tGGQfG89UgIv/RF?= =?us-ascii?Q?mHMd4ZouUGNxL6LgWeJYOCIs34qZ+Z+rWsUmEQnElhMx7wbXVqXAfSSfCpaV?= =?us-ascii?Q?101axDzcIFbBlUqm+TU2VhkYhbmYSUFaO7HQ1MPARtHNFBx0mbikEDPtPDYi?= =?us-ascii?Q?81540cjSh425BcyZe7xoOey2ziwENTpdlTtvjfWXFr3oIRKnCtCryRW8h1Fd?= =?us-ascii?Q?+LACuPmnU42c9TiuodKm36J78JNy+9K0zJtkMIi/zjhs5ICZt3mL+vQGyHZq?= =?us-ascii?Q?9x1s9JuHr1zLcZw5eGRfFqRKKLPdPeT/LL65/VLUMgIdtxxUsfqGR1TZcZ+t?= =?us-ascii?Q?DcjIzYolaGVA7nDzWEZ+XJ2I+6dBNGR6+EmCJGe3ir6Cgy+Up2iLinf78wV6?= =?us-ascii?Q?gPNwJmDEjs9fEsZMmW68YSitUDA5tQzfMSIsYZXVA5Z3J6cKnZeoPSLhmOQz?= =?us-ascii?Q?gWp42GKaGBMDRUQQsZ1LAxWyIfhKoN9StoSbNjzf?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 988e53ad-4cbc-43a3-bd50-08dbf78d54f7 X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Dec 2023 01:30:59.5540 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SwzqxcqGE1I35K0kU/F/JurJB/tzz4PRB6yz2zN8tpCrHXn3Wa1pvu9XMsFF+jRIuoZJHr9xL5ejXJCuENbIAQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5828 X-Rspamd-Queue-Id: 7373E80008 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 939srn1tygx49m8h6esepuguapnjd8ob X-HE-Tag: 1701999063-395985 X-HE-Meta: U2FsdGVkX18mkSrCV7B2g0tyx1MVj6P3vBscje27g2bYpEHbJU8lKdqFTy3eHWYmDgy9Y6rtoI0LAlYyXT+SF473B8jvYdLgotgfmJMIz+8wx+ifhT0iyDI/7m+nlmoAkifEzvXnD///b9ghdKDzodyt1sR3I5RA9GbWyFTOzPhCUcTqjf05NwftzUPNFQ8ShfNi/kJ1G6t3FklSqB8koI+fcUWplOE7Zy2GcTEylDi/t6RGJt+sjv7dFE7qa9bHSEagSZUKyn7lrkT+MSrdPgoatfqP9Vn8BjTiSdTZ/l5HKFd8Zh/YFKPjDbQC0z7NCGDF9sNzCWrZMe1qcvOCm55uqxeBFYrezsxIzM+cF+nqKFQbWz0ovurzX5j3YXj/TY1GZZILLNpB+jb/+B1p3vdjcMM+uJ4Oevz09DaZ430RmDBXBwlDBBdIm8xiZ5U6NrJfrJECGzjemfODTYBrNzA/7MEFM86ePbQJLmLTbfJGMlwJSHGTaz1Z8DoPfqKV9QBI+9QNuBnvyAED6YiE4XDAatz1PNKngots91zvX6eA5+gKfnPrRuQolSLNmZwg+tS3CGIbEWdu34oedruFOtx3JJQQwMcUHJJBxR73Wg+SBitRTB7B+krPgnoRoftf9Pj4Jzb4ON4FQw0YV8pS7hRyQc3wX2Q3qk/mjZD3n32Y7OKWE3hDCV3QXhKvjtkt3DAojp5g7GFdm98oZcx/tXEjwTkWJxfY3tHqRzX8y/GfLE/1tLpa+g7ybtMAi0ywicbEDBoLkJ4jq4HKiO/+ma+2oET6WvpqeE/Fo9TRsAeYQZszvBFnetx5JnJzhJIEqyvsj8jKS+MpfIWaOYUAL7Jp0atJlyY2/LE9ytdRr+O5pvpUa5/JUammMbyXLE8iFk23ecpvFcT58+FZ71PF4U/aeGkRT7hTFV812z4OloyZtJSbWkgtjCISRvw7DemtQq8Cy2A0rkKJii4a2BZ XEd5cuAg b7oaZgIQmfmbzHE69Dx1wn+cUwz+Vsf0YDPa/BYOqK33YmPyQZWdIPTAFYA4RaBfKmpQxyZxfo30tX/rM1+/inUyUU52Q0W0SJk+WqGLjiifUvgP3G3rrpfyNojV2eLB6UDvQisZbo4V5qlotnN5Eig/Dg6iNmZcxkfnJNYP+2NTCK7equ63fayTVmHlCt1ElqsUHIz/az3IepStLMbt64tJh/2RViLx8Yfp+cnWvDQk0DvpQezqpvUGcrFACgWt6dnOm0s+hq5wd5JqWJNiWGCI2mSGOHsnbfaQDxST10PR3H3o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Ryan Roberts writes: > Convert zap_pte_range() to clear a set of ptes in a batch. A given batch > maps a physically contiguous block of memory, all belonging to the same > folio. This will likely improve performance by a tiny amount due to > removing duplicate calls to mark the folio dirty and accessed. And also > provides us with a future opportunity to batch the rmap removal. > > However, the primary motivation for this change is to reduce the number > of tlb maintenance operations that the arm64 backend has to perform > during exit and other syscalls that cause zap_pte_range() (e.g. munmap, > madvise(DONTNEED), etc.), as it is about to add transparent support for > the "contiguous bit" in its ptes. By clearing ptes using the new > clear_ptes() API, the backend doesn't have to perform an expensive > unfold operation when a PTE being cleared is part of a contpte block. > Instead it can just clear the whole block immediately. > > This change addresses the core-mm refactoring only, and introduces > clear_ptes() with a default implementation that calls > ptep_get_and_clear_full() for each pte in the range. Note that this API > returns the pte at the beginning of the batch, but with the dirty and > young bits set if ANY of the ptes in the cleared batch had those bits > set; this information is applied to the folio by the core-mm. Given the > batch is garranteed to cover only a single folio, collapsing this state Nit: s/garranteed/guaranteed/ > does not lose any useful information. > > A separate change will implement clear_ptes() in the arm64 backend to > realize the performance improvement as part of the work to enable > contpte mappings. > > Signed-off-by: Ryan Roberts > --- > include/asm-generic/tlb.h | 9 ++++++ > include/linux/pgtable.h | 26 ++++++++++++++++ > mm/memory.c | 63 ++++++++++++++++++++++++++------------- > mm/mmu_gather.c | 14 +++++++++ > 4 files changed, 92 insertions(+), 20 deletions(-) > diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c > index 4f559f4ddd21..57b4d5f0dfa4 100644 > --- a/mm/mmu_gather.c > +++ b/mm/mmu_gather.c > @@ -47,6 +47,20 @@ static bool tlb_next_batch(struct mmu_gather *tlb) > return true; > } > > +unsigned int tlb_get_guaranteed_space(struct mmu_gather *tlb) > +{ > + struct mmu_gather_batch *batch = tlb->active; > + unsigned int nr_next = 0; > + > + /* Allocate next batch so we can guarrantee at least one batch. */ > + if (tlb_next_batch(tlb)) { > + tlb->active = batch; Rather than calling tlb_next_batch(tlb) and then undoing some of what it does I think it would be clearer to factor out the allocation part of tlb_next_batch(tlb) into a separate function (eg. tlb_alloc_batch) that you can call from both here and tlb_next_batch(). Otherwise I think this overall direction looks better than trying to play funny games in the arch layer as it's much clearer what's going on to core-mm code. - Alistair > + nr_next = batch->next->max; > + } > + > + return batch->max - batch->nr + nr_next; > +} > + > #ifdef CONFIG_SMP > static void tlb_flush_rmap_batch(struct mmu_gather_batch *batch, struct vm_area_struct *vma) > {