From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D78AAC636CC for ; Tue, 31 Jan 2023 17:54:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72F506B0074; Tue, 31 Jan 2023 12:54:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6E0B46B0075; Tue, 31 Jan 2023 12:54:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55A116B007D; Tue, 31 Jan 2023 12:54:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 470F66B0074 for ; Tue, 31 Jan 2023 12:54:57 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0793616031B for ; Tue, 31 Jan 2023 17:54:57 +0000 (UTC) X-FDA: 80415845034.24.D06C7E5 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2085.outbound.protection.outlook.com [40.107.96.85]) by imf21.hostedemail.com (Postfix) with ESMTP id 28B2A1C001A for ; Tue, 31 Jan 2023 17:54:53 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=qqN7g1Tf; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf21.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.96.85 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675187694; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KY1lpu9KlXeOVvOP2jpOYx174PWpQ62RPpzjLzusRyU=; b=5GaCCrtBpX32Mkw9DHsN94la2QUKgUryemwNCq/T4kgw+gitIb5ClO7WeggHCs+zwuVr8D gDumXtbm2mrNqG9rL9uLH4V12TT8LTsCIjMbN0uAdD1vc5VMMkjDZskKsNz2HMhHMrkej9 5GnrZYtlRiITibmDVn0LqQCa9BJkgRM= ARC-Authentication-Results: i=2; imf21.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=qqN7g1Tf; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf21.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.96.85 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1675187694; a=rsa-sha256; cv=pass; b=3KNDR8JIRtCT8dz4Tc16Wtw1QOko4M3IuPtJ4+oHgcZkEtkuGClv6fF0iOHxZVYPA3UlCp Xx3Gm8zCRIzdsWKX8GT7zSfOjEbbh6yqqZ3zCrZSP9E1Aw+r3WrH3nJWyzFX74XmYYv7yj pycZp1qLkZ7Aw2MqFzIS4cx8FzATcx8= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NZoCLESe1IogTK8Qaort1O0mQ7xdtd82q2dLGM5ekLcZu4SNwYrh7/Y78zP6Lu4tv709lc5ZN8+soXRft+qxJBwVtw4818xzGnnqzIab3N+rMhXdbRKlVe3ZLhZPCc44lEZaxSEAXt2I3Ralsj1iNSCfIs/M3OFJs3W1MtDeZYIl+mytIidniDMts9r0BtL20MUiU7hPCe01MrH51WPUmGbLrenT+IaBOtR6ljLYg467qNMf3PRkDC6cW1zAbQ5XZWAJ9n1L78hPxWzYs8IacbXPJ0OiOkTdEt5r6FWJjJAWfx001goQGaXBGO5XEhgoHL3elbdhhP5V+XFAeZ1VIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KY1lpu9KlXeOVvOP2jpOYx174PWpQ62RPpzjLzusRyU=; b=IWaPelMNlyRaWLfaVk7HuQd5V1IMtoqhhLstLYV0tOQ3aRjAYDj1dVqfdgNb/Ey/GOsDaPNDYpRi2cIwvVs5u7yzHstFXPAfaZ2WDw+MkLbbk7kQjz/bdna2rQ/jmsXa4iOHQ2Kubz5U9ljo345OXhrO48yd5W9kwFc1IR+GuC7FTPxzYgzKTpHjJ9jZGfZsTv0/jmvzjZ2tEINhRSrx3LWm+r6nVA50aBCCQC5NyPBDBw7tQODvVMkQrOcAFE/przIyoCyOyKeuM9vrLYcMK+ybiIiUofSfgHBjoyXVYKuUpm+BiS7yzPhzjbWvo0eUNRKgOx3fGcjsNPd7MlGCnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KY1lpu9KlXeOVvOP2jpOYx174PWpQ62RPpzjLzusRyU=; b=qqN7g1TfW+QoJX/lzbAAY1ShRbEgttt8UvL3msI7mqz1wrSdstF2ASHAsLnfJRM7S9LqFbuxkbO5qcGbbWdMkoIwNdTtVGsiDaHgQQsi+pULn0nJ+3Hd55Yf02m0V1fmroyykRkmzwljbquluYaLsUll/JuWfI1KLggAtC09qpw1gobmPqWVGZjDhrkIzeZ1S0Ez1PYry2V+yRbdW6c2kOFwbLf/R5n3U3icXT/NRasSOY2JUrh9pKWhlOboBnpHykaHkf1YPSd+oSh9ISgTFCpBsKtPvUckAiuKisXAFZoxg7pUQTqNbxgxYQaPmc+E2Vl7LP7ZIhCnbekU+W7CRw== Received: from BY5PR12MB4130.namprd12.prod.outlook.com (2603:10b6:a03:20b::16) by IA1PR12MB6330.namprd12.prod.outlook.com (2603:10b6:208:3e4::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6043.36; Tue, 31 Jan 2023 17:54:51 +0000 Received: from BY5PR12MB4130.namprd12.prod.outlook.com ([fe80::3b11:5acd:837b:c4c7]) by BY5PR12MB4130.namprd12.prod.outlook.com ([fe80::3b11:5acd:837b:c4c7%9]) with mapi id 15.20.6043.038; Tue, 31 Jan 2023 17:54:51 +0000 Message-ID: <0fc7fded-f6f5-e7e4-1b3d-3dfd224d9458@nvidia.com> Date: Tue, 31 Jan 2023 09:54:48 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.0 Subject: Re: [GIT PULL] iov_iter: Improve page extraction (pin or just list) Content-Language: en-US To: Jan Kara , David Hildenbrand Cc: Jens Axboe , David Howells , Al Viro , Christoph Hellwig , Matthew Wilcox , Jason Gunthorpe , Logan Gunthorpe , Jeff Layton , linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <3351099.1675077249@warthog.procyon.org.uk> <3520518.1675116740@warthog.procyon.org.uk> <040ed7a7-3f4d-dab7-5a49-1cd9933c5445@redhat.com> <20230131122857.6tchetnsgepl3ck3@quack3> From: John Hubbard In-Reply-To: <20230131122857.6tchetnsgepl3ck3@quack3> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: SJ0PR03CA0363.namprd03.prod.outlook.com (2603:10b6:a03:3a1::8) To BY5PR12MB4130.namprd12.prod.outlook.com (2603:10b6:a03:20b::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY5PR12MB4130:EE_|IA1PR12MB6330:EE_ X-MS-Office365-Filtering-Correlation-Id: e68d6f18-92fc-4013-3b64-08db03b44039 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Mnawtus+fVEYLlsoyASMJTufIh082mDCAndZ6L2FyA1MldOzhfPP3P2QBM1NITBlb3CXdipriV90JPpb6B0xPhdlYkS4COOwJ3OsrkKlnA0/CfLMMCW8txhKrKjWwe4rxJXwqA1wTB3zrdKJro8eSrvXRqI4ndTAFlMm6IUtidNo9CU4jJf4z6N1yY0kdfEjCzQS3mDYS3wH5aCIO8RBzXmKWNA9pky2vkBMw1x12gPvjmqFpYHU6OW/manzxD7rV+UwjgH4eJwo8zz+mg1II6q80Vh0JIV+HZ6xejiBU6ceVchSWtr7KtBZ7vdTf1BF0YjGi+3yRVt9WbJPccL8BtrStEfB8ZhpAFzATSvWmOnkoqDKwYXaSP0FRuOqpZYmMXyMPC5GyDy3Hl1kfGI4pBYNjjJigwLoHGvkiUocgpsaJWG3VLpkJ9VmhlJCaqLCMg3qW6FwE6ajFDq35QrK7Fx9o1y2ygek9PGbkdiyjepkG6BAXw+SBeyOuB8hrjlhAM5SmzPjyc/4vDChvGRjLrYhUc3ya3SzyyLcoCab8JXVpc+8WvxULt6XpDE8AdAg+BHrJXBt3dj9Ir0pCBb85KZxh87YtNKnT3xCGEZH3H2I2XDeiW2dmZD5QjCN8RzcT0/8Sy1PoWXi9QFEooI6CJHPCFlcyzNOZe9ckIGS8TyDdQ6UlbdAq6zCg5sPK6h8tviJvjM23HEh5JdxCMrqO1JdphOviKjharpOdVc8r1U= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR12MB4130.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(396003)(39860400002)(136003)(366004)(346002)(376002)(451199018)(31686004)(2906002)(54906003)(26005)(186003)(6512007)(478600001)(6486002)(316002)(110136005)(31696002)(36756003)(86362001)(38100700002)(66946007)(4326008)(8936002)(66476007)(2616005)(66556008)(8676002)(6666004)(41300700001)(53546011)(6506007)(7416002)(5660300002)(83380400001)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bnY3OHRBY2MrOG1iUnR1bmo2Qko5Ym50NE42V0RwKytBWHJkUU83cXNCZnhB?= =?utf-8?B?MmptL1NNZldtQ1ZHZkdrS0F3UkVRam9zdFJiS1FCeWRhUzJML1hsYWkxNk9R?= =?utf-8?B?a3dOS3ozalVmM0g3Wk9mckQxMHB0OWdiL3pucnFiaHpDS1VROXJpa3lUWEpq?= =?utf-8?B?eFRRd1dlNkxTN2xtZ2lCS1VjZ2ZyYWV6a0lPRU9CQllaREZ2d0ZpNXh4TEZW?= =?utf-8?B?UVlGZ1lualBSYlVrNXR4aTBPOHZDK0FQaWdDeTJmQW03ZHpxbFVmbWVGK2VH?= =?utf-8?B?Slh0Y2hpWTN2NjhXbEk4OG5IYVNPRWRmTGxTU205Ymc1UmRiV2JxWjc3N25y?= =?utf-8?B?NHROZFlHU0N4dkFCbWtKcFJpa055TCtIUjJTYVBVOVNGNnhRZENTbGtFei9R?= =?utf-8?B?dmxNUk9WTmhWcDJKbEZCQ1FxZFlFamtqRjV0Z3o3ejlOTnNyeUY4by9lZ1Ba?= =?utf-8?B?dEpHRWpRSU9ZSGJEQ3VCK1EyRXNBdVlxVTRlVktucmdUUFZTcC9MQkRIMjdj?= =?utf-8?B?L0tQOExaK1NTUHhCVGtoakpnQU1SNUJ6Nml6MlFOR093VXVXc3VkUFV3SFY5?= =?utf-8?B?MFNPaFlxZmdINVh2UHBYYU5Ja21XWllMMHN5VDd0ZmRTZ0ZoYWQ1MWFrTEJX?= =?utf-8?B?bStMaitLbnVURWMvVDBoZCs3MVpYOUZBcXRZZFJtVUY1dmFnQ3RYVHdjcHZ4?= =?utf-8?B?eFo3QTZSODJCT1pwM0ppYnVsU21JVFFlaG4ycHc3TFRHMU5wWTBMbVRFWVZH?= =?utf-8?B?WnBRbTMxRGVoZEUxdlIyZzY0ZjZ4WkhXVklWMm9meW13THo1dTB5d2NBZWZu?= =?utf-8?B?MHI5WG1OYzg3YnpERmd5WElSN1YvYTI3M2VOY0x0RXJSNlJEOVo3S255U2JB?= =?utf-8?B?Y3dkcEcxSHdubmxpTndwamJ1a29TL3preDJGV1EvUmRVelpyTGhlQ1FYZ0Zn?= =?utf-8?B?OHlBNnF5Ui9YSHlVRjlEYU5RQWY3bW0waHNCdTdXcHFJelZ0MVd1ZTV6MCs4?= =?utf-8?B?cTVYa1ZvQ0VNMjZBL3p1bStwQjBkNXliUFNwaTNBa2NvbkFObG1GN2s3YzJl?= =?utf-8?B?NE1uRUNkeG1UMmd2dzJFc0V2eG5CZUZhcVhMZzJkaktMbVJMUVVKTTUrOGE4?= =?utf-8?B?MFg4QnJYYk03ZnBsV3ExNFlteGx4cVFDWGJVTStFY2t4N0ZKT3ozbXpCYys0?= =?utf-8?B?MHRuLzdsNVFPNG1QOEtWREJVcWNaMVJlRnpVclRYbUoxd0FPOTlIZDNPcm90?= =?utf-8?B?WVBJYXc5NzQ3ZlVXQWlhT0FrbTgrSWJiNTZTMFFRbGV2bVFaMGtETmdING1W?= =?utf-8?B?STRPMDVKeWRHV0duWkRtNmh6dFdzY2JuM2JmN0dGaUE0OTFneC9YZi93czA4?= =?utf-8?B?NTF5b3RCS0twVThOZDlhc3NJSGxLeVZPeEZsV0dNVTdRc211WHVzWk9VcmE5?= =?utf-8?B?QlRoZTVwaTMzRHU3OHlhUVZ2VE4yaFNEVm8yd2dDOFNYVmFDaDVsWUZsTjhz?= =?utf-8?B?cjBha3JmU1ZBTzl0am05YjZEL2RGMXR2azZLMnFzYlZHNzQ3MUVFbzJ1ODlx?= =?utf-8?B?QU9EYkVTOWJZQzhhanduQXI4cjR0bmljRFVob0EvTFZ3K29rUXY4ZW9MUklz?= =?utf-8?B?MmxOMEpzQUpxVGdySzJZcTJiSTVvSnFyYVluSDdNMzNZUTZwZ0ttdE5zUVBB?= =?utf-8?B?emN1ZjBhNXpBaHlmNkxLcC9TbHY3YThlNWt5UFBNNDhvd2RESVlsTGtaUm5Y?= =?utf-8?B?YjF6ZmhjY0VxMmx1N1BrM0NRbGFWRHZyelZ0TDNscEdFOXM5T1ErdWlxZUE3?= =?utf-8?B?QnFaNzMvUU8zTE5MMXc1bmtRMVBYUzJPMElRRDZvS0F2RzhCOGI5VEo0UXhS?= =?utf-8?B?YnVJZGU2MXdJUnNaSXFyUldtU1drTzVZRjRuSDVpS1U0ODc0TGordHhrY015?= =?utf-8?B?TFdud04vdlE3Mit5UTZzcWNyRjUvV0RVNXBnMUtPVlpqYktzUHh3WCtDY0FT?= =?utf-8?B?eVMxa216UVJWVmRqQjBHdmZSTEF4NVZQc2tyYkdwNGdrMVAvZFNnZ1NKTXVL?= =?utf-8?B?ZE5wMHNsL015WVF4eEFjNkhlWE9MenBxZit5T21kT2FjOWU2RVJVRU0vSm50?= =?utf-8?Q?9O+rhEw72uXDnJHKU1Q8VYG0z?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e68d6f18-92fc-4013-3b64-08db03b44039 X-MS-Exchange-CrossTenant-AuthSource: BY5PR12MB4130.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Jan 2023 17:54:51.1264 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: JmWGNpCLv7S9Wq9r9nOpeoLPx+qIy/NPeYX34uqYJ8IOsg3Ma/8jcg0JJM/QTRiII6/HqJbmdU29WLFYWXTdwQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6330 X-Rspamd-Queue-Id: 28B2A1C001A X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: f19zibxyidggi4p9h6mh8uk1y5ur6ieq X-HE-Tag: 1675187693-127208 X-HE-Meta: U2FsdGVkX1+WnFUfGVBG8sgaSmUbVR1LlxlY1jB5kNCsIdKVq+IaZTwbe0AKFFPQQVTq8ciNHlwvqTVlL5CaYYlTw0X5wqvBxuQgWSkjkWbJ+fRgGyG/aJO1Wca6ZeH1JF+Z7Zx3wy6KAuoV+hJPInlZ4aiR1bxG1xE7YNUD3S/4jOLPfzj7kF0Qukq/Y6DXFsq7kNsalWsbCUT+nbtI9e2jkp/LojEBt3ScU1/6MJlcRoqYUHHaf55RncFT830IQMQI5kZoLI6n4AZvlOsDU5BxT6kh49BglaCVgZlks8q5e9DU5WhAdWd6S2TQJnGaBLodGXXSzKtueyDxtWqz1CjZDQSixuo5PpVbTA3hm2539wSH2UXiounCNDJCtR2A11pKo+MFRAYb5VY0pLDeT79zXpcRA2leLGh9WYTbCulA21WSKS7M4hVdx0w0F8N+2bsCgPpRT+7ALJhND+HLRQqd4487m+exIjU79tdevqUqonJaFxcU0KP80sV47dBk4cUjlGPPAZaYmr8H9+2w3qIbZrXXT8VGeyZi3EKOtu9CKUs5QuGO9TanHwChqc+BSpPlKgRnnNzS6aJcuUfZHCtDicnoQDWP12EC9xhpaIkwWDGdyWbZiKmLj5Z4sfyomJRz7Jg8OuO2D9WNhOM87IqNUm0K4ruHo86TdtpbQSU+GvA5DGZSD/dnw7BBNQzWUwc60qFF4vIwgloY7irBzy9Emj6/pjEwYjFJsoBx+DvslXoTUq1SEz8fUcWdOjoAP+raI11F7xPX8ia+V3Ifn2Tf+reBiZKbFSxsgB2NagQ3lMKtEA3oduKkCBzvhx07mx+NtZG1wDtI3Ln7bCyC0TOr7rybsFhMpZ+r6vU3d1W5M3nyhQQbAoJoAVlFqklfj9IfZyPlQ0BKAEBIKtI7kjIcxb28eOPMvyNApXGDQLnOKNnmjekQ3UbWbWl9x8ihAMXdD5JYw3Y0z6VgVse UDGcoqQ0 XX4M0CkUtGqo++ygkUVKCbM9pXG2pBVgFimhOO1XUTp9GnGFgqKxf4givyrbjL0fguO2oaerRls/Cd7TH3OZOAxmkXYSxhHFEg9mVwSITRYQxrtc7JO6HtTcC5o2koTC0cIOjxcDfJL77zxV/TZQfJPIeWkiRVk2zo3m5kUADOZJTFyajiCRXgwSEGJLUUbaGv8HfTElNF7B7UVPiuRmim1bW9scEo8CFEqEtswCLPk49ifVre6mgtLAMLwhj9TQqIn611ufHG/pYSzgsk4wVZ7HqC5O3HAYJ1ty9O25eQ63SdOk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 1/31/23 04:28, Jan Kara wrote: > On Tue 31-01-23 09:32:27, David Hildenbrand wrote: >> On 30.01.23 23:15, Jens Axboe wrote: >>> On 1/30/23 3:12 PM, David Howells wrote: >>>> John Hubbard wrote: >>>> >>>>> This is something that we say when adding pin_user_pages_fast(), >>>>> yes. I doubt that I can quickly find the email thread, but we >>>>> measured it and weren't immediately able to come up with a way >>>>> to make it faster. >>>> >>>> percpu counters maybe - add them up at the point of viewing? >>> >>> They are percpu, see my last email. But for every 108 changes (on >>> my system), they will do two atomic_long_adds(). So not very >>> useful for anything but low frequency modifications. >>> >> >> Can we just treat the whole acquired/released accounting as a debug >> mechanism to detect missing releases and do it only for debug kernels? > > Yes, AFAIK it is just a debug mechanism for helping to find out issues with > page pinning conversions. So I think we can put this behind some debugging > ifdef. John? > Yes, just for debugging. I wrote a little note just now in response to the patch about how we ended up here: "yes, it's time to hide these behind an ifdef". thanks, -- John Hubbard NVIDIA