From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60DC8EB64DD for ; Sun, 16 Jul 2023 22:30:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BCA28D0002; Sun, 16 Jul 2023 18:30:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6454E8D0001; Sun, 16 Jul 2023 18:30:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 498978D0002; Sun, 16 Jul 2023 18:30:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 370EF8D0001 for ; Sun, 16 Jul 2023 18:30:32 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0D282C01BA for ; Sun, 16 Jul 2023 22:30:32 +0000 (UTC) X-FDA: 81018920304.04.51D425A Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2052.outbound.protection.outlook.com [40.107.244.52]) by imf08.hostedemail.com (Postfix) with ESMTP id 28F14160008 for ; Sun, 16 Jul 2023 22:30:28 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=aDsNBPYO; spf=pass (imf08.hostedemail.com: domain of jgg@nvidia.com designates 40.107.244.52 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689546629; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cT6dL6NjKoIS/Lnw15pwIZwQG595wZvCh0+SZHCQAb8=; b=ecoTUHEJJcyng3jXgrVXh+Sk39qRWibdI4/EEH4MuUjlewRTLW5PQflLaJkolgr+e8sR0f RMlc8RNS6SSVGFVY+7O8RaxKNu1h1561xMcNS+pU1OFZmRmABz3g4PvJosZtwqZhNrs9x9 XC8npiAgW840JE53Wah6scnyY6uAPQs= ARC-Authentication-Results: i=2; imf08.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=aDsNBPYO; spf=pass (imf08.hostedemail.com: domain of jgg@nvidia.com designates 40.107.244.52 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1689546629; a=rsa-sha256; cv=pass; b=pj2sL2cD3NoRK94Awu4MK3y1sDSTdb8kgjmBZIl0yFgylCSlTxoO9xNr0FULFfk3N/Jpgc QxbT49zCf3Go4Gmg+XGFT22xISjfKbXMNkKG9j51f2iE1m/wJi3DJYKE/Tj7NASbrlZS0K Iq1SH/EyDx7T/YAip+IrznmVXm+8R60= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DB2OZSVyPWxc/s37D/crIr9iV3GV6K12Bol62h7ZJgZ9kAJpeWyvdoqNbIa6/YaPZ/M827qcjOBtE1RJYhJN0T7RKmj2YPCUJX11kKzGPp3zvoCWjD21YFm0WFCN4/l9e7WALqe6CsJjbbOQ6DmuToJNvztkAok704YSaa3mpyzQSObWwMw4YKeMZHTphJF6QTl0Z10Q8lMAtDrL5WH/73AgRbXAoJWlyfmKYufM3+dxMEfVwqgKzHGA2e7pvlQHal3X5zv53jP90VWgCUVvAlrZuiBsRmbSeCscZHFEhudI8NJL4kxO8mliOk24s2C4FdiREI9UU3WIBnmlwm5daQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cT6dL6NjKoIS/Lnw15pwIZwQG595wZvCh0+SZHCQAb8=; b=P1yT4D1xSBCkJwuzRvVJR+jZQUyS+BgHYwomdToyRDUrdjU0gVYDIfYdd5lK2BdaVa08AOMpo/NMJS7bSOr+su7mjhR2BRh1dQV6dxke67JB3gCVfc7rRkeBYN2LuJcvG9l4g9dXKDlXSwhlXJkjv1IwJnfI06s9jeJip0+UJsbwNb1l9HaXwmQ0NpO9aIx3KgclTbpHNZcF8IDorPqGK6xrp6mhcua0G8/3OjpL4B0Vi8YiOvv1BEaEFzFveiV3CICZt2ZGFTPbd/3otLRGAxCLdSeQ1iratcc737SJbG1/n/usB7XlaSKvHz9HJus8+C8QGqTGWRh/se/aEHKLZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cT6dL6NjKoIS/Lnw15pwIZwQG595wZvCh0+SZHCQAb8=; b=aDsNBPYOZEi33/1g7PqNJODssHsHfLmDjgQ0OQ7/Bwe5UpaBxt/0lUBNibEc35+zOa9dJI6VkuHvN49Jdg5wbRMHzkYATcQY4tOkRoNJPEKm+bWJf2iBZiu6yumbHjm0wULkTgJfu6YLIlivw9zfABusVlaa6QgDHYaPCNvJZeAD0FLk9Xu+bzMnHQsTSIAEXozjMs7qyqm44GuIuSSHBNHJWJsIIgxCvM8FYhP9Li16G5FwGVEZMPLt++v1fT7adWL5ZKEMeEJKGRda3Wb/EcY04j5zdfhwWWb77xmjWBKTkW75oEE7PqoCMgB+N2xjv57aAbnNugleUKV/ssbc7w== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by IA0PR12MB7508.namprd12.prod.outlook.com (2603:10b6:208:440::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6588.31; Sun, 16 Jul 2023 22:30:25 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::5111:16e8:5afe:1da1]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::5111:16e8:5afe:1da1%6]) with mapi id 15.20.6588.031; Sun, 16 Jul 2023 22:30:25 +0000 Date: Sun, 16 Jul 2023 19:30:23 -0300 From: Jason Gunthorpe To: Catalin Marinas Cc: Benjamin Herrenschmidt , Marc Zyngier , ankita@nvidia.com, alex.williamson@redhat.com, naoya.horiguchi@nec.com, oliver.upton@linux.dev, aniketa@nvidia.com, cjia@nvidia.com, kwankhede@nvidia.com, targupta@nvidia.com, vsethi@nvidia.com, acurrid@nvidia.com, apopple@nvidia.com, jhubbard@nvidia.com, danw@nvidia.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, Lorenzo Pieralisi , Clint Sbisa , osamaabb@amazon.com Subject: Re: [PATCH v3 1/6] kvm: determine memory type from VMA Message-ID: References: <20230405180134.16932-1-ankita@nvidia.com> <20230405180134.16932-2-ankita@nvidia.com> <86r0spl18x.wl-maz@kernel.org> <67a7374a72053107661ecc2b2f36fdb3ff6cc6ae.camel@kernel.crashing.org> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: YT3PR01CA0107.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:85::20) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|IA0PR12MB7508:EE_ X-MS-Office365-Filtering-Correlation-Id: b720841f-411c-4c5a-afeb-08db864c4011 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: G239vWAex6cj3lUh83fgAVlys5n4ScRq3qJEYmuLVJRtNUlsK56vvOL4qcgCAtUbww3tNdA1L2x4X8KeBF6gxGzD7doiiiaKcA8Qvnz2FBcUpZmLJLNPfLcSrWXND8KHDHNZ5a+G0t6ER5wuj5hqRi58W5NQKMLEihVnXeWdhKUMMSwSUgdKm45U0v0I00l3LlHb4LJGidrw70n3x+Lh5TjPF70yx0FDQOvFXJnhbevfbgPbtHqW40ovAj+xrye2QUB/JYI/c7pnkvqT2bcKiUY1Z88pKUZ8mdiJbUMKEtgxG0GSG5CBEw01HgA+Cxk0pKK1hw9YFipTneFf2JkM1ZUBftNoLOz0vTU5Iazmiu5+r/nk73D6xX5AOz5TBcyVAMAoKhDMaVIkzLlXHEl+PyL2P8D2FYVPmakFpb57ArNwBc56eos9eaFdhyWfXLP8iE+jSEzfbiEYHAzxuIoXBX0UIrScPG6JvkZWn2oO7aLjCXBWDg0nhctHe29u0fD6eXiFx9oLjsRsoUZan6wZQZvF/qZjksL7u0eORnIwY2PDXqIo1vM4+TzeDXbHXqaI X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(136003)(366004)(396003)(346002)(376002)(39860400002)(451199021)(6512007)(6506007)(26005)(54906003)(66476007)(66556008)(66946007)(36756003)(38100700002)(7416002)(5660300002)(86362001)(2906002)(316002)(6916009)(4326008)(8676002)(8936002)(41300700001)(6486002)(478600001)(2616005)(83380400001)(186003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?dOmWtTcJ7dWn4naN4pDr0YpY4LZKyiuyKq+4I/gSp/VgN/LcVsw8uy47K/lC?= =?us-ascii?Q?KWgMgz1AOdHRFnPg9geFQyNV+6YWS3A/pPUBCAcdsh1sreed/z9nqnxdHdgR?= =?us-ascii?Q?8IRS9oklcCFZjAJdM1LE1I9PZs3WFsqfiASzqVWoBkN6dMk3odD/i7EvGqXR?= =?us-ascii?Q?LSTZdYJ2ncLU7FMi86Nh3436ElelBWP2HUv0+VGxajieRS3P1Ntvy4OMo4e/?= =?us-ascii?Q?lQmiQRalVXZx+jxVRUdS2Y59xpMvjUBvgK3Df5EhAjhiPEREutTzHIcVEc+I?= =?us-ascii?Q?h6GEPeVOHG1ayQVySnTA9COA03EohdgSIt105nO5YIY4XD2XAkREfZiVZXWJ?= =?us-ascii?Q?48/J7zuOYKXGn003tosbKbWHj+WRbJh46jeykM8k0WJSt0JEP4fQM+cosDWz?= =?us-ascii?Q?9e8lHkFtR/2zQLeHvA/2mCm0G0zAMIuOJsHxJJjDRQrgI93STYhMax66JgKi?= =?us-ascii?Q?6X4D604Shlt0Z++ogTXs3G1hE1XBrZfyFSKrB0yexHdTH2sFXy3fRzXsJIoE?= =?us-ascii?Q?bKxx2E/33HTbPw2oicyiKNbHTT+goJxuOJpNsbh7fn7sWxwtU5/F+uGATVWC?= =?us-ascii?Q?Kk4mHyZ7l9ZtaAASa0ZeeeIHnrYCFQP7DbCjkGYuuoPw+DGSyC3v6ZnwgtWA?= =?us-ascii?Q?yctkqOsEhKM2JHj2NRjDytuqymrr2FZiHlhNa0NNP3/7yC8L0bWrGP+pyY2F?= =?us-ascii?Q?1JXU6toF2zilulm3nGQYFjF1XFCenGAoy0JVyW0FH8jGeyIyQJoCZJgJ13lV?= =?us-ascii?Q?pn7Xy0+TEAUJhL5mhqLhPReh5N+SSZjzzuxF3drZhAlR1ov9UZ0mXb79Z3wK?= =?us-ascii?Q?/CWdeHsdu4CsA7iGvcpyB+VOsc3Y1rPhI2dJJVVJ+d58MVQY3Z/AMw7w1W8y?= =?us-ascii?Q?4YplSjMuWqpuHxBWIXRfoa/dCBl7qAl8DGMFAc9qL0+/lhwzNH6RDJC7HOwa?= =?us-ascii?Q?/E1aRCCgXxAWtKToVmchakf0azaweAv8CldcVJ5i2eeJ2yskEizK0HlwFRtK?= =?us-ascii?Q?/6v9e3JNA6OgudnJJjjVmgbdvf/Ohzey2zFZsRf/clCrFPBOO1leInxCPfSj?= =?us-ascii?Q?3f2L3AA+Dd+9UHqnZeA3GidFVmTqabpDFy8yG3pnIqtgXupAujcfqyQqlpui?= =?us-ascii?Q?LUtv9aSIIT0u/SFuhUk139MpMoji7oOIZyAj119GsQPUO17XlMC/Qn+TemJE?= =?us-ascii?Q?dwaGhL/akLRWXCdE2vo3V3jQ+XeYr83/TH0WkgoVuphAEOqSkjY0AKww29l1?= =?us-ascii?Q?z0fqZDHQKz54W11YzGBACa4MVhUMTgS1Z8JZtXxJMgvVWcb09U9JRTa9LBX7?= =?us-ascii?Q?qE7sHHTIdj1QOFMzS5DUnKEMwQ8PTeImDVK8Gx5d9O0XR+wxxRHpxmcSRbID?= =?us-ascii?Q?ocw+/Iy1h/gW/XJySszNOeK8xM6KKjq3DZUdTJNlfrAMWXp3tE7xkuIWmXOt?= =?us-ascii?Q?/gGY+vQSMrq/WMtmxyNcb32ixsMg63sHkQc8PjWcscO2kAIhI77wxicNGg9y?= =?us-ascii?Q?Sa8H8iT1BZd2YJduvVMazcYMq89O0F80soMiDe+HvDj+5I8BDIjIoQmoIqQ/?= =?us-ascii?Q?l5hpgQmHxznb57ifWRc7t/MdfxD8fyR7DjEbf0uJ?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: b720841f-411c-4c5a-afeb-08db864c4011 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jul 2023 22:30:25.6646 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fd4X/IlFgBMMJZYuaRei3VfUzQApRSea22BJk+eSgfT6wEdxjKkn1fUK3hfGa3c4 X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB7508 X-Rspamd-Queue-Id: 28F14160008 X-Rspam-User: X-Stat-Signature: exnxyjj6myeziycyymgbg7ifwu858x5f X-Rspamd-Server: rspam01 X-HE-Tag: 1689546628-238499 X-HE-Meta: U2FsdGVkX18jbsipdtylwRyq9UGaUuHhfAzj9NRXfr0Iwdqr4DSP+4fRLCLeW2s6cnN4JYqO5xW+NFqpfFDeOMYK202INQcQ2zEiT2AAAu7LJiSLDCQKT5akuy8rPswCrak8QbcZ6KPdJXcJKkVAsyaOSPf2cS4CQqWUlxFwA1T96TxMC414ZLUsqeJPNELftaA78zRixlKaSO2DceX3qro6yi2DoNLbzZKWZ7/jm0QgrQ4WQCHq6F72D/eM/RdoVGuN73XC7oclrXE1FQ3xNVc6he3e1nMyz8muOs5Ut6avqP/K1iXO5sVD25pVKyNJgbReSuHiX9auPf4hLcp1z8uytUGfGTj6xUjlEUUmY3yKyWl50hn1TD04bJsCcTcnS9/jDohIqQtjkjbweT7p0QSQQyrwB5dxLgpU6hBsp3rJe7+0g3Mo6KghFdR++7xFRImzYil/BpUEERvyeJmyhFS/55/CPvLzfKC/JZpPbGrPEt2+9+qhT0ettje4qzcjvjijMCPj3hLnXaCmxzX0E9QYzcxKbifskmPgA5szR0cAbAnK8Z7rwUnqk9MV6F8eW4dqJ/oTWl84+yaryqlAKMrqr3qSYQW26dG7FLKdP/iOWXjHEeBFerairVXf8VAgfcUHRT60yMpdV+V5pmsUun30vUfjuCh1R2ectu6XRybiVwOMrej6XC6i4XAiUC40KP15/efLhpF+RSaytdMJe0KMjUn0jC4LKBt/y0Lt5Jc6BhQu4xk009/Xya1yEODUGUsyDMUNrLNwGOhdCedrCbtzrAhos/Iwnn/M8efIwO1YH1YeFTGKGo6kmVNSt0DzDNljzH0FlBm6HVmAeWgR57hmvGe7c0KWktKqJlQuos/P4NLCDztERnfYXwcawTBwO42Bxq7O42YcOG3DA2tV95ek6ZHhRr/FcLX9I+RENRzxsv2c5g51B9e04muACXCgkyFUTocp/tRyXL4+p34 TfBD+u9O fADjuKxvs870yQq5cIrupASWWDndeJHDR5oZSTyFXyocGDicZSO/brTgL6U4oI1y6esEjK/mnPhtoxbz0dsVV5rklRDENUP2rTRJVwKpFrKScAiLxemt8v7F5bmkiZetea/0vIKCHYUN7pFfG8jdPla+xXW7W5eSh1uIkuOBgzgLyusbrbwE8kp3Xs4ZDuFXfh7wU+JJbrymiINZYb2BwarWSwA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, Jul 16, 2023 at 08:09:02AM -0700, Catalin Marinas wrote: > In terms of security for arm64 at least, Device vs Normal NC (or nc vs > wc in Linux terminology) doesn't make much difference with the former > occasionally being worse. The kernel would probably trust the DPDK code > if it allows direct device access. RDMA and DRM already allow device drivers to map WC to userspace on demand, we expect the platform to support this. > > So the userspace component needs to be responsible for selecting the > > mapping, the same way using the PCI sysfs resource files today allows > > to do that by selecting the _wc variant. > > I guess the sysfs interface is just trying to work around the VFIO > limitations. I think just nobody has ever asked for VFIO WC support. The main non-VM user is DPDK and none of the NIC drivers have wanted this (DPDK applications areis more of throughput than latency focused typically) > > This is particularly suited for the case (which used to exist, I don't > > know if it still does) where the buffer that wants write combining > > reside in the same BAR as registers that otherwise don't. > > IIUC that's still the case for some devices (I think Jason mentioned > some Mellanox cards). Right, VFIO will have to allow it page-by-page > I think this interface would help KVM when we'll need a cacheable > mapping. For WC, we are ok without any VFIO changes. Yes, it may be interesting to map cachable CXL memory as NORMAL_NC into userspace for similar reasons. Jason