From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CC6EC531DC for ; Tue, 20 Aug 2024 16:56:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C4F2B6B0083; Tue, 20 Aug 2024 12:56:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BFED46B0085; Tue, 20 Aug 2024 12:56:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AC6976B0088; Tue, 20 Aug 2024 12:56:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 923226B0083 for ; Tue, 20 Aug 2024 12:56:22 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 3F2BDA035D for ; Tue, 20 Aug 2024 16:56:22 +0000 (UTC) X-FDA: 82473227004.22.021B577 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by imf16.hostedemail.com (Postfix) with ESMTP id C9C9C180003 for ; Tue, 20 Aug 2024 16:56:19 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=Q4GwamGV; spf=pass (imf16.hostedemail.com: domain of quic_eberman@quicinc.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=quic_eberman@quicinc.com; dmarc=pass (policy=none) header.from=quicinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724172900; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jzY5T8RZv1PP7op27sD8n0SONGxQfSzjHm+fnS2Ly20=; b=lU44CgZRbdh6MPQNijDdGMAZmUM0iJC8PP6A/Gome9sRMA0S5jjLeTdn9AksYcT7S+gHvu AP7omfbpFG8gGVlwNUqdsXCeHL29PqdgT7BLqnC1FVDqz1aQ0DPiPQRDHD9Vi4fismJvjc ZmsGDytSkzhap0fdfvzUW8DuUpEqJ4Q= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724172900; a=rsa-sha256; cv=none; b=5IRqLdOC3rvfIUkRtSTvMYBs69gaxWvTcaaWvRCc5NjajziHvvDnhx+bcimbsi/zkimInL R09dnZksNVYbDYKaPincPwN5WKuaIfTp4p7uguM0PMOXadNfsqOTdbgmoEz8MX0Z5USASR Mv+ZYCCjymwX1d540TPUkouO7wjGtzA= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=Q4GwamGV; spf=pass (imf16.hostedemail.com: domain of quic_eberman@quicinc.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=quic_eberman@quicinc.com; dmarc=pass (policy=none) header.from=quicinc.com Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 47KEYqTD022348; Tue, 20 Aug 2024 16:56:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= cc:content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to; s=qcppdkim1; bh=jzY5T8RZv1PP7op27sD8n0SO NGxQfSzjHm+fnS2Ly20=; b=Q4GwamGVnUYHezWuHUjtjza6A/lLhwDxn1B0wORk bTlnIlWzNfi+wdhxzaABCNC/EMT3P7WJx6vG6pV1Fwn5PVyJfdT53/ZOpvsxCh+x yRRMs/d5cYjyhzv1WRhQYp6A20O8lgMVpsicruTmmAgl5STF1ExdY9JD5rLzcptl OB9oWUcGUEINslDeow42p7Z9UJLQH3+HKEI15Zq8UVGnNQvk4XaLi16dJ4zRtuEX PU9LejQ20kkidZ0OSAwhbBsd/qS3WT2NmyjJdtJyhxXbWsqaupLPxecTuE6SkdVb XB7+CPgBlSpH/P156LOeoP6utSCQjj5j7lq1U40Zb9BwRA== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 414pdm9ngv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 20 Aug 2024 16:56:12 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA04.qualcomm.com (8.18.1.2/8.18.1.2) with ESMTPS id 47KGuBDx007487 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 20 Aug 2024 16:56:11 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.9; Tue, 20 Aug 2024 09:56:10 -0700 Date: Tue, 20 Aug 2024 09:56:10 -0700 From: Elliot Berman To: Mike Rapoport CC: Andrew Morton , Paolo Bonzini , Sean Christopherson , Fuad Tabba , David Hildenbrand , Patrick Roy , , Ackerley Tng , , , , , Subject: Re: [PATCH RFC 3/4] mm: guest_memfd: Add option to remove guest private memory from direct map Message-ID: <20240820094213541-0700.eberman@hu-eberman-lv.qualcomm.com> References: <20240805-guest-memfd-lib-v1-0-e5a29a4ff5d7@quicinc.com> <20240805-guest-memfd-lib-v1-3-e5a29a4ff5d7@quicinc.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: BsZBnYdjUKxH8zs16Q3X3BsOufoDwHwH X-Proofpoint-ORIG-GUID: BsZBnYdjUKxH8zs16Q3X3BsOufoDwHwH X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-08-20_12,2024-08-19_03,2024-05-17_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=850 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 priorityscore=1501 mlxscore=0 impostorscore=0 bulkscore=0 lowpriorityscore=0 adultscore=0 clxscore=1011 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2407110000 definitions=main-2408200125 X-Stat-Signature: bnhi1icswwncgw7ymxoh9869tp3tw4wy X-Rspamd-Queue-Id: C9C9C180003 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1724172979-332038 X-HE-Meta: U2FsdGVkX1/atc2suVTfPTa+fofNm63CLovEy5IkR1imoRvxiAu4a2AtHFlUNt1nSuobGKVIWPKVcstpI5sxTEY/Vh+XbR7lbExlsSH8f+Yvkf6gShoxjibUjxZM6/UoST5Fv/GmOXyFq8aSknjNRZB1gbbG3FBehvKrjKxDQV6OzSXyAxJEU6HeESvcWYbEeokZEh01oyXzxvxR4Ll4WL/vlzSMlj64/Y9k051dlIfEV5J8TSJsgiKRFAFXLa/zRWeWiutwU++DynxMj3G//e6HogY5cPgDLzvIOUc7Yy042FpjIobgm049U9Ga8YBav3bsn4+SokuhLnEVrolIE2lMnW9GuldQZlnVSsUPewfNQb3BzDFzagRDreTIwjKC6dgLE12QHquLMwtPoqbHNodCkA+P8rb0iMPAeuD3YZd86fVlbV1G3EJfevKhczrbZQOp9xAuV8742hvo1TMGeifCmUpfJ6Srtq96IkMDW5Rx796MXr+JYjV7k9ixMDBErdyiqOm9ScOhDj3UDN8Glu5xDiuUDuYnQ7iEosTWFZJq45GykvfzG0VAYan2XB15iY7oRbULmO+HpEZjSrhuFx34KKn0gbQnKcJXWknovY+MKO6CiUErEBtfbekG9vQ1Dr66iTctBKcgUrUc8NqF7tukWsOnGu+pTbLMZA2mPqmBTJfi+/vnF40awwEY5zFggMXZ5TyeG7dMYPRyMu6BZipvoOcIt/ayncLgjDmqNotG3QMVubHbf8m9Muh/Pxd852dvUPTa83LwDQQJCPrN65TlNB/5OpMS86tGC41d8tO3nc0emDAwSM1DOK/5WriGi8WYjqOxfAkaTI5G9NNaVdy2773RjrV3Go4jkjVcn34SYuRhSQvQhJIk5wAGAXGjygIb1kJ4DR3kbjT4AN6DGsp+THAdfv6GM+YXaxzBmQbYFMoPFOHGDg9m1iGtlwVUKZ/rvIG/umB/kPRgIsQ hpDNf9f7 VXHUNUpaAlqJs2kFWj1J6nN5Sxqq0WVYl1WqOOenPxln7H/FDlLsEK63kD1ztnC6SrtJAt9fqvaYyDKRerMBeUYHeRDuqH8Ry1WWAO3OlAzo2P1Cl/n71NkXrsQQFnQAPIF1HBAqVmQb0I7Mj+MJ+YIZWZ9BizWS5wP7qk1OYESfhSzYBsQleXHvmJ4zlTr1YnrxC5BGplyRppPhmNlHjS+MWolQuwlMIzhPec6TBjWQJVSyNIuwnKKorQLz8ojFf8bTbj1dXBKxy5ztKMV8G0SYJ2wYVp70Zg17w3TZldo3GabDcRnjAGsk+aPeulCqaJCYk8vztppUXKbUolWrGu6q1ndsQYSDEVK2mINN+bOMV3Co9c8RCjIfMGABlo+iIGNZi501Cms94GNTjbtc9gsXnp92D6hVyTXfEvEx3xbzcV/QEsUZvxEwyCC3jEa/Bpydi185fa3JsFKApydwlyowQ3fslGpvGxVbYvaDJgBvMfPc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Aug 19, 2024 at 01:09:52PM +0300, Mike Rapoport wrote: > On Mon, Aug 05, 2024 at 11:34:49AM -0700, Elliot Berman wrote: > > This patch was reworked from Patrick's patch: > > https://lore.kernel.org/all/20240709132041.3625501-6-roypat@amazon.co.uk/ > > > > While guest_memfd is not available to be mapped by userspace, it is > > still accessible through the kernel's direct map. This means that in > > scenarios where guest-private memory is not hardware protected, it can > > be speculatively read and its contents potentially leaked through > > hardware side-channels. Removing guest-private memory from the direct > > map, thus mitigates a large class of speculative execution issues > > [1, Table 1]. > > > > Direct map removal do not reuse the `.prepare` machinery, since > > `prepare` can be called multiple time, and it is the responsibility of > > the preparation routine to not "prepare" the same folio twice [2]. Thus, > > instead explicitly check if `filemap_grab_folio` allocated a new folio, > > and remove the returned folio from the direct map only if this was the > > case. > > > > The patch uses release_folio instead of free_folio to reinsert pages > > back into the direct map as by the time free_folio is called, > > folio->mapping can already be NULL. This means that a call to > > folio_inode inside free_folio might deference a NULL pointer, leaving no > > way to access the inode which stores the flags that allow determining > > whether the page was removed from the direct map in the first place. > > > > [1]: https://download.vusec.net/papers/quarantine_raid23.pdf > > > > Cc: Patrick Roy > > Signed-off-by: Elliot Berman > > --- > > include/linux/guest_memfd.h | 8 ++++++ > > mm/guest_memfd.c | 65 ++++++++++++++++++++++++++++++++++++++++++++- > > 2 files changed, 72 insertions(+), 1 deletion(-) > > > > diff --git a/include/linux/guest_memfd.h b/include/linux/guest_memfd.h > > index be56d9d53067..f9e4a27aed67 100644 > > --- a/include/linux/guest_memfd.h > > +++ b/include/linux/guest_memfd.h > > @@ -25,6 +25,14 @@ struct guest_memfd_operations { > > int (*release)(struct inode *inode); > > }; > > > > +/** > > + * @GUEST_MEMFD_FLAG_NO_DIRECT_MAP: When making folios inaccessible by host, also > > + * remove them from the kernel's direct map. > > + */ > > +enum { > > please name this enum, otherwise kernel-doc wont' be happy > > > + GUEST_MEMFD_FLAG_NO_DIRECT_MAP = BIT(0), > > +}; > > + > > /** > > * @GUEST_MEMFD_GRAB_UPTODATE: Ensure pages are zeroed/up to date. > > * If trusted hyp will do it, can ommit this flag > > diff --git a/mm/guest_memfd.c b/mm/guest_memfd.c > > index 580138b0f9d4..e9d8cab72b28 100644 > > --- a/mm/guest_memfd.c > > +++ b/mm/guest_memfd.c > > @@ -7,9 +7,55 @@ > > #include > > #include > > #include > > +#include > > + > > +static inline int guest_memfd_folio_private(struct folio *folio) > > +{ > > + unsigned long nr_pages = folio_nr_pages(folio); > > + unsigned long i; > > + int r; > > + > > + for (i = 0; i < nr_pages; i++) { > > + struct page *page = folio_page(folio, i); > > + > > + r = set_direct_map_invalid_noflush(page); > > + if (r < 0) > > + goto out_remap; > > + } > > + > > + folio_set_private(folio); > > + return 0; > > +out_remap: > > + for (; i > 0; i--) { > > + struct page *page = folio_page(folio, i - 1); > > + > > + BUG_ON(set_direct_map_default_noflush(page)); > > + } > > + return r; > > +} > > + > > +static inline void guest_memfd_folio_clear_private(struct folio *folio) > > +{ > > + unsigned long start = (unsigned long)folio_address(folio); > > + unsigned long nr = folio_nr_pages(folio); > > + unsigned long i; > > + > > + if (!folio_test_private(folio)) > > + return; > > + > > + for (i = 0; i < nr; i++) { > > + struct page *page = folio_page(folio, i); > > + > > + BUG_ON(set_direct_map_default_noflush(page)); > > + } > > + flush_tlb_kernel_range(start, start + folio_size(folio)); > > I think that TLB flush should come after removing pages from the direct map > rather than after adding them back. > Gunyah flushes the tlb when it removes the stage 2 mapping, so we skipped it on removal as a performance optimization. I remember seeing that pKVM does the same (tlb flush for the stage 2 unmap & the equivalent for x86). Patrick had also done the same in their patches. Thanks, Elliot