From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A48ECC04A94 for ; Tue, 1 Aug 2023 20:48:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F06E62800F5; Tue, 1 Aug 2023 16:48:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E8F452800C8; Tue, 1 Aug 2023 16:48:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D082B2800F5; Tue, 1 Aug 2023 16:48:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BB2242800C8 for ; Tue, 1 Aug 2023 16:48:41 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 8BBD7405FD for ; Tue, 1 Aug 2023 20:48:41 +0000 (UTC) X-FDA: 81076724442.29.BCE31C2 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by imf05.hostedemail.com (Postfix) with ESMTP id C788E100016 for ; Tue, 1 Aug 2023 20:48:37 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b="D/JbRKJ3"; dmarc=pass (policy=none) header.from=quicinc.com; spf=pass (imf05.hostedemail.com: domain of sudaraja@qualcomm.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=sudaraja@qualcomm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690922917; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=oJCS+x3JhjDoIexaW/EX09NVFqCt3LH9qSJgusuqsoU=; b=Cpnc8zk8GwITWqYqf9DJi2niWMnorYEP6lpJ5046otEkIZurbeTkbfcv5dNgrg7gegObWa XnnL50vrRwv4uofKkiHCvKlh1+uV0cTtDkVyjxk1PmtA5saYVI4rpnasH1YR9Kw9bBfUhW +RTrnnwIbpxCy9CYT91WuztP4m1mbd4= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b="D/JbRKJ3"; dmarc=pass (policy=none) header.from=quicinc.com; spf=pass (imf05.hostedemail.com: domain of sudaraja@qualcomm.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=sudaraja@qualcomm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690922917; a=rsa-sha256; cv=none; b=lJ65jOXsYOtRoKmN7CkxX0b/fGhdNuyltSTbIGl4hRb0SH54RhvWZZxiha8zMZrtOnxdAI J8PC0pukH2kTwKegsZcc5tm5SVQIIy4zdf9XwSsIc2yN6nBuMrbTMOUIc9D3CPTPmlTwsU fEhEPGpM2K01Wb0gRcRWjZNnynU25JY= Received: from pps.filterd (m0279871.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 371KExxM008438; Tue, 1 Aug 2023 20:48:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : mime-version : content-type : content-transfer-encoding; s=qcppdkim1; bh=oJCS+x3JhjDoIexaW/EX09NVFqCt3LH9qSJgusuqsoU=; b=D/JbRKJ3+jjzbkQ7l/hu0Lxnfxe+tU59mhKAixs4RVch39TpCKe+4Wtfdh96yzbpKZHA oB6YRi5BtBin34HN2XV7aqaZ/mpVrZNfWJi80fbicczJBzc+VynfgEIsETnjTFpymMFH piu2KZrvn98AFzdaIb1duOGr75C1yK+B9g6qDxUYt4zI2t2VgYU/9HBN6IxqZs0qA7ds 0N4ZB/rfRQ/1bGs9kAUWidSugyguLa+y8H5rVady7L42AW5DWe4LoBtGeH4krY2wOMDB QSJdJlcMGSYjQEuM/iP87EHaGftvcOr6hkA8haAOXL1mjSrw3R5QRm20gm3g+rfEW7Yq iA== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3s75b30hyq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 01 Aug 2023 20:48:29 +0000 Received: from pps.filterd (NASANPPMTA05.qualcomm.com [127.0.0.1]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 371Kcg7K031024; Tue, 1 Aug 2023 20:48:28 GMT Received: from pps.reinject (localhost [127.0.0.1]) by NASANPPMTA05.qualcomm.com (PPS) with ESMTP id 3s6g0pxy7p-1; Tue, 01 Aug 2023 20:48:28 +0000 Received: from NASANPPMTA05.qualcomm.com (NASANPPMTA05.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 371KjCpv006589; Tue, 1 Aug 2023 20:48:28 GMT Received: from th-lint-014.qualcomm.com (th-lint-014.qualcomm.com [10.63.177.51]) by NASANPPMTA05.qualcomm.com (PPS) with ESMTP id 371KmRkN010395; Tue, 01 Aug 2023 20:48:28 +0000 Received: by th-lint-014.qualcomm.com (Postfix, from userid 455663) id C5DDE6846; Tue, 1 Aug 2023 13:48:27 -0700 (PDT) From: Sudarshan Rajagopalan To: "T . J . Alumbaugh" , David Hildenbrand , Johannes Weiner , Suren Baghdasaryan , Mike Rapoport , Oscar Salvador , Anshuman Khandual , "mark . rutland @ arm . com" , "will @ kernel . org" , "virtualization @ lists . linux-foundation . org" , "linux-mm @ kvack . org" , "linux-kernel @ vger . kernel . org" , "linux-arm-kernel @ lists . infradead . org" , "linux-arm-msm @ vger . kernel . org" Cc: Sudarshan Rajagopalan , Trilok Soni , Sukadev Bhattiprolu , Srivatsa Vaddagiri , Patrick Daly , Elliot Berman Subject: [RFC PATCH 0/2] vmrd: dynamic guest VM memory resizing daemon Date: Tue, 1 Aug 2023 13:47:12 -0700 Message-Id: X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: UkeaIKi4HYzp1e7cDt_v9jUSqjLTJGCW X-Proofpoint-ORIG-GUID: UkeaIKi4HYzp1e7cDt_v9jUSqjLTJGCW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-08-01_19,2023-08-01_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0 suspectscore=0 phishscore=0 lowpriorityscore=0 priorityscore=1501 mlxscore=0 spamscore=0 mlxlogscore=550 bulkscore=0 clxscore=1011 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2306200000 definitions=main-2308010185 X-Rspamd-Queue-Id: C788E100016 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: pj4d8u8pr7hhhxq6ab4856atpix1b5jg X-HE-Tag: 1690922917-682089 X-HE-Meta: U2FsdGVkX1/nbJ5+fVDYLvKb9McDSw7Io83OGlhHu9szsCmTgx4zxGHd4Nr1r0Mgg6a2dFNGX1VUR67FAw+zGGtB3yjKAVpJ3Vq4OZVxy2OYxNak74ILUgQ7XqfUgZnlvTXNnTYfSUwI213YK8niZ5qZsQLMRrdb5rcbZlLo1J07/9W9lfeOXCSjefnnThTXfHiJDFH63kygMrC4cHScqKwoH87v1dmYCUB7kVUADXtElVp8pFWVM3gh1n/RWoKYykclrFFGhiNeKeEvLxnZj2SKSdTl24s+B60xTE9fMNlvuOyprREQh4xmfki4n+Bhtyqhm1eU53EEwu1x4M5lq18k+96lQ0oZ00cMBKQ3EQiDXeJXARcxwP9I6hAsmLZFifmnZfCsR8OPfVpipxlML0ckHYU56EjcOo4xVuRhq3XQ5BXH7nrYNtR0o3xM2J8XoSRHhdPH1iPPnspMd5swr2oYTDFGokih869JpTNwEWWqmHsQixRXS0tpPbdLPmzezPECl4VpOW7IuJXEV3D3lDryCelaG/+ONFE0w77A42XsUOzDKHi69z5wCMNMi4cA46i6/XMgSaWvXfp9FVYgB2pcWf1oapIvBnCN0WjoyCEuw293OdGgEqwP0Y9U6L8NF60QnE4wv8LzpJmOHHMxYPAenQMGZiDWuNAG4R64U2vtGgjhL/yiOFDnDDsKwrinjZZ5blGuT19irIpPOQj0cZItRmkKh4FmndTK161Z3XVxuRWU8snq+63+Ml/pehpSN9607yZb4aFNVsGIlTS3axTuHBSEtKN7CAi4AZkXidi28+ZSO13W4iwtl1XKZqP2sANo+n4m0UkbM4Fdmk/aNHr3i2qtFM0vYSgbz70LebhkEJgB+QHxNY9o3WG+hNQoB/AZSarNE8mnE246wn6KKJXp2g633r9cIQM9H1OfaZCE4UD7pZ/ahWUzHufFB9P3xxdyvd6VeTZq9+qRzp5 HTtwxshq AMOASY+g36IPCtHmNYTMPgGUrnhMv21kIu98CqOZNJ9Kt/5c1VEZnh2qiClzEFwn9Jef54kJkUt+yYtdyyzRGQX3pB6VrM+puRVkd+PgDTxXKium6mkVa1fUsFESBrS7Fg/MDHpJ/17BkrSmR5waRH5fQN5NfuR53KEP04z5uyln3N/XrvOH707HylU3dyl6di1eNZdP7abvEFp858iRguBmb/tMkpbP9um66rDIgXLFNmxy2tlmntEJ9GaZGKxAUpXDbVS8zqoS/Jlb4M5tfldrOqH65s6Feza5IZMdXHMBgroNU+3VNw/NtJbFkGBt15i1gCkYF4C62XojbgEHSdCH6JiCFATrF0AwBD/dALf/aeJqn8ePtU54x9Bd7tjVKKDP/1MmwpREUNlQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: VM Memory Resizing Daemon (vmrd) Initial idea of dynamic memory resizing for guest VMs was discussed here in this thread below as a RFC. https://lore.kernel.org/linux-arm-kernel/1bf30145-22a5-cc46-e583-25053460b105@redhat.com/T/ Posting the source-code for this userspace daemon written in native C++ as RFC on such an approach for resizing the VM. We will be merging this into Github or CodeLinaro after gathering all the review comments. The userspace daemon registers to Linux kernel's PSI mechanism and it should work on any systems running Linux with CONFIG_PSI enabled. This daemon adds memory to guest VM when demand for memory increases, and tracks for memory pressure going down as the usecase ends and frees memory to release memory back to host proactively. virtio-mem is used as interface for adding/removing memory in the guest VM. More details on the detail and implementation in patch commit text and RFC discuss thread mentioned above. PATCH 2 shows the vendor specific implementation by Qualcomm on the memory plugin interfaces for adding/removing memory to VM by using modified virtio-mem driver, which uses mem-buf driver for communication between the VMs. More details on PATCH 2. This vendor specific implementation is shown just as a reference and different user/vendor can have their own implementation. We also intend to make these memory plugin interfaces generic so that we have an upstream-friendly mechanism for lending memory from the host to guest VMs. This is still on the works and wanted to seek upstream community's inputs on how we can achieve this. This way, vmrd can run on any guest running Linux with CONFIG_PSI and virtio-mem enabled which will be agnostic to the underlying Hypervisor. Qualcomm has been using this daemon into its guest VMs for memory resizing. This has been currently tested with Gunyah Hypervisor and not on any other Hypervisor. Currently, the guest VM memory is resized by expanding Movable zone only, since virtio-mem uses memory hotplug and Movable zone guarantees memory blocks to be isolated and offlined. Next incremental patches of this feature will be shared soon where both Normal and Movable zone can be resized, thereby increasing headroom for both DMA (unmovable) and userspace (movable) memory. Sudarshan Rajagopalan (2): vmrd: dynamic guest VM memory resizing daemon vmrd: add implementation of memory plugin interfaces Makefile.am | 14 + vmrd.cpp | 1259 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 1273 insertions(+) create mode 100644 Makefile.am create mode 100644 vmrd.cpp -- 2.7.4