From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23C54C4828F for ; Wed, 7 Feb 2024 15:29:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A53546B0085; Wed, 7 Feb 2024 10:28:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A02A56B0087; Wed, 7 Feb 2024 10:28:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A46F6B0088; Wed, 7 Feb 2024 10:28:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 78BF06B0085 for ; Wed, 7 Feb 2024 10:28:59 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 20BB01C0DAE for ; Wed, 7 Feb 2024 15:28:59 +0000 (UTC) X-FDA: 81765390798.05.3E0F492 Received: from mail-oi1-f178.google.com (mail-oi1-f178.google.com [209.85.167.178]) by imf13.hostedemail.com (Postfix) with ESMTP id 2043F20016 for ; Wed, 7 Feb 2024 15:28:56 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=WRTkRiyM; dmarc=none; spf=pass (imf13.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.167.178 as permitted sender) smtp.mailfrom=jgg@ziepe.ca ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707319737; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KgcZE+tbSx7VYsRad6qU230OGLkbrGEcFdePnG3XBqw=; b=zaFtaiajsyRoU5+nZCOWVQWnYjnP+oxpRATRXYQYUYmuXXP/F9pOlpeOWQquF366e76wSJ FVTqY1882RObYoA/EaZBT0zpVjopX2d76oRfAXxFxZXdbK2kHXxUq/sspzryDxUyfma+43 BdGyxTifulkQBF7le2ylvv3sbBDa0Cg= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=WRTkRiyM; dmarc=none; spf=pass (imf13.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.167.178 as permitted sender) smtp.mailfrom=jgg@ziepe.ca ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707319737; a=rsa-sha256; cv=none; b=GdbY+KJnZ6eB9gPaKd0qM/DvRMXbokWjS+vH3uGPeNPxBr3lTAU68V5QqUmOXjU6SvZADY DJbW8EVAkRyIjb427+p2yJDN/USXcWs9bFitLO2PXbl6zsmuBQfy6VuoZZTZcXMVo+dEKS mnSvdH0ee5PJySm1K38+y7w6zIW1dSI= Received: by mail-oi1-f178.google.com with SMTP id 5614622812f47-3be61772d9aso355182b6e.3 for ; Wed, 07 Feb 2024 07:28:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1707319736; x=1707924536; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=KgcZE+tbSx7VYsRad6qU230OGLkbrGEcFdePnG3XBqw=; b=WRTkRiyMRRE5JwBIFft2QStkdskc92eOVM9yULLfo3Z6eWxIliYXsTOnju5atmoZo1 I5QmAa+CdyxCgI/XWK3IjiaWfbPxHAWVxWCPnl653iIcheDp4VDXh3AFbYha+PhK+pVR d1N+dmBNUEZPcX6fgczL5jsuXhue7qJpvJb80eTXkkWOs72TMFXaPty49tqh0MtdRDEf idMZb9By7aIPVY6lpLhu9QpQNB4THEpFhYA76LbKSNjrIPJCYXYspwyd3JubyQvlckjT JpcOx02sQYcpvDoOCPI19B+qy/T2SiCmhaQK+VX8aJk1c0ml65RjHz6NZn3rfunSLCro 4t9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707319736; x=1707924536; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=KgcZE+tbSx7VYsRad6qU230OGLkbrGEcFdePnG3XBqw=; b=onolypXnxdtuRJIyLT9DFJSohdI/3Nmlz8o9XL+Y+R9JqlHjCiqIUiI3DoGidBD+sT bCCXg64X0c/oIDft5qsa8hvBIAuuxL78ffE6mvFBRX5jmeAZlKc4JTaLbCZysm6ORdLx oILQvWJX67FISh9/0WEE54Ax8rpbf8l0/69PTCIM07Xlit2Oy3KfJeORX+N7EVasIm8y BvbcjmQCrLxntxLj3Tfky0zoBw8U/g9xIYR7J2GzsgZ8Czp3mrhB51IvHqWk6f38VPwh BOjuRW6VO8I4oK4qRP2RPp6K6qs63fQsPt7rZWJV/5BK3oE0vlisEksxBvtMloioaMTN aMYA== X-Gm-Message-State: AOJu0Yz5Qw3HnPqSLhoDO/BHsEX1mqXeVIRR2OzpSsvvfU1iOOXMVyZE +HAfRVOM7zxyoorKIgByvMemCgF46ZVn16CmNHvF53+4JAEjgZ95QZVEgXOfc/8= X-Google-Smtp-Source: AGHT+IHEL7CFznI2Gw3SaxhMoycGQY1ZAMJ2M/MoHIzeasU/nr/2JepWSFEuheW8Ams28t9PtymRRw== X-Received: by 2002:a05:6870:bac7:b0:219:83cc:287c with SMTP id js7-20020a056870bac700b0021983cc287cmr6065943oab.18.1707319736190; Wed, 07 Feb 2024 07:28:56 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVUY+po1giBXhl03Vf0urto5lU8KkUE0xa/D+uEwt3aAnuuk7whgR0HTCfY+MTVQpslUqqUs/wsMp+t3ROMVbRdiv139tCe2ZBXKAtq4ZceV9PQJGgSNRI8IXanfewIYC0pI3FzZsKmRPGCkekyIVSuYunEdq/NKi0/3z9aOOnSjGhG1HWtDCKx6tkf7sG69wd0voFNXVKcPnx/Nm+iJxlnF6OfGX8tZ9W5Ef2TdHxCT40iWFJmfW6S8uQKXx/coePehrn7fBGc1ApNtwTAo8s/s5HRc5UVF5kRUR22WO8nHBRkrzcJX/Ul/xP8hPJikjNY5otCJsvQrEVBk2ZaVzx3MkqaDTyR4Ul7c2lNd1sQoJE9R4hMA5FyNT4k6cGm9+PELO3Pd2X6HZW5Jm8qV/VAErzvt6LcaoXye2Wua/2AYMT/wp1V94k8M1zgi8oPWuI9edPTgsfIAIs+qSWZnIHymtDR0ZDoa4pbEKoLlGJm39ZpBPkOgypT7AHFEI8rU/GAr/BBLOvIN75lw9qDVd4OTS7E4DjE+P+TXjW/ejHtWFFjkRAxscCfnxXGlxTCjMW8flXh31SrWNWQb7kthgrk5FZ0nc1rQCTq6Q8fWQP0Awd0gAJf2PwiCs47eD05aetIREJ1Gf81uCiJ/HxdH+LBZUmXJT18VOvJdRMBqtyz Received: from ziepe.ca (hlfxns017vw-142-68-80-239.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.80.239]) by smtp.gmail.com with ESMTPSA id p6-20020a9d76c6000000b006e112c93b1esm228799otl.6.2024.02.07.07.28.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Feb 2024 07:28:55 -0800 (PST) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1rXjr4-008ccK-Kk; Wed, 07 Feb 2024 11:28:54 -0400 Date: Wed, 7 Feb 2024 11:28:54 -0400 From: Jason Gunthorpe To: "Gowans, James" Cc: "alex.williamson@redhat.com" , "kexec@lists.infradead.org" , "kvm@vger.kernel.org" , "brauner@kernel.org" , "Graf (AWS), Alexander" , "iommu@lists.linux.dev" , "anthony.yznaga@oracle.com" , "skinsburskii@linux.microsoft.com" , "steven.sistare@oracle.com" , "akpm@linux-foundation.org" , "linux-kernel@vger.kernel.org" , "seanjc@google.com" , "Woodhouse, David" , "pbonzini@redhat.com" , "linux-mm@kvack.org" , "joro@8bytes.org" , "ebiederm@xmission.com" , =?utf-8?B?U2Now7ZuaGVyciwgSmFuIEgu?= , "will@kernel.org" , "linux-fsdevel@vger.kernel.org" , "usama.arif@bytedance.com" Subject: Re: [RFC 00/18] Pkernfs: Support persistence for live update Message-ID: <20240207152854.GL31743@ziepe.ca> References: <20240205120203.60312-1-jgowans@amazon.com> <20240205101040.5d32a7e4.alex.williamson@redhat.com> <6387700a8601722838332fdb2f535f9802d2202e.camel@amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6387700a8601722838332fdb2f535f9802d2202e.camel@amazon.com> X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 2043F20016 X-Stat-Signature: xxprfire848temp61m65h43w1as6777c X-HE-Tag: 1707319736-113573 X-HE-Meta: U2FsdGVkX1/4w2JjwDR/UG+RHZsTvJKeG6DhnTWNQlqk7yDvaH+QWJK8PWOvrLvv3Vd67F8HuQwF294ERPchzVI7SWEMHNcTrX8qE64Q5S1NZR+30B+FNGYxezGr/o25Wd9zk96ut3W/nyKlnABLJ0lWm9OX34xIKZeNFQqZI6qYyUyJdSIy5kH6GF3FwOSUw8GaJ4WDPBnv8Mv5A5x2skhkwJQffT481N9h4eioouKYenlGjvqW5zSPhu9aSWod1AqseSfR4RWawRWNvV4qhbPcMTukyNO5TsmIR6Xrt3cm0q49pp5Rt3OgrQJLCfs8aBTrSo5d+QaB9WCfSr0Xg+dqy8tDPbqVWKzLJ1E+58lKS1uidHkPXatjfJqyy5ciPfmYzNMFLLSYJ+ag2XAvWxieQAIKCDD0YTYp/2DQ7hkn1AlTDrfYbE1OdRPOrsaQuBF+q6SgwVyOPnY3Sr5stzDN16YiEnvxJ1nDL5CTqnb27apj49D6VrZq+O7Fi2dStN6NE17tpjfY7SmXuaaNvRQTcR4AytO+/WzYU9Sm3K4AHYRK5ksdUUc8ClFFUoxBFUjJGOEitkY9DSAbCe8HVe6JzewD9W+sDhXRNr8pr9ygyhHhEuQ8OTsKRUjK9peNAbBkxGYJT1zbPasy3QqR9aQeBOEKO8hx+oEn2JUzLSeL4l91NkiUpA9mp6uPtiWY6mUUjm03CVGO1XoB+q+qrOe9HuRMcklWZfKLGIqAittTDWTBlXUsFYOkqG/mn020Lt3ifTA1EQojA5V40ZNL9Vrr3VdGMzJtxdLvPIT1p6fKQn7EkvKUj01ZaiBt5hSaLlWF7rM3MnWQtdIHzyw91jJItfMS+Qt+TdmOBwIR1eJLzIsWYO2J5cjywGedXWc8U60YvKMMRbMILkRCbvLWapt/M/k6284gRfD5OVfe5Zn5371eM3mcIPWUdsABeKp6DtWhvc4L/Kh2FBrke9B Xqa3/9KL Bgj5YuC9DkXTEusW0GS71Y2BgCRAL/eagKaS6SBhDU8p5tkiwjBOp7Ax3w8CQAOI8vUGSf+8yauGlMDobMTByCE69fD5/m2NdlCTUGdwPFBgr+5dH0olmvjKgNYVTKo/M/ifiEYC1rm4Xq7yOGkwGFnn6h8lVwo9yzrqP32cZa2wlmnvz7M0gK7zN/YKyh97WYnZycyHuAl7y+1TmLLEUKycRQIv2kgf6whwM8kHJKULNfQVfB9jW2CBBElc6SSzGqp7zdwZzjLqhZHKIyjNUp9IsFGmjZHsEbE0vAWlBVV9hqdKCYdW75VqRMS4b5+s6Tn4Umz5TpTIV5U8EBoqEtpWslGbuo5VFJoSPfT/r5ggRYzOSTHiCrXLLJxGCp2PoAKfK X-Bogosity: Ham, tests=bogofilter, spamicity=0.041156, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Feb 07, 2024 at 02:56:33PM +0000, Gowans, James wrote: > 2. Tell VFIO to avoid mapping the memory in again after live update > because it already exists. > https://github.com/jgowans/qemu/commit/6e4f17f703eaf2a6f1e4cb2576d61683eaee02b0 > (the above flag should only be set *after* live update...). Definately no to that entire idea. It completely breaks how the memory lifetime model works in iommufd. iommufd has to re-establish its pins, and has to rebuild all its mapping data structures. Otherwise it won't work correctly at all. This is what I was saying in the other thread, you can't just ignore fully restoring the iommu environment. The end goal must be to have fully reconstituted iommufd with all its maps, ioas's, and memory pins back to fully normal operation. IMHO you need to focus on atomic replace where you go from the frozen pkernfs environment to a live operating enviornment by hitlessly replacing the IO page table in the HW. Ie going from an IOMMU_DOMAIN_PKERFS to an IOMMU_DOMAIN_PAGING owned by iommufd that describes exactly the same translation. "adopting" an entire io page table with unknown contents, and still being able to correctly do map/unmap seems way too hard. Jason