From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C1CAC28B30 for ; Thu, 20 Mar 2025 18:03:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E260D280005; Thu, 20 Mar 2025 14:03:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D88B8280001; Thu, 20 Mar 2025 14:03:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C029B280005; Thu, 20 Mar 2025 14:03:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9F0B4280001 for ; Thu, 20 Mar 2025 14:03:49 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5D940161104 for ; Thu, 20 Mar 2025 18:03:50 +0000 (UTC) X-FDA: 83242702620.24.20048CC Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) by imf05.hostedemail.com (Postfix) with ESMTP id 51F22100034 for ; Thu, 20 Mar 2025 18:03:48 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=soleen-com.20230601.gappssmtp.com header.s=20230601 header.b="ie61/vSd"; spf=pass (imf05.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.172 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=pass (policy=none) header.from=soleen.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742493828; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aEWRrYL81JihAJazUhZRKND0YCO46+skLrt/0zhqckA=; b=gYiErnKch8PKODfbxS/pe9nH9dkv52zT0w4MSkQAmMcxwwNq2GJxqH1NGk4CrXgRq2ZPfK Ncxztrrof1o76nH1O9NIDTnWRhHRafmsJbI2EyDMjgIGD0rphzJPknLiaUoyZr1TruGGFG bvPpNwgvMru+6DmbhYcAv/EqMdLYThA= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=soleen-com.20230601.gappssmtp.com header.s=20230601 header.b="ie61/vSd"; spf=pass (imf05.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.172 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=pass (policy=none) header.from=soleen.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742493828; a=rsa-sha256; cv=none; b=rzGNmsy7T2LDgG7B3c4/GryOXDWYmACCAdwPcfWL5zNoP8I9lXQzlsx1dBbcdSg8ohFeIZ nvCdLQL6nyCXD7FhWlvSoLixkQeKqNNaRYJWMtxjGVVizlFeAtQMHZwVgPyazb5uhssaxO Y2VBcu6RgiYHT6J/rq6Wtxn6Kz+cxvM= Received: by mail-qt1-f172.google.com with SMTP id d75a77b69052e-476a1acf61eso10412531cf.1 for ; Thu, 20 Mar 2025 11:03:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen-com.20230601.gappssmtp.com; s=20230601; t=1742493827; x=1743098627; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=aEWRrYL81JihAJazUhZRKND0YCO46+skLrt/0zhqckA=; b=ie61/vSdw6dvDGX9lzkqKAU0WmDyfd6HxEt2H8oR5VglIqvw5DN6+T92x7my+tlo7o PKtiS5wn750oVCEEb84eA/fovqBRPvt6sUVFkacTf8c+ALgsrcei/L6Hkzm2poCwym1F VyvecLsr2hL3350IsYpCEjGVumjLX5MOKaj8cyG1g7lyfNPgmyxeeZVNbwyzAbN3K6WK LxQcSc/Y5vyhadV4CjJMhhiZRO2WVzQ1MrIXF592RNA0MpjmaJvyv8kECsnPPMgCBkaX rSPM80R+89d+V4JBLfsyDgfHaDY7vnplu0pGnXV0Yy7gVzcGPGkF+fbjizIAh/O1NSZj qzcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742493827; x=1743098627; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aEWRrYL81JihAJazUhZRKND0YCO46+skLrt/0zhqckA=; b=Ic8tF1999zwqJKuPGl1jRR8VJbn1s/HhVrqDrZdD9XpWV6eTj/OqRgqJ6FTQkjEb/7 TXe8KIUKfynZ5JA8+6ZS8S5IyPfviVMUS4EeJHW8myPMPO+bhnN2kcsJk5GSKt2u/sZz wv8H72fKiMl3aMCCr/nZq/FycT49eiDKt21F6wn+C28sFmghA7MgEIqzEQgwf8nYe11l gk9ZQ6COC5q4qLlnaIksDXw9l3qKF24lh8+8MvMBiEFFOd+UBhsgqIWsY21lqU8mn30y UBdwlPp+sXasbChpOsOnV51En5ulD3ku192s+1hUGejliALVNvLhboQlK7x+RG3IcsIp ep1Q== X-Forwarded-Encrypted: i=1; AJvYcCX/9BYB4R34Ctoq17Qqv0Ww6x+/4MgDwsUqGBMz0aCqRd1G4RrWiN7oYvsof7e1sjGJOfllMetFgw==@kvack.org X-Gm-Message-State: AOJu0YxdpVoKVbvGrXY/c3zedBfpT01d2eGOEALr1lMIUTF+N6XZIsmN M9T8nKYCU78Xg5BV23zKOUR6fkoTGOj02yYTITChut22GNfbzR4R4Dq0QtEx6I6PnRUdoRou1gS uhdhM9XuiI9XXSiMMgPeuDNspqmk4naxw+jmdvA== X-Gm-Gg: ASbGncv3gKTouJPY4vl84fBVhIrfACBZwa8iF1HSgNzTLpfQdyg4AESRTidtjm02UYm cC0xBaU7cQHf5bKcH2wiAyJfsP/Sa0wXhpQLZ6pjUH6fmIfiFD/J60vEpP8I+2IBIob70ROQIr5 NnvQGWfAobEaC0ibeVLNaLLog= X-Google-Smtp-Source: AGHT+IGY2UOOzyotLrqjin40PYelcsYa/VhmCLulBzUFzsbC3V6wg+CyJm8VRhjJJFkAkC9pww3riPd+ntiYsa9NfTA= X-Received: by 2002:a05:622a:5c8b:b0:474:e255:db2c with SMTP id d75a77b69052e-4771dd8dd21mr3600651cf.26.1742493827149; Thu, 20 Mar 2025 11:03:47 -0700 (PDT) MIME-Version: 1.0 References: <20250320024011.2995837-1-pasha.tatashin@soleen.com> <20250320024011.2995837-3-pasha.tatashin@soleen.com> <2025032058-reassure-veneering-0fdb@gregkh> In-Reply-To: <2025032058-reassure-veneering-0fdb@gregkh> From: Pasha Tatashin Date: Thu, 20 Mar 2025 14:03:10 -0400 X-Gm-Features: AQ5f1JqUt7jlSeOStqUEKgBoUzbkc-shkgv5lvEs9-PxXsZsyZ9P_iNSvHCE62k Message-ID: Subject: Re: [RFC v1 2/3] luo: dev_liveupdate: Add device live update infrastructure To: Greg KH Cc: changyuanl@google.com, graf@amazon.com, rppt@kernel.org, rientjes@google.com, corbet@lwn.net, rdunlap@infradead.org, ilpo.jarvinen@linux.intel.com, kanie@linux.alibaba.com, ojeda@kernel.org, aliceryhl@google.com, masahiroy@kernel.org, akpm@linux-foundation.org, tj@kernel.org, yoann.congal@smile.fr, mmaurer@google.com, roman.gushchin@linux.dev, chenridong@huawei.com, axboe@kernel.dk, mark.rutland@arm.com, jannh@google.com, vincent.guittot@linaro.org, hannes@cmpxchg.org, dan.j.williams@intel.com, david@redhat.com, joel.granados@kernel.org, rostedt@goodmis.org, anna.schumaker@oracle.com, song@kernel.org, zhangguopeng@kylinos.cn, linux@weissschuh.net, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, rafael@kernel.org, dakr@kernel.org, bartosz.golaszewski@linaro.org, cw00.choi@samsung.com, myungjoo.ham@samsung.com, yesanishhere@gmail.com, Jonathan.Cameron@huawei.com, quic_zijuhu@quicinc.com, aleksander.lobakin@intel.com, ira.weiny@intel.com, andriy.shevchenko@linux.intel.com, leon@kernel.org, lukas@wunner.de, bhelgaas@google.com, wagi@kernel.org, djeffery@redhat.com, stuart.w.hayes@gmail.com, jgowans@amazon.com, jgg@nvidia.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Queue-Id: 51F22100034 X-Rspamd-Server: rspam03 X-Stat-Signature: 8iyh8atshwrxjo357pf8ew1xcshoydrj X-HE-Tag: 1742493828-863702 X-HE-Meta: U2FsdGVkX1891/84pOI+K6H6ZuYfNVgy6HI3MpC0NFJ2Z7SiLpkVNOrEas9SCxn6EQ6MLBb5IyZMJosQb46SrL0kehvercRA2Ezt8sAGs0D8wtqhHXmKs7BielLLI0xR8UmZmJAXFLqoLD5w8jQgJQuPdPIgNczugPjT/KylSDjjERFgbpi137kOhnaGe35lLq9s934W0cnR5aKGNJgLhOxY5ZHZn4w7rWGLw/cWCg/2JT3rpGdlj57UShl3Unxqvu/tde6CRSTgagT0z/2bP9fqcMmkxaQGxz7vMK+kj71O9Gf6tNEPIljROW7X05vd/cca4rVPrfZ97AlqgQK62lkLhi74nG29R8Ax99nOcDlothM4OQzg4wqBDcp0uA+8maaXt0/tBqEYffX1Gv4QAhDpPSyYIdUlo6N8MtZ7Q9epf4uFqMMs7JiDnAQF27Irw3Qlkr7mWMJUL5Ko8JC03SZmICVNQMv1Sdil3Nj962k0qZf3l2wzos5HNBzUoP77CSWuxYSlfpHf9C5eKzku/577A9xm2W8K3NA24KFKQDDmSjVioNjNQ2fd2ixE290RqxHu9/ly8pt6S7Ws1xPkiHOWaY90r6VyTJWV7m2btYG1Ft+/zw/QFLBCSYWH1RDm0oXP0b+Bb+/J590p0VVEc0zpk/cNEGBj/0P1RwkGCcLF/vKJTie/4cUG2cCGxjCpcK6L/hs8jhzbI6XiATglmOhz/1cXI7uVpcNzAsXXS4+lbftZzEAU8kHubRsVL2mfKjSP2f3iGlf8X0I47lIGBhRbT/RjhBFzVgUZ+ow4xBGI6d5UWufSRRNKupOIe9bQb0UhsSU34xuLtD2DVm16/WbUWQv+DyJumD4Bf0/LQ1LG5pdXxfH0AWSdCOftQHOpcB1zC01YvQ+anCvNfJqzxLSZqXAiKecXVRVGhIXH3/Yy8EfXw2Q8/mby7EvW/UT1AQV/SiGBG5aW556aLBW nWOcFFmv GOzrcBQmK8dIIvetG0qWm8GxsVTskUXojaz7YW9cywpdlqFABDr+T311uQa7EFB2gQdyHF0YQItJPcKLUKVdI3nqwt/twGbZ8ZawUH4jsLWhMf+8/Zt+g5179wH9JkBVd3WnpzXgNQy6cIbQHKbSwPtDZiPgq6Pc7RLLCm0ycKAYzOPFcf0Nset20lWmEMrBL0VIRXoWKOG85KvacSYRbAzsnC9T+pdnCrKHMqIBQlzs1khhUvjG7eEqRjG5OXtI7bkYbXqOp2QxipdB/bnJ8lsNdlmtCqNxDSxPUCkNekkXFXgoBmFmd79aIlb35HD/9dckCHByI1CKNyu2CfUcI9LGieVvLUHYJvjL5ivLaA7zQZ1HnwN5h36GTGw5TwhoSove+t2lOHXoaw51TdClqA6GJw4wA5ZQKeyJrBbClewaVhQdiki55tFNjR8VG0Ic02Dz42X61rww+Z4lvMIUSyUp7A7gOM8F39ELpwafnM/JYwoMM0HpGuDelvyZbfUulCohA7eok7tBrBJlsZprCSgsZLAYCAMiXCIVbkGLnFiojyOZONYe8W8Sa1nXrrNe9HBTaLqKWHYWZ0lA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Mar 20, 2025 at 9:36=E2=80=AFAM Greg KH wrote: > > On Thu, Mar 20, 2025 at 02:40:10AM +0000, Pasha Tatashin wrote: > > Introduce a new subsystem within the driver core to enable keeping > > devices alive during kernel live update. This infrastructure is > > designed to be registered with and driven by a separate Live Update > > Orchestrator, allowing the LUO's state machine to manage the save and > > restore process of device state during a kernel transition. > > > > The goal is to allow drivers and buses to participate in a coordinated > > save and restore process orchestrated by a live update mechanism. By > > saving device state before the kernel switch and restoring it > > immediately after, the device can appear to remain continuously > > operational from the perspective of the system and userspace. > > > > components introduced: > > > > - `struct dev_liveupdate`: Embedded in `struct device` to track the > > device's participation and state during a live update, including > > request status, preservation status, and dependency depth. > > > > - `liveupdate()` callback: Added to `struct bus_type` and > > `struct device_driver`. This callback receives an enum > > `liveupdate_event` to manage device state at different stages of the > > live update process: > > - LIVEUPDATE_PREPARE: Save device state before the kernel switch. > > - LIVEUPDATE_REBOOT: Final actions just before the kernel jump. > > - LIVEUPDATE_FINISH: Clean-up after live update. > > - LIVEUPDATE_CANCEL: Clean up any saved state if the update is > > aborted. > > > > - Sysfs attribute "liveupdate/requested": Added under each device > > directory, allowing user to request that a specific device to > > participate in live update. I.e. its state is to be preserved > > during the update. > > As you can imagine, I have "thoughts" about all of this being added to > the driver core. But, before I go off on that, I want to see some real, > actual, working, patches for at least 3 bus subsystems that correctly > implement this before I even consider reviewing this. > > Show us real users please, otherwise any attempt at reviewing this is > going to just be a waste of our time as I have doubts that this actually > even works :) > > Also, as you are adding a new user/kernel api, please also point at the > userspace tools that are written to handle all of this. As you are > going to be handling potentially tens of thousands of devices from > userspace this way, in a single system, real code is needed to even > consider that this is an acceptable solution. Hi Greg, Thanks for the feedback on this RFC. I understand your hesitation about adding this to the driver core without seeing concrete implementations. The primary goal of posting this RFC now is to get early feedback on the overall state machine and rules concept. We have a bi-weekly meeting [1] where the "Live Update Orchestrator" is scheduled for presentation. I wanted to give people a chance to look at the framework ahead of those discussions. Regarding your request for real, working patches, we are actively working on that. Our current efforts are focused on adding live update support for LUO for these subsystems: KVM, Interrupts, IOMMU, Devices Within the devices subsystem, we are targeting generic PCI, VFIO, and a few other device types (real and emulated) to demonstrate the implementation. I absolutely agree that demonstrating a real use case is important. However, this is a complicated project that involves changes in many parts of the kernel, and we can't deliver everything in one large patchset; it has to be divided and addressed incrementally. So far, we have the following pieces of the Live Update puzzle: KHO (for preserving kernel memory), LUO (for driving the live update process), and Dev_Liveupdate (for managing device participation in live update), IOMMU preservation [2], guest memory [3], and we are planning to add support for interrupts, PCIe, VFIO, some drivers, and other components. On the user side, we are planning to propose the necessary changes to VMMs such as CloudHypervisor and QEMU. Thanks, Pasha [1] https://lore.kernel.org/all/a350f3e5-e764-4ba6-f871-da7252f314da@google= .com [2] https://lpc.events/event/18/contributions/1686 [3] https://lore.kernel.org/all/20240805093245.889357-1-jgowans@amazon.com