From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A340DC433F5 for ; Wed, 18 May 2022 16:55:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3953F6B0072; Wed, 18 May 2022 12:55:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 344D68D0001; Wed, 18 May 2022 12:55:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20CFB6B0074; Wed, 18 May 2022 12:55:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 124ED6B0072 for ; Wed, 18 May 2022 12:55:57 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D65BD614DD for ; Wed, 18 May 2022 16:55:56 +0000 (UTC) X-FDA: 79479465912.25.48D0069 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) by imf28.hostedemail.com (Postfix) with ESMTP id E14E0C00C4 for ; Wed, 18 May 2022 16:55:29 +0000 (UTC) Received: by mail-ej1-f47.google.com with SMTP id ck4so734983ejb.8 for ; Wed, 18 May 2022 09:55:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=+RcN5J0ur4waoOrpP5dEneSJvtTucrgzrBX0vuvDNzg=; b=jqT/3rdeSfMtEpH8kcA4/nn+EfeZSUuf8LPPqhLKI9etfw6hQt/jE/5RB9+wAn7N+Z Ehawr2Lnj8Xy/V9rEAUasHDJAEfZoSITtIIF1X9hmH3oDXIBNzCPI0/8wWj68x+PdNBs CYcJUtEpJeYfXt4e7pO3nCwPPYgiL7h53DO5Fuujh2Y6lzOqlHXACtVhqvzimDu3Oual tfhtqDL4wOAzaNxnKSix47hX+w5GxeHrTuVV+uT6Gaq249kr69g+Sjjj703raStyIOke zlWbjfU2A5GEbVZtSf4efh8q1kIlRtND9i+ESBSanFHpCYBuGyVgZ0dkOIm+//g3Sp7O /xlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+RcN5J0ur4waoOrpP5dEneSJvtTucrgzrBX0vuvDNzg=; b=mPHMU5qOQfkY1lfX3Sgo9Lxpx1/BF1vRMOEqEBzdn62zknEHzueqjnDC5hqmQxZRs8 AmV2VajoC1wIWbavQTRUU2ioXRXXFYHGbihm0ufNePAkcRDOTm5FQC0R9q/dDX05UFqA cuPNU8P/q+19FF/nPIdCHLGF8bCB03eRZPlt3oW4E3MFZfrHOPeT8JQeZRt2nq068H09 54uFJCIxXnFOChPUVZSlld6eEAg5XRfC9jyZ/UongCui5t6HPrla7fueRF5JDnSzjv29 Gi2hSxY0rUG5JUmNN9lVEPmQ14rXceWoGZUlh095wCtf72RCfCadfc0jxjTt8/9wTopt Qqrw== X-Gm-Message-State: AOAM530M3ZrKS1Fv5iZ/752EG+DnQL31mneXEI4nKKvlNHorqlUqZuXi zpkTnIRKsGlIIgYCz/jGEtBhDmk1SmIyGBFCqh0Bog== X-Google-Smtp-Source: ABdhPJzoDOL5onqSCpBWW4lu7fFIE11dPS6M11PWkiCu5oUw/XsvL6IBRO8qySJCvzfyvrHLY+M/j+7SW0BKghv5pwQ= X-Received: by 2002:a17:907:1c8a:b0:6e9:2a0d:d7b7 with SMTP id nb10-20020a1709071c8a00b006e92a0dd7b7mr479411ejc.572.1652892954798; Wed, 18 May 2022 09:55:54 -0700 (PDT) MIME-Version: 1.0 References: <1652860121-24092-1-git-send-email-quic_vivekuma@quicinc.com> In-Reply-To: <1652860121-24092-1-git-send-email-quic_vivekuma@quicinc.com> From: Pasha Tatashin Date: Wed, 18 May 2022 12:55:18 -0400 Message-ID: Subject: Re: [RFC 0/6] Bootloader based hibernation To: Vivek Kumar Cc: Jonathan Corbet , Catalin Marinas , Will Deacon , Thomas Gleixner , Marc Zyngier , Jens Axboe , "Rafael J. Wysocki" , Andrew Morton , Linux Doc Mailing List , LKML , Linux ARM , linux-block , linux-pm@vger.kernel.org, linux-mm , len.brown@intel.com, Pavel Machek , paulmck@kernel.org, Borislav Petkov , Kees Cook , Muchun Song , Randy Dunlap , damien.lemoal@opensource.wdc.com, Fuad Tabba , Ard Biesheuvel , tsoni@quicinc.com, quic_psodagud@quicinc.com, quic_svaddagi@quicinc.com Content-Type: text/plain; charset="UTF-8" Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b="jqT/3rde"; spf=pass (imf28.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.218.47 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none X-Rspam-User: X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: E14E0C00C4 X-Stat-Signature: x5s335bkzagr7hyoee4j7a7ydtsa9byo X-HE-Tag: 1652892929-130342 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Vivek, Interesting stuff. On Wed, May 18, 2022 at 3:49 AM Vivek Kumar wrote: > > Kernel Hibernation > > Linux Kernel has been already supporting hibernation, a process which > involves freezing of all userspace tasks, followed by quiescing of all > kernel device drivers and then a DDR snapshot is taken which is saved > to disc-swap partition, after the save, the system can either shutdown > or continue further. Generally during the next power cycle when kernel > boots and after probing almost all of the drivers, in the late_init() > part, it checks if a hibernation image is present in the specified swap > slot, if a valid hibernation image is found, it superimposes the currently > executing Kernel with an older kernel from the snapshot, moving further, > it calls the restore of the drivers and unfreezes the userspace tasks. > CONFIG_HIBERNATION and a designated swap partition needs to be present > for to enable Hibernation. > > Bootloader Based Hibernation: > > Automotive usecases require better boot KPIs, Hence we are proposing a > bootloader based hibernation restore. Purpose of bootloader based > hibernation is to improve the overall boot time till the first display > frame is seen on the screen or a camera application can be launched from > userspace after the power on reset key is pressed. This RFC patchset > implements a slightly tweaked version of hibernation in which the > restoration of an older snapshot into DDR is being carried out from the > bootloader (ABL) itself, by doing this we are saving some time > (1 second measured on msm-4.14 Kernel) by not running a > temporary kernel and figuring out the hibernation image at late_init(). I wonder where most of the time is spent? Is it initializing struct pages? Potentially we could enlighten bootloader to determine whether hibernation image is stored or not on the swap device, and change boot parameter for the kernel accordingly. The booting kernel would know from the very beginning of boot that it will eventually resume a hibernated image, and therefore skip some of initilization parts, and perhaps limit amount of memory that it initializes. > In order to achieve the same bootloader checks for the hibernation > image at a very early stage from swap partition, it parses the image and > loads it in the DDR instead of loading boot image form boot partition. > Since we are not running the temporary kernel,which would have done some > basic ARM related setup like, MMU enablement, EL2 setup, CPU setup etc, What boot loader is used? I suspect bootloader enables MMU to load the hibernated image into memory, otherwise the performance would be very poor. After the image is loaded, and prior to jumping into the entry address of loaded image the MMU is probably disabled. > entry point into hibernation snapshot image directly from bootloader is > different, on similar lines, all device drivers are now re-programming > the IO-mapped registers as part of the restore callback (which is > triggered from the hibernation framework) to bring back the HW/SW sync. > > Other factors like, read-speed of the secondary storage device and > organization of the hibernation image in the swap partition effects the > total image restore time and the overall boot time. In our current > implementation we have serialized the allocation of swap-partition's slots > in kernel, so when hibernation image is being saved to disc, each page is > not scattered across various swap-slot offsets, rather it in a serial > manner. For example, if a DDR page at Page frame number 0x8005 is > located at a swap-slot offset 50, the next valid DDR page at PFN 0x8005 > will be preset at the swap-slot offset 51. With this optimization in > place, bootloader can utilize the max capacity of issuing a disc-read > for reading a bigger chunk (~50 MBs at once) from the swap slot, > and also parsing of the image becomes simpler as it is available > contiguously. This optimization seems generic enough that it would benefit both types of resume: from bootloader and from kernel. Thanks, Pasha > > > > Vivek Kumar (6): > arm64: hibernate: Introduce new entry point to kernel > PM: Hibernate: Add option to disable disk offset randomization > block: gendisk: Add a new genhd capability flag > mm: swap: Add randomization check for swapon/off calls > Hibernate: Add check for pte_valid in saveable page > irqchip/gic-v3: Re-init GIC hardware upon hibernation restore > > Documentation/admin-guide/kernel-parameters.txt | 11 ++ > arch/arm64/kernel/hibernate.c | 9 ++ > drivers/irqchip/irq-gic-v3.c | 138 ++++++++++++++++- > include/linux/blkdev.h | 1 + > kernel/power/snapshot.c | 43 ++++++++ > kernel/power/swap.c | 12 +++ > mm/swapfile.c | 6 +- > 7 files changed, 216 insertions(+), 4 deletions(-) > > -- > 2.7.4 >