Re: [Bug 221055] New: Hibernate image write 10x slower since commit 0ff67f990bd4 ("mm, swap: remove swap slot cache")

Re: [Bug 221055] New: Hibernate image write 10x slower since commit 0ff67f990bd4 ("mm, swap: remove swap slot cache")

[Andrew Morton]

On Fri, 06 Feb 2026 19:51:38 +0000 bugzilla-daemon@kernel.org wrote:

> https://bugzilla.kernel.org/show_bug.cgi?id=221055

Dear Deity, I do wish kernel MM bugzilla would just go away.  Sigh.

Canned response follows:

(switched to email.  Please respond via emailed reply-to-all, not via the
bugzilla web interface).


>             Bug ID: 221055
>            Summary: Hibernate image write 10x slower since commit
>                     0ff67f990bd4 ("mm,  swap: remove swap slot cache")
>            Product: Memory Management
>            Version: 2.5
>           Hardware: All
>                 OS: Linux
>             Status: NEW
>           Severity: normal
>           Priority: P3
>          Component: Other
>           Assignee: akpm@linux-foundation.org
>           Reporter: carstengrohmann@gmx.de
>         Regression: No
> 
> === Summary ===
> 
> Hibernate image write 10x slower since commit 0ff67f990bd4 ("mm, 
> swap: remove swap slot cache")

Thanks for all the work you put into this.

> === Description ===
> 
> Since kernel 6.15, writing the hibernate image to swap has become
> approximately 10x slower. The regression was introduced by commit
> 0ff67f990bd45726e0d9e91111d998e7a3595b32 ("mm, swap: remove swap slot
> cache") authored by Kairui Song and merged by Andrew Morton.
> 
> === Performance Data ===
> 
>                  Before (6.14)   After (6.15+)
> Hibernate        28-32 seconds   290-342 seconds
> Effective rate   ~60 MB/s        ~4.5 MB/s
> 
> The swap partition's raw sequential write speed (measured with dd)
> is 255-450 MB/s, confirming the hardware is not the bottleneck.
> 
> === Bisect Result ===
> 
> A full git bisect on the vanilla kernel repository
> (git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git)
> between v6.14 (good) and v6.15 (bad) with 14 iterations
> consistently identified this commit:
> 
>   # first bad commit: [0ff67f990bd45726e0d9e91111d998e7a3595b32]
>   #   mm, swap: remove swap slot cache
> 
> The parent commit 1b7e90020eb7 ("mm, swap: use percpu cluster as
> allocation fast path") is fast (28s), while 0ff67f990bd4 itself
> is slow (307s).
> 
> === Kernel Version Tests ===
> 
>   Kernel              Duration   Result
>   6.12.1-arch1-1      32 sec     OK
>   6.13.1-arch1-1      42 sec     OK
>   6.14.1-arch1-1      32 sec     OK
>   6.15.1-arch1-1     302 sec     SLOW
>   6.18.1-arch1-1     290 sec     SLOW
> 
> === Reproduction ===
> 
> 1. System with swap partition (not swap file)
> 2. Boot kernel >= 6.15
> 3. Run: echo reboot > /sys/power/disk
> 4. Run: echo 1 > /sys/power/pm_print_times
> 5. Run: date; echo disk > /sys/power/state; date
> 6. Compare hibernate duration with kernel <= 6.14
> 
> For best visibility of the issue, use hibernate=nocompress as
> kernel parameter to disable image compression.
> 
> === Hardware ===
> 
> - CPU: Intel Core (Ivy Bridge)
> - RAM: 16 GB
> - SSD: Samsung SSD 830 Series (SATA II, 3.0 Gbps)
> - Swap: 20 GB partition (/dev/sda2)
> - Arch Linux x86_64
> 
> === Kernel Log (6.18, with hibernate=nocompress) ===
> 
>   PM: hibernation: hibernation entry
>   PM: hibernation: Preallocating image memory
>   PM: hibernation: Allocated 2217260 kbytes in 0.39 seconds (5685.28 MB/s)
>   PM: hibernation: Creating image:
>   PM: hibernation: Need to copy 506945 pages
>   PM: hibernation: Wrote 1794356 kbytes in 397.97 seconds (4.51 MB/s)
> 
> -- 
> You may reply to this email to add a comment.
> 
> You are receiving this mail because:
> You are the assignee for the bug.

Re: [Bug 221055] New: Hibernate image write 10x slower since commit 0ff67f990bd4 ("mm, swap: remove swap slot cache")

[Kairui Song]

On Fri, Feb 06, 2026 at 12:11:51PM +0800, Andrew Morton wrote:
> On Fri, 06 Feb 2026 19:51:38 +0000 bugzilla-daemon@kernel.org wrote:
> 
> > https://bugzilla.kernel.org/show_bug.cgi?id=221055
> 
> Dear Deity, I do wish kernel MM bugzilla would just go away.  Sigh.
> 
> Canned response follows:
> 
> (switched to email.  Please respond via emailed reply-to-all, not via the
> bugzilla web interface).
> 
> 
> >             Bug ID: 221055
> >            Summary: Hibernate image write 10x slower since commit
> >                     0ff67f990bd4 ("mm,  swap: remove swap slot cache")
> >            Product: Memory Management
> >            Version: 2.5
> >           Hardware: All
> >                 OS: Linux
> >             Status: NEW
> >           Severity: normal
> >           Priority: P3
> >          Component: Other
> >           Assignee: akpm@linux-foundation.org
> >           Reporter: carstengrohmann@gmx.de
> >         Regression: No
> > 
> > === Summary ===
> > 
> > Hibernate image write 10x slower since commit 0ff67f990bd4 ("mm, 
> > swap: remove swap slot cache")
> 
> Thanks for all the work you put into this.

Thanks for letting me know!

> 
> > === Description ===
> > 
> > Since kernel 6.15, writing the hibernate image to swap has become
> > approximately 10x slower. The regression was introduced by commit
> > 0ff67f990bd45726e0d9e91111d998e7a3595b32 ("mm, swap: remove swap slot
> > cache") authored by Kairui Song and merged by Andrew Morton.
> > 
> > === Performance Data ===
> > 
> >                  Before (6.14)   After (6.15+)
> > Hibernate        28-32 seconds   290-342 seconds
> > Effective rate   ~60 MB/s        ~4.5 MB/s
> > 
> > The swap partition's raw sequential write speed (measured with dd)
> > is 255-450 MB/s, confirming the hardware is not the bottleneck.
> > 
> > === Bisect Result ===
> > 
> > A full git bisect on the vanilla kernel repository
> > (git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git)
> > between v6.14 (good) and v6.15 (bad) with 14 iterations
> > consistently identified this commit:
> > 
> >   # first bad commit: [0ff67f990bd45726e0d9e91111d998e7a3595b32]
> >   #   mm, swap: remove swap slot cache

Oh, I took a look and I think I know the problem now, the new added
fast path in that series is only used by allocation from vmscan,
for hibernation it always use the slow path, as I didn't expect the
slow path is below hibernation's requirement.

Or maybe, it not the allocation path that is slow, but because
cluster rotation in slow path splatted the allocation to every
cluster makes the IO much slower.

Both are not too hard to fix. Is that's SSD only issue? And
should we fix this in every LTS or just mainline? It could
be a bit trouble some if every LTS needs a fix.

Re: [Bug 221055] New: Hibernate image write 10x slower since commit 0ff67f990bd4 ("mm, swap: remove swap slot cache")

[Kairui Song]

[-- Attachment #1: Type: text/plain, Size: 2201 bytes --]

On Wed, Feb 11, 2026 at 2:39 AM Kairui Song <ryncsn@gmail.com> wrote:
>
> On Fri, Feb 06, 2026 at 12:11:51PM +0800, Andrew Morton wrote:
> > On Fri, 06 Feb 2026 19:51:38 +0000 bugzilla-daemon@kernel.org wrote:
> >
> > > https://bugzilla.kernel.org/show_bug.cgi?id=221055
> >
> > > === Summary ===
> > >
> > > Hibernate image write 10x slower since commit 0ff67f990bd4 ("mm,
> > > swap: remove swap slot cache")
> >
> > Thanks for all the work you put into this.
>
> > > === Bisect Result ===
> > >
> > > A full git bisect on the vanilla kernel repository
> > > (git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git)
> > > between v6.14 (good) and v6.15 (bad) with 14 iterations
> > > consistently identified this commit:
> > >
> > >   # first bad commit: [0ff67f990bd45726e0d9e91111d998e7a3595b32]
> > >   #   mm, swap: remove swap slot cache
>
> Both are not too hard to fix. Is that's SSD only issue? And
> should we fix this in every LTS or just mainline? It could
> be a bit trouble some if every LTS needs a fix.

Hi All,

After many attempts, I haven't (fully) reproduced it yet. But I do
have a patch, see attachment.

After this patch:
PM: hibernation: Wrote 1559088 kbytes in 0.55 seconds (2834.70 MB/s) - NVME
PM: hibernation: Wrote 2038748 kbytes in 4.04 seconds (504.64 MB/s) - SATA

After 0ff67f990bd4 (the regression commit):
PM: hibernation: Wrote 2224184 kbytes in 0.90 seconds (2471.31 MB/s) - NVME
PM: hibernation: Wrote 2215472 kbytes in 4.44 seconds (498.98 MB/s) - SATA

Before 0ff67f990bd4:
PM: hibernation: Wrote 2222772 kbytes in 0.84 seconds (2646.15 MB/s) - NVME
PM: hibernation: Wrote 2230700 kbytes in 4.47 seconds (499.03 MB/s) - SATA

It seems a bit faster / slower indeed, or maybe just noise. On my
machines 0ff67f990bd4 is not causing an major issue like the one
described in the bugzilla link. I'll try to see if there are more detailed
issues with the test setup.

And can anyone help verify this fix? I'll also try some other SSD later
maybe that's only reproducible with SSD that have very poor 4K performance.

HDD should not be affected because the allocator has a special
way to do HDD allocation.

[-- Attachment #2: hibernate-perf.patch --]
[-- Type: application/x-patch, Size: 5772 bytes --]

Re: [Bug 221055] New: Hibernate image write 10x slower since commit 0ff67f990bd4 ("mm, swap: remove swap slot cache")

[Carsten Grohmann]

12.02.2026 20:01:48 Kairui Song <ryncsn@gmail.com>:

> And can anyone help verify this fix? I'll also try some other SSD later
> maybe that's only reproducible with SSD that have very poor 4K 
> performance.
I'll try out your patch over the weekend.

> HDD should not be affected because the allocator has a special
> way to do HDD allocation.
I can connect an older hard drive to my PC and test this as well. Should 
I do that? If it's not necessary, then I can save myself the effort.

Re: [Bug 221055] New: Hibernate image write 10x slower since commit 0ff67f990bd4 ("mm, swap: remove swap slot cache")

[Kairui Song]

Carsten Grohmann <carstengrohmann@gmx.de> 于 2026年2月13日周五 04:37写道：
>
> 12.02.2026 20:01:48 Kairui Song <ryncsn@gmail.com>:
>
> > And can anyone help verify this fix? I'll also try some other SSD later
> > maybe that's only reproducible with SSD that have very poor 4K
> > performance.
> I'll try out your patch over the weekend.

Thanks a lot!

>
> > HDD should not be affected because the allocator has a special
> > way to do HDD allocation.
> I can connect an older hard drive to my PC and test this as well. Should
> I do that? If it's not necessary, then I can save myself the effort.

That's not necessary, let's just verify the issue you reported. I have
been running swap tests on HDD for a while and hibernate uses the same
allocation method, the performance is good so far, I can double check
that.

Re: [Bug 221055] New: Hibernate image write 10x slower since commit 0ff67f990bd4 ("mm, swap: remove swap slot cache")

[Carsten Grohmann]

12.02.2026 21:37:04 Carsten Grohmann <carstengrohmann@gmx.de>:

> 12.02.2026 20:01:48 Kairui Song <ryncsn@gmail.com>:
>
>> And can anyone help verify this fix? I'll also try some other SSD 
>> later
>> maybe that's only reproducible with SSD that have very poor 4K 
>> performance.
> I'll try out your patch over the weekend.

I tested your patch with the current HEAD from git.kernel.org. The first 
run is with your patch, the second without:

Date:     2026-02-15 08:29:33
Commit:   770aaedb461a (Merge tag 'bootconfig-v7.0' of 
git://git.kernel.or)
Kernel:   6.19.0-20260214-153617-09922-g770aaedb461a-dirty
Duration: 35 Sekunden
Result:   good
----------------------------------------
Date:     2026-02-15 09:01:25
Commit:   770aaedb461a (Merge tag 'bootconfig-v7.0' of 
git://git.kernel.or)
Kernel:   6.19.0-20260215-083120-09922-g770aaedb461a
Duration: 324 Sekunden
Result:   bad

With your patch, hibernation is fast again. It solves the problem I 
reported.

Thank you very much for taking care of my annoying problem.

Regards,
Carsten

Re: [Bug 221055] New: Hibernate image write 10x slower since commit 0ff67f990bd4 ("mm, swap: remove swap slot cache")

[Kairui Song]

On Sun, Feb 15, 2026 at 4:56 PM Carsten Grohmann <carstengrohmann@gmx.de> wrote:
>
> 12.02.2026 21:37:04 Carsten Grohmann <carstengrohmann@gmx.de>:
>
> > 12.02.2026 20:01:48 Kairui Song <ryncsn@gmail.com>:
> >
> >> And can anyone help verify this fix? I'll also try some other SSD
> >> later
> >> maybe that's only reproducible with SSD that have very poor 4K
> >> performance.
> > I'll try out your patch over the weekend.
>
> I tested your patch with the current HEAD from git.kernel.org. The first
> run is with your patch, the second without:
>
> Date:     2026-02-15 08:29:33
> Commit:   770aaedb461a (Merge tag 'bootconfig-v7.0' of
> git://git.kernel.or)
> Kernel:   6.19.0-20260214-153617-09922-g770aaedb461a-dirty
> Duration: 35 Sekunden
> Result:   good
> ----------------------------------------
> Date:     2026-02-15 09:01:25
> Commit:   770aaedb461a (Merge tag 'bootconfig-v7.0' of
> git://git.kernel.or)
> Kernel:   6.19.0-20260215-083120-09922-g770aaedb461a
> Duration: 324 Sekunden
> Result:   bad
>
> With your patch, hibernation is fast again. It solves the problem I
> reported.
>
> Thank you very much for taking care of my annoying problem.

Thank you very much for the testing! I'll post the fix with your tested-by.