Re: [PATCH RFC 2/2] zram: support compression at the granularity of multi-pages

[Sergey Senozhatsky <senozhatsky@chromium.org>]

On (24/04/11 19:49), Barry Song wrote:
> > > This question brings up an interesting observation. In our actual product,
> > > we've noticed a success rate of over 90% when allocating large folios in
> > > do_swap_page, but occasionally, we encounter failures. In such cases,
> > > instead of resorting to partial reads, we opt to allocate 16 small folios and
> > > request zram to fill them all. This strategy effectively minimizes partial reads
> > > to nearly zero. However, integrating this into the upstream codebase seems
> > > like a considerable task, and for now, it remains part of our
> > > out-of-tree code[1],
> > > which is also open-source.
> > > We're gradually sending patches for the swap-in process, systematically
> > > cleaning up the product's code.
> >
> > I see, thanks for explanation.
> > Does this sound like this series is ahead of its time?
> 
> I feel it is necessary to present the whole picture together with large folios
> swp-in series[1]

Yeah, makes sense.

> > These partial reads/writes are difficult to justify - instead of doing
> > comp_op(PAGE_SIZE) we, in the worst case, now can do ZCOMP_MULTI_PAGES_NR
> > of comp_op(ZCOMP_MULTI_PAGES_ORDER) (assuming a access pattern that
> > touches each of multi-pages individually). That is a potentially huge
> > increase in CPU/power usage, which cannot be easily sacrificed. In fact,
> > I'd probably say that power usage is more important here than zspool
> > memory usage (that we have means to deal with).
> 
> Once Ryan's mTHP swapout without splitting [2] is integrated into the
> mainline, this
> patchset certainly gains an advantage for SWPOUT. However, for SWPIN,
> the situation
> is more nuanced. There's a risk of failing to allocate mTHP, which
> could result in the
> allocation of a small folio instead. In such cases, decompressing a
> large folio but
> copying only one subpage leads to inefficiency.
> 
> In real-world products, we've addressed this challenge in two ways:
> 1. We've enhanced reserved page blocks for mTHP to boost allocation
> success rates.
> 2. In instances where we fail to allocate a large folio, we fall back
> to allocating nr_pages
> small folios instead of just one. so we still only decompress once for
> multi-pages.
> 
> With these measures in place, we consistently achieve wins in both
> power consumption and
> memory savings. However, it's important to note that these
> optimizations are specific to our
> product, and there's still much work needed to upstream them all.

Do you track any other metrics? Memory savings is just one way of looking
at it. The other metrics is utilization ratio of zspool
	compressed size : zs_get_total_pages(zram->mem_pool)

Compaction and migration can also be interesting, given that
zsmalloc is changing.

> > Have you evaluated power usage?
> >
> > I also wonder if it brings down the number of ZRAM_SAME pages. Suppose
> > when several pages out of ZCOMP_MULTI_PAGES_ORDER are filled with zeroes
> > (or some other recognizable pattern) which previously would have been
> > stored using just unsigned long. Makes me even wonder if ZRAM_SAME test
> > makes sense on multi-page at all, for that matter.
> 
> I don't think we need to worry about ZRAM_SAME.

Oh, it's not that I worry about it, just another thing that is
changing. E.g. having memcpy() /* current ZRAM_SAME handing ling */
vs decomp(order 4) and then memcpy().

> mTHP offers a means to emulate a 16KiB/64KiB base page while
> maintaining software
> compatibility with a 4KiB base page. The primary concern here lies in
> partial read/write
> operations. In our product, we've successfully addressed these issues. However,
> convincing people in the mainline community may take considerable time
> and effort :-)

Do you have a rebased zram/zsmalloc series somewhere in public access
that I can test?