Re: [Lsf-pc] [LSF/MM/BPF TOPIC] Optimizing Page Cache Readahead Behavior

[Kalesh Singh <kaleshsingh@google.com>]

On Tue, Feb 25, 2025 at 8:36 AM Jan Kara <jack@suse.cz> wrote:
>
> On Mon 24-02-25 13:36:50, Kalesh Singh wrote:
> > On Mon, Feb 24, 2025 at 8:52 AM Lorenzo Stoakes
> > > > > > OK, I agree the behavior you describe exists. But do you have some
> > > > > > real-world numbers showing its extent? I'm not looking for some artificial
> > > > > > numbers - sure bad cases can be constructed - but how big practical problem
> > > > > > is this? If you can show that average Android phone has 10% of these
> > > > > > useless pages in memory than that's one thing and we should be looking for
> > > > > > some general solution. If it is more like 0.1%, then why bother?
> > > > > >
> >
> > Once I revert a workaround that we currently have to avoid
> > fault-around for these regions (we don't have an out of tree solution
> > to prevent the page cache population); our CI which checks memory
> > usage after performing some common app user-journeys; reports
> > regressions as shown in the snippet below. Note, that the increases
> > here are only for the populated PTEs (bounded by VMA) so the actual
> > pollution is theoretically larger.
> >
> > Metric: perfetto_media.extractor#file-rss-avg
> > Increased by 7.495 MB (32.7%)
> >
> > Metric: perfetto_/system/bin/audioserver#file-rss-avg
> > Increased by 6.262 MB (29.8%)
> >
> > Metric: perfetto_/system/bin/mediaserver#file-rss-max
> > Increased by 8.325 MB (28.0%)
> >
> > Metric: perfetto_/system/bin/mediaserver#file-rss-avg
> > Increased by 8.198 MB (28.4%)
> >
> > Metric: perfetto_media.extractor#file-rss-max
> > Increased by 7.95 MB (33.6%)
> >
> > Metric: perfetto_/system/bin/incidentd#file-rss-avg
> > Increased by 0.896 MB (20.4%)
> >
> > Metric: perfetto_/system/bin/audioserver#file-rss-max
> > Increased by 6.883 MB (31.9%)
> >
> > Metric: perfetto_media.swcodec#file-rss-max
> > Increased by 7.236 MB (34.9%)
> >
> > Metric: perfetto_/system/bin/incidentd#file-rss-max
> > Increased by 1.003 MB (22.7%)
> >
> > Metric: perfetto_/system/bin/cameraserver#file-rss-avg
> > Increased by 6.946 MB (34.2%)
> >
> > Metric: perfetto_/system/bin/cameraserver#file-rss-max
> > Increased by 7.205 MB (33.8%)
> >
> > Metric: perfetto_com.android.nfc#file-rss-max
> > Increased by 8.525 MB (9.8%)
> >
> > Metric: perfetto_/system/bin/surfaceflinger#file-rss-avg
> > Increased by 3.715 MB (3.6%)
> >
> > Metric: perfetto_media.swcodec#file-rss-avg
> > Increased by 5.096 MB (27.1%)
> >
> > [...]
> >
> > The issue is widespread across processes because in order to support
> > larger page sizes Android has a requirement that the ELF segments are
> > at-least 16KB aligned, which lead to the padding regions (never
> > accessed).
>
> Thanks for the numbers! It's much more than I'd expect. So you apparently
> have a lot of relatively small segments?

Hi Jan,

Yeah you are right the segments can be relatively small.

I took one app on my device as an example:

adb shell 'cat /proc/$(pidof com.google.android.youtube)/maps' | grep
'.so$' | tee youtube_so_segments.txt

cat youtube_so_segments.txt | ./total_mapped_size.sh
Total mapping length: 147980288 bytes

cat youtube_so_segments.txt | wc -l
1148

147980288/1148/1024 = 125.88 KB

Let's say very roughly on average it's 128KB per segment; the padding
region can be anywhere from 0 to 60KB of that.

--Kalesh

>
> > Another possible way we can look at this: in the regressions shared
> > above by the ELF padding regions, we are able to make these regions
> > sparse (for *almost* all cases) -- solving the shared-zero page
> > problem for file mappings, would also eliminate much of this overhead.
> > So perhaps we should tackle this angle? If that's a more tangible
> > solution ?
> >
> > From the previous discussions that Matthew shared [7], it seems like
> > Dave proposed an alternative to moving the extents to the VFS layer to
> > invert the IO read path operations [8]. Maybe this is a move
> > approachable solution since there is precedence for the same in the
> > write path?
>
> Yeah, so I certainly wouldn't be opposed to this. What Dave suggests makes
> a lot of sense. In principle we did something similar for DAX. But it won't be
> a trivial change so details matter...
>
>                                                                         Honza
> --
> Jan Kara <jack@suse.com>
> SUSE Labs, CR