Re: [QUESTION] Resizing shared mapping without clashing with others

[Dmitry Dolgov <9erthalion6@gmail.com>]

> On Mon, Dec 02, 2024 at 04:40:30PM GMT, David Hildenbrand wrote:
> On 02.12.24 16:04, Dmitry Dolgov wrote:
> > > On Mon, Dec 02, 2024 at 12:07:01PM GMT, David Hildenbrand wrote:
> > >
> > > Likely it's best to look into reserving a large VMA space using
> > > mmap(MAP_PRIVATE|MA_ANON, PROT_NONE); it's reserved but inaccessible, so it
> > > cannot get reallocated for different purposes. Then converting pieces of
> > > that into actually usable shared anonymous memory (e.g., MAP_FIXED).
> >
> > Yes, we've considered this option, preparing an initial "reserving"
> > mapping, then take pieces out of it for usage. As I've mentioned to
> > Matthew, there seems to be arguments against this approach. In a few
> > words:
>
> Note that this approach is extremely common ;)

That's why I'm asking your folks, trying to get some wisdom :) Although
I have to translated this knowledge it into the database world, is it
common for access patterns like we see in databases (large amount of
memory used as a buffer cache) as well?

> > * In certain cases this will be counted against a reservation limits,
> >    although we don't use the memory yet (e.g. with hugetlb insige a
> >    cgroup with hugetlb.2MB.rsvd.limit_in_bytes set).
>
>
> With  mmap(MAP_PRIVATE|MAP_ANON, PROT_NONE)? Unlikely. In any case, you
> might be able to tame reservations using MAP_NORESERVE.

Oh, interesting, I didn't realize this flag controls that as well. This
might just work, although there are still annoying parts -- postgres
actually needs reservation limits checking at the start to figure out of
any huge pages are available at all (there were too many complains about
getting SIGBUS, when limitations applied at page fault time). But maybe
I can work something out. Thanks!