Re: [PATCH V5 06/10] Per-memcg background reclaim.

[Ying Han <yinghan@google.com>]

[-- Attachment #1: Type: text/plain, Size: 3152 bytes --]

On Mon, Apr 18, 2011 at 4:32 PM, Minchan Kim <minchan.kim@gmail.com> wrote:

> On Tue, Apr 19, 2011 at 6:38 AM, Ying Han <yinghan@google.com> wrote:
> >
> >
> > On Sun, Apr 17, 2011 at 8:51 PM, Minchan Kim <minchan.kim@gmail.com>
> wrote:
> >>
> >> On Sat, Apr 16, 2011 at 8:23 AM, Ying Han <yinghan@google.com> wrote:
> >> > +
> >> > +               sc->nr_scanned = 0;
> >> > +               shrink_zone(priority, zone, sc);
> >> > +               total_scanned += sc->nr_scanned;
> >> > +
> >> > +               /*
> >> > +                * If we've done a decent amount of scanning and
> >> > +                * the reclaim ratio is low, start doing writepage
> >> > +                * even in laptop mode
> >> > +                */
> >> > +               if (total_scanned > SWAP_CLUSTER_MAX * 2 &&
> >> > +                   total_scanned > sc->nr_reclaimed +
> sc->nr_reclaimed
> >> > / 2) {
> >> > +                       sc->may_writepage = 1;
> >>
> >> I don't want to add more random write any more although we don't have
> >> a trouble of real memory shortage.
> >
> >
> >>
> >> Do you have any reason to reclaim memory urgently as writing dirty
> pages?
> >> Maybe if we wait a little bit of time, flusher would write out the page.
> >
> > We would like to reduce the writing dirty pages from page reclaim,
> > especially from direct reclaim. AFAIK, the try_to_free_mem_cgroup_pages()
> > still need to write dirty pages when there is a need. removing this from
> the
> > per-memcg kswap will only add more pressure to the per-memcg direct
> reclaim,
> > which seems to be worse. (stack overflow as one example which we would
> like
> > to get rid of)
> >
>
> Stack overflow would be another topic.
>
> Normal situation :
>
> The softlimit memory pressure of memcg isn't real memory shortage and
> if we have gap between hardlimit and softlimit, periodic writeback of
> flusher would write it out before reaching the hardlimit. In the end,
> direct reclaim don't need to write it out.
>
> Exceptional situation :
>
> Of course, it doesn't work well in congestion of bdi, sudden big
> memory consumption in memcg in wrong [hard/soft]limit(small gap)
> configuration of administrator.
>
> I think we have to design it by normal situation.
> The point is that softlimit isn't real memory shortage so that we are
> not urgent.
>

This patch is not dealing with soft_limit, but hard_limit. The soft_limit
reclaim which we talked about during LSF
is something i am currently looking at right now. This patch is doing the
per-memcg background reclaim which
based on the watermarks calculated on the hard_limit. We don't have the
memcg entering the direct reclaim each
time it is reaching the hard_limit, so we add the background reclaim which
reclaiming pages proactively.


> How about adding new function which checks global memory pressure and
> if we have a trouble by global memory pressure, we can change
> may_write with 1 dynamically in memcg_kswapd?
>

Like I mentioned, the may_write is still needed in this case otherwise we
are just put this further to per-memcg
direct reclaim.


Thanks

--Ying

> --
> Kind regards,
> Minchan Kim
>

[-- Attachment #2: Type: text/html, Size: 4527 bytes --]