Re: [PATCH] memcg: rearrage fields of mem_cgroup_per_node

[Shakeel Butt <shakeel.butt@linux.dev>]

On Wed, May 22, 2024 at 09:35:57PM -0700, Yosry Ahmed wrote:
> On Wed, May 22, 2024 at 8:48 PM Shakeel Butt <shakeel.butt@linux.dev> wrote:
[...]
> >
> >  struct mem_cgroup_per_node {
> > -       struct lruvec           lruvec;
> > +       /* Keep the read-only fields at the start */
> > +       struct mem_cgroup       *memcg;         /* Back pointer, we cannot */
> > +                                               /* use container_of        */
> >
> >         struct lruvec_stats_percpu __percpu     *lruvec_stats_percpu;
> >         struct lruvec_stats                     *lruvec_stats;
> > -
> > -       unsigned long           lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS];
> > -
> > -       struct mem_cgroup_reclaim_iter  iter;
> > -
> >         struct shrinker_info __rcu      *shrinker_info;
> >
> > +       /* memcg-v1 only stuff in middle */
> > +
> >         struct rb_node          tree_node;      /* RB tree node */
> >         unsigned long           usage_in_excess;/* Set to the value by which */
> >                                                 /* the soft limit is exceeded*/
> >         bool                    on_tree;
> > -       struct mem_cgroup       *memcg;         /* Back pointer, we cannot */
> > -                                               /* use container_of        */
> 
> Do we need CACHELINE_PADDING() here (or maybe make struct lruvec
> cache-aligned) to make sure the false cacheline sharing doesn't happen
> again with the fields below, or is the idea that the fields that get
> read in hot paths (memcg, lruvec_stats_percpu, lruvec_stats) are far
> at the top, and the memcg v1 elements in the middle act as a buffer?
> 
> IOW, is sharing between the fields below and memcg v1 fields okay
> because they are not read in the hot path? If yes, I believe it's
> worth a comment. It can be easily missed if the memcg v1 soft limit is
> removed later for example.
> 

For 6.10, I wanted to keep the change simple and yes, the memcg v1 stuff
as a buffer between the pointers and lruvec/lru_zone_size fields. For
6.11 or later kernels, I am planning to use some asserts to make sure
these fields don't share a cacheline, so later when we remove the
v1-only stuff, the asserts will make sure we keep the separate cacheline
property intact.