RE: [PATCH net-next 1/2] net: Keep sk->sk_forward_alloc as a proper size

["Zhang, Cathy" <cathy.zhang@intel.com>]

> -----Original Message-----
> From: Shakeel Butt <shakeelb@google.com>
> Sent: Wednesday, May 10, 2023 1:58 AM
> To: Zhang, Cathy <cathy.zhang@intel.com>; Linux MM <linux-
> mm@kvack.org>; Cgroups <cgroups@vger.kernel.org>
> Cc: Eric Dumazet <edumazet@google.com>; Paolo Abeni
> <pabeni@redhat.com>; davem@davemloft.net; kuba@kernel.org;
> Brandeburg, Jesse <jesse.brandeburg@intel.com>; Srinivas, Suresh
> <suresh.srinivas@intel.com>; Chen, Tim C <tim.c.chen@intel.com>; You,
> Lizhen <lizhen.you@intel.com>; eric.dumazet@gmail.com;
> netdev@vger.kernel.org
> Subject: Re: [PATCH net-next 1/2] net: Keep sk->sk_forward_alloc as a proper
> size
> 
> On Tue, May 9, 2023 at 8:07 AM Zhang, Cathy <cathy.zhang@intel.com>
> wrote:
> >
> [...]
> > >
> > > Something must be wrong in your setup, because the only small issue
> > > that was noticed was the memcg one that Shakeel solved last year.
> >
> > As mentioned in commit log, the test is to create 8 memcached-memtier
> > pairs on the same host, when server and client of the same pair
> > connect to the same CPU socket and share the same CPU set (28 CPUs),
> > the memcg overhead is obviously high as shown in commit log. If they
> > are set with different CPU set from separate CPU socket, the overhead
> > is not so high but still observed.  Here is the server/client command in our
> test:
> > server:
> > memcached -p ${port_i} -t ${threads_i} -c 10240
> > client:
> > memtier_benchmark --server=${memcached_id} --port=${port_i} \
> > --protocol=memcache_text --test-time=20 --threads=${threads_i} \ -c 1
> > --pipeline=16 --ratio=1:100 --run-count=5
> >
> > So, is there anything wrong you see?
> >
> 
> What is the memcg hierarchy of this workload? Is each server and client
> processes running in their own memcg? How many levels of memcgs? Are
> you setting memory.max and memory.high to some value? Also how are you
> limiting the processes to CPUs? cpusets?

Here is the full command to start memcached instance:

docker run -d --name ${memcached_name} --privileged --memory 1G --network bridge \
-p ${port_i}:${port_i} ${cpu_pinning_s[set]} memcached memcached -p ${port_i} \
-t ${threads_i} -c 10240

We have a script to get CPU set from the same NUMA node, both CPU count and thread
count for each instance are equal to Num(system online CPUs) / Num(memcached instances).
That is, if we run 8 memcached instances, 224 / 8 = 28, so each instance will get 28 CPUs and
28 threads assigned.

Here is the full command to start memtier instance:
docker run --rm --network bridge ${cpu_pinning_s[set]} --memory 1G \
redislabs/memtier_benchmark memtier_benchmark --server=${memcached_id} --port=${port_i} \
--protocol=memcache_text --test-time=20 --threads=${threads_i} -c 1 --pipeline=16 --ratio=1:100 \
--run-count=5 --hide-histogram

Each instance has the same CPU set as the server it connects to, and it has the same threads
count.

That is all for server and client settings.