From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBCCAC7EE22 for ; Wed, 10 May 2023 15:13:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 434356B0072; Wed, 10 May 2023 11:13:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E46A6B0074; Wed, 10 May 2023 11:13:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2ACDC6B0075; Wed, 10 May 2023 11:13:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1CCCD6B0072 for ; Wed, 10 May 2023 11:13:22 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8C4C01C7656 for ; Wed, 10 May 2023 15:13:21 +0000 (UTC) X-FDA: 80774689002.07.6321454 Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) by imf28.hostedemail.com (Postfix) with ESMTP id BE462C08B8 for ; Wed, 10 May 2023 15:07:36 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=VHa+zJPx; spf=pass (imf28.hostedemail.com: domain of edumazet@google.com designates 209.85.128.49 as permitted sender) smtp.mailfrom=edumazet@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683731256; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+TNYqcSs6PC6BWjv/cKNCEBTiGNlpvDG9qZWo/NwWgk=; b=iMPFSk/8j9i+mTQ/60U+7SMdhBEVV4W+RgyBYSc7U6LvTJKS2FWD9Aj6GcJBIwXZbjvhyG DHOpgAojZGizLl7VqpMk+uu3eRnpTucdo/8qs0S3Makq0C+KYDiatEBJOjPH6GGTRxQLJ8 /Md15ch9l+hGfwS3zxtEZGhjk8koPT4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683731256; a=rsa-sha256; cv=none; b=w3edDdh6D62RhjCl/H1kzVqRPN2v0N7CDHG0FJVdjsggo/F2rjGVQMVgOSHirM6bltNkku zYjeQRi9PyBbM6wrPy05CNxVrYcdo4JVTk3EnaKt+uPRDn1W8+rnGs/1dSLgkdSpVC+zlT p61GW0Ec2IbZ239biZZdZdLdHBOz1ss= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=VHa+zJPx; spf=pass (imf28.hostedemail.com: domain of edumazet@google.com designates 209.85.128.49 as permitted sender) smtp.mailfrom=edumazet@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-3f423c17bafso125215e9.0 for ; Wed, 10 May 2023 08:07:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683731255; x=1686323255; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=+TNYqcSs6PC6BWjv/cKNCEBTiGNlpvDG9qZWo/NwWgk=; b=VHa+zJPx3Vz6wytMa3SIhZzYfgU2/Niwh71tnoKAYIeWA2H5epFTFhHSQvFN43kk6B cFDZen9tkH7p0WZnnrOzwIENa38J8Y14PbDUEwOakn8Oj0yZBW9fs6jWD/rDzE/vB43V Qz1d1XAKURN9MomiGhS9L4fJncNHQsNIPFngC2R0ThLp3yRrL/r6pOGWoH0IgAa/XqMm CVNitkcf2R+0SqHeNhSZUmYJ8hLSdPSSfbH8jGOIcZpl6IWOPynuR1hReNmhPg5z0teg 5Ktw1dOsG4oFDuFWcd42P5fNifriJTDIeKvlkgfQQq0BmKMypIUJTRWWVWw1yJyVBdFh vP7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683731255; x=1686323255; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+TNYqcSs6PC6BWjv/cKNCEBTiGNlpvDG9qZWo/NwWgk=; b=KM/ZO++EBaC3Pu4Se1jSICMzeETjOW6RMDaD25zRWhqBsv5e6tDqSGnJIYVVjWNNRV XA5pY6rvBfC9DsYQCmEr5j8U7qnxwBJx9X4k0yuyz7Vt65OfgzNNR+Swcdit8HNcJDU4 XBlGM4NaALHdAXrA+eFKkiXZL/Gk8hBgv6o/BzE1vd95OCdn2mEmX/CkdQYmv33jnIib 8wJl4LhkFUIFk0vztPEccJuEHHAnyQBlwzFg09DlrQJo3z9qSL824mr2mt10kLxHOmVC tJ8eAIYmftA5Bc09ipX6h9CkJYZ0OFN8vQ1jX2JIZXLKsqqDiClE2NQs/9cj2zi0XSG0 9hQQ== X-Gm-Message-State: AC+VfDyV5qo6FHtYhdARCcF39zJpW8DvEZZmvmegIIERTLQwYx/MPfPl KMY72xr5+BFimlAPlhcx480UvOdIoQSXxNSQjQjvGQ== X-Google-Smtp-Source: ACHHUZ7ReeeiFKcAq8guOqqbvtXHzsH9pveN7COQFJ9sbuKpLeN4UdPfY8kTUiKY5rf8VminXW7vGdnkqTnUKbnBebE= X-Received: by 2002:a05:600c:4f42:b0:3f4:2594:118a with SMTP id m2-20020a05600c4f4200b003f42594118amr226785wmq.2.1683731254627; Wed, 10 May 2023 08:07:34 -0700 (PDT) MIME-Version: 1.0 References: <20230508020801.10702-1-cathy.zhang@intel.com> <20230508020801.10702-2-cathy.zhang@intel.com> <3887b08ac0e55e27a24d2f66afcfff1961ed9b13.camel@redhat.com> In-Reply-To: From: Eric Dumazet Date: Wed, 10 May 2023 17:07:22 +0200 Message-ID: Subject: Re: [PATCH net-next 1/2] net: Keep sk->sk_forward_alloc as a proper size To: "Zhang, Cathy" Cc: Shakeel Butt , Linux MM , Cgroups , Paolo Abeni , "davem@davemloft.net" , "kuba@kernel.org" , "Brandeburg, Jesse" , "Srinivas, Suresh" , "Chen, Tim C" , "You, Lizhen" , "eric.dumazet@gmail.com" , "netdev@vger.kernel.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 3unsrg4ga7s8cowx5dwzwa8iy8az4ok4 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: BE462C08B8 X-Rspam-User: X-HE-Tag: 1683731256-72759 X-HE-Meta: U2FsdGVkX1+IVy6HIF4qMaj9rAIHPY6gcanN15rcDXIzBj6Zx4Kki+C+CF3sESREhCNLXGn+i4n8IjXWbBfHNXOG3qwAxAC3W7NlJqoxANhbScOkcQvIwuWZWVdF3kR3yvQcmpW2JhvUgW1DLl/bV8ruA9XxrBo+f13SxlFVrDVlELgTR9AxOj43bMk84HaEspXpvCTyFj9/+wngEMm9wNxNVcez4Ete6QSfgWd0mgzN5M24fU/0aDCJLqIgG2j6mgWlFM8dshsT03yNdahny6vDCdgH4sudyrivgn3R46N3+y1TDVcwpN5SvTgJWaQBykcfLIrR+tsD5DLuQRKyuIwIY11iBguL3UOddS2/6UoOPwuzAWbMu7u3FSz2F536vgYmxXWKPR/fKe9SdImc7D4iL/hKjBB8JuSArSiZB4RNepVD+pSGukzx3l07Lqopdabyp6BLBYkc7D143qDLbGWdJWvgl+BjO3WOem2LasUqRoEdMmsLSXlWZUOL2XGDCPGLEkLHhGK/QI/oMJfU8xy1ydP6DTbDfuGDTg+KroqXN7EbRaYqBIPVUb4zTC8WuJErGPkEKmcrOKnBy2UA8pJiBJr6LWDbKGMXtPwcKTb/Er9gS5Nc6pbRl+pkbQ6XQs7FzjQnnGKWQyKg7yayWd+2N+38raxw5Q/oRH8spyS2UEUJG6PmYjhuhRfQdsXubclrCp12mFaBGfl8abCLwNLjA5YWcOJx6l2NJmmihJTBbcI19BmNbXgKnvnCBeA6ZlRInPLiSl+G8k/uZ1aOpyuvrtnNoEnMxSOk0mpARmT86cGKo2b1Ojo6ZPO/IkL0jzREXQoHiz1ouUEVLanP3SBgOoxY99tDSvBA6xjdg/2+a20DJsyLihsSmfft38aZW+Lrz7AGZRDIrYmUji3x+NJh0iNhf6ZowFhaUyeIJl04hq1N0o3P/ZOFePbBExG+5YNVbidh3sprIvE5I4i bG/FbrTy 33CxF9TGZkWov5yjy6c5LpylkfkFQ/YC8LdyeS26BLmqu+lvr5Zo/iApTqdRMSdRpvcXp0FyWqP2Ph6lI2rw9pLUYreD7TMVKpV1gBsNuZVpiuqnJlj4yOZcZ3iV8CbGGtQiK0MDIhJiCSm0dA6HR2TPgmNqdA6ajo5Lm+09IRat6hnuo3ZdCJrXlmzeJ0c43sRpMs8j02HYPm19bM8oacj6mp/6BHbGtWOjuEUxRT18qbxtfiG8w3NeC++M91+MiGnEjBZj9XWs6F4Sk2Tw64rH58bRTXpocstdAUG6cDmCnY1IHsN46W5RPHpJCzcxaR9uryXnFk0ciCKALoAEJ5Fmwka64I4SMtY4RL9dGtVya4vTAhKBmPCxlm2UK/afAKr55O7u+ea2m/p8NyH+IlTecgPkKmcgFIQDDxQfCb1ej1Y8Hfz//b2TD2DvD2fifL2VROpoN5sepEnNzZeKEncpiisM5e6E7i12ExKdU+U0sIffImVxKZY/LeXdDqE+F+bOmkhvdYhn3BWLRkh4katLTw1GgOxpDCAO4JnAA6nmCLfdDjRVOfDX5O8stt3z99wFTwCoPpCnQyiCteH8lMIdf41fENR5QuvnTfVR8VhL/Jg5jECkn7/mu509QzHfwyz51wvuUrbQ2+UnZ8/iDx4OC7A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, May 10, 2023 at 3:54=E2=80=AFPM Zhang, Cathy wrote: > > > > > -----Original Message----- > > From: Eric Dumazet > > Sent: Wednesday, May 10, 2023 7:25 PM > > To: Zhang, Cathy > > Cc: Shakeel Butt ; Linux MM ; > > Cgroups ; Paolo Abeni ; > > davem@davemloft.net; kuba@kernel.org; Brandeburg, Jesse > > ; Srinivas, Suresh > > ; Chen, Tim C ; You, > > Lizhen ; eric.dumazet@gmail.com; > > netdev@vger.kernel.org > > Subject: Re: [PATCH net-next 1/2] net: Keep sk->sk_forward_alloc as a p= roper > > size > > > > On Wed, May 10, 2023 at 1:11=E2=80=AFPM Zhang, Cathy > > wrote: > > > > > > Hi Shakeel, Eric and all, > > > > > > How about adding memory pressure checking in sk_mem_uncharge() to > > > decide if keep part of memory or not, which can help avoid the issue > > > you fixed and the problem we find on the system with more CPUs. > > > > > > The code draft is like this: > > > > > > static inline void sk_mem_uncharge(struct sock *sk, int size) { > > > int reclaimable; > > > int reclaim_threshold =3D SK_RECLAIM_THRESHOLD; > > > > > > if (!sk_has_account(sk)) > > > return; > > > sk->sk_forward_alloc +=3D size; > > > > > > if (mem_cgroup_sockets_enabled && sk->sk_memcg && > > > mem_cgroup_under_socket_pressure(sk->sk_memcg)) { > > > sk_mem_reclaim(sk); > > > return; > > > } > > > > > > reclaimable =3D sk->sk_forward_alloc - > > > sk_unused_reserved_mem(sk); > > > > > > if (reclaimable > reclaim_threshold) { > > > reclaimable -=3D reclaim_threshold; > > > __sk_mem_reclaim(sk, reclaimable); > > > } > > > } > > > > > > I've run a test with the new code, the result looks good, it does not > > > introduce latency, RPS is the same. > > > > > > > It will not work for sockets that are idle, after a burst. > > If we restore per socket caches, we will need a shrinker. > > Trust me, we do not want that kind of big hammer, crushing latencies. > > > > Have you tried to increase batch sizes ? > > I jus picked up 256 and 1024 for a try, but no help, the overhead still e= xists. This makes no sense at all. I suspect a plain bug in mm/memcontrol.c I will let mm experts work on this. > > > > > Any kind of cache (even per-cpu) might need some adjustment when core > > count or expected traffic is increasing. > > This was somehow hinted in > > commit 1813e51eece0ad6f4aacaeb738e7cced46feb470 > > Author: Shakeel Butt > > Date: Thu Aug 25 00:05:06 2022 +0000 > > > > memcg: increase MEMCG_CHARGE_BATCH to 64 > > > > > > > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h in= dex > > 222d7370134c73e59fdbdf598ed8d66897dbbf1d..0418229d30c25d114132a1e > > d46ac01358cf21424 > > 100644 > > --- a/include/linux/memcontrol.h > > +++ b/include/linux/memcontrol.h > > @@ -334,7 +334,7 @@ struct mem_cgroup { > > * TODO: maybe necessary to use big numbers in big irons or dynamic ba= sed > > of the > > * workload. > > */ > > -#define MEMCG_CHARGE_BATCH 64U > > +#define MEMCG_CHARGE_BATCH 128U > > > > extern struct mem_cgroup *root_mem_cgroup; > > > > diff --git a/include/net/sock.h b/include/net/sock.h index > > 656ea89f60ff90d600d16f40302000db64057c64..82f6a288be650f886b207e6a > > 5e62a1d5dda808b0 > > 100644 > > --- a/include/net/sock.h > > +++ b/include/net/sock.h > > @@ -1433,8 +1433,8 @@ sk_memory_allocated(const struct sock *sk) > > return proto_memory_allocated(sk->sk_prot); > > } > > > > -/* 1 MB per cpu, in page units */ > > -#define SK_MEMORY_PCPU_RESERVE (1 << (20 - PAGE_SHIFT)) > > +/* 2 MB per cpu, in page units */ > > +#define SK_MEMORY_PCPU_RESERVE (1 << (21 - PAGE_SHIFT)) > > > > static inline void > > sk_memory_allocated_add(struct sock *sk, int amt) > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: Shakeel Butt > > > > Sent: Wednesday, May 10, 2023 12:10 AM > > > > To: Eric Dumazet ; Linux MM > > > mm@kvack.org>; Cgroups > > > > Cc: Zhang, Cathy ; Paolo Abeni > > > > ; davem@davemloft.net; kuba@kernel.org; > > > > Brandeburg, Jesse ; Srinivas, Suresh > > > > ; Chen, Tim C ; > > > > You, Lizhen ; eric.dumazet@gmail.com; > > > > netdev@vger.kernel.org > > > > Subject: Re: [PATCH net-next 1/2] net: Keep sk->sk_forward_alloc as > > > > a proper size > > > > > > > > +linux-mm & cgroup > > > > > > > > Thread: https://lore.kernel.org/all/20230508020801.10702-1- > > > > cathy.zhang@intel.com/ > > > > > > > > On Tue, May 9, 2023 at 8:43=E2=80=AFAM Eric Dumazet > > > > wrote: > > > > > > > > > [...] > > > > > Some mm experts should chime in, this is not a networking issue. > > > > > > > > Most of the MM folks are busy in LSFMM this week. I will take a loo= k > > > > at this soon.