From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3663DC433DF for ; Tue, 13 Oct 2020 03:53:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8E38A20838 for ; Tue, 13 Oct 2020 03:53:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="Jvs6pDvj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E38A20838 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A8C53940007; Mon, 12 Oct 2020 23:53:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A13EE900002; Mon, 12 Oct 2020 23:53:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B352940007; Mon, 12 Oct 2020 23:53:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0051.hostedemail.com [216.40.44.51]) by kanga.kvack.org (Postfix) with ESMTP id 53DA2900002 for ; Mon, 12 Oct 2020 23:53:07 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DD4DC1EE6 for ; Tue, 13 Oct 2020 03:53:06 +0000 (UTC) X-FDA: 77365531572.19.foot95_0d13bca27200 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id C17591AD1B5 for ; Tue, 13 Oct 2020 03:53:06 +0000 (UTC) X-HE-Tag: foot95_0d13bca27200 X-Filterd-Recvd-Size: 7042 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Tue, 13 Oct 2020 03:53:05 +0000 (UTC) Received: by mail-pf1-f195.google.com with SMTP id c20so5224685pfr.8 for ; Mon, 12 Oct 2020 20:53:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=I976gwN/15KL+QNMsgGgqLZsKsiYJk1WAAkrJcvqpXg=; b=Jvs6pDvj0IsDRmKV94si7Hmy7rSC+vZBw6IORU+6GuiBz6nFVyhqIqcqJDs0PDrtGr nXgB5Gg3YnIhlpunv6ok32b0qmXBGoCwLNqj8YhtpNO34HbUEe0fj0rk5j4aFEsdLznp VoXOaX98Lakuk83ySfZj3iYVBGQSYWKOXyIXuOrEyRswmVxiaN0fpvQLOEbiBPlkB0f0 SaQzLf6nCGYW1cdweYZEMOz6t3FkN6D8ke5ashIIre24atnH4a15lepW6k2kSOlNz9zr SS6NCvPuDVbJ/UO8vSYODHhBG1Za37JNzNmmpoq6Rj8IhJ3hedl5GM6iDMNEd51QrfKm Hzeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=I976gwN/15KL+QNMsgGgqLZsKsiYJk1WAAkrJcvqpXg=; b=byImiWksdN0p2CpPBBbhyzy7bt27YLIi3qd9cVsc9JeFq3ZmsD5oGmyzw/OKtP7D+x cLeoDvaU7mr0Xfz2/Z+fWMCA9k8sZa2mfosKl5SM4NoDlpQ3K1Emvw5jyYDOTIrNBv7x m7+Rl2eERXcoYnaXZ99leomwLiWFXNRfi2dda0QukO6km4CJm9HSn6weTd3OBhlK+Dju j9yYqPmQV5Cp1SeAbTMX6M7iN955ANWkCnOEu2D48dVwCMEd6WxvHR5tZgAYUCewcUB5 Dzw739wR3LyqkgfqyHLRv5bQshugRWrGBm+itD9eNlv+k1Eboae+GbHsQje6o5eG+97n Nk2w== X-Gm-Message-State: AOAM531LL6kHlv4/vhMaHWfBgrXS29PmOQa1EHCRaO0UzTY60wypaXn1 o7A/sZdeN2IuOs5ax4u97sb8ORDSihtfDtCbIqxuLQ== X-Google-Smtp-Source: ABdhPJxTj1LpD8Fj5IklJNe6tlIaSL6h8M3LDjY1NdFgvnmFbpO9hutDnYHrS/qSHLk7UFCnLzxPbWYodXJxV9DZ0fk= X-Received: by 2002:a17:90a:4749:: with SMTP id y9mr10977409pjg.229.1602561184587; Mon, 12 Oct 2020 20:53:04 -0700 (PDT) MIME-Version: 1.0 References: <20201010103854.66746-1-songmuchun@bytedance.com> <9262ea44-fc3a-0b30-54dd-526e16df85d1@gmail.com> In-Reply-To: From: Muchun Song Date: Tue, 13 Oct 2020 11:52:28 +0800 Message-ID: Subject: Re: [External] Re: [PATCH] mm: proc: add Sock to /proc/meminfo To: Cong Wang Cc: Eric Dumazet , Eric Dumazet , Greg KH , rafael@kernel.org, "Michael S. Tsirkin" , Jason Wang , David Miller , Jakub Kicinski , Alexey Dobriyan , Andrew Morton , Alexey Kuznetsov , Hideaki YOSHIFUJI , Steffen Klassert , Herbert Xu , Shakeel Butt , Will Deacon , Michal Hocko , Roman Gushchin , Neil Brown , Mike Rapoport , Sami Tolvanen , "Kirill A. Shutemov" , Feng Tang , Paolo Abeni , Willem de Bruijn , Randy Dunlap , Florian Westphal , gustavoars@kernel.org, Pablo Neira Ayuso , Dexuan Cui , Jakub Sitnicki , Peter Zijlstra , Christian Brauner , "Eric W. Biederman" , Thomas Gleixner , dave@stgolabs.net, Michel Lespinasse , Jann Horn , chenqiwu@xiaomi.com, christophe.leroy@c-s.fr, Minchan Kim , Martin KaFai Lau , Alexei Starovoitov , Daniel Borkmann , Miaohe Lin , Kees Cook , LKML , virtualization@lists.linux-foundation.org, Linux Kernel Network Developers , linux-fsdevel , linux-mm Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Oct 13, 2020 at 6:12 AM Cong Wang wrote: > > On Mon, Oct 12, 2020 at 2:53 AM Muchun Song wrote: > > We are not complaining about TCP using too much memory, but how do > > we know that TCP uses a lot of memory. When I firstly face this problem, > > I do not know who uses the 25GB memory and it is not shown in the /proc/meminfo. > > If we can know the amount memory of the socket buffer via /proc/meminfo, we > > may not need to spend a lot of time troubleshooting this problem. Not everyone > > knows that a lot of memory may be used here. But I believe many people > > should know /proc/meminfo to confirm memory users. > > Well, I'd bet networking people know `ss -m` better than /proc/meminfo, I agree with you. But if someone(not networking people) faces the same problem. He may suspect that there is a memory leak or think that a certain driver allocates memory but has no statistics. He only saw the memory disappeared via /proc/meminfo. > generally speaking. > > The practice here is that if you want some networking-specific counters, > add it to where networking people know better, that is, `ss -m` or /proc/net/... > > Or maybe the problem you described is not specific to networking at all, > there must be some other places where pages are allocated but not charged. Yeah, it is not charged. The allocation path is as follows. This allocation consumes 25GB memory on our server. And it belongs to the network core. Thanks. __alloc_pages_nodemask+0x11d/0x290 skb_page_frag_refill+0x68/0xf0 sk_page_frag_refill+0x19/0x70 tcp_sendmsg_locked+0x2f4/0xd10 tcp_sendmsg+0x29/0xa0 sock_sendmsg+0x30/0x40 sock_write_iter+0x8f/0x100 __vfs_write+0x10b/0x190 vfs_write+0xb0/0x190 ksys_write+0x5a/0xd0 do_syscall_64+0x5d/0x110 entry_SYSCALL_64_after_hwframe+0x44/0xa9 > If so, adding a general mm counter in /proc/meminfo makes sense, but > it won't be specific to networking. > > Thanks. -- Yours, Muchun