From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF6A2C77B7C for ; Fri, 12 May 2023 05:07:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4FD1C6B0075; Fri, 12 May 2023 01:07:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4AD346B0078; Fri, 12 May 2023 01:07:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 374026B007B; Fri, 12 May 2023 01:07:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 286146B0075 for ; Fri, 12 May 2023 01:07:00 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E93E6160E6F for ; Fri, 12 May 2023 05:06:59 +0000 (UTC) X-FDA: 80780418558.19.6C4EFBF Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf20.hostedemail.com (Postfix) with ESMTP id 2E5A81C000A for ; Fri, 12 May 2023 05:06:57 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=h2TzF1Sm; spf=pass (imf20.hostedemail.com: domain of 3ccldZAgKCKQWLEOIIPFKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--shakeelb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3ccldZAgKCKQWLEOIIPFKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--shakeelb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683868018; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XEaR3Dr2C8Lj1JCkXuxURcDiSzXLw0/h58n+0eTFpXk=; b=ZO7tR7aznWkxouAHbjJ1RtYfoNdczEgLC7zVduxSQjsB4BgmU5wo7k6/nR1D8S6halcUWM pH3qlHY/yAfF7awK8yKYesq+DX2Mb6niKUjq9ymcpb7f6XmJhMHAP3M8opuA+Ql/M3a/Ul uRA5hxvKcMxxntzD69lKYfsU4nAoD+U= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=h2TzF1Sm; spf=pass (imf20.hostedemail.com: domain of 3ccldZAgKCKQWLEOIIPFKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--shakeelb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3ccldZAgKCKQWLEOIIPFKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--shakeelb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683868018; a=rsa-sha256; cv=none; b=lIkSCp0y8Gdwl7yMFRctF53Thw/pKccAZtxnx2qFPBwXxPEmSUMHjLN7aOifMwaLrUOLSn LTrvATGI5ug7gRpr39roZns8kuN5l58drQulvNzEWQHOOAZ6Jzr5Zg+yFUYrUM+tv+46Sx pTTcpLiInTaoGphplWGZZjiZ97QFmOE= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-b9963a72fd9so20840366276.0 for ; Thu, 11 May 2023 22:06:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683868017; x=1686460017; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=XEaR3Dr2C8Lj1JCkXuxURcDiSzXLw0/h58n+0eTFpXk=; b=h2TzF1Sm4wgRUFzC402/r2JRCTOK95t1n6LIDMbVzltdZxZ1uErK2N2BwmqWhMnZ2x gDJIw0J+ieOpsXu8IU+C+9duObCvbbPPNDpYL988Bbh3KOzpvj3WlzGRiCq3UgH/fKdN jnwqcwBkrJpjqMeb1gRz3IRnnjZdtsx6i43myLWpj4bq54RB0Rl+g5e4CVUWGbG7PuB/ R7uGjC+ttKIstS0IoBo1yuXXZ2eT57tZ2egiGWEnJXomBv7Vlig+QVZOkQZLtcw28KJY LFbjerQqdZ+UkjE5fPUKDWc2gKvjdO5IUEWfiXlrRyC0zWLen+gMgJyctE6BqErCTj8Y hz+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683868017; x=1686460017; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XEaR3Dr2C8Lj1JCkXuxURcDiSzXLw0/h58n+0eTFpXk=; b=brM9TUqtf7aiinoS8jcIrC+r3qIZRLpcjo7OfJl1WxQgYIk1rkBPjDdIustZeh/M7X 8Um4VmVSkTLJnPh+g6GNpUwwbxzjL9OJ9cqXnyzRMQIm2aOR6h5Yj/8FhcrLerkZKeIM F1g6aTkOvQjmhHHgwFXAXh7QwVNz23PLcROFRvo5DtEJ4IxsHpmhTIgCnzttDEKUlq+S zrJfAxn/ykAUkQI78kzQyeGuPQKWAZ6UA/GGLIgWVOxhvkm+Z6G+CqRkIzekeKf5UOIE XmdsPEJly0hGUh8YZ0tBkkSdh460bE5mwB0uRwVOo/pEsVyKSx4w7tbnjxLzoVKzZaJm 4BdQ== X-Gm-Message-State: AC+VfDwD9RSuNLaC3itJSSUhIMZsvZ/oefuDHL4N7fqfx1G8Aoq2DAhD 4jhAMWssxYJAyB9krfn5gCcXeLnkIgYzBg== X-Google-Smtp-Source: ACHHUZ7DT3BCZCjNOk8XAjSIGAmryIlHycJZyc76A2YAWgJLUvsduVjlwCBoIYJxBL1Q7Z3AnsVenTZUavGcRg== X-Received: from shakeelb.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:262e]) (user=shakeelb job=sendgmr) by 2002:a25:6b11:0:b0:b9d:a7a1:cab8 with SMTP id g17-20020a256b11000000b00b9da7a1cab8mr14344595ybc.7.1683868017143; Thu, 11 May 2023 22:06:57 -0700 (PDT) Date: Fri, 12 May 2023 05:06:55 +0000 In-Reply-To: Mime-Version: 1.0 References: <20230511211338.oi4xwoueqmntsuna@google.com> Message-ID: <20230512050429.22du3gt6rrq6e37a@google.com> Subject: Re: [PATCH net-next 1/2] net: Keep sk->sk_forward_alloc as a proper size From: Shakeel Butt To: cathy.zhang@intel.com Cc: Eric Dumazet , Linux MM , Cgroups , Paolo Abeni , davem@davemloft.net, kuba@kernel.org, Brandeburg@google.com, Brandeburg Jesse , Srinivas Suresh , Chen Tim C , You Lizhen , eric.dumazet@gmail.com, netdev@vger.kernel.org Content-Type: text/plain; charset="us-ascii" X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 2E5A81C000A X-Stat-Signature: a7zcw9c4mwq4jtkfs3cua6imawa4yot1 X-HE-Tag: 1683868017-683579 X-HE-Meta: U2FsdGVkX18eFh9Zz4qy9ukCCP/bOQ/+GGNJaQXM/s0KM4w+cVxrJVEbhsup4CwF1wwBu3279iNL8mOWBTN5HMkTNrEw99fvrBNNIawnV/giENGH/AiVvJeUZKskY6ebHzRGxYmkrK91LhyLgFW8VkEORkUdUqe57G7hdB5Sy6Sn9cYaDOpWmpWK/5z8Qc0/NGTmX46QE2gWKDmBt1ptTD8dkoQymIm0iGK3pcdHSt3xHboFjd4VIauJSwxTVKP84sY8tpKBArzqVw6ivLNNffDPzLxlqaJasZPW5yj7zKdm8ioUPpcI5uM74ZP+a43xpiwLTb6u3+H2BxsTvA1RY9GR5gJsba06qQFo0B73V1h/Y0EWE5wUN5Eokg8JEOG+C6dOewI2OPeiFJc4HKjXivH5fUy13SU8grCCPK+XLVMshvDh2YmmL0aewZWxcTB0nDWctEAQTyg5SKDuj8/TMEDG7ArGKiWcATXSrYLn4SkzrOFsItoONwZzf//TpagdnltrMVbqmIKfg0M8cjwUFdE1YckLq8wSbCHTDTQUlq7t9/BPUWCk1JKtNnUqWwQnyfcMlHpZq+timYdbBdb9gBnbj0OIsOV3KaaLrXnEu/wTyw19brvrcc/O/buTb85Jbp0OLb0pcxIRN4c0qarqrB7+H6S1MExOkTp7lVBH12ooNPR8z6cCO2c3RhUusQ9hYj691HRyY5v0takxpqnDRR/uH6ay+WBABVKSMx5df5lsTbhWdqAf9io40qqfzhUsrm052UztpFKwpXF+l4Cgacs7VhaUBOR9b0P+hgwm94RP/rZqrPx3JW2kn8Wd/wMJe2XMYzXrl2tT5pPK4vO9Wvu3Q62ppIdHsKrr2sMfyogVByTXAiVaLFigCB2QmYfE207aTIvt5aFF0g+P8V1ZiGBwujzi1soeuCA0kwJHlM6mnX1NsOp7RkEi7NDPAdZGbOlHaaauVhSKiMTDG/d V0tNGu2G Brkfq9/wU9g9GbMgDDs1096iQRd2vHkWZR6jLLvHXAkBw7nHMOsJKkcki8nSMH92G+h5GLu7RpdtEMlalvucwr8QO52aVWD02K09deWZL7Z86TNUXz64IHZiwnbIgowisMtqOEpjJjLn6RCppmGbW1f3TICAwHiBOvwX8nOOtmtKxbhhsYffgrB3/I4kRlSX+54tPyjVyQ3FZ5w4zHzYEXSd6KGoY+UFiKLXLrQqLa+QtTERw5tnbpqM7s3Cm1HgmmnxKQ3xnoSYVPELafUVXC6RF/86s0t205RWbN7Ja/m0RcGSCX/CplHtnKXrmRrk590GYcolbrH/1Xyfb+YgL1LRgvgzkGt8ipprBfEWQmlpkyx/49kRL0lZY1ZsXdo1XmyYm1Mt5ljHT0o3Vimlr3U+jRgUdABui3BhLQ4oLR56WE9672g0adWTeWjXHRv6fXXgEKvAf1nekRpM9PVBs2fXfvCEUdvnGFmuXTyR3M+92OKtbPTNCR9FsS8l7fSODYTV0pnkKC9miu+w/X19qyj4XNuWzYxG2P4lAf0qsPkB6pVxwJ8FY//MC+arpDtrEJE4anUZDrjxeqzn4d407XVNELN6nBw47LbMQh6Zvnr7ymxx2QKVpAtw3HaI62wKwlkNNlBzuALv6LMYf9PsiuX3Q/7jL/tw9Sktn07eOUuc7WdT7Do62dAegYB9UwlKODHFDBadZmAPO5Dvrrrg2Q2YDbw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, May 12, 2023 at 03:23:45AM +0000, Zhang, Cathy wrote: > Remove the invalid mail addr added unintentionally. > Sorry that was my buggy script. [...] > > > > Hi Shakeel, > > > > Run with the temp change you provided, the output shows it comes to > > drain_stock_1(), Here is the call trace: > > > > 8.96% mc-worker [kernel.vmlinux] [k] page_counter_cancel > > | > > --8.95%--page_counter_cancel > > | > > --8.95%--page_counter_uncharge > > drain_stock_1 > > __refill_stock > > refill_stock > > | > > --8.88%--try_charge_memcg > > mem_cgroup_charge_skmem > > | > > --8.87%--__sk_mem_raise_allocated > > __sk_mem_schedule > > | > > |--5.37%--tcp_try_rmem_schedule > > | tcp_data_queue > > | tcp_rcv_established > > | tcp_v4_do_rcv > Thanks a lot. This tells us that one or both of following scenarios are happening: 1. In the softirq recv path, the kernel is processing packets from multiple memcgs. 2. The process running on the CPU belongs to memcg which is different from the memcgs whose packets are being received on that CPU. BTW have you seen this performance issue when you run the client and server on different machines? I am wondering if RFS would be good enough for such scenario and we only need to worry about the same machine case.