From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6E29C61DA4 for ; Mon, 6 Feb 2023 11:53:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F21816B0072; Mon, 6 Feb 2023 06:53:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EAA266B0073; Mon, 6 Feb 2023 06:53:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D24F86B0074; Mon, 6 Feb 2023 06:53:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BFADC6B0072 for ; Mon, 6 Feb 2023 06:53:24 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 8B7BDA0DAE for ; Mon, 6 Feb 2023 11:53:24 +0000 (UTC) X-FDA: 80436706728.05.B3DDE78 Received: from mail-qv1-f43.google.com (mail-qv1-f43.google.com [209.85.219.43]) by imf12.hostedemail.com (Postfix) with ESMTP id DC39A40011 for ; Mon, 6 Feb 2023 11:53:22 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=jTk8Ff7B; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf12.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.219.43 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675684402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WpOlzlgprXcBwzGCC3ZZLrYR97mq4YuOjOiXOKGzquY=; b=ZEP3hh4s6WES5/lKaGq519awNgHyE5sJq+lt8wCwJTMMTsgmULo5thChay4xcqgNABhLIn a0kODSvWaA7UHSncpoSSEEGIEbnd7Qb689hecyKUQ25CvvxYAs8ANztK8u3fizBJtMrmt3 oiya73m87pKWBpCc3Y+OLE6yxNoOUrI= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=jTk8Ff7B; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf12.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.219.43 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675684402; a=rsa-sha256; cv=none; b=RDjI45CyAGZK0RdA0pUVEy76V2tw2h6mSYp8LbyuakMRAg20oPsBgR0JL0OrH3MXfVU8kF FEsSDvCmIvLqUBGbS/PrjU+dTdLGA4KuWJRtozNNasIm3WcAZqRNhkF0BygMObb1c3eox4 Q4BRloz3xh+nAa5yUgXWsYQYNeSpl+w= Received: by mail-qv1-f43.google.com with SMTP id e6so4788404qvp.1 for ; Mon, 06 Feb 2023 03:53:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=WpOlzlgprXcBwzGCC3ZZLrYR97mq4YuOjOiXOKGzquY=; b=jTk8Ff7B/f0cUIwybEahiD0B53JzmMiPAESl+iqnoN2oJbQUsoCKwyFUdfntCHN1Tq RKlSnBIWljYqtCMddu1MfamKAz50vJAh4XxRBgY6BLzVwvXy0BZltifH5NDUAwrcl+1a KOXJ8QtvVSVAtVBxRkDYLajTGYd4JKe40twh0FJcVD+atIriki0tSUDbVVn3620UflNc hGAOXRWZCwOi6kStCYUyUk96yTb6EsU3cbXdAR/ccLyzZAGiGqJLjNJ8benii3BiIk80 14Y7N7Vx589pYhsJ6hE4wILwW3aMUXk62LftPQNX4cEjpSYyMabRCOKqX6CPoDnFe3i5 TH2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=WpOlzlgprXcBwzGCC3ZZLrYR97mq4YuOjOiXOKGzquY=; b=jCUkmJkN0YpUhXeuRvUJeIlLIdjMpNMeOHOlSDmgTtRGu1zeqjwD+8W8RqTaZzSto1 JvGnnCko/DbQvLCz/3FlVspzwdHCYwBqJS1x8uKCrQUAZ4oA+eEHy0ecQCL6g7QIxbeY lm5siBE1DYw/6RAYio84HijRsvD2y0du1Icf1vFSlA3flGodx+ihi+WOiV8wJCrU9Fi8 VM9q9v8M0Yzbob44Pecph02VbYqPS0BcTcc/tbhMHdeJxhTM8+a8/1oFir4UDxWP879S CFqSJulwMUfMfE4zRMmuuH5+LCsQ4Wbx+JXJ5UaULdFMLOElKZuIrd9bytP/LOjKn1X9 ORIw== X-Gm-Message-State: AO0yUKVvHuST6J0Ga8IYXtrisLxvgWbLKdqyTsoxaLDBrBGpWWixTpKD KJ3X3QLj7yhhIQS+YS9xZw3eHe8+rMVNJ4OHiMU= X-Google-Smtp-Source: AK7set/qqvbkHL+8abukp/KqPpIpfPhfODhJgn087gy4RZ1tmJw+8wgEKfnzxP7KHhivFpyDl9AZY5So/eb0/7ex3SA= X-Received: by 2002:a0c:f40f:0:b0:56a:eed2:704f with SMTP id h15-20020a0cf40f000000b0056aeed2704fmr540110qvl.5.1675684401966; Mon, 06 Feb 2023 03:53:21 -0800 (PST) MIME-Version: 1.0 References: <20230202014158.19616-1-laoar.shao@gmail.com> <20230202014158.19616-8-laoar.shao@gmail.com> In-Reply-To: From: Yafang Shao Date: Mon, 6 Feb 2023 19:52:45 +0800 Message-ID: Subject: Re: [PATCH bpf-next 7/7] bpf: hashtab memory usage To: Cong Wang Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, tj@kernel.org, dennis@kernel.org, cl@linux.com, akpm@linux-foundation.org, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, vbabka@suse.cz, urezki@gmail.com, linux-mm@kvack.org, bpf@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: DC39A40011 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: xejcwedbo5m6w63713te8bh861dkzef3 X-HE-Tag: 1675684402-558116 X-HE-Meta: U2FsdGVkX18bbnGcTeHZvEUf/M1uXoNy4TDIT27s1YiuUn5Y1wAwlOq13cVpiN0ZvnWVBk7gNV7IEhjnP50uUTy6+KvNdbvWGfeCk3n6E7/PAAZH6FW+CVLoCURy4hysCeUja8qLxBe/InaYnE3o6BxyZrDln9fQ0hEcjkCpsZ0ShqPddX6eWsVcHU27boLXDSAYCFbbc62lGOFK5j8K+Iwa5Agkjh1DKd37lUNUU9+SMF9fj5tKuzGZ9qOF5j0fxlGvzsXXkA9rgir5tmgnoDPrSUKj4p+QmnqNravdxDJsskWWL38FwbmpAks49iLONEAXTDRO2exOkXG7uDwLEuCWViy6lM0GOuntp38kS0myJ0racPsg27vVChoAfjK2bVMkRobLAdYdF4/1uEI5NX5jPrESdNvrPNCmPtkozwKN9Bvobmkl/JgAnBHnGWtbuCGKI/+q6z2NEwIhZ6Nr9ytbEz6Yt9wUBoedjmyO3OhJC/jsnt/lwt9RNt2v/KBk7ZvtoxaeodAp3F9v4NpIouWmf1Oh07LljdcBva4Bs2rNW7CNxbh8uv3xynAcJBchzYpldhJcvjZidQIML3nzCc7pYlW3r4lxrtuEIyUchaLOhNy1HtHY4bICtTKuK+ppIXui3py0X0Vyu3F3u8mWelh7iCV3VCfJImvwy2lkQZI5+4neilg0ep072il9aRGrY9xXfH/iHqz1w4G6qubUnJx60hAR4ierEwW+R1tqf/Dwbyy+mSmQ3MQjUkE1gwsj5Ykz+vaLSbjqeGobYniUpIULdW18BXzs6qbJimooBKD3tyes2PdAbDIkJzjJ/q+uNhPu4CP/eXUXulCL0fkMbVm8MCtrETUfsHXrCZnmHg/IVu3IVAeMIF9NUbbGXzxJiWBRaVjp8E8/A2uuT77Dg/WlQB9xfyBShFg0ctakMJw6/Ie+2dqPOQs+O/bwEo9nPBGaKMYLG/pM3EK7aXd X+iN+aLr 0YdFgktqgpSYfsz3oqexTzAignTmI+f2CO1lzJwliX5mSlPwFuk4OqSMtDKRvv776AJk4f7/7QE3ReXXfNnbe9brYOUONQhSRKIg4YAidIYIS7X47GiT6Kch1jLz0uXuWr+tXvGB5H8nF1ivQY3sQqE1txhN6RCVUqqVUBiCQE0k6arFvSwHWDOqWvuttl9tolFxJwVW1j+i6vHN+e9YX21CyFSXD4UnbpUni8FQZEXSWJbWyDO+znHSzQ2q1PllXYgpWCWCsPRczN8BNylR6Y8QZ+Pjrw4EfiJJs7VgSru7XmnmHO+vk54YCfipm5YrJjuZI3KCSasedCwFjppuwE9ISWTeeWrbaX3QeM17Mmt8T4p5V6P5SHAulbp+O05PxZ3i27SmwQnONIEx65yEETinR4w0Y/E/3OfJAjpjrtwF64HoZu/PWryVFUZBRhQN8zdzn9MAiry/Z9+YbjUypyKkAELaZxvaIggt3i9scF8Z2zqNg5ZnEF0XFLA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Feb 6, 2023 at 6:14 AM Cong Wang wrote: > > On Thu, Feb 02, 2023 at 01:41:58AM +0000, Yafang Shao wrote: > > Get htab memory usage from the htab pointers we have allocated. Some > > small pointers are ignored as their size are quite small compared with > > the total size. > > > > The result as follows, > > - before this change > > 1: hash name count_map flags 0x0 <<<< prealloc > > key 16B value 24B max_entries 1048576 memlock 41943040B > > 2: hash name count_map flags 0x1 <<<< non prealloc, fully set > > key 16B value 24B max_entries 1048576 memlock 41943040B > > 3: hash name count_map flags 0x1 <<<< non prealloc, non set > > key 16B value 24B max_entries 1048576 memlock 41943040B > > > > The memlock is always a fixed number whatever it is preallocated or > > not, and whatever the allocated elements number is. > > > > - after this change > > 1: hash name count_map flags 0x0 <<<< prealloc > > key 16B value 24B max_entries 1048576 memlock 109064464B > > 2: hash name count_map flags 0x1 <<<< non prealloc, fully set > > key 16B value 24B max_entries 1048576 memlock 117464320B > > 3: hash name count_map flags 0x1 <<<< non prealloc, non set > > key 16B value 24B max_entries 1048576 memlock 16797952B > > > > The memlock now is hashtab actually allocated. > > > > At worst, the difference can be 10x, for example, > > - before this change > > 4: hash name count_map flags 0x0 > > key 4B value 4B max_entries 1048576 memlock 8388608B > > > > - after this change > > 4: hash name count_map flags 0x0 > > key 4B value 4B max_entries 1048576 memlock 83898640B > > > > Signed-off-by: Yafang Shao > > --- > > kernel/bpf/hashtab.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++++++- > > What about other maps like regular array map? > I haven't finished the work to support all bpf maps. Most of the maps have fixed entries, so we can get the usage directly from an map pointer or get the element size and then calculate it, for example, - arraymap usage size = kvsize(array); - percpu arraymap usage size = kvsize(array); size += percpu_size(array->pptrs[0]) * array->map.max_entries; But there's special case like cgroup_storage, the max_entries of which is 0, for example, 122: cgroup_storage flags 0x0 key 16B value 8B max_entries 0 memlock 0B For this case, we should calculate the count of entries first. -- Regards Yafang