From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7C4BC05027 for ; Sat, 4 Feb 2023 02:15:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD21B6B0071; Fri, 3 Feb 2023 21:15:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D82486B0072; Fri, 3 Feb 2023 21:15:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C4B2D6B0073; Fri, 3 Feb 2023 21:15:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B5A006B0071 for ; Fri, 3 Feb 2023 21:15:58 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 8827714074C for ; Sat, 4 Feb 2023 02:15:58 +0000 (UTC) X-FDA: 80427993996.21.A39CDB9 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf14.hostedemail.com (Postfix) with ESMTP id B594E100016 for ; Sat, 4 Feb 2023 02:15:56 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="qYTOq6/l"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of john.fastabend@gmail.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=john.fastabend@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675476956; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2YS7B6O70LhbZ3Kdw/RBfmMHaegKvoCD2PlKa2IVQqU=; b=ow641kCTjr2QSCxbmUVN/Q3uZeAEx4X3cBUKdX7OVDzNCm5m6G8ent/QNmkSo3PEJCWW/+ R2G6BskegeCUc9Wsl8U6MDVY06renkAe7qB5AchK/OZdU6k4GfU/vUyzpr5YZTsPIqW5oI g0NkHfkFCwU5x2OsK4A4YmVA1ZTn8pw= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="qYTOq6/l"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of john.fastabend@gmail.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=john.fastabend@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675476956; a=rsa-sha256; cv=none; b=kXvd5w9dZ/HSvlE4Do8HRpKcen89VuGiFEt2enG8iZ0HMK6LcXcpQ52sw7oxuiuwK9WtJ8 xZd6teEeFplVZX1wP4Qjb+2iSNuByL4Hr1h11DQ7rty6IvbCvqS8Vf7/F7j9RyprBiOJKN 5PVqtZQiMLKW5hw+0VTzOWlrBepP6xE= Received: by mail-pl1-f172.google.com with SMTP id m2so7081360plg.4 for ; Fri, 03 Feb 2023 18:15:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:subject:references :in-reply-to:message-id:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=2YS7B6O70LhbZ3Kdw/RBfmMHaegKvoCD2PlKa2IVQqU=; b=qYTOq6/le9oPQ5cZxjM5k208A3lNIcnE88Ui8w2oxjhwPm31u27bSWiF596qVa4xju YXngzA06e3W2tWpHiu7P47PIwxg7bQg/kCCZd8dwMxf5LQsQjRerHDLlQ8r75UqyocjL zAvkzavYGBFyX31Th/2HNeBIWHrRpB6BJOKxx/HKBYU7PodKbv7+4oDqR4kcBny67kFe Z0J4Z9RdWNaj+3jwoauBU83bYC9FVYEsrusz9uTvY491zB2UL5C7ltnoskLP6w3pcvUR RxUv2d62EuoJ16HZi4TttjCiQpftmhVSVJZtVJTYXQ3s9Xj3c2DiTD41ibBMUFuqwAgB rHAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:subject:references :in-reply-to:message-id:cc:to:from:date:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=2YS7B6O70LhbZ3Kdw/RBfmMHaegKvoCD2PlKa2IVQqU=; b=VGpWPhfZnEdW2Y0W9CGHdho0Sri72iROQT21F0NLd2xvyHbf99J1hGbu5xH4AweoBN 0aMW1TJwUx/st+tetXwXlw5yUoeJNfP0hIGnUU4JG/FxbKVrJvEtGRF8W7V2WV5pi79o qs5Hn5Ka3/LA6h3JLMGx1ae3HDIQDioYP46yEqNLKMF4VzUk6UodLFoFNIruEdlr/29O qnaKJjLAdEQoSHqANOKP0Dkk7L0hwTOU9y6eWry6av2Xt0bBU7ZY3LedekNY8PyfoV4q 0RdoMI3TU+9lE0DDvq3W0iMijF4CMOnf9YrR2exidgFuM4samaHD/LOfM/1nKlZtna1f W3oA== X-Gm-Message-State: AO0yUKVg6Z0i7NtRw8y6e3PPApYkQRCBgAWS2KLyN7PoXVO97cWnMZjG Qpnfd8E5oyZ6Lxcyi8btFw4= X-Google-Smtp-Source: AK7set9hK8dUsqOb7e3jfmbRblXFhWo9Dfbm1nGlu1Cd3l9cKMuAWF7ZDYR2dX0FffYAvhrXAntZig== X-Received: by 2002:a17:902:c402:b0:198:e63d:9a3c with SMTP id k2-20020a170902c40200b00198e63d9a3cmr3551939plk.44.1675476955500; Fri, 03 Feb 2023 18:15:55 -0800 (PST) Received: from localhost ([98.97.116.12]) by smtp.gmail.com with ESMTPSA id d19-20020a170902c19300b001947c22185bsm2252508pld.184.2023.02.03.18.15.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Feb 2023 18:15:54 -0800 (PST) Date: Fri, 03 Feb 2023 18:15:53 -0800 From: John Fastabend To: Yafang Shao , ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, tj@kernel.org, dennis@kernel.org, cl@linux.com, akpm@linux-foundation.org, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, vbabka@suse.cz, urezki@gmail.com Cc: linux-mm@kvack.org, bpf@vger.kernel.org, Yafang Shao Message-ID: <63ddbfd9ae610_6bb1520861@john.notmuch> In-Reply-To: <20230202014158.19616-1-laoar.shao@gmail.com> References: <20230202014158.19616-1-laoar.shao@gmail.com> Subject: RE: [PATCH bpf-next 0/7] bpf, mm: bpf memory usage Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: B594E100016 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: 4xaehgzr1uczqr6gbykmmydxbpbdgs97 X-HE-Tag: 1675476956-99989 X-HE-Meta: U2FsdGVkX18MB9jiYN1w3wBfBqnmkau+1j/Ao9upnAk6nYgzKuX3xju9KP+SqJa9mDXRjeuZbED3Ea/u123MxMZYA2Rpi5NpdoXrzMNZDpGfe8YtGyc/pkEb1g8HxzyrpZk+98pM2wDz/RqD2BIvgF7SSlbGBpUHzgwlPiaQtO3vU6/4tQrQN/aR0T5yOX9XwWRI4ZFfIyquWB/+yyRrirZkVR8H87H+QUo3Y74hcl/sbY5JK+IPIZvnktdl5+zhMwz07rRc+VqP9Ls2SiZ7CXwivUZyYiQIrNzixVUPCJ44sfoYKQIIPgXCSBLJbg5dzs8vGRRY3J9uQsNrYsODpda9XHmHfJBHCmIGOC71gVb7gcP6uLU7HwgnC/41VGmoA34PDEr6L6qrZz/v3QJjdindcimNpj/UMLIeM7IO/O52QbCF29ON5Xb3CvyOrHsp3X8PwodXiNFq5KCDLG+4jYFnMyv/DV7PCh+1SmyAB90tpVq/eEzQ0JN8wi0AwdhxCKCIAgYtkEHA+JhBLn/lpBoO719BltuY5ZNVk56/DLp6TXWwPK+eDugk+08TiOm6hJXgiWBMOiVyymmqBbf1fNAMVAjkly79e8TlANPWG+Zwn+Y836DIZfVPYnQ/tbjGYy4DbD99XY3L/dqUbmkf8Bj+Z09xRFyz2o5NNhGnLVciWoC4KS7j9lOTISasYR5C0en5XUkO8NsiNCKBSiPDTpAKxtIwBFiZffln9Gz/ZflmiXqRyMIzNqyuGwHK5/NN2KSrtNLRbfgw/YJDBjuLrLjbGllKFD3BtD+NP2mT/uHjnZ6HDtYtD3iYOx1uwerjg8G7b/RaLguAzyJABVP5TZjYt+bY12FXlmEEIsWUJPWhzdmJQ378BEXd1wlejFtvH4h1RijtilfCXB8zLRdkFGeOerh3KtE4IYjCfvTnpwdmdDvdq3DvmtodOxdMINzNYSAnzgg+U036y/30EfG tZUZ1j76 YmYFVDhXmuK3v5aMTto2EvvBgG4++Xj+80CfXEGWI4soRV3vH3heyVHw9xe2d9Q5xWXOBI0tsUQtCJ47EopuebOp+LFBhkf5naeAdzPJE9MnMcf3PxNNfH+yhcr6Bm5G8V0GHNRlUlbHfJjrXpolsasPGpWmtS5ZoQyhMLFpfJrWTzHF9cP+OT3CKCp6daYHIyMjswnyLJ8yVTzxV6/4PL6MjsMjq2WTwXcuxAxT8jHA215n+lugLS37vbx2rkEqKBvAmhZXlBEBw8IuZ3W9UUx0Lrcw53lhyxyLfcoHOloWcZSlChdxX4R21BYLbp5mQpR2Mol43VHdi9WDp37Pv+dtXLx44+qasGCsyuk7SlQjrmjoi6g2DKvxCMJDuGd1I+uyKUG7X3KX3iWl4S+2n184zSmqSTGB+/zNx+rHzknXRl/QqytfckqdzG4e0hUbI/GwY24nwQCc5yWM2+E2Xxdi1gZjKNZcFpRw4B2JYXfaJRpaQoNkXIaoyFgtwoIqrTp8p3YVxBGwUiGpUOwIh1nVuLDk5DKDEb4rKhcRyBDO9p0RXVGdE1l17CdaD+YTL4mEKo+cZzdu+oIQWw8tK9DdpsQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Yafang Shao wrote: > Currently we can't get bpf memory usage reliably. bpftool now shows the > bpf memory footprint, which is difference with bpf memory usage. The > difference can be quite great between the footprint showed in bpftool > and the memory actually allocated by bpf in some cases, for example, > > - non-preallocated bpf map > The non-preallocated bpf map memory usage is dynamically changed. The > allocated elements count can be from 0 to the max entries. But the > memory footprint in bpftool only shows a fixed number. > - bpf metadata consumes more memory than bpf element > In some corner cases, the bpf metadata can consumes a lot more memory > than bpf element consumes. For example, it can happen when the element > size is quite small. Just following up slightly on previous comment. The metadata should be fixed and knowable correct? What I'm getting at is if this can be calculated directly instead of through a BPF helper and walking the entire map. > > We need a way to get the bpf memory usage especially there will be more > and more bpf programs running on the production environment and thus the > bpf memory usage is not trivial. In our environments we track map usage so we always know how many entries are in a map. I don't think we use this to calculate memory footprint at the moment, but just for map usage. Seems though once you have this calculating memory footprint can be done out of band because element and overheads costs are fixed. > > This patchset introduces a new map ops ->map_mem_usage to get the memory > usage. In this ops, the memory usage is got from the pointers which is > already allocated by a bpf map. To make the code simple, we igore some > small pointers as their size are quite small compared with the total > usage. > > In order to get the memory size from the pointers, some generic mm helpers > are introduced firstly, for example, percpu_size(), vsize() and kvsize(). > > This patchset only implements the bpf memory usage for hashtab. I will > extend it to other maps and bpf progs (bpf progs can dynamically allocate > memory via bpf_obj_new()) in the future. My preference would be to calculate this out of band. Walking a large map and doing it in a critical section to get the memory usage seems not optimal > > The detailed result can be found in patch #7. > > Patch #1~#4: Generic mm helpers > Patch #5 : Introduce new ops > Patch #6 : Helpers for bpf_mem_alloc > Patch #7 : hashtab memory usage > > Future works: > - extend it to other maps > - extend it to bpf prog > - per-container bpf memory usage > > Historical discussions, > - RFC PATCH v1 mm, bpf: Add BPF into /proc/meminfo > https://lwn.net/Articles/917647/ > - RFC PATCH v2 mm, bpf: Add BPF into /proc/meminfo > https://lwn.net/Articles/919848/ > > Yafang Shao (7): > mm: percpu: fix incorrect size in pcpu_obj_full_size() > mm: percpu: introduce percpu_size() > mm: vmalloc: introduce vsize() > mm: util: introduce kvsize() > bpf: add new map ops ->map_mem_usage > bpf: introduce bpf_mem_alloc_size() > bpf: hashtab memory usage > > include/linux/bpf.h | 2 ++ > include/linux/bpf_mem_alloc.h | 2 ++ > include/linux/percpu.h | 1 + > include/linux/slab.h | 1 + > include/linux/vmalloc.h | 1 + > kernel/bpf/hashtab.c | 80 ++++++++++++++++++++++++++++++++++++++++++- > kernel/bpf/memalloc.c | 70 +++++++++++++++++++++++++++++++++++++ > kernel/bpf/syscall.c | 18 ++++++---- > mm/percpu-internal.h | 4 ++- > mm/percpu.c | 35 +++++++++++++++++++ > mm/util.c | 15 ++++++++ > mm/vmalloc.c | 17 +++++++++ > 12 files changed, 237 insertions(+), 9 deletions(-) > > -- > 1.8.3.1 >