From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DDEBC48260 for ; Tue, 13 Feb 2024 23:16:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2F6148D0020; Tue, 13 Feb 2024 18:16:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2CD808D000E; Tue, 13 Feb 2024 18:16:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1958E8D0020; Tue, 13 Feb 2024 18:16:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 06D0F8D000E for ; Tue, 13 Feb 2024 18:16:06 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D2D83802E5 for ; Tue, 13 Feb 2024 23:16:05 +0000 (UTC) X-FDA: 81788340690.26.8A1303B Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf09.hostedemail.com (Postfix) with ESMTP id 3971414002C for ; Tue, 13 Feb 2024 23:16:04 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=KWqzPDiq; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf09.hostedemail.com: domain of andrii.nakryiko@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=andrii.nakryiko@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707866164; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=D8bdMenaLJTIatKTmEETJ8kQRfZNPeSS9xW0bhOuIBk=; b=0Q2qk1OkJ16wK4f4QKqUXBg2Onf1A0+wXoYqVm8PmwACPLIX6l+fxbPLWRJXMB0VsC0SAI FBs6OoB5EHJQH5U+6uGqmVdGpn67yMUEC/OSPhHZ3flJ4ijr9YWvHjSiMde03sBqTlaKZF 7tBV3DfdrCe6iKMuFla4gOGybGQ1CDg= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=KWqzPDiq; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf09.hostedemail.com: domain of andrii.nakryiko@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=andrii.nakryiko@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707866164; a=rsa-sha256; cv=none; b=WVNIC4yu8PWEYdJSBmpcqoBPDZC2AH3imFhoeq+pfqoEKQ/iSGUVX3vdLkXPeDg3ErRi9k dxmeJFn7E1SA2qu3x2g/HOyDQOmkF9f0ZIxiGicT4dcpeNrDOfLiTBZF1jgsgMkCWiHWSQ 2qLlshXs1DYa90I9UjQFLSZ4HXoRrRg= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1d8aadc624dso36872155ad.0 for ; Tue, 13 Feb 2024 15:16:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707866163; x=1708470963; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=D8bdMenaLJTIatKTmEETJ8kQRfZNPeSS9xW0bhOuIBk=; b=KWqzPDiqzKAxaFhDaj5XI4/pO9zocdB+V4Cxbn0/z2EI6sngdga3xH1GJiYuu5HkyY 3bun0buVTLKIuxEXREzKWINz/R0w9JKSJx7Yad+UQEil7NIXHRSKW3KflgyBxijP4eMy dYUsB6xV53LBH3M2hgWDO3AH6W2kNLhW5wT8BWPCM2en02u2JlE1nyOT+c+wkMU424FC 9oIOUfmwxoCx0Q2U6RWtGaiRenDx5gUHvugbUmhKm8nUynZKyXVIuxuJQXbcUfArRTKp n/lBsT2wnEQ/O2oJz0x6iv7rteF59LTcUGiwOPq+UQzIo2b14Injxh+QbTukX6CBwKc5 j8Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707866163; x=1708470963; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D8bdMenaLJTIatKTmEETJ8kQRfZNPeSS9xW0bhOuIBk=; b=r8BAbePahkXlIL+sPRol8tGoGKWdmiqL5F0FCp/NREewnvb8STEL/9kJoP0tf4sM5T SfDZiJ3plUsSgSZuKK5yqjPFtmv3td33NlMBlM2BRByolV7omztjkr9mAkie/eKG0PFQ hfGWM0gzYPIXo46787p7MERxJE+s3P1mPgecLa1No8xfavFIvS3c9qUIVG+jSJU0mywk NacbdYg50Pf2moVw2HesxxsGK+uVe+oBrnPuO+2K6ZosewqvguS6KkzvgkybX9HxjlXE jZmbXFKsF4/nWVElzSPtwre7byFnlyfwq5/Kd/85T6O7Paf5aJqrnxgC/3bcd1rvmfS5 JwAg== X-Forwarded-Encrypted: i=1; AJvYcCUklfXjxIBgoKve6DqeGlQDy25eaQ1XNRDuebsdWW9Pt3EzSHEUWL2bsF9vDbJ8CPqe6yS0E/nN9sPYCCFlX8RFn5Y= X-Gm-Message-State: AOJu0Yy7dmVtiCkURIff/MU+N0PNe+ObOSW39MXRAW41ADzsgS0Iboek IdUlyEsNFtJ9DF0V79CRzL98qDjGk0GEOTJ8K/l7e/yDel3eTCFHbLq0nVnD814TSZAMICYV6YT f5ViCwbXoe9/Fh3psyT6S3mcpEmo= X-Google-Smtp-Source: AGHT+IEXYUiNeFFzRPbr4yj3nY290hUu3Wi26SeGJQowsI5gRz0aZwKSyg90d50LHpSjNnJbppPSCKNYoK2PHXKCSYI= X-Received: by 2002:a17:90a:cc01:b0:298:d6b5:7fb0 with SMTP id b1-20020a17090acc0100b00298d6b57fb0mr4815pju.16.1707866162997; Tue, 13 Feb 2024 15:16:02 -0800 (PST) MIME-Version: 1.0 References: <20240209040608.98927-1-alexei.starovoitov@gmail.com> <20240209040608.98927-15-alexei.starovoitov@gmail.com> In-Reply-To: <20240209040608.98927-15-alexei.starovoitov@gmail.com> From: Andrii Nakryiko Date: Tue, 13 Feb 2024 15:15:51 -0800 Message-ID: Subject: Re: [PATCH v2 bpf-next 14/20] libbpf: Recognize __arena global varaibles. To: Alexei Starovoitov Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, memxor@gmail.com, eddyz87@gmail.com, tj@kernel.org, brho@google.com, hannes@cmpxchg.org, lstoakes@gmail.com, akpm@linux-foundation.org, urezki@gmail.com, hch@infradead.org, linux-mm@kvack.org, kernel-team@fb.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 3971414002C X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 1jt4iy889qjfy34bj86ebnmdot8djk68 X-HE-Tag: 1707866164-660801 X-HE-Meta: U2FsdGVkX1/PeS+DJdB8aDIw2QciAMYRDY71EwKr2wypXDo9Mpp2jypC4djD6Uq253YkKKTrzP2S7ALOWlxn7785xKlg+OU7udQFyhYryLwsPLYBvVjEwiKIYwSEmhyLtMyBaq0HyIs7W8BDMnHMLFs4C2fPZjQBpvsv9sXBC3RPEc4y+vpLQ5+zDJ3767TWLsSnF0ye6AxCBIX/HhNYmnhdfL2tiMzeWByZz23oCufFUbFtudJ3ffcCTltM7at4N8r7vq4Rh+e0s+FSHIdJXRfE1A6hd6UJ5loaiGFzQ58QDRh0riyFNPin8hQFWTCWklarF3pRi5tmQehi93cti0kDtW09PGQNHW+fWd6CAMg1X2v6WDdyxrvj/xMq7/fWhwA1+wlzrOLNWFeMEIJvdu0UM0vtdQ+/TkC5RN4WyrkIsmslm18qiLqTqqJMa2L02HTT8GyQaZCKLvt0d1a5/3PH950+BOrXbQCW1ETdtvl439UOX3xgviEdXXllODSTloLkhT5TaSTd6scC28+ITErw3mtZq5Gbe3o6TCA+JZ0JP8m8r9zWrAuO69cZ8ZDpCE9oiu2Mh/1VquFqc8Ef9Ij8yPDKFmCD78hsuo6qZ4BbntEeREJQKneqSFGufQKcXqcjGLGDLR+oXl2mp+pSmzbiBslp+mSbHaAH5Knb69wBXKPrHljzNB07F4hw6AWBQmoqu0v5WlQPrk0UmeWtUWE8JY9eDlLK7qALto4Q5ougcZrvtaI0HZZOj7jNgLRpdbdVU1s0OS0q1OkS50ERW0AKeEGlMwCaL0oxE3JPYDagRMVnNKohW5R5w4lhL7UVOqC+pJea/QzCg7Anfb7ufSmdMnk1mcjGyAl98Nr6jl8d+9bYxY/X5HmZi/asZ0gE5y43i/fqiz5EKCysEFIc6sIr/eA4irgl7swoZrnM+E73JevQ5WDnMP+8Vqzd+Wz8ENanf37qrTdO4dv1UUC tb6r4880 aFlz9gwGWDGZb7EYDKmBbFjD8Oc4CM1jxMuXEesuzKx2cyNJfriZ8D19yJ9TkU/96LDQUKn09NkjhPyUDoRXrPtn9M9g3BRrGxtQsUdn1E6b0livYV9Sj6P/CbewaydXYonVpfULMocM73bHWpW8m0aTdR9WlxMAzI9K42uQZzixFjCXMMMjwZV39A73HRSSPyoV8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Feb 8, 2024 at 8:07=E2=80=AFPM Alexei Starovoitov wrote: > > From: Alexei Starovoitov > > LLVM automatically places __arena variables into ".arena.1" ELF section. > When libbpf sees such section it creates internal 'struct bpf_map' LIBBPF= _MAP_ARENA > that is connected to actual BPF_MAP_TYPE_ARENA 'struct bpf_map'. > They share the same kernel's side bpf map and single map_fd. > Both are emitted into skeleton. Real arena with the name given by bpf pro= gram > in SEC(".maps") and another with "__arena_internal" name. > All global variables from ".arena.1" section are accessible from user spa= ce > via skel->arena->name_of_var. This "real arena" vs "__arena_internal" seems like an unnecessary complication. When processing .arena.1 ELF section, we search for explicit BPF_MAP_TYPE_ARENA map, right? Great, at that point, we can use that map and it's map->mmapable pointer (we mmap() anonymous memory to hold initial values of global variables). We copy init values into map->mmapable on actual arena struct bpf_map. Then during map creation (during load) we do a new mmap(map_fd), taking into account map_extra. Then memcpy() from the original anonymous mmap into this arena-linked mmap. Then discard the original mmap. This way we don't have fake maps anymore, we initialize actual map (we might need to just remember smaller init mmap_sz, which doesn't seem like a big complication). WDYT? BTW, I think bpf_object__load_skeleton() can re-assign skeleton's arena data pointer, so user accessing skel->arena->var before and after skeleton load will be getting correct pointer. > > For bss/data/rodata the skeleton/libbpf perform the following sequence: > 1. addr =3D mmap(MAP_ANONYMOUS) > 2. user space optionally modifies global vars > 3. map_fd =3D bpf_create_map() > 4. bpf_update_map_elem(map_fd, addr) // to store values into the kernel > 5. mmap(addr, MAP_FIXED, map_fd) > after step 5 user spaces see the values it wrote at step 2 at the same ad= dresses > > arena doesn't support update_map_elem. Hence skeleton/libbpf do: > 1. addr =3D mmap(MAP_ANONYMOUS) > 2. user space optionally modifies global vars > 3. map_fd =3D bpf_create_map(MAP_TYPE_ARENA) > 4. real_addr =3D mmap(map->map_extra, MAP_SHARED | MAP_FIXED, map_fd) > 5. memcpy(real_addr, addr) // this will fault-in and allocate pages > 6. munmap(addr) > > At the end look and feel of global data vs __arena global data is the sam= e from bpf prog pov. > > Another complication is: > struct { > __uint(type, BPF_MAP_TYPE_ARENA); > } arena SEC(".maps"); > > int __arena foo; > int bar; > > ptr1 =3D &foo; // relocation against ".arena.1" section > ptr2 =3D &arena; // relocation against ".maps" section > ptr3 =3D &bar; // relocation against ".bss" section > > Fo the kernel ptr1 and ptr2 has point to the same arena's map_fd > while ptr3 points to a different global array's map_fd. > For the verifier: > ptr1->type =3D=3D unknown_scalar > ptr2->type =3D=3D const_ptr_to_map > ptr3->type =3D=3D ptr_to_map_value > > after the verifier and for JIT all 3 ptr-s are normal ld_imm64 insns. > > Signed-off-by: Alexei Starovoitov > --- > tools/bpf/bpftool/gen.c | 13 ++++- > tools/lib/bpf/libbpf.c | 102 +++++++++++++++++++++++++++++++++++----- > 2 files changed, 101 insertions(+), 14 deletions(-) > [...] > @@ -1718,10 +1722,34 @@ static int > bpf_object__init_internal_map(struct bpf_object *obj, enum libbpf_map_ty= pe type, > const char *real_name, int sec_idx, void *d= ata, size_t data_sz) > { > + const long page_sz =3D sysconf(_SC_PAGE_SIZE); > + struct bpf_map *map, *arena =3D NULL; > struct bpf_map_def *def; > - struct bpf_map *map; > size_t mmap_sz; > - int err; > + int err, i; > + > + if (type =3D=3D LIBBPF_MAP_ARENA) { > + for (i =3D 0; i < obj->nr_maps; i++) { > + map =3D &obj->maps[i]; > + if (map->def.type !=3D BPF_MAP_TYPE_ARENA) > + continue; > + arena =3D map; > + real_name =3D "__arena_internal"; > + mmap_sz =3D bpf_map_mmap_sz(map); > + if (roundup(data_sz, page_sz) > mmap_sz) { > + pr_warn("Declared arena map size %zd is t= oo small to hold" > + "global __arena variables of size= %zd\n", > + mmap_sz, data_sz); > + return -E2BIG; > + } > + break; > + } > + if (!arena) { > + pr_warn("To use global __arena variables the aren= a map should" > + "be declared explicitly in SEC(\".maps\")= \n"); > + return -ENOENT; > + } so basically here we found arena, let's arena->mmapable =3D mmap(MAP_ANONYMOUS) here, memcpy(arena->mmapable, data, data_sz) and exit early, not doing bpf_object__add_map()? > + } > > map =3D bpf_object__add_map(obj); > if (IS_ERR(map)) > @@ -1732,6 +1760,7 @@ bpf_object__init_internal_map(struct bpf_object *ob= j, enum libbpf_map_type type, > map->sec_offset =3D 0; > map->real_name =3D strdup(real_name); > map->name =3D internal_map_name(obj, real_name); > + map->arena =3D arena; > if (!map->real_name || !map->name) { > zfree(&map->real_name); > zfree(&map->name); [...] > @@ -13437,6 +13508,11 @@ int bpf_object__load_skeleton(struct bpf_object_= skeleton *s) > continue; > } > > + if (map->arena) { > + *mmaped =3D map->arena->mmaped; > + continue; > + } > + yep, I was going to suggest that we can fix up this pointer in load_skeleton, nice > if (map->def.map_flags & BPF_F_RDONLY_PROG) > prot =3D PROT_READ; > else > -- > 2.34.1 >