From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81BD8C48BC1 for ; Wed, 14 Feb 2024 17:24:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EDEDD6B00B0; Wed, 14 Feb 2024 12:24:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E8BFE6B00B1; Wed, 14 Feb 2024 12:24:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D54296B00B2; Wed, 14 Feb 2024 12:24:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C5F0F6B00B0 for ; Wed, 14 Feb 2024 12:24:27 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 12F37C0205 for ; Wed, 14 Feb 2024 17:24:27 +0000 (UTC) X-FDA: 81791083374.15.4DF7746 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf09.hostedemail.com (Postfix) with ESMTP id 217EC140036 for ; Wed, 14 Feb 2024 17:24:24 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=YopayZLR; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf09.hostedemail.com: domain of andrii.nakryiko@gmail.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=andrii.nakryiko@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707931465; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=S6qAbhUtem/s1YgKm360zEs1FN7y6mvxL+lYin6s8rw=; b=fYxejBRI18L5Qom4NsqRlBotAnlfeYyEAog5cK7d0YfEaQjs65D+BebknJyPAyT1RcKegi qTik53FqVZeJ7XcWunBi+L/jf1j/8PHHcum+RmLF4niWNAglp2Te+DE4cWI5GT+YMuBaHL bWyJ7fcWp2aiaWBiEkHm9JY3d6cC4fQ= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=YopayZLR; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf09.hostedemail.com: domain of andrii.nakryiko@gmail.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=andrii.nakryiko@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707931465; a=rsa-sha256; cv=none; b=AlSrK0KCgzS7T+SdmaunSzdU+O8JUYRfwReQWBLAT5QEAA7uooTORfPzK/dUPrHqVGV2rF BaHTAbrAp8brmv31O09POdNVnNbbl1kKdXNA9BUagi5iD62O1Z5RbuwzRDzxpW983xDbni 3WD/tNc75JtAd/YCWl+TSap5PlRP4JM= Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-6e0507eb60cso9886b3a.3 for ; Wed, 14 Feb 2024 09:24:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707931464; x=1708536264; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=S6qAbhUtem/s1YgKm360zEs1FN7y6mvxL+lYin6s8rw=; b=YopayZLRTZSkZ7B3H+rWzhkBxpmYX5BLKierCMl8HT6OZwnOq3wH8QUBqH9txjudlt +XyMjJJ1YsMFuFCOKDNWrBgMmk6rxlRbhIGB5Mi867ouz5WIgFPZ4X+RNbjQrVIE+TK+ yWnA9zlL4yA4AyRGjj925WEGm0GbYQSyXZwGJKL7lSQ3G7M2gCtAax9+Iw/rc5ev7NOo joF6vu+egirt2fS7dk6BdQNOYUFBHvSS7+5wb8Ar51H/UeK8u+8z9HfJvkmSjvXZj2E9 pJRiRhKXk6hnqh9pC3QOTadhTGPhN3E06f83uhcI1Ml/UZdJo+UfEMYd7ySGCFGl3Bvv B52g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707931464; x=1708536264; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S6qAbhUtem/s1YgKm360zEs1FN7y6mvxL+lYin6s8rw=; b=HMX1spa418TVXsfQFKA/ODqAZeALbtYYQH3n46CPNtbNjgvEaqTHbWoEKWdBwxUDFT zhY5kJCXGKATJ1bFK5C6paxVdyvp+eOPq0KCbASyMidYxmry7rktkBf2LAtGdhTTp6JO mUrBUaG8xViJOTm/atPaQPKg9KS2da4Wc/rOETKVJS+MLWJWoLe4RNNyrZK1saoRNPNt jV6HLo8v6AeTAMYlYv5B+VnOVEXXCdh09Sj0jROJ+bhZ5utdb2oMMgryw9ap7T2zG6EF w5ySaS1BqYdsby9TonoOry3P+kZQi+asYbkr1B0PROjBRZ3OLW7ps8Ymg4jOJCEqc8u1 gLoA== X-Forwarded-Encrypted: i=1; AJvYcCVgEfouhT8l9WlcTARfVSHkxydxzlKoxAfsnKCY7uEjE8bwA9BSJwYIS6uzGmGVjXTCY2WX43Qh4/4BX5pu/Dkjdic= X-Gm-Message-State: AOJu0YwJuRdctdCLX/fN3BYXZnaySMrckbB4LcU8mwoK6nfPDnfoESsh kRmNYu7+1MlIbW46brfE+yCTDlGvTYaIfWfW7i0JhEPwFwCbs0SZfB7/of7uZuEbG8W1OyxjRGy tlSPveDkUf0O7FQGm72qyBDEwBns= X-Google-Smtp-Source: AGHT+IGwGGW7d+E2Ow0+w4Z2OiiBv3qgTxH7Wd3Ycop1PVBco9owHvyGcMfM/zsGNspZhGCVjiyWgo3IsSv+hbpu8CM= X-Received: by 2002:a05:6a20:e68c:b0:19e:b96e:e6b with SMTP id mz12-20020a056a20e68c00b0019eb96e0e6bmr4228071pzb.19.1707931463713; Wed, 14 Feb 2024 09:24:23 -0800 (PST) MIME-Version: 1.0 References: <20240209040608.98927-1-alexei.starovoitov@gmail.com> <20240209040608.98927-15-alexei.starovoitov@gmail.com> <7af0d2e0cc168eb8f57be0fe185d7fa9caf87824.camel@gmail.com> In-Reply-To: From: Andrii Nakryiko Date: Wed, 14 Feb 2024 09:24:11 -0800 Message-ID: Subject: Re: [PATCH v2 bpf-next 14/20] libbpf: Recognize __arena global varaibles. To: Alexei Starovoitov Cc: Eduard Zingerman , bpf , Daniel Borkmann , Andrii Nakryiko , Kumar Kartikeya Dwivedi , Tejun Heo , Barret Rhoden , Johannes Weiner , Lorenzo Stoakes , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , linux-mm , Kernel Team Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 217EC140036 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: szds88t6z5146nrg7wwdk79t86erww66 X-HE-Tag: 1707931464-818011 X-HE-Meta: U2FsdGVkX19dZfPlM++SKCuFSpVhksVMaPs4DLCP7D9emYS/3gjt8oiZ5N15pCbJIYb4j/JNeba9E96WzYPC8UH9cMkUKZVee4I4InWCKyqaVePZG6ugiee/hWPIuT8nP6vN7C65tiFUCmHEwKsDs0uG0he4kVsKRUJuhLot/c+QKPybFT1Orxb9a5igxayrxkADmf1LfEin4FtOTqz9HVHRoVpwtTbnoKNrJCMZXZPt02vhUwVsW5PcaPu2eMNDtrSnEIyAgZEYyyzMVDTNAhSnnkCzy+0h966TIIFVfmhSnVfa91ZMl3HeJRn24gVTfBob/sO9z0tfXZ8OYhtOhGzkG9CnagSlb8d6eRBFGNdUnuVNYQzRaho4et8fcSH73ArqCMjE9lUy1QAVZHlfJuxJw9rBvezfDPurK+s5D8mw6V12y22dsa7oueKl7KdbNEYvOCI9ndOajX6qstLeSs3hR53ao5wwDPqiZsLzAKYId01vwxzlO2zl3JzqBh0azGL0/9eXqyUCmwkTAOTVSLUXXc2qDppL2r2VgZwoOlf0bziyGJAlhthoWfX/HXhYklmDwwLXi+GXroB2JvNyORrcrsDYrp4EsDOiSXsxEVlq3oreVQOMWENcqrBjJt13gEM9bYpmZ8j662zlFOcfrMDmm21rPS/RAax1MxwlzAgHJSqwaqzbq+t6iCkMOZxqg84Q+tnmrE1BOfi9zPXJtmvTaAhzhEK1wWdrZbn13rJQl/et6kCSVYkKL903uJZyKDbEYg3bKeIZR44hAJH0nfdeDsrRiqfWni0ejUL0IRojN73Cn7U4qoUtCctx/dSOpqjkENKreX/E+Ff4J4YnGXqPKk0VUxatHiktaD7Y8W5fVSNHQiIa5uXx1k+A6/yDMPAJZpHdnGFt8qyvxl+6bBemh83jnDTkpDZl+ezOTWhzXssKUuboZMjP19NIhbOB2tmns4FnCyySIFw4JYS /ape0jqd y6VuJXmVPJL+fmAP+qERAcu74ZIV5UjshlL7UbBEBfrB3vu+Ahij8+ogKjDs8B9D+GvNjAZklKRTI53aWPidKqJHvmHrVeuChNAZMYB4XtOikpfTT+huUEKqMel1vJ0MQ4A4kFl126mD4Tw2mMal9AliF/sjs+WR3sLZAypBoj8DEQtZ6yoLIzW66dSIhhhO4yxgwcJKJkyvIwAI5fMbIhpB3yUkcsSf05YenQ5bQXZif9NlAliqeaJeygeKRsLsbKGm9Cx3E+Fj1sdU3WaC5GjNyTH9o9pXaKhdFELhNao/lj9ehI2qUsnGxWdm9PoKEB80LT1kR4iUZ+ZyQUPH/E5L1Bfir/yIxL6A6sIObdX2Y4IH8Wi1rMDY5vCconuW6Dv/hB1D5BRC6w5FlNaxlAYIVNZJm1ITwao3ZUH75TiWJAx1sZ0V375Of0iWL56UFBq4sjpE+27hNFl7VKczqUhJzvUBwXqZnnvfJjbwp79oIiXEfztf4vrgpNfKWStj2iKwAfn8nN4dVjzm/PmVsNS5D7g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Feb 13, 2024 at 5:24=E2=80=AFPM Alexei Starovoitov wrote: > > On Tue, Feb 13, 2024 at 4:09=E2=80=AFPM Andrii Nakryiko > wrote: > > > > On Tue, Feb 13, 2024 at 3:37=E2=80=AFPM Eduard Zingerman wrote: > > > > > > On Tue, 2024-02-13 at 15:17 -0800, Andrii Nakryiko wrote: > > > > > > [...] > > > > > > > > So, at first I thought that having two maps is a bit of a hack. > > > > > > > > yep, that was my instinct as well > > > > > > > > > However, after trying to make it work with only one map I don't r= eally > > > > > like that either :) > > > > > > > > Can you elaborate? see my reply to Alexei, I wonder how did you thi= nk > > > > about doing this? > > > > > > Relocations in the ELF file are against a new section: ".arena.1". > > > This works nicely with logic in bpf_program__record_reloc(). > > > If single map is used, we effectively need to track two indexes for > > > the map section: > > > - one used for relocations against map variables themselves > > > (named "generic map reference relocation" in the function code); > > > - one used for relocations against ".arena.1" > > > (named "global data map relocation" in the function code). > > > > > > This spooked me off: > > > - either bpf_object__init_internal_map() would have a specialized > > > branch for arenas, as with current approach; > > > - or bpf_program__record_reloc() would have a specialized branch for = arenas, > > > as with one map approach. > > > > Yes, relocations would know about .arena.1, but it's a pretty simple > > check in a few places. We basically have arena *definition* sec_idx > > (corresponding to SEC(".maps")) and arena *data* sec_idx. The latter > > is what is recorded for global variables in .arena.1. We can remember > > this arena data sec_idx in struct bpf_object once during ELF > > processing, and then just special case it internally in a few places. > > That was my first attempt and bpf_program__record_reloc() > became a mess. > Currently it does relo search either in internal maps > or in obj->efile.btf_maps_shndx. > Doing double search wasn't nice. > And further, such dual meaning of 'struct bpf_map' object messes > assumptions of bpf_object__shndx_is_maps, bpf_object__shndx_is_data > and the way libbpf treats map->libbpf_type everywhere. > > bpf_map__is_internal() cannot really say true or false > for such dual use map. > Then skeleton gen gets ugly. > Needs more public libbpf APIs to use in bpftool gen. > Just a mess. It might be easier for me to try implement it the way I see it than discuss it over emails. I'll give it a try today-tomorrow and get back to you. > > > The "fake" bpf_map for __arena_internal is user-visible and requires > > autocreate=3Dfalse tricks, etc. I feel like it's a worse tradeoff from = a > > user API perspective than a few extra ARENA-specific internal checks > > (which we already have a few anyways, ARENA is not completely > > transparent internally anyways). > > what do you mean 'user visible'? That __arena_internal (representing .area.1 data section) actually is separate from actual ARENA map (represented by variable in .maps section). And both have separate `struct bpf_map`, which you can look up by name or through iterating all maps of bpf_object. And that you can call getters/setters on __arena_internal, even though the only thing that actually makes sense there is bpf_map__initial_value(), which would just as much make sense on ARENA map itself. > I can add a filter to avoid generating a pointer for it in a skeleton. > Then it won't be any more visible than other bss/data fake maps. bss/data are not fake maps, they have corresponding BPF map (ARRAY) in the kernel. Which is different from __arena_internal. And even if we hide it from skeleton, it's still there in bpf_object, as I mentioned above. Let me try implementing what I have in mind and see how bad it is. > The 2nd fake arena returns true out of bpf_map__is_internal. > > The key comment in the patch: > /* bpf_object will contain two arena maps: > * LIBBPF_MAP_ARENA & BPF_MAP_TYPE_ARENA > * and > * LIBBPF_MAP_UNSPEC & BPF_MAP_TYPE_ARENA. > * The former map->arena will point to latter. > */ Yes, and I'd like to not have two arena maps because they are logically one= .