From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 152B9C5AD49 for ; Thu, 29 May 2025 16:31:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8A1246B0083; Thu, 29 May 2025 12:31:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 878C96B0085; Thu, 29 May 2025 12:31:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7932C6B0088; Thu, 29 May 2025 12:31:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 5B7856B0083 for ; Thu, 29 May 2025 12:31:57 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id D58B9EA814 for ; Thu, 29 May 2025 16:31:56 +0000 (UTC) X-FDA: 83496487032.24.F72E084 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf17.hostedemail.com (Postfix) with ESMTP id 1FBA54000A for ; Thu, 29 May 2025 16:31:54 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=xy+ybMvm; spf=pass (imf17.hostedemail.com: domain of almasrymina@google.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=almasrymina@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748536315; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NVDGu5ERwU5ZgtkOKGr5oJl2eiCK7dgYvbFJ0GbeMic=; b=IFIeBghTVVY9swR99QEd1HB+pb67rMhhoSb74pctel9jEEQ04tCg8k/IB+1Oa7w1twrPuG d5qYeHNK6TQY3mqLCG7V5mmM19paWrTM9O+ogOHXW/QJhgSk/BwbMtYAdxC9UtYwQkUpER xtmHWEXDUOdnp2SEyQ0lKz/I4SgMdbk= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=xy+ybMvm; spf=pass (imf17.hostedemail.com: domain of almasrymina@google.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=almasrymina@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748536315; a=rsa-sha256; cv=none; b=DHm8X8YNtjUap4H/JE1DMtnCk7jYIOWGCWJS2C80uPRmlpOLJjCSpNN2N5kv5ffX2TNevG vItWdAiZRJVmsefq17X43c/r30O+WtWahq2lGIemO0s7SV+z8drxOC7kHjgktVQ1oj7xXX dmjB4ecv9owIDAdmPdqjTA/5mTpEBQw= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-231ba6da557so210325ad.1 for ; Thu, 29 May 2025 09:31:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1748536314; x=1749141114; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=NVDGu5ERwU5ZgtkOKGr5oJl2eiCK7dgYvbFJ0GbeMic=; b=xy+ybMvmQ3PVMsU9CFVI8oMnzkDMxTogJiS0LRxYRmstnu37D3fYECPB9GymKq0TTW eSDA7ORWacGCbLhSJoL6s695vUSLPWtM2nk7DfVar2hQAZNK8opnmwvBqn9hJ1D5kfUZ C1EEm+oNWOy+uf7GenQN8lvH2yX7IGUp3fOlhNxgBVKaeEVDY/heqNQhp6hRBX5jPHtI PvKYZzmhxa6J4960nyVjbB7xIHXLUl6/fhoK0OPPT4R2HV6Iao40/4WShPcXxf/GPruy uBnGsIv/3dgG/l12pKJnT/92cDNSOrbdL6PIOK6ut7rz+pryUS29A5orhTyiPD8dWmNQ MPMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748536314; x=1749141114; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NVDGu5ERwU5ZgtkOKGr5oJl2eiCK7dgYvbFJ0GbeMic=; b=GWsjGvDs/exn9wNvDDAHO76nxJ47fkfYCZKIaTCfQKM7TVj4pGYuzT7a49aGyism4k ZOSanTZGc/rowMcvglfXUAVcqgPwyJBMM+Htggg/dvyfONAqNQASyZ+TcBZMLS4WPEfe hlnfg9Xxjp54oOGl3iO5VISvpVeGNnyioOnNyD5rjNoCei9cl2W1VedgpQIb71uv0F63 QxghSjpnXgz3C7yMmHCWLaC5yAzAFCoBMVFM88AzuIMfWiRz+2DgScV93tZdUiL0Dsfq 4WycbStnDp06Y+8nBd6hmMU74Q/zqyW49X+B8lgLOiJ4mAIdn+A/RPB0vuIW1Tlxp/Y3 s3XA== X-Forwarded-Encrypted: i=1; AJvYcCVs3w3mgE2hr6G/VL4tqm/G+OlN4cBRfcu82upqpJgJ1udT0zeYdnK+UuyVdVYpE8Fzcak+HwnfhA==@kvack.org X-Gm-Message-State: AOJu0Yyk3u772qS9gbwBQcMf24hYhrlaDlfFwhuEuOEd9em7PooSzYa4 o5gkJKt+ZYhgz7JH3LH33/Xibu2gFuM0j5VYdh1hVPAV6E0qqUyqwsPtI7MYchu/VK91m9BY/ME m5ZFphoA55srBEgHBCyNEU02im3kFi1b4povKkd05 X-Gm-Gg: ASbGncsxCkg1tYrW95bxhTCijLWUlqvltnTzGcLzSE4RDJxxIFNtT4B/Blvg5nmH2OF kWL6BaoctXs34jbrX1c5Pg0Z2KClQRcpgSlu5xOzl5M6AtUrvNzN5EiyBqMrN5ufZgw1gIv06f3 Rwhw1hJS+Vg0hbOhlce1/FrK2EJ7Jy9Ipus65cc6H74psk X-Google-Smtp-Source: AGHT+IFb5ehY2LBDui4OEWlKyCcK0nJ/FtdW4knYpW4p37PTcpUggU1f3J8Fa109z1I2zx4rCQYS2wIX4+Tp0LJygZ8= X-Received: by 2002:a17:902:e84e:b0:234:7653:aff4 with SMTP id d9443c01a7336-23506a95855mr3173125ad.23.1748536313565; Thu, 29 May 2025 09:31:53 -0700 (PDT) MIME-Version: 1.0 References: <20250529031047.7587-1-byungchul@sk.com> <20250529031047.7587-2-byungchul@sk.com> In-Reply-To: <20250529031047.7587-2-byungchul@sk.com> From: Mina Almasry Date: Thu, 29 May 2025 09:31:40 -0700 X-Gm-Features: AX0GCFu6I95UuYW0dpMwLmIgYy9IdXbwL1i3YzcJ0K7HMx8drU0_s9IWKIaa7gM Message-ID: Subject: Re: [RFC v3 01/18] netmem: introduce struct netmem_desc mirroring struct page To: Byungchul Park Cc: willy@infradead.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, asml.silence@gmail.com, toke@redhat.com, tariqt@nvidia.com, edumazet@google.com, pabeni@redhat.com, saeedm@nvidia.com, leon@kernel.org, ast@kernel.org, daniel@iogearbox.net, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, horms@kernel.org, linux-rdma@vger.kernel.org, bpf@vger.kernel.org, vishal.moola@gmail.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 1FBA54000A X-Stat-Signature: wykybmiynkn76bk3jr553oz6botznbp5 X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1748536314-376332 X-HE-Meta: U2FsdGVkX19KPu5SOCKnueLylQI7oYogUYVqetPF4kLpwOmXhk8F8CawZiT1YtKl/7kUQy68Ry9pcKu32Chdn3wuEtS8sHic7iRU8UTFsjGsM5z+mm95/ctqGZubam2cPDJw32uun7eaMRQcXh/ydjkVoh7Dx+dQXm3aJxnyyHXdPHlmnPDtWDKyBkLQOvXh8I7Rpn2A7kKEvoTqKX1cjcxtu6XOKohkfQ6xotVk23OB/4Jl+tjW2d8XouQpLhZvroJRiVwvuQfqALDMx6wjnio7zZaORSS3PyzOAm3p/gx/+Wj8TfYU4viysjvja2+bQfOjPZW7s/Vo4vKDs7G0SWi+58VW7GLWlWKlLNafqjTK45HXAcTFLAI6uTnBMr4sgThoVYWcSGRxj6tHHH59UJEehk/FtA5l4dL2oGneivHPFXk+hAY0Om02y9pGdbXSZSNS1Gx1uAR++s7nGp9mnpUbqsySX2SX5J5R2o+BXhfazLNeB/EJwGCZIj7HdBjUwrt61W+K2K2zD6oCCCdRc8FaFJ60DFybliUlN5lbHM2qqy+J0A7cCXXYe4VtV4qs6zDw/Si3KFMESWVZA/QF4FgQJ94MUHXy5BEieOTVSCbKjLWJ7sGMu76HCuxP5LCbjN56afGBF1tOee+VGYWJKgoxYGB59m2YzcHIVg9+gSlpfXl8nYjvn3wLjTnt2blRgtp+fz291VhSXL7rBzp1CRyhYWKmg8oo+6AUsQRNif5a5uHRDivu3o/gaZny3iT4IfWgwS8OqwBeMwB1muS7Bu4FCo1/7Hs2GOXxyYPI0Lgc8K1doBgdVgE48qcoIBFZuqTp/EjlgfXa2nssUpXxo0725ASlgkZP9Aa1M+YbbsDPFO+LfD6mCaAKhxUl++pIC89oqtFfmdqxMG+JOk+hFafP24hcZGxvNPjSHNBI1bwP23ztKXAAf732R6ztFnMsSgmai/7FZPc+Qy7S00g 6ycD/oAh r5LSmG+KJnsZAyN0rqpHZbrM/Fc5eOOD2TDqAFNHDJlYokca2A42NuayPv9toPNyCqHA/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, May 28, 2025 at 8:11=E2=80=AFPM Byungchul Park w= rote: > > To simplify struct page, the page pool members of struct page should be > moved to other, allowing these members to be removed from struct page. > > Introduce a network memory descriptor to store the members, struct > netmem_desc and make it union'ed with the existing fields in struct > net_iov, allowing to organize the fields of struct net_iov. The final > look of struct net_iov should be like: > > struct net_iov { > struct netmem_desc; > net_field1; /* e.g. struct net_iov_area *owner; */ > net_field2; > ... > }; > > Signed-off-by: Byungchul Park This version looks much better from networking POV, and I think with small adjustments we can make it work I think. I'll leave Matthew to confirm if it's aligned with the memdesc plans. > --- > include/net/netmem.h | 101 +++++++++++++++++++++++++++++++++---------- > 1 file changed, 79 insertions(+), 22 deletions(-) > > diff --git a/include/net/netmem.h b/include/net/netmem.h > index 386164fb9c18..d52f86082271 100644 > --- a/include/net/netmem.h > +++ b/include/net/netmem.h > @@ -12,6 +12,47 @@ > #include > #include > > +/* These fields in struct page are used by the page_pool and net stack: > + * > + * struct { > + * unsigned long pp_magic; > + * struct page_pool *pp; > + * unsigned long _pp_mapping_pad; > + * unsigned long dma_addr; > + * atomic_long_t pp_ref_count; > + * }; > + * > + * We mirror the page_pool fields here so the page_pool can access these > + * fields without worrying whether the underlying fields belong to a > + * page or netmem_desc. > + */ > +struct netmem_desc { > + unsigned long _flags; > + unsigned long pp_magic; > + struct page_pool *pp; > + unsigned long _pp_mapping_pad; > + unsigned long dma_addr; > + atomic_long_t pp_ref_count; > +}; > + > +#define NETMEM_DESC_ASSERT_OFFSET(pg, desc) \ > + static_assert(offsetof(struct page, pg) =3D=3D \ > + offsetof(struct netmem_desc, desc)) > +NETMEM_DESC_ASSERT_OFFSET(flags, _flags); > +NETMEM_DESC_ASSERT_OFFSET(pp_magic, pp_magic); > +NETMEM_DESC_ASSERT_OFFSET(pp, pp); > +NETMEM_DESC_ASSERT_OFFSET(_pp_mapping_pad, _pp_mapping_pad); > +NETMEM_DESC_ASSERT_OFFSET(dma_addr, dma_addr); > +NETMEM_DESC_ASSERT_OFFSET(pp_ref_count, pp_ref_count); > +#undef NETMEM_DESC_ASSERT_OFFSET > + > +/* > + * Since struct netmem_desc uses the space in struct page, the size > + * should be checked, until struct netmem_desc has its own instance from > + * slab, to avoid conflicting with other members within struct page. > + */ > +static_assert(sizeof(struct netmem_desc) <=3D offsetof(struct page, _ref= count)); > + > /* net_iov */ > > DECLARE_STATIC_KEY_FALSE(page_pool_mem_providers); > @@ -31,12 +72,33 @@ enum net_iov_type { > }; > > struct net_iov { > - enum net_iov_type type; > - unsigned long pp_magic; > - struct page_pool *pp; > - struct net_iov_area *owner; > - unsigned long dma_addr; > - atomic_long_t pp_ref_count; > + union { > + struct netmem_desc desc; > + > + /* XXX: The following part should be removed once all > + * the references to them are converted so as to be > + * accessed via netmem_desc e.g. niov->desc.pp instead > + * of niov->pp. > + * > + * Plus, once struct netmem_desc has it own instance > + * from slab, network's fields of the following can be > + * moved out of struct netmem_desc like: > + * > + * struct net_iov { > + * struct netmem_desc desc; > + * struct net_iov_area *owner; > + * ... > + * }; > + */ We do not need to wait until netmem_desc has its own instance from slab to move the net_iov-specific fields out of netmem_desc. We can do that now, because there are no size restrictions on net_iov. So, I recommend change this to: struct net_iov { /* Union for anonymous aliasing: */ union { struct netmem_desc desc; struct { unsigned long _flags; unsigned long pp_magic; struct page_pool *pp; unsigned long _pp_mapping_pad; unsigned long dma_addr; atomic_long_t pp_ref_count; }; struct net_iov_area *owner; enum net_iov_type type; }; > > struct net_iov_area { > @@ -48,27 +110,22 @@ struct net_iov_area { > unsigned long base_virtual; > }; > > -/* These fields in struct page are used by the page_pool and net stack: > +/* net_iov is union'ed with struct netmem_desc mirroring struct page, so > + * the page_pool can access these fields without worrying whether the > + * underlying fields are accessed via netmem_desc or directly via > + * net_iov, until all the references to them are converted so as to be > + * accessed via netmem_desc e.g. niov->desc.pp instead of niov->pp. > * > - * struct { > - * unsigned long pp_magic; > - * struct page_pool *pp; > - * unsigned long _pp_mapping_pad; > - * unsigned long dma_addr; > - * atomic_long_t pp_ref_count; > - * }; > - * > - * We mirror the page_pool fields here so the page_pool can access these= fields > - * without worrying whether the underlying fields belong to a page or ne= t_iov. > - * > - * The non-net stack fields of struct page are private to the mm stack a= nd must > - * never be mirrored to net_iov. > + * The non-net stack fields of struct page are private to the mm stack > + * and must never be mirrored to net_iov. > */ > -#define NET_IOV_ASSERT_OFFSET(pg, iov) \ > - static_assert(offsetof(struct page, pg) =3D=3D \ > +#define NET_IOV_ASSERT_OFFSET(desc, iov) \ > + static_assert(offsetof(struct netmem_desc, desc) =3D=3D \ > offsetof(struct net_iov, iov)) > +NET_IOV_ASSERT_OFFSET(_flags, type); Remove this assertion. > NET_IOV_ASSERT_OFFSET(pp_magic, pp_magic); > NET_IOV_ASSERT_OFFSET(pp, pp); > +NET_IOV_ASSERT_OFFSET(_pp_mapping_pad, owner); And this one. (_flags, type) and (_pp_mapping_pad, owner) have very different semantics and usage, we should not assert they are not the same offset. However (pp, pp) and (pp_magic,pp_magic) have the same semantics and usage, so we do assert they are at the same offset. Code is allowed to access __netmem_clear_lsb(netmem)->pp or __netmem_clear_lsb(netmem)->pp_magic without caring what's the underlying memory type because both fields have the same semantics and usage. Code should *not* assume it can access __netmem_clear_lsb(netmem)->owner or __netmem_clear_lsb(netmem)->type without doing a check whether the underlying memory is page/netmem_desc or net_iov. These fields are only usable for net_iov, so let's explicitly move them to a different place. --=20 Thanks, Mina