From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9167CC83F1A for ; Mon, 14 Jul 2025 19:02:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2FBFB8D000C; Mon, 14 Jul 2025 15:02:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2AC298D0001; Mon, 14 Jul 2025 15:02:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 19BCD8D000C; Mon, 14 Jul 2025 15:02:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 06CFB8D0001 for ; Mon, 14 Jul 2025 15:02:28 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7304757FC3 for ; Mon, 14 Jul 2025 19:02:27 +0000 (UTC) X-FDA: 83663791134.08.DAD3638 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf06.hostedemail.com (Postfix) with ESMTP id 809DA180003 for ; Mon, 14 Jul 2025 19:02:25 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CGKqSBWK; spf=pass (imf06.hostedemail.com: domain of almasrymina@google.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=almasrymina@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752519745; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7VNuZztsVJnCZDih7ufg2t56wYe/NpWWBgR9l5kOONY=; b=lsFCpVqdH/VD5aFPSGpvfSN4OwivxtAy+Zen7mFgGd3VZBGGzDKguHYDOYriysa8+IYA67 ytWsdLXwb6c83WgnHHJoYnoTWpimdCDs4son96/awLiw+bRmpqOOgZFvQbFJuESJpbFsBd Ky5jWBGmUy1GNDpX13Sa3R3LL/qBipo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752519745; a=rsa-sha256; cv=none; b=pgIZmYusEA8zwNx57cRxsFr4ya3kVigxHydZiQGlezSVbDN94ROO8cEkAFPzKnnMzC3LHH 7dE21rKPbyFW9D9kVwV4aLDZIDEyN7w8dVuRkXCLzbs9Gc3y0jZ/W66uO3Y/jzP6NAxr+b vGZI58LwITJo6mj1BSuvb/Xlmc0pWj4= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CGKqSBWK; spf=pass (imf06.hostedemail.com: domain of almasrymina@google.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=almasrymina@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-23dd9ae5aacso27785ad.1 for ; Mon, 14 Jul 2025 12:02:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1752519744; x=1753124544; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=7VNuZztsVJnCZDih7ufg2t56wYe/NpWWBgR9l5kOONY=; b=CGKqSBWKS6Bk2PqhthIesrnACBcZ8xfhQa3U829YHNMq5nxbof5cTWYtJRSBjipdkB jPVpPsjnaBrW4ck3K40xZClAbpEzyt5y++ErBZgt866yT+wQ/M8zBdLZDFYTJxgte11B 69435acs2/X/nuhy3J8fm+lhVoCwbtST8rg9qKnjhzhILQYute+qapgTMVknDI2tDYvc XKjJxT1srQZR8khCtYJPnkMciTRKkPtKb2cw3GAE97OXiE4NrBmtS7Y0vycljWzGwgkQ IUxJKHZndf95WiJeCIWSjk2K3XS3zlMRCr2GTNhvKWClv4B0ShUaJ8MtQiET77h6F7SK BR8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752519744; x=1753124544; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7VNuZztsVJnCZDih7ufg2t56wYe/NpWWBgR9l5kOONY=; b=sUbrcFnhW1O7GvuBOXSKKuariuedWjBZkYWIFlghfUL6XoPtqADsN1vzH9mc90OBvt vu6WmMI5/m6BYKFI9ryngxNN3uEbH4ZRO827UMUgpXqXniobRZO0BJP42d7PbO2Vutmh UOIl72/ZtV45mD6XoOtVBTfADogRe13HjLmptKPl8rALQQhG1TjsqbZRKusyXACZjIIO RE7SKg5NurjEvPVlMsc/36Phw/jC8zbd4CJ/WNnW4GCZE75V8YjyZ66f2v+O5ADeIYhU azwVDNXWzTquWdidVECsTYHK3kKqlvkY4d38qqEqIiS6GluYV3QneQ9cNcLovVe/G+id KynQ== X-Forwarded-Encrypted: i=1; AJvYcCUJFjFY8qvMo/90JBd/aufDHr1H6vdYnzQKX1JznLnTnIGyKviovW7Fz6z8ZbREQqkQLGILCnuuOg==@kvack.org X-Gm-Message-State: AOJu0YwWe9TZFjN1VIiyO8piRKkPoh+7bpBs1x6hGwVhGgpcUOiUpNC0 1+NWwzeVKKJ4OIhrzMrHQ/RtQjGQ2vQhnHXfy2heS9HuUM1ufixoIocMVvMvZ5CPJdjQYmoiOId wnOSzYNqq2jLi80vpyPahJ2dK1wsUr8VGimDwgN4f X-Gm-Gg: ASbGncuvyVNYWPXRMCikHMBDqYEQw2hV5EoRFYymGkDnInCwDILxN33FCp4hFj2Q67I fQb4mU80xzdVjwdSVVB7/XdICLRX36gnBO6IaB5AhdgJUHkP5n3bcSltrqjqbINTi8c/p4BkE5w JKGK8o9VR2VMas3MBC51nzyNYMr8AftlbJiHd2o3k2H27kiIjnIRk3sBI3ytsPpG1AUHKAqkkEb N5wJo+3ktQJ9v9w9HucvDEkOLTtJKlNH32HJQ== X-Google-Smtp-Source: AGHT+IE6b3u0FEYbE/5zU2GFDjTpZUgGKFIuxC3GHJP9usTARJ45zz7VYQPM3TtdZ3Tr5Ox8uT4ZdAfccFl5CsvEYCc= X-Received: by 2002:a17:902:e552:b0:22e:766f:d66e with SMTP id d9443c01a7336-23e1ab000f4mr410435ad.12.1752519743690; Mon, 14 Jul 2025 12:02:23 -0700 (PDT) MIME-Version: 1.0 References: <20250710082807.27402-1-byungchul@sk.com> <20250710082807.27402-7-byungchul@sk.com> <20250711013257.GE40145@system.software.com> In-Reply-To: <20250711013257.GE40145@system.software.com> From: Mina Almasry Date: Mon, 14 Jul 2025 12:02:02 -0700 X-Gm-Features: Ac12FXxeXh1UoBnGrAUfq0hNcg_aduZtehgNTdEPgSN61td1YzR9QhWexF1R0b0 Message-ID: Subject: Re: [PATCH net-next v9 6/8] mlx4: use netmem descriptor and APIs for page pool To: Byungchul Park Cc: willy@infradead.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, asml.silence@gmail.com, toke@redhat.com, tariqt@nvidia.com, edumazet@google.com, pabeni@redhat.com, saeedm@nvidia.com, leon@kernel.org, ast@kernel.org, daniel@iogearbox.net, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, horms@kernel.org, linux-rdma@vger.kernel.org, bpf@vger.kernel.org, vishal.moola@gmail.com, hannes@cmpxchg.org, ziy@nvidia.com, jackmanb@google.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 809DA180003 X-Stat-Signature: wg654xdqqpyj64udbg9u3amuu1prrcj7 X-Rspam-User: X-Rspamd-Server: rspam07 X-HE-Tag: 1752519745-36899 X-HE-Meta: U2FsdGVkX18R595B0HLPKrMsjw0jEjBeJO9EPu5yqoVwoXX8vhSXCqn8bC/1tkOcD4TRo8xqOQ2OLa49A3Qj4yVc9S5RLBcaJUPoaxC5A+oy+kCfGC+GyohrJyyp5u1mxmetLnDh8/kOgYi/8bWF7MnJKAMYe974bS0iDOnPsTJOUV8o24Doe2wlFETu+TrVO9d5pORGIY/X9GG7f1bcOl/IaHIh/O+dfIQQXYhnjbT1JJivv5/GIu13ifOvJ3p1YGR50GzRdp9NpMBz4TCFO+rJLHOwbBEAIQjmzWn62R84i3vWupSNWdonR0yyXPBDH5T6PT9oKccX8qbU2vwgi1kCPPqZ85SE57vuN32/NWNLuaxRES7lxa0p7QQ2p/YloJ22mpiiyGryzBBBYoKfq8ts5SVM/7iQETcUyG8wWcVW25b9oLD9pXwQWzd716nYf0cqeRomilOhccr7dHieoZk7VZWnuQBszPJIWK9UMF7qUpjRvKzpb1csbmu07HlmdruDlIKSqo3oRBwVInCy/3PrXZOn1glMhaY/eWKncPdApd4JwtXiKEbkpb2AxytObX84M/Z2LWz8dQdYMMafSmzdkjQVFJBncJ8dYHIXzKI7MCLrA7xjvonGAZEn80aCMlBqJjUi9g3DLaeTQFQCK5jqBWHpc7YYpn5xOUS/tEUwfEpfR6ZsseeGPY40Yv41uu96pIj3UKm91Tji5eMVAU9lJPN4F3vc8gE4ndBqRp1TjAohRVGROyF9agTKTcxDxNLXZ3wB491hliMay2D6G9U5Rrz5Yx3nuGhIID/Q8aFn5F6Out9Iep4mOSFOKHVzpTulLCcQxzR39rX9noFWPBxbYc8i/hIn3AqW04pzZfXBmruSaeogGDNf1zbzlsgNwouc28Gf6GyJ4NE/6q4SvTY2G09HxzceY3CRP28TNXN1g0Wch+sZQVXN/Kt66waDpMPrWS4cpFc2GopWrAe aTiWxM1B vWuwGUgr4Fn4PmmDhOlycwh/OBi/bgk5IPy8O4FEGvqQOfHg+sbt2AVDmPU+rH9Gnd9yO1KNlHCIYnFZ3X+zDYCUaFLiH6a0Wo8lWLcpVRy7ZNyYzwkp+O136P3Su1fnumtCcj0V73vLtvcP29QU/I7HlobcnN9kJqYghZuo1BPXEa4gaVz/8RsW3bhb7VVrtXdwB/fWWnAp0Mzm29OTU7yv7vm4JmlbrBp13/tp1G2N6VzaM74jlG19IHC5isT6ahgpauIdrroZZkx/y0/c5JfMsE09f+8tSa9fWdAh97Dsmrn0T0+WIu/uA6i8RXRYyC6OXyHB2BfiS8Rk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jul 10, 2025 at 6:33=E2=80=AFPM Byungchul Park w= rote: > > On Thu, Jul 10, 2025 at 11:29:35AM -0700, Mina Almasry wrote: > > On Thu, Jul 10, 2025 at 1:28=E2=80=AFAM Byungchul Park wrote: > > > > > > To simplify struct page, the effort to separate its own descriptor fr= om > > > struct page is required and the work for page pool is on going. > > > > > > Use netmem descriptor and APIs for page pool in mlx4 code. > > > > > > Signed-off-by: Byungchul Park > > > --- > > > drivers/net/ethernet/mellanox/mlx4/en_rx.c | 48 +++++++++++-------= -- > > > drivers/net/ethernet/mellanox/mlx4/en_tx.c | 8 ++-- > > > drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 4 +- > > > 3 files changed, 32 insertions(+), 28 deletions(-) > > > > > > diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net= /ethernet/mellanox/mlx4/en_rx.c > > > index b33285d755b9..7cf0d2dc5011 100644 > > > --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c > > > +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c > > > @@ -62,18 +62,18 @@ static int mlx4_en_alloc_frags(struct mlx4_en_pri= v *priv, > > > int i; > > > > > > for (i =3D 0; i < priv->num_frags; i++, frags++) { > > > - if (!frags->page) { > > > - frags->page =3D page_pool_alloc_pages(ring->p= p, gfp); > > > - if (!frags->page) { > > > + if (!frags->netmem) { > > > + frags->netmem =3D page_pool_alloc_netmems(rin= g->pp, gfp); > > > + if (!frags->netmem) { > > > ring->alloc_fail++; > > > return -ENOMEM; > > > } > > > - page_pool_fragment_page(frags->page, 1); > > > + page_pool_fragment_netmem(frags->netmem, 1); > > > frags->page_offset =3D priv->rx_headroom; > > > > > > ring->rx_alloc_pages++; > > > } > > > - dma =3D page_pool_get_dma_addr(frags->page); > > > + dma =3D page_pool_get_dma_addr_netmem(frags->netmem); > > > rx_desc->data[i].addr =3D cpu_to_be64(dma + frags->pa= ge_offset); > > > } > > > return 0; > > > @@ -83,10 +83,10 @@ static void mlx4_en_free_frag(const struct mlx4_e= n_priv *priv, > > > struct mlx4_en_rx_ring *ring, > > > struct mlx4_en_rx_alloc *frag) > > > { > > > - if (frag->page) > > > - page_pool_put_full_page(ring->pp, frag->page, false); > > > + if (frag->netmem) > > > + page_pool_put_full_netmem(ring->pp, frag->netmem, fal= se); > > > /* We need to clear all fields, otherwise a change of priv->l= og_rx_info > > > - * could lead to see garbage later in frag->page. > > > + * could lead to see garbage later in frag->netmem. > > > */ > > > memset(frag, 0, sizeof(*frag)); > > > } > > > @@ -440,29 +440,33 @@ static int mlx4_en_complete_rx_desc(struct mlx4= _en_priv *priv, > > > unsigned int truesize =3D 0; > > > bool release =3D true; > > > int nr, frag_size; > > > - struct page *page; > > > + netmem_ref netmem; > > > dma_addr_t dma; > > > > > > /* Collect used fragments while replacing them in the HW desc= riptors */ > > > for (nr =3D 0;; frags++) { > > > frag_size =3D min_t(int, length, frag_info->frag_size= ); > > > > > > - page =3D frags->page; > > > - if (unlikely(!page)) > > > + netmem =3D frags->netmem; > > > + if (unlikely(!netmem)) > > > goto fail; > > > > > > - dma =3D page_pool_get_dma_addr(page); > > > + dma =3D page_pool_get_dma_addr_netmem(netmem); > > > dma_sync_single_range_for_cpu(priv->ddev, dma, frags-= >page_offset, > > > frag_size, priv->dma_di= r); > > > > > > - __skb_fill_page_desc(skb, nr, page, frags->page_offse= t, > > > - frag_size); > > > + __skb_fill_netmem_desc(skb, nr, netmem, frags->page_o= ffset, > > > + frag_size); > > > > > > truesize +=3D frag_info->frag_stride; > > > if (frag_info->frag_stride =3D=3D PAGE_SIZE / 2) { > > > + struct page *page =3D netmem_to_page(netmem); > > > > This cast is not safe, try to use the netmem type directly. > > Can it be net_iov? It already ensures it's a page-backed netmem. Why > is that unsafe? > Precisely because it can be net_iov. The whole point of netmem_ref is that it's an abstract type that can be something else underneath. Converting the driver to netmem only to then say "well, the netmem is just pages, cast it back" is just adding a lot of useless typecasting and wasted cpu cycles with no benefit. In general netmem_to_page casts in drivers are heavily discouraged. > With netmem, page_count() and page_to_nid() cannot be used, but needed. > Or checking 'page =3D=3D NULL' after the casting works for you? > I'm really hoping that you can read the code and be a bit familiar with the driver that you're modifying and suggest a solution that works for us. netmem_to_page() followed by page_count() without a NULL check crashes the kernel if the netmem is not a page underneath... so it shouldn't work for anyone. What may be acceptable here is netmem_count helper and what not like we have netmem_is_pfmemalloc which handles net_iov correctly. However, it looks like this driver is trying to check if the netmem is recyclable, we have a helper like that already in __page_pool_page_can_be_recycled. Maybe that can be reused somehow. --=20 Thanks, Mina