From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E430C83F1A for ; Fri, 11 Jul 2025 01:33:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8FF626B009B; Thu, 10 Jul 2025 21:33:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 881736B009E; Thu, 10 Jul 2025 21:33:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 722B46B00A0; Thu, 10 Jul 2025 21:33:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 5E11D6B009B for ; Thu, 10 Jul 2025 21:33:07 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E5779141131 for ; Fri, 11 Jul 2025 01:33:06 +0000 (UTC) X-FDA: 83650260372.22.9B0A330 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf06.hostedemail.com (Postfix) with ESMTP id 3A02E180009 for ; Fri, 11 Jul 2025 01:33:03 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; spf=pass (imf06.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752197585; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=587DXL54TKRvPL2I/4JwhWUQANHvnVlOKz8Qt2e2LRQ=; b=DXJMY16v6CeI1bnBvZoLnFZgOEcpCO0bbswuB7rGcje8y/vFpouUh1H6WP0znO0KHKF25G c7i5SSFu6vkPuns1oSKTWX60l7yqV0xlvLdiccsciLSbh3A7b5oCUaeKbu7JThB5kfzZgm L7C3N5LVDsYNRp1kvfihnNYfdG6TpyE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752197585; a=rsa-sha256; cv=none; b=wXSear7Ly+F8QnZ0M39Z89nUmr0QZQEq5O91vCi9cOWsygmjdhsIVPp1XdwosNBXLIce+T KzOU7QICN9IKuirt3D8c5kEtPgWmQ6Ss1Z9H38c2lAWU5TCHDVtdm5foHR9SnKnnUCao9r JCZarqvc3KtxzooEdsijfwnW7DFs0wE= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf06.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com X-AuditID: a67dfc5b-681ff7000002311f-68-687069ce6d1f Date: Fri, 11 Jul 2025 10:32:57 +0900 From: Byungchul Park To: Mina Almasry Cc: willy@infradead.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, asml.silence@gmail.com, toke@redhat.com, tariqt@nvidia.com, edumazet@google.com, pabeni@redhat.com, saeedm@nvidia.com, leon@kernel.org, ast@kernel.org, daniel@iogearbox.net, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, horms@kernel.org, linux-rdma@vger.kernel.org, bpf@vger.kernel.org, vishal.moola@gmail.com, hannes@cmpxchg.org, ziy@nvidia.com, jackmanb@google.com Subject: Re: [PATCH net-next v9 6/8] mlx4: use netmem descriptor and APIs for page pool Message-ID: <20250711013257.GE40145@system.software.com> References: <20250710082807.27402-1-byungchul@sk.com> <20250710082807.27402-7-byungchul@sk.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Brightmail-Tracker: H4sIAAAAAAAAA02SSUxTURSGc/tGKjWPinKFhUkFDERxiCEnapSdd2PU4EaJQ5WnrTI0LSCg JiDEoQoOmIilkIcDVCA2VqFIlGipCI6IwRSVQRAhWkVEGgGntoTI7sv5z/2/s7g8pbzJhPLa lDRRn6JOUrFyWv4lsHzJc61Os8z5KQbM1hoWqn9mQmVvPQPmqjoEP8bfcjDqfMTClXIPBeYX +TSMWScoGGju46DatgF6Kj7ScPe4nYK+My0sFORPUnBv/CsHR+stMmirK2TgwsQ1Cuw5vRy8 ajCz0F3zl4GPjgIaWk3XaegpjINmaR54nrgROK12GXhOl7JQ1C6x0J/fg6C9qY+GktxCBNZG FwOTP70dJQ+7ubhw0uQepsjt650ycsfUxRHJlk5uWaKJ0dVOEVvVSZbYvp/nyLvXd1nSUjxJ kzv1ozJSkPeVJSMDb2gy3NjBEuvtDpo8lZzcpqBt8jWJYpI2Q9QvXbtLrpGM83VvwzPfj+Ac 9DvEiAJ4LKzEJ9wXZNPc+WOQ8jEtROCGU3bkY1ZYhF2ucf88WIjCVxvPMUYk5ynBxOKOhj7a F8wRtuKi3MeMjxUC4Aa7S+ZbUgoWhI3SkGwqCMKtlz74H1De1l9l7d5W3sthuPIPPzVegPNq S/yyAGEzHu8p8q/PFRbi+3WP/J1YaOGx1VLBTV09Hz+wuOizKMg0Q2GaoTD9V5hmKCREVyGl NiUjWa1NWhmjyUrRZsbsSU22Ie9HqjjyK6EefW+LdyCBR6pARVx1qkbJqDMMWckOhHlKFay4 sUGnUSoS1VnZoj51pz49STQ4UBhPq0IUKzwHE5XCPnWaeEAUdaJ+OpXxAaE56ERpBJ/wrbV/ 7LhdMxEcGttWu2SdZNquGGrpdBtq/xQRZ3FTyaFZf7X7+aGRW2WnAjcpB3PXOrp0e92Xox9G fv4kvjj8IIFabx4dO7ZqdVNb2MXhWYt3rD4vryk+Hcw3bslb846L9Vif7S6LlJrf2DZGRmkG nVGzX3oSmMzs+MHtKtqgUS+PpvQG9T+HrZnjRAMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA02Sa0iTYRTHed7bXkeTt2X1pkS0ssjSLhScMkryQw+VkVBGEuSoNzdyc2wq GiSmC2mpZXaxabKozKk0WOqW5hK1iwVdVtZMc2rZjVDLZV6zTYn89uP8/+d3vhyWlBbQgaxS nSRo1fIEGSOmxLvDs0KfKTWKNQ2eICi2VDJQMZwKt7rsNBSX1yDwjLSLYLD5EQPXrw2RUPxc T8EvyygJvQ97RFBhjQJ36ScK7mXbSOg5+5iBXP0YCfUjfSLItJcR0HS1hYYXNXk0XBi9SYIt o0sEr2qLGeisnKThU2MuBS1GMwXuvAh4aJoHQ0+/I2i22AgYyrnKQIHTxMAHvRuBs6mHgqKT eQgsDhcNY8NeR9GDTlFEMG763k/iKnMbge8a34uwyZqM75SFYIPLSWJr+WkGW3+eF+GON/cY /LhwjMJ37YMEzs3qY/CP3ncU7ne0Mvj6lwECW6paqT3SWPHmI0KCMkXQrt4SJ1aYDAs07UtT u3/wGWhivgH5sTy3nm/zfCZ9THHBfO0ZG/Ixwy3nXa6RqXkAt4K/4cinDUjMkpyR4Vtreyhf MIc7wBecfEL7WMIBX2tzEb6SlCtDvMH0hZgOZvMtVz5OLZBe63iJ02tlvRzE3/rDTo8X8VnV RVPH/LhofsRdMFWfyy3hG2oeEeeQv3GGyTjDZPxvMs4wmRBVjgKU6hSVXJmwIUx3TJGmVqaG HU5UWZH3VUpPjOfbkefV9kbEsUg2SxJRkaiQ0vIUXZqqEfEsKQuQ3I7SKKSSI/K044I28ZA2 OUHQNaIglpLNl+zYL8RJuXh5knBMEDSC9l9KsH6BGWjft6iG7vpdq/wnyCquu26ysP1D08Lk 3pc3nnVyMZsuf83v37Dy4ERp/dv4cO7tmtLshbH4dUlmx+vurR8jf1eZnaub24qctptmWUm8 I6lfH7JY4386OjrmvlkVGla3czI/xL888tSlnL3agXXL0ivUn+0Xt82r1jt729OlR6NKNlpk lE4hXxtCanXyv0FOTGwmAwAA X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 3A02E180009 X-Stat-Signature: nfifpr664xke3nb1pbbrr69pn9x7fknh X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1752197583-329201 X-HE-Meta: U2FsdGVkX1/db54YPLHqeARthCPNyXzeSKF91N0/Uiwn9c1ItnbaRhEQG66xrW2xSLsX+ZZh8htBPVkX/q/cve9vyN+KXwWu7zbip6/FiOg/F/tSzBBDJW7ayquaMeqGKHHflWZrhwUtQ3YY/YAd2AwhoNeTvQBeg1KUVLFM+6iJXi/wdwK04hyFLCetNjGzC4adPHfqA2HOC+vxYctcmzkiZbCnyLWJqkwwEFHk0QmhxNGX20RnPhCbcoGo5FbHIGOd6qSCtvCtCcl45S4hs0qwIFbtJB/ZO5S4fLv2VpF/xCF0QbFetXp91f2H6mX57NqzKTOjh4UzyIbiF/6+k9aLdPK9FHGroGiMZJpNd5Al0eG8VumAt6TCfRa2Tz+HO5GZJalVVtjmXj1VkGPw/kuR5piZ2s3li+IbVObhBj60e0WnVzKtISKu7tLuxCOAQFWVHwcs5nS3L0PAIaFPzsjUOgEWahbHC3/+ER8Z/QRgtmYSN0ac/lM+q6/HNMp8xpUPl60gsXveZaWkTrVBAyTZUBqg6mvHjcYvg1akdIWeoW9QF6Rx2NloFikk6QVp+gUEBJZS3rvLIS6BdwUQrxbJRoYsI8yhNHdIgApd1RknYl3lasuwj10g9rsIoV0a38xp+OTPa26T2odcdk9Ci6T4OOM2OTmjlNEILm9Otm6lSw7/yD9WlBe+4pqI1+Ya0n+2U/Cd6sbaZ6MYVZ47e2WmFOBDmNnWheAGySFMRiRsmZg4bRlLBIKF35mtePKgzy4+JhxtM2nGQwuJUgDVNXZXd2LfPbVjDsIqaBEOL4ZECo2TJmwmInMZI8g81ek57U/MytPssqmtb2TMk0aQz3MfXjMvkbik84yi8Rc3tD/6d23pQZmThqtrk+9zbHy++6NSMmR3YQaEStUQ5h+nh+GpqVQe4wjLIUSjUfRn5VewkMtiG/lKL9re9FKwQ5WnuuCHwPe41bsnYThJKzK UBllqhqT K96j8xHDemu5SHYnkYDk75Li2vxMBfM5SJYeC X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jul 10, 2025 at 11:29:35AM -0700, Mina Almasry wrote: > On Thu, Jul 10, 2025 at 1:28 AM Byungchul Park wrote: > > > > To simplify struct page, the effort to separate its own descriptor from > > struct page is required and the work for page pool is on going. > > > > Use netmem descriptor and APIs for page pool in mlx4 code. > > > > Signed-off-by: Byungchul Park > > --- > > drivers/net/ethernet/mellanox/mlx4/en_rx.c | 48 +++++++++++--------- > > drivers/net/ethernet/mellanox/mlx4/en_tx.c | 8 ++-- > > drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 4 +- > > 3 files changed, 32 insertions(+), 28 deletions(-) > > > > diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c > > index b33285d755b9..7cf0d2dc5011 100644 > > --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c > > +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c > > @@ -62,18 +62,18 @@ static int mlx4_en_alloc_frags(struct mlx4_en_priv *priv, > > int i; > > > > for (i = 0; i < priv->num_frags; i++, frags++) { > > - if (!frags->page) { > > - frags->page = page_pool_alloc_pages(ring->pp, gfp); > > - if (!frags->page) { > > + if (!frags->netmem) { > > + frags->netmem = page_pool_alloc_netmems(ring->pp, gfp); > > + if (!frags->netmem) { > > ring->alloc_fail++; > > return -ENOMEM; > > } > > - page_pool_fragment_page(frags->page, 1); > > + page_pool_fragment_netmem(frags->netmem, 1); > > frags->page_offset = priv->rx_headroom; > > > > ring->rx_alloc_pages++; > > } > > - dma = page_pool_get_dma_addr(frags->page); > > + dma = page_pool_get_dma_addr_netmem(frags->netmem); > > rx_desc->data[i].addr = cpu_to_be64(dma + frags->page_offset); > > } > > return 0; > > @@ -83,10 +83,10 @@ static void mlx4_en_free_frag(const struct mlx4_en_priv *priv, > > struct mlx4_en_rx_ring *ring, > > struct mlx4_en_rx_alloc *frag) > > { > > - if (frag->page) > > - page_pool_put_full_page(ring->pp, frag->page, false); > > + if (frag->netmem) > > + page_pool_put_full_netmem(ring->pp, frag->netmem, false); > > /* We need to clear all fields, otherwise a change of priv->log_rx_info > > - * could lead to see garbage later in frag->page. > > + * could lead to see garbage later in frag->netmem. > > */ > > memset(frag, 0, sizeof(*frag)); > > } > > @@ -440,29 +440,33 @@ static int mlx4_en_complete_rx_desc(struct mlx4_en_priv *priv, > > unsigned int truesize = 0; > > bool release = true; > > int nr, frag_size; > > - struct page *page; > > + netmem_ref netmem; > > dma_addr_t dma; > > > > /* Collect used fragments while replacing them in the HW descriptors */ > > for (nr = 0;; frags++) { > > frag_size = min_t(int, length, frag_info->frag_size); > > > > - page = frags->page; > > - if (unlikely(!page)) > > + netmem = frags->netmem; > > + if (unlikely(!netmem)) > > goto fail; > > > > - dma = page_pool_get_dma_addr(page); > > + dma = page_pool_get_dma_addr_netmem(netmem); > > dma_sync_single_range_for_cpu(priv->ddev, dma, frags->page_offset, > > frag_size, priv->dma_dir); > > > > - __skb_fill_page_desc(skb, nr, page, frags->page_offset, > > - frag_size); > > + __skb_fill_netmem_desc(skb, nr, netmem, frags->page_offset, > > + frag_size); > > > > truesize += frag_info->frag_stride; > > if (frag_info->frag_stride == PAGE_SIZE / 2) { > > + struct page *page = netmem_to_page(netmem); > > This cast is not safe, try to use the netmem type directly. Can it be net_iov? It already ensures it's a page-backed netmem. Why is that unsafe? With netmem, page_count() and page_to_nid() cannot be used, but needed. Or checking 'page == NULL' after the casting works for you? Byungchul > -- > Thanks, > Mina