From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF672C5B559 for ; Wed, 4 Jun 2025 02:53:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 71AD66B054F; Tue, 3 Jun 2025 22:53:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 694836B055A; Tue, 3 Jun 2025 22:53:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13D246B054F; Tue, 3 Jun 2025 22:53:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3700E6B0558 for ; Tue, 3 Jun 2025 22:53:08 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D97258BB58 for ; Wed, 4 Jun 2025 02:53:07 +0000 (UTC) X-FDA: 83516196414.10.CED83E1 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf10.hostedemail.com (Postfix) with ESMTP id 0DEBEC0003 for ; Wed, 4 Jun 2025 02:53:05 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf10.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1749005586; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=ueF9QV8+htH0zc5y6nO5fQbShyINdncB/FrNdj2eDxM=; b=HbmvWLBVnypRy0wcsAWnUoeTqhH29nTzjDhvoCmHL4vvRskUmS/z89S59+TL3OLUohipei 7LnAkMTq2fQF9Azyu/sjcqR+inwrBPkYjlWBsQt59fJUu6eL9ujTNsrQROGxDsSzl86lfR 0KkdJv1VEML6zOufFDv5e0Irv4j44C4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1749005586; a=rsa-sha256; cv=none; b=JSyOiq8KI8LqZ7xRlaQE4nLKTI3nQrc4ZAqvAdV3t0c1ktzgJnQaxPuXd2jnMqPqs5A1sb KYcoXSxYOjqDS4U6cJLU1czAjjqtNeH3s0gxLOLlgLSV+g8L8X5D7brrJ4/jVPMx7mb+qX YwAiqCg+F1XV021KMA+AKQqdOnkC53g= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf10.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com X-AuditID: a67dfc5b-681ff7000002311f-28-683fb50a2444 From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, asml.silence@gmail.com, toke@redhat.com, tariqt@nvidia.com, edumazet@google.com, pabeni@redhat.com, saeedm@nvidia.com, leon@kernel.org, ast@kernel.org, daniel@iogearbox.net, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, horms@kernel.org, linux-rdma@vger.kernel.org, bpf@vger.kernel.org, vishal.moola@gmail.com Subject: [RFC v4 11/18] mlx4: use netmem descriptor and APIs for page pool Date: Wed, 4 Jun 2025 11:52:39 +0900 Message-Id: <20250604025246.61616-12-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250604025246.61616-1-byungchul@sk.com> References: <20250604025246.61616-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWRW0hTcRzH/e+cnbMNF6cl7aSQtIrSzGVZ/LpgQi+HQoiUHjKyqac2mpc2 tRkE1gTRmoqJia6Yeb/AYnmZoWLLa0qN2WyVpk2mD+V1Zt7AnOLbh++Xz/fly8NE3bgvT5GY wqoSZUoJIcAF095lxwVNF+UnimwU6I0NBNQva6B63MwFfV0zgsWVHyS4u3oJKC9bwkD/OROH v8ZVDFw9ThLGqiZxaMtqwcCZ10eALnMNgyfmGg5Ym3O5ULhaiUFLxjgJQ+/0BPxs2ODCpEWH Q39JLQ5jueHQY9gLSwN/EHQZWziw9OwlAc9tBgImMscQ2D44cSh9nIvA2OHgwtqyngg/wDTW fuMwrSWjJGMwpTJvawKZHIcNY0x12QRjWiggmZHhNoLpK17DmVazm8PotDMEM+/6jjOzHXaC MTbacWbQ0EUybtP+q9QNwYV4VqlIY1XSsNsCubZ0Ck+uPKcpn3+NMlC2NAfxeTQVSv8edJA7 rG1q43qYoI7QDscK5mEfKoR2O3vxHCTgYdQMl3bp1zieYg91mXZZO7dknDpMr5i+Ig8LqTP0 REUHsT3qT9e/6dwa4m/mIzMvtlwRdZrWmb9gnlGamibpdksh2hb20e9rHHg+EhqQVx0SKRLT EmQKZWiwPD1RoQmOS0owoc1zqx6tR5vRgjXSgigekngLzSNhchFXlqZOT7AgmodJfIT+AZuR MF6W/pBVJcWoUpWs2oL8eLhELDy59CBeRN2VpbD3WDaZVe20HB7fNwMZPk1WVH8syjfw8wbv P3UGUde954LvcPj/IoyRsRhBv4paL2i/Zgs4tmEQ81MWdQOj/RF+u8K04vNiOupKzJS9KJas tZaiBv+jqrjJJE34wQVKEjrbl0f237xUH3n2VkZ+z3CYlzSWDeo+pM36Re4eIirnWGktitYL TxXbJbhaLgsJxFRq2X+ipmzg2AIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWRW0hTcRzH/e9cNoeT47I82YMxkEhIE1J+mJjZg4chwxLRIsihhzacUzbv YFguQnF2pUInzfuVJmu6o4iXKepKyLzONBVF8yEsL5nXzBW9ffh8+X5fvgJMXId7C5TqdFaj lqskpBAXyi4XXBC2XlFcLL8HBlMzCU072VC3wBFgaGxDsLU7w4fN/kESqiq2MTB81OHw07SH wfLAIh/ma1dw6HxkxWDx8RAJet0+Bg+4eh70ldsJGGkrIeDFXg0G1vwFPox1GEiYaz4iYMWm x8Fe2oDDfEk4DBhPwfaHbwj6TVYebBeXk/B81EjCkm4ewWjfIg5l90sQmLocBOzvGMhwCWNp mOYx7aVf+IzRnMG8q/djihyjGGNuLCQZ88YzPjM72UkyQ6/3caad2+Qx+oI1kllf/owz37sm SKZq9QePMVkmcGbY2M+P9rglDE1iVcpMVhMQliBUFJR9xdNqQrKr1itRPioMKEKuApq6RBe0 dhJOJqlztMOxiznZkwqkNxcH8SIkFGDUGkEvG/Z5zuAEJaWXR3r4TsYpX3rXPIWcLKKC6aXq LvLfqA/d1NLzd8j12M+uvfrbFVNBtJ4bx54goRG5NCJPpTozRa5UBflrkxU5amW2f2Jqihkd 31ebd/CUQ1tjkTZECZDETcTNhinEhDxTm5NiQ7QAk3iKfM4fK1GSPCeX1aTe0WSoWK0NnRHg Ei+RNI5NEFN35elsMsumsZr/KU/g6p2P/Oe8KFulC26//uYh974i6FoTe1KqjrG/ze26Ics6 O9wZ2RsboRO/jCc6tqa4OffujembMW5xiSF5Q/YoCJq9miguzkqbiV3pnmlZr62ePOzN9DiI 8k3KHbN+qrSftqzoycPw27/de8ZXh37JuJaQoxJZaERwX3RzfG+31MJKcK1CHuiHabTyP/Wm b1y6AgAA X-CFilter-Loop: Reflected X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 0DEBEC0003 X-Stat-Signature: 4t4xuxesfiik8dzbr4xwg8bptak8xmaa X-Rspam-User: X-HE-Tag: 1749005585-887812 X-HE-Meta: U2FsdGVkX18mQYhDK8KQgwYKav8zShKEc5gpLIZM7LvZXBV/VVKWBTL0vn3jxQfI0hTeTEFhfOPM9zHt5KTTL0BeOWgO062GuXsOPuJ3EFOivZ7RfulkABVMc0HdQvqw6BGVeEAP52uPXUEtR3iAIxQg4+qlWdN2XOWKZRyo8Ql0O+IqmD1DQIT59mMMLILDq5LcUYrr0/8AhaaSORqSFDVXXhsTjH1F/3BN/DKCbE+4BIKJkgMpim0hRDtRmYPIjN8MDntMvXcTRw++x+LrVzGXk2DuwpGVoJiVgSVbEJpTt7cZhJQwanNO39bZGi9f7LC+qYvjhl5lm3Ov9QQYLroNAAXWkQ5ZTik02hS45v6Lg0MuC9JDXeh/r+OCtKRQiS+j/I3recwC8Zs/RBZXLpGwsQa6CEHS57AuWjhwsFJf/JvovKWvuTP1tS15i45VxsC37LTsbuHzhQ4Fi7OmI481kgbv294X0u/2CI69A3BB1dj7CCJmh5aettS0GtSLlfZOW/K8Ag5uNe20dtBVNJRX3vMA7cNwlPL0YwLMDAk1Iq8heJotDfEsj2cxaVCfyvABycIlb/gLu9kTDkyeG9cXYDp3EUzM4wRH2isfcsAG47Qt7e5hViMoq6wOAILXJeqE9vjmQhYZUWY5og6OKuJa9z6lvNWbRt6uZwdhoZDj4sEuJisfSaxazECRY7i3ueQ/hUMDlatggC9urSipEg0ow5g3slKAH2Akk23yI4xT2oqcvXAYDn2SucbaiBEHQnRp3uWsCTOgRzZA9hdecMgUwVbevNNcevS0uM9ZQDYmmLWVUiA2a2CoU131ygxIheucX50aptSQ/Ed0gQ7fXC0GlJUfrpT/WWPshaFX/4tRnida3Dbzf7AP9BEDbX9+h7AeshrffM6/V9pbr9vgqdr43ss4rSmdocoXrkNLlLkvuLjRLDAk/yMSjq7dLnMurjQl7aGmIXnHotpoVs5 pXzyYP8+ Hlg1MYtspklcf9kY6rX/fbMgergsuGgbNBYe6gZc14Ut6w+VP9YqbnR6uzJCO3IO987SO X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To simplify struct page, the effort to separate its own descriptor from struct page is required and the work for page pool is on going. Use netmem descriptor and APIs for page pool in mlx4 code. Signed-off-by: Byungchul Park --- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 48 +++++++++++--------- drivers/net/ethernet/mellanox/mlx4/en_tx.c | 8 ++-- drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 4 +- 3 files changed, 32 insertions(+), 28 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c index b33285d755b9..7cf0d2dc5011 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -62,18 +62,18 @@ static int mlx4_en_alloc_frags(struct mlx4_en_priv *priv, int i; for (i = 0; i < priv->num_frags; i++, frags++) { - if (!frags->page) { - frags->page = page_pool_alloc_pages(ring->pp, gfp); - if (!frags->page) { + if (!frags->netmem) { + frags->netmem = page_pool_alloc_netmems(ring->pp, gfp); + if (!frags->netmem) { ring->alloc_fail++; return -ENOMEM; } - page_pool_fragment_page(frags->page, 1); + page_pool_fragment_netmem(frags->netmem, 1); frags->page_offset = priv->rx_headroom; ring->rx_alloc_pages++; } - dma = page_pool_get_dma_addr(frags->page); + dma = page_pool_get_dma_addr_netmem(frags->netmem); rx_desc->data[i].addr = cpu_to_be64(dma + frags->page_offset); } return 0; @@ -83,10 +83,10 @@ static void mlx4_en_free_frag(const struct mlx4_en_priv *priv, struct mlx4_en_rx_ring *ring, struct mlx4_en_rx_alloc *frag) { - if (frag->page) - page_pool_put_full_page(ring->pp, frag->page, false); + if (frag->netmem) + page_pool_put_full_netmem(ring->pp, frag->netmem, false); /* We need to clear all fields, otherwise a change of priv->log_rx_info - * could lead to see garbage later in frag->page. + * could lead to see garbage later in frag->netmem. */ memset(frag, 0, sizeof(*frag)); } @@ -440,29 +440,33 @@ static int mlx4_en_complete_rx_desc(struct mlx4_en_priv *priv, unsigned int truesize = 0; bool release = true; int nr, frag_size; - struct page *page; + netmem_ref netmem; dma_addr_t dma; /* Collect used fragments while replacing them in the HW descriptors */ for (nr = 0;; frags++) { frag_size = min_t(int, length, frag_info->frag_size); - page = frags->page; - if (unlikely(!page)) + netmem = frags->netmem; + if (unlikely(!netmem)) goto fail; - dma = page_pool_get_dma_addr(page); + dma = page_pool_get_dma_addr_netmem(netmem); dma_sync_single_range_for_cpu(priv->ddev, dma, frags->page_offset, frag_size, priv->dma_dir); - __skb_fill_page_desc(skb, nr, page, frags->page_offset, - frag_size); + __skb_fill_netmem_desc(skb, nr, netmem, frags->page_offset, + frag_size); truesize += frag_info->frag_stride; if (frag_info->frag_stride == PAGE_SIZE / 2) { + struct page *page = netmem_to_page(netmem); + atomic_long_t *pp_ref_count = + netmem_get_pp_ref_count_ref(netmem); + frags->page_offset ^= PAGE_SIZE / 2; release = page_count(page) != 1 || - atomic_long_read(&page->pp_ref_count) != 1 || + atomic_long_read(pp_ref_count) != 1 || page_is_pfmemalloc(page) || page_to_nid(page) != numa_mem_id(); } else if (!priv->rx_headroom) { @@ -476,9 +480,9 @@ static int mlx4_en_complete_rx_desc(struct mlx4_en_priv *priv, release = frags->page_offset + frag_info->frag_size > PAGE_SIZE; } if (release) { - frags->page = NULL; + frags->netmem = 0; } else { - page_pool_ref_page(page); + page_pool_ref_netmem(netmem); } nr++; @@ -719,7 +723,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud int nr; frags = ring->rx_info + (index << priv->log_rx_info); - va = page_address(frags[0].page) + frags[0].page_offset; + va = netmem_address(frags[0].netmem) + frags[0].page_offset; net_prefetchw(va); /* * make sure we read the CQE after we read the ownership bit @@ -748,7 +752,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud /* Get pointer to first fragment since we haven't * skb yet and cast it to ethhdr struct */ - dma = page_pool_get_dma_addr(frags[0].page); + dma = page_pool_get_dma_addr_netmem(frags[0].netmem); dma += frags[0].page_offset; dma_sync_single_for_cpu(priv->ddev, dma, sizeof(*ethh), DMA_FROM_DEVICE); @@ -788,7 +792,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud void *orig_data; u32 act; - dma = page_pool_get_dma_addr(frags[0].page); + dma = page_pool_get_dma_addr_netmem(frags[0].netmem); dma += frags[0].page_offset; dma_sync_single_for_cpu(priv->ddev, dma, priv->frag_info[0].frag_size, @@ -818,7 +822,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud if (likely(!xdp_do_redirect(dev, &mxbuf.xdp, xdp_prog))) { ring->xdp_redirect++; xdp_redir_flush = true; - frags[0].page = NULL; + frags[0].netmem = 0; goto next; } ring->xdp_redirect_fail++; @@ -828,7 +832,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud if (likely(!mlx4_en_xmit_frame(ring, frags, priv, length, cq_ring, &doorbell_pending))) { - frags[0].page = NULL; + frags[0].netmem = 0; goto next; } trace_xdp_exception(dev, xdp_prog, act); diff --git a/drivers/net/ethernet/mellanox/mlx4/en_tx.c b/drivers/net/ethernet/mellanox/mlx4/en_tx.c index 87f35bcbeff8..b564a953da09 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_tx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_tx.c @@ -354,7 +354,7 @@ u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv, struct page_pool *pool = ring->recycle_ring->pp; /* Note that napi_mode = 0 means ndo_close() path, not budget = 0 */ - page_pool_put_full_page(pool, tx_info->page, !!napi_mode); + page_pool_put_full_netmem(pool, tx_info->netmem, !!napi_mode); return tx_info->nr_txbb; } @@ -1191,10 +1191,10 @@ netdev_tx_t mlx4_en_xmit_frame(struct mlx4_en_rx_ring *rx_ring, tx_desc = ring->buf + (index << LOG_TXBB_SIZE); data = &tx_desc->data; - dma = page_pool_get_dma_addr(frame->page); + dma = page_pool_get_dma_addr_netmem(frame->netmem); - tx_info->page = frame->page; - frame->page = NULL; + tx_info->netmem = frame->netmem; + frame->netmem = 0; tx_info->map0_dma = dma; tx_info->nr_bytes = max_t(unsigned int, length, ETH_ZLEN); diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h index ad0d91a75184..3ef9a0a1f783 100644 --- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h +++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h @@ -213,7 +213,7 @@ enum cq_type { struct mlx4_en_tx_info { union { struct sk_buff *skb; - struct page *page; + netmem_ref netmem; }; dma_addr_t map0_dma; u32 map0_byte_count; @@ -246,7 +246,7 @@ struct mlx4_en_tx_desc { #define MLX4_EN_CX3_HIGH_ID 0x1005 struct mlx4_en_rx_alloc { - struct page *page; + netmem_ref netmem; u32 page_offset; }; -- 2.17.1