From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1915BC71155 for ; Fri, 20 Jun 2025 04:27:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB5FC6B007B; Fri, 20 Jun 2025 00:27:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A66966B0089; Fri, 20 Jun 2025 00:27:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 955226B008A; Fri, 20 Jun 2025 00:27:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 87EC96B007B for ; Fri, 20 Jun 2025 00:27:42 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0F2DA81551 for ; Fri, 20 Jun 2025 04:27:42 +0000 (UTC) X-FDA: 83574495564.10.96CCB01 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf20.hostedemail.com (Postfix) with ESMTP id 4C4B11C0003 for ; Fri, 20 Jun 2025 04:27:38 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750393660; a=rsa-sha256; cv=none; b=41tASO7H0dCTEjt9TbxYfrezGSqu6DLpjVpL8v3qiuDGLFvLvV7IS448DA/e/mRPzAQvUQ cF12HEXQmySfNCp8KvniL8gblgieR4d5qJ2rlU+y+uV7Z7H0Wkdz0tNPW43gwBXA6dDk/K PJ3BosHv+bLRPZulMwBTaopz2hthP9s= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750393660; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5RvjQXXCMMAtM5c8T107vk31eRk/0191tfWMh4Pp3Ts=; b=z36+EIUTCw3veWTowQlg2bZ4HIjNCcMeFXgm2XgxZWZLwnTUAkokUIWKB5+WSqEfpsw6F8 f/O5gC4WnfN8vV7rfAScfjylH76BeiUklJpm97blyRIdqkDsHddiY1MNdxf8V9OE6438GR KdcosflrkrQCF8owDP4sdj3DGx7vMwU= X-AuditID: a67dfc5b-669ff7000002311f-53-6854dfb21c96 From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, asml.silence@gmail.com, toke@redhat.com, tariqt@nvidia.com, edumazet@google.com, pabeni@redhat.com, saeedm@nvidia.com, leon@kernel.org, ast@kernel.org, daniel@iogearbox.net, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, horms@kernel.org, linux-rdma@vger.kernel.org, bpf@vger.kernel.org, vishal.moola@gmail.com, hannes@cmpxchg.org, ziy@nvidia.com, jackmanb@google.com Subject: [PATCH net-next v6 1/9] netmem: introduce struct netmem_desc mirroring struct page Date: Fri, 20 Jun 2025 13:12:16 +0900 Message-Id: <20250620041224.46646-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250620041224.46646-1-byungchul@sk.com> References: <20250620041224.46646-1-byungchul@sk.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Brightmail-Tracker: H4sIAAAAAAAAA02SbUhTYRTHeXaf++JocF2RN4OKlYSGVmJ5gij7INwoJTAssrCVFzfUGZua RoGpGJmzKKOcM65Kvk2dTc1ZJjktFQXfKGaWjpX6IStMG75RbUrktz//c36/8+UwhLyI9GXU mhRBq1EmKigpln7bUBpomTit2jfTEQRGcy0FpoV0qHRYSTDWPEcwvzhGw1xXNwXlpS4CjAM5 GH6ZlwiYfOukwWSJgImKKQxtt1oIcN7toUCfs0zAq8XvNGRZqyQw+LyAhMKlpwS0ZDpoGHlh pGC89g8JUzY9hl5DNYaJgjB4K24GV98Mgi5ziwRc+SUUPBgWKficM4FguNOJofhmAQJzu52E 5QW3o/jNOB22i++c+UHwTdWjEr7V8InmRUsq31gVwOfZhwneUnOb4i0/79P8x/dtFN/zeBnz rdY5Ca/P/k7xs5MfMP+j/R3Fm5veYb5f7KJPeZ+THo4TEtVpgnbvkYtS1cusu+iK0T89N7sP ZaL2HXnIi+HYEG66uZr+lxvLekhPptjdnN2+SHjyJnY/N+fsxnlIyhBsHcV11Y6tAhvZGG5+ ctYNMAxm/bg/NzM8tYw9wImmCrTm3M6ZGl6verzYg9xs1kvKk+XunYWyHHJt35vrLfqCPRrC fdf8RO6pCTea3VxMeM5ybBvD3XKt4DXnFq6jyo7vIdawDjf8xw3rcBERNUiu1qQlKdWJIUGq DI06PehycpIFuR+m4sZKjBX9HIyyIZZBig0y63yUSk4q03QZSTbEMYRik6y8J1Ill8UpM64J 2uRYbWqioLOhrQxW+MiCXVfj5Gy8MkVIEIQrgvbfVMJ4+WaiR/F3HDMnZNcbRN3O7oTzbP/8 wMmR6KlL5YqiuoHf7Zf2pEXXh3uH+uCzTHzy9HCgw7/utuPCaNMz8WGdJKjc0jqb+VEMjezb tjRyqCaM1JQ0F4Qf3623JUfktzacMRU5c3/FVpoPH/MLLtMf9RsSCluyY4c09W39o50BfYee flVgnUq5P4DQ6pR/AddoijYsAwAA X-Brightmail-Tracker: H4sIAAAAAAAAA02Se0hTcRzF+e3e3XtdjW5L6qJZMrJQSLMyvlCEkODN0IzsgSE59eKGc8am 4orwsaUkaU+i5pSlab5yNU2XL8Qt3SqoLMvUVDStIDWdiY/AtiTqv8M5n3O+/3wpTLSMe1Ay RSqnVEjkYkKACyL2a3aahqOku/ofbwG9sZaAmoUMeDBi5oO+uhHB3OIACQ5rNwFl9+Yx0L/S 4vDTuITBeNcoCTWmcBiumMChNa8Jg9GrNgIKtMsYtC1OkZBjruSBpdjOh9eNhXy4tVSOQVPW CAlvm/UEDNWu8GGiswAHu64Kh+HCYOgybIT5F98RWI1NPJi/UkzAzR4DAWPaYQQ9llEcirIL ERjb+/iwvODcKHo2RAb7sJbv0xjbUPWRxz7VfSJZgymNra/0Y/P7ejDWVH2ZYE2zN0h28H0r wdruLOPsU7ODxxZopgh2ZrwfZ6fbewm27OsPHmts6MUjRdGCAwmcXJbOKQMOxgqkLTlX0Tm9 b0au5gXKQu3e+ciNYui9TH2pje/SBL2D6etbxFzanQ5kHKPdeD4SUBj9kGCstQOkK9hAn2Hm xmecBYrCaR9mJVvtsoV0EGOoqUCrm1uZmkcdf3bc6H3MTE4L4dIiJ7NQquWv8usZ+93PuGsG c941lohcNuasap4UYdeQUPcfpftH6f6jDAirRu4yRXqyRCYP8lclSdUKWYZ/fEqyCTlfouLi r+tmNPc2tBPRFBKvFZrnjktFfEm6Sp3ciRgKE7sLy2wRUpEwQaI+zylTzirT5JyqE3lSuHiT MOwUFyuiEyWpXBLHneOUf1Me5eaRheIsR+sybWGH/L2iM9/sp9Ydvl9sceyRHm72Ot3YG2UI SCvuPXlku8O+NTlFY0VvwofiPUPLj63pqN6NJo9lNYe8jIxpG9TplupGvsXpykKKYk5MTdgW 5r0Dt82WLPUrvsgb9n0g0/OeJ15YdPhOvqPGcm+veZ0YLgzYrN10yf0AT4yrpJJAP0ypkvwG Xn+cjQ4DAAA= X-CFilter-Loop: Reflected X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 4C4B11C0003 X-Stat-Signature: 9w75gbgwx68eo7rwh75zo6fjx7b6d398 X-Rspam-User: X-HE-Tag: 1750393658-304291 X-HE-Meta: U2FsdGVkX1984VKi0Hrgu78xlO2bLE4gRHYqrENo21vMUV6NJPSo++dOM34yqa/SIPh3vj8Z3lAF45jphbPZhKQfXbJFHbSlP2a3DiIgTWUfGi/EpsZWEdA6dmwHUnhhsWJI5O3CMOgYuFemOwfsjgqsSXofRVbkzCO7jvPQHjZ0ojGSw9ycWlQL+o/zsEkpx/+br2L6Af6ve8u9hOVUj+bP21+f1HSBl2d9wLTxCWrKbMh0J9vFWyU2wmQjmHzrOAaJ2+x1RI6ZcIWNdPolCl16DXt7YY7GbmZ8H6BSaiBgDGAnj078eBzOumjaV8TpWg9nXlht7EU5TQ3YWV6tmHFzc+w89PvPFiawHQ3Gt+6CCotRqpPrUOr7WJf1n0sJMeXaThlXT+I2In+jBMeiECm0eBd9k1oKu8VvaJDRw+egb3F1n3AsN1kfx6tb+PqjQ7UrnrE8g3MmIqen5OOgZ3joGqMIBBWpxuB2SJ+qX+al1yyqW1Knws9cj20BQMvN0mTBnGq/oUygiZ0pJPcYZEBuaG2up4UNkTfoUWTEwCymtSIw1oJa11oT7PFZZLMtg3h2YdIBfb3LQROrmvRQ6L3E39xo2zWrDelCHjUA7aEJ7BWpyHuIhaw3F2cc9ojE7ohZXsbnTLqU7ou48TzM8a41QeUkty5WLw/FSe/5CxxIETt9/xUS0utkT+errOi3B5rgvTf6wx1VbpVxRCr/p7SdENFxoTC45avaxHpDEx1RdDJ6v8wx87z11zkZoNHKN5riv6FskHle6VDUQgOsrfJsO8eZ/Mmxa5XtLISCpioGZ+doIhpC/33ud36AXwdwbzUPb1NQ9KJSdOylqD5FmybwyPxiY2BTpV6FnBlq5THfNAXBNMCNZ0m48uyUl07m0je3f1GdxoQ92UifZAqiBScSbKdo8Uun+fpiyiBynrN8tsHO9k1kb1ErXp1n8iA9cqVHZqQDMotZfD8dm7i i5KZlswn 4GIZsCXh2yhWYz6PxE/ZZR9UHxP/r0PyPeHq+tEaKxy4m/kjTXWr+9pAINx7LcvghW9ylJTIfCtsu+AOV5kw4d8pu7K1EvbHF9WnSw9uQ63fuUWRh5cqV4gnFT19Y+Zb9HgVIcWqvLzkvCmYeLoNvx9L0rnSfWu687FBU3OJRiED0RtgNn5L/ry9jUYnlCsSaLYXJam0FEk2CjZMdwpJdvkvR/OTKUOTe13GOhf32xwkdXqJQ76+CwtoqoQsXqGV3P5nHDN1+kISbw+6GngJmahF1iJXfxsRK0Nzk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To simplify struct page, the page pool members of struct page should be moved to other, allowing these members to be removed from struct page. Introduce a network memory descriptor to store the members, struct netmem_desc, and make it union'ed with the existing fields in struct net_iov, allowing to organize the fields of struct net_iov. Signed-off-by: Byungchul Park Reviewed-by: Toke Høiland-Jørgensen Reviewed-by: Pavel Begunkov Reviewed-by: Mina Almasry Reviewed-by: Vlastimil Babka Acked-by: Harry Yoo --- include/net/netmem.h | 94 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 73 insertions(+), 21 deletions(-) diff --git a/include/net/netmem.h b/include/net/netmem.h index 7a1dafa3f080..e0aa67aca9bb 100644 --- a/include/net/netmem.h +++ b/include/net/netmem.h @@ -12,6 +12,50 @@ #include #include +/* These fields in struct page are used by the page_pool and net stack: + * + * struct { + * unsigned long pp_magic; + * struct page_pool *pp; + * unsigned long _pp_mapping_pad; + * unsigned long dma_addr; + * atomic_long_t pp_ref_count; + * }; + * + * We mirror the page_pool fields here so the page_pool can access these + * fields without worrying whether the underlying fields belong to a + * page or netmem_desc. + * + * CAUTION: Do not update the fields in netmem_desc without also + * updating the anonymous aliasing union in struct net_iov. + */ +struct netmem_desc { + unsigned long _flags; + unsigned long pp_magic; + struct page_pool *pp; + unsigned long _pp_mapping_pad; + unsigned long dma_addr; + atomic_long_t pp_ref_count; +}; + +#define NETMEM_DESC_ASSERT_OFFSET(pg, desc) \ + static_assert(offsetof(struct page, pg) == \ + offsetof(struct netmem_desc, desc)) +NETMEM_DESC_ASSERT_OFFSET(flags, _flags); +NETMEM_DESC_ASSERT_OFFSET(pp_magic, pp_magic); +NETMEM_DESC_ASSERT_OFFSET(pp, pp); +NETMEM_DESC_ASSERT_OFFSET(_pp_mapping_pad, _pp_mapping_pad); +NETMEM_DESC_ASSERT_OFFSET(dma_addr, dma_addr); +NETMEM_DESC_ASSERT_OFFSET(pp_ref_count, pp_ref_count); +#undef NETMEM_DESC_ASSERT_OFFSET + +/* + * Since struct netmem_desc uses the space in struct page, the size + * should be checked, until struct netmem_desc has its own instance from + * slab, to avoid conflicting with other members within struct page. + */ +static_assert(sizeof(struct netmem_desc) <= offsetof(struct page, _refcount)); + /* net_iov */ DECLARE_STATIC_KEY_FALSE(page_pool_mem_providers); @@ -31,12 +75,25 @@ enum net_iov_type { }; struct net_iov { - enum net_iov_type type; - unsigned long pp_magic; - struct page_pool *pp; + union { + struct netmem_desc desc; + + /* XXX: The following part should be removed once all + * the references to them are converted so as to be + * accessed via netmem_desc e.g. niov->desc.pp instead + * of niov->pp. + */ + struct { + unsigned long _flags; + unsigned long pp_magic; + struct page_pool *pp; + unsigned long _pp_mapping_pad; + unsigned long dma_addr; + atomic_long_t pp_ref_count; + }; + }; struct net_iov_area *owner; - unsigned long dma_addr; - atomic_long_t pp_ref_count; + enum net_iov_type type; }; struct net_iov_area { @@ -48,27 +105,22 @@ struct net_iov_area { unsigned long base_virtual; }; -/* These fields in struct page are used by the page_pool and net stack: +/* net_iov is union'ed with struct netmem_desc mirroring struct page, so + * the page_pool can access these fields without worrying whether the + * underlying fields are accessed via netmem_desc or directly via + * net_iov, until all the references to them are converted so as to be + * accessed via netmem_desc e.g. niov->desc.pp instead of niov->pp. * - * struct { - * unsigned long pp_magic; - * struct page_pool *pp; - * unsigned long _pp_mapping_pad; - * unsigned long dma_addr; - * atomic_long_t pp_ref_count; - * }; - * - * We mirror the page_pool fields here so the page_pool can access these fields - * without worrying whether the underlying fields belong to a page or net_iov. - * - * The non-net stack fields of struct page are private to the mm stack and must - * never be mirrored to net_iov. + * The non-net stack fields of struct page are private to the mm stack + * and must never be mirrored to net_iov. */ -#define NET_IOV_ASSERT_OFFSET(pg, iov) \ - static_assert(offsetof(struct page, pg) == \ +#define NET_IOV_ASSERT_OFFSET(desc, iov) \ + static_assert(offsetof(struct netmem_desc, desc) == \ offsetof(struct net_iov, iov)) +NET_IOV_ASSERT_OFFSET(_flags, _flags); NET_IOV_ASSERT_OFFSET(pp_magic, pp_magic); NET_IOV_ASSERT_OFFSET(pp, pp); +NET_IOV_ASSERT_OFFSET(_pp_mapping_pad, _pp_mapping_pad); NET_IOV_ASSERT_OFFSET(dma_addr, dma_addr); NET_IOV_ASSERT_OFFSET(pp_ref_count, pp_ref_count); #undef NET_IOV_ASSERT_OFFSET -- 2.17.1