From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31537C61DB2 for ; Mon, 9 Jun 2025 04:32:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E637A6B008C; Mon, 9 Jun 2025 00:32:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E3AFA6B0092; Mon, 9 Jun 2025 00:32:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CDAD76B0095; Mon, 9 Jun 2025 00:32:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id AE1096B008C for ; Mon, 9 Jun 2025 00:32:40 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 41A97C0890 for ; Mon, 9 Jun 2025 04:32:40 +0000 (UTC) X-FDA: 83534591280.08.055E153 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf02.hostedemail.com (Postfix) with ESMTP id D7EFC8000A for ; Mon, 9 Jun 2025 04:32:37 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1749443558; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NKk21oGHamnBbIw1VJOTGn+CWeAdxZlSCJlHqNTTXTQ=; b=Ra6IOQYTu2JqsURJU27BA0fY1/plj+XlMc2ln2YozVzTKi5jMaf6ohzH1q0XoISjFj/TJ5 IJ9VyIFCVRcCqCU9Qzv3+bvXvy8ITvNdX+1G9ksq1kMALs/MypdpCSTKec0tO6+VjBXv8C JsbZUkGGYieIqVepCWDbmofTWqFKuPs= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1749443558; a=rsa-sha256; cv=none; b=XVXiOgXbCkQ2OuwKq8X02QOxsrwnqB+jC0n+1+iXW1ZuEPKWJt10it1WHMUhIlDMK8GpVu WZkaJe7p9KdpBc1+KMla4kjpzo3r+FtncIosX1OXvsKNv6QjxPRhige2tYVix9oc6qa+tP eWl0QARLTK/92A+9FWm8hspX7dbPaNM= X-AuditID: a67dfc5b-669ff7000002311f-4e-684663e399ec From: Byungchul Park To: willy@infradead.org, netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, almasrymina@google.com, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, asml.silence@gmail.com, toke@redhat.com, tariqt@nvidia.com, edumazet@google.com, pabeni@redhat.com, saeedm@nvidia.com, leon@kernel.org, ast@kernel.org, daniel@iogearbox.net, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, horms@kernel.org, linux-rdma@vger.kernel.org, bpf@vger.kernel.org, vishal.moola@gmail.com Subject: [PATCH net-next 1/9] netmem: introduce struct netmem_desc mirroring struct page Date: Mon, 9 Jun 2025 13:32:17 +0900 Message-Id: <20250609043225.77229-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250609043225.77229-1-byungchul@sk.com> References: <20250609043225.77229-1-byungchul@sk.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Brightmail-Tracker: H4sIAAAAAAAAA02Sa0hTcRjG++9/dnY2HJyW1cmicBSGpWVpvZSIlMbBPmT0Jayooce22lQ2 L7MovIWoeam0cpsxs3JearG8lkgu0UmFZlnLsslMP1RauRIvA9uKqG8Pz/v8nufLS2FJAd+P UiSlcuokmVJKigjRpM/NoLH4ffKtl+aCwWBuJKFhVgu1o218MNS3IPgx904Aru5eEmqqZzAY +vMI+GmexzDe4xSA484EAR35rRicpTYSivMWMOS0mXgw0FLCh/L52xhas0YF8PKhgYQPjYt8 mLAWE9CnqyPAURIJPcYVMPP0C4JucysPZi5WkXBl0EjCWJ4DweATJwH67BIE5k47HxZmDWSk P9tU95bHtutGBKzRksY+MAWyhfZBzFrqC0jWMn1ZwL5/3UGytusLBNve5uKxxblTJPt9fJhg v3YOkay5aYhgnxm7BazLsjaWjhOFJ3BKRTqn3hJxQiR/PjGEU7o2akd/3MVZaHJdIRJSDB3K fCrqJ//qIfdlwqtJOoCx2+ewV/vSIYzL2evxRRSmp/jMuGGB5z0sow8z+WWPf4cIegPjLr2H ChFFiekwptaN/nSuYxrueyMUJaR3MKP2VK8t8SSsr0y/STG9lOmr/Eh4I9gza74h8drYQ+Y2 67F3laHrKOZCjgP/qVzFdJnsRBmidf/hun+47j/ciHA9kiiS0lUyhTI0WJ6ZpNAGxyerLMjz IHfOuY+0oemBQ1ZEU0jqIz5xLVou4cvSNZkqK2IoLPUV0469cok4QZZ5hlMnH1enKTmNFa2m COlK8baZjAQJfVKWyp3muBRO/ffKo4R+WWjr7okBZlejuzekv0GTmHfsUoeraHPodMWStPCY qDVnl2v1Qr1/9C1hclDV4KK4vDaDl7jpniompIyWrVdENT8L6EsUl576tlidoor7/MZJZUcU 1+yJtOl9lp+Pn4wV7TxQpFPbBH6Pws7XF+w/XHlybLZi2ObuPGrgRqIOvth+VUpo5LKQQKzW yH4Bi6JqIhwDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA02Sa0hTcRjG+e9/dnZcrk7L7GCUNEjDsrKyXihEiuoQFSXd6OrQQxs6tU2H GpU1SRp5ya7OGQvJppaTZTrFLs7hJbvOFssyzRuWZhfNvJFtReS3h+f5Pe/75aGwuJ/woeQx 8ZwyRhotIYWEcPtaTWBnxCbZ8l8P54PedJuE4pFEuNVu4YO+qBzB0OhbAQza6knIvzGMQf88 lYAfpjEM3XUdAmgr6CGgOq0CQ0dmAwnpqeMYzliMPKjNa+TDi/IMPlwau4mhIqVdAM1VehLe 357kQ481nYBGXSEBbRmhUGfwhuGmfgQ2UwUPhs/nkXDRbiChM7UNgb22g4Dc0xkITA+cfBgf 0ZOhEras8A2PrdS1CliDOYG9awxgtU47Zs1F50jW/D1bwL57XU2yDdfGCbbSMshj0zUDJPut u4VgvzxwkGx+71ceaypzEOwTg02wY+Z+4bpILlqu5pTLQsKFsqc9DhxXsyixfegOTkGffbXI g2LoVYxjIptwa5L2Z5zOUezWXnQQM9hR7/KFFKYH+Ey3fpznDmbR+5i0rEd/IIJeyExkliAt oigRHczcmkB/b/oyxaVuhKI86NVMuzPebYtdhPWV8U9TRM9kGnO6CDeCXW9N18VuG7uamnu5 OAuJdFMo3X9KN4UyIFyEvOQxaoVUHh28VBUlS4qRJy6NiFWYkWsDBScmLljQUPNmK6IpJPEU hV/dKBPzpWpVksKKGApLvER02waZWBQpTUrmlLFHlAnRnMqK5lKEZI5oy14uXEwflcZzURwX xyn/pTzKwycF7XEcP3C52xioNCpm+P3MyWy9773tRPG3Bdow/my/3f4Hi5tChIEeXVd5e8I+ Te+cNdl77Mr7FVXzz9q7TiYXPHo5zX5I05ez0kbEflQ8DlKXlhRiea0nrV2fnFZ9elTj32Je 3PruK93nk7dr3taumtGdISGn+g6vsUymjXyoWpJheSYhVDJpUABWqqS/AXFNIgf/AgAA X-CFilter-Loop: Reflected X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: D7EFC8000A X-Stat-Signature: 6ezixws78huftoffkuibhqm41knp46sd X-Rspam-User: X-HE-Tag: 1749443557-364453 X-HE-Meta: U2FsdGVkX19cQiNIo9FLfaxXrvP7+WF9RpK4ID8mqDznOI4ZbHJ2KFTFJvvmqPpFDwtAVpJHIFyMMR/7m97RMxJprNgQ/YMDjmYXXyBfhBdN/zCG7aC7z3/KTKZNy+ddRN6zFdk6vMMRXdB1jQ56cdkmIURLRbehlVH0PJykB1EbILGYGPCGedK0Vqu8nfpihna1VWEufO6et/l/asS2fNbFF0ISTAHGHrxPl1a3Obp3sZ+SonRadyleX2AzPnNZWU+TygU8OQiwC9Lj5dnMrHn5HoVaDdAfDBjtRk+VHJ8GL2xenNJUmhZXizBaqqG+yE1cjku1UWfVE6Ips8LDUHMxRRz5dZ4q1TalGTMAWbWdh+v0u07aNAh2lUssRie4uW7kuk7qJ3H0DfaqWAlwRmM3pfn5Jl94EccIuHjmTmdihr6yjhTM8IlPI2lh6N49GyPfL15o8Ga6gs+4/MGHcSo/CubzSJ/zC2abgGOdesjOfDYneisKzv8d34mv97mJYdT4elq9/nmzRerykPVK0zB+mlu/vdPyqz3a7DiCqpeOkVs5Bl3lfxwnd83MwbQCEwGETXbCYMyUEs64cmFnJqZnSBcA1T3W92NxQPZyigPMyl58g3x7VvvM8b1+AvPIgA5F+VhJ/24M46piq/gRIWM54eufHm0UyEIi4LmQ03BIcHqBONrU7p9js3+3bU9so47pTkLfV7AgoYa8gyQlNFTkPEWIL/xsQFghJOJqZV39pSJiS+STfV7iKe2eSAyJ+EUluuXtPe0e/wJpgk+1bxt6PfvxT4jPu6Uucsy/JlLPWjqN9++iF7/FpqaUjDHVxR4PIJr23hpCzi6XgXlP7LDOyVhCllgqorLG9pUu20xzWQ8PP2g6kY7xhikvQRidRXJxwfn9KyoADN4Arz8nNlBgvMXkLwAhvDQjliG7bay74VeRcslbT6qtNE9itf8H//uFz1Pmfr8oN0j0NwJ TYEMV3UT VEznXn5+Xy6qRsyxhJdUW9qnzRteh5gU+IPtnxiW9el9VItLgwdTwudh1zF9LJ3EzRk3jtvlKHk1g8ns9ZEUuN1pmPMnnrhUlRz3kt46wEg6zJ99wiYI8oszw1D7QGJiTT4ftpoCqX7JR+ywMwpICYjBK/sIlLM9aY/kP+95tlW3y4jfC/HS+chx1e1AqVJ/JfoYDpEBSVeP3Z1irPLfYlnjLaX4J0YDT26/OqQtLEfzPRO+I+nvktpmPnrllom+WUsuWyVrHZAGi8Ho= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To simplify struct page, the page pool members of struct page should be moved to other, allowing these members to be removed from struct page. Introduce a network memory descriptor to store the members, struct netmem_desc, and make it union'ed with the existing fields in struct net_iov, allowing to organize the fields of struct net_iov. Signed-off-by: Byungchul Park Reviewed-by: Toke Høiland-Jørgensen Reviewed-by: Pavel Begunkov Reviewed-by: Mina Almasry --- include/net/netmem.h | 94 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 73 insertions(+), 21 deletions(-) diff --git a/include/net/netmem.h b/include/net/netmem.h index 386164fb9c18..2687c8051ca5 100644 --- a/include/net/netmem.h +++ b/include/net/netmem.h @@ -12,6 +12,50 @@ #include #include +/* These fields in struct page are used by the page_pool and net stack: + * + * struct { + * unsigned long pp_magic; + * struct page_pool *pp; + * unsigned long _pp_mapping_pad; + * unsigned long dma_addr; + * atomic_long_t pp_ref_count; + * }; + * + * We mirror the page_pool fields here so the page_pool can access these + * fields without worrying whether the underlying fields belong to a + * page or netmem_desc. + * + * CAUTION: Do not update the fields in netmem_desc without also + * updating the anonymous aliasing union in struct net_iov. + */ +struct netmem_desc { + unsigned long _flags; + unsigned long pp_magic; + struct page_pool *pp; + unsigned long _pp_mapping_pad; + unsigned long dma_addr; + atomic_long_t pp_ref_count; +}; + +#define NETMEM_DESC_ASSERT_OFFSET(pg, desc) \ + static_assert(offsetof(struct page, pg) == \ + offsetof(struct netmem_desc, desc)) +NETMEM_DESC_ASSERT_OFFSET(flags, _flags); +NETMEM_DESC_ASSERT_OFFSET(pp_magic, pp_magic); +NETMEM_DESC_ASSERT_OFFSET(pp, pp); +NETMEM_DESC_ASSERT_OFFSET(_pp_mapping_pad, _pp_mapping_pad); +NETMEM_DESC_ASSERT_OFFSET(dma_addr, dma_addr); +NETMEM_DESC_ASSERT_OFFSET(pp_ref_count, pp_ref_count); +#undef NETMEM_DESC_ASSERT_OFFSET + +/* + * Since struct netmem_desc uses the space in struct page, the size + * should be checked, until struct netmem_desc has its own instance from + * slab, to avoid conflicting with other members within struct page. + */ +static_assert(sizeof(struct netmem_desc) <= offsetof(struct page, _refcount)); + /* net_iov */ DECLARE_STATIC_KEY_FALSE(page_pool_mem_providers); @@ -31,12 +75,25 @@ enum net_iov_type { }; struct net_iov { - enum net_iov_type type; - unsigned long pp_magic; - struct page_pool *pp; + union { + struct netmem_desc desc; + + /* XXX: The following part should be removed once all + * the references to them are converted so as to be + * accessed via netmem_desc e.g. niov->desc.pp instead + * of niov->pp. + */ + struct { + unsigned long _flags; + unsigned long pp_magic; + struct page_pool *pp; + unsigned long _pp_mapping_pad; + unsigned long dma_addr; + atomic_long_t pp_ref_count; + }; + }; struct net_iov_area *owner; - unsigned long dma_addr; - atomic_long_t pp_ref_count; + enum net_iov_type type; }; struct net_iov_area { @@ -48,27 +105,22 @@ struct net_iov_area { unsigned long base_virtual; }; -/* These fields in struct page are used by the page_pool and net stack: +/* net_iov is union'ed with struct netmem_desc mirroring struct page, so + * the page_pool can access these fields without worrying whether the + * underlying fields are accessed via netmem_desc or directly via + * net_iov, until all the references to them are converted so as to be + * accessed via netmem_desc e.g. niov->desc.pp instead of niov->pp. * - * struct { - * unsigned long pp_magic; - * struct page_pool *pp; - * unsigned long _pp_mapping_pad; - * unsigned long dma_addr; - * atomic_long_t pp_ref_count; - * }; - * - * We mirror the page_pool fields here so the page_pool can access these fields - * without worrying whether the underlying fields belong to a page or net_iov. - * - * The non-net stack fields of struct page are private to the mm stack and must - * never be mirrored to net_iov. + * The non-net stack fields of struct page are private to the mm stack + * and must never be mirrored to net_iov. */ -#define NET_IOV_ASSERT_OFFSET(pg, iov) \ - static_assert(offsetof(struct page, pg) == \ +#define NET_IOV_ASSERT_OFFSET(desc, iov) \ + static_assert(offsetof(struct netmem_desc, desc) == \ offsetof(struct net_iov, iov)) +NET_IOV_ASSERT_OFFSET(_flags, _flags); NET_IOV_ASSERT_OFFSET(pp_magic, pp_magic); NET_IOV_ASSERT_OFFSET(pp, pp); +NET_IOV_ASSERT_OFFSET(_pp_mapping_pad, _pp_mapping_pad); NET_IOV_ASSERT_OFFSET(dma_addr, dma_addr); NET_IOV_ASSERT_OFFSET(pp_ref_count, pp_ref_count); #undef NET_IOV_ASSERT_OFFSET -- 2.17.1