From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BCFAC54FB3 for ; Fri, 30 May 2025 01:10:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A9E146B007B; Thu, 29 May 2025 21:10:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A4FB06B0082; Thu, 29 May 2025 21:10:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 919576B0083; Thu, 29 May 2025 21:10:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7123C6B007B for ; Thu, 29 May 2025 21:10:13 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 32F5A5FECC for ; Fri, 30 May 2025 01:10:13 +0000 (UTC) X-FDA: 83497793106.02.25AE155 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf20.hostedemail.com (Postfix) with ESMTP id 709711C0005 for ; Fri, 30 May 2025 01:10:10 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748567411; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fLi/JhvGigU23+YmR9kz4QPEgNsoZUG+aWIz9S3rf/Q=; b=PqX8f2m7ZlNG/E38veEZE0f7NCSGO+MQyeHq+OmTYQdujK8tyvV4TCehxiOP69zkSWUcgm ie67ERDgkLBF/TuweWPxx1Z5r95wLWMeadQ3PDOOu3dB1WXmb+VIRYq99DxrHTGnzTCrKt scioWfNmo78ZFDmSnilD9qOylYV8Meg= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748567411; a=rsa-sha256; cv=none; b=AmKf05E2jO6HAIwDRCrHdXQLqmQuWCJuSJGXFqzSrJLVWAf5SyI7cGRpTh5PP0CB48to9c DfoUVynw9P2z5zJhwlmkOD6T4+YOHkw8PgaF2Mgcfwr1tTAfnvlP5+6vCVR+2KGTHei0D+ E5sobzLBWV7aRTaG8KZhnrStCjEgM3Q= X-AuditID: a67dfc5b-669ff7000002311f-eb-6839056f5554 Date: Fri, 30 May 2025 10:10:02 +0900 From: Byungchul Park To: Mina Almasry Cc: willy@infradead.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, ilias.apalodimas@linaro.org, harry.yoo@oracle.com, hawk@kernel.org, akpm@linux-foundation.org, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, asml.silence@gmail.com, toke@redhat.com, tariqt@nvidia.com, edumazet@google.com, pabeni@redhat.com, saeedm@nvidia.com, leon@kernel.org, ast@kernel.org, daniel@iogearbox.net, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, horms@kernel.org, linux-rdma@vger.kernel.org, bpf@vger.kernel.org, vishal.moola@gmail.com Subject: Re: [RFC v3 01/18] netmem: introduce struct netmem_desc mirroring struct page Message-ID: <20250530011002.GA3093@system.software.com> References: <20250529031047.7587-1-byungchul@sk.com> <20250529031047.7587-2-byungchul@sk.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Brightmail-Tracker: H4sIAAAAAAAAA02Sa0iTYRTHefZe9jodva7bo5LRouxqGWJHCrUger5kdvkgCtVLvrTlnDEv aRGstJs5u2iSc8Yq71mLZTqtpJaYYmooxeyislQIynCapXZziuS3H+f8z/mdD4ejFLcZX06t TRZ1WkGjZGW07KvXrfWJTKhqY/3vUDBZqli4+zMNyvpsDJgqaxCMjr+XwkjjSxbu3BqjwNSR ScN3ywQFA01OKfSWDtLw5HwtBc7LzSwYMicpOGMrl8DrmhwG8iZKKKjV90mhq97EQk/VXwYG 7QYaWowVNPTmRECTeRGMtX5B0GiplcBYdhELuZ1mFj5l9iLofOGkofB0DgJLg4OByZ8mNmIZ qa7olpA640cpMVtTyMPyNSTL0UkRa+VFllhd16Tkw9snLGm+MUmTOtuIhBgyhlgyPPCOJt8a 3rDEUv2GJq/MjVIyYvWP4mNkW+NEjTpV1G0IOyRTvejQHrOtSmt53kbp0We/LOTBYT4YXyqw U7Pc1vOUdjPNr8Alj/Klbmb5AOxwjE9nFvCrcXHDVcbNFN/L4HbTUTfP56Nx+b0miZvl/Gbs KrRN5WWcgi9F2Pp5QjrT8MYtBf30zHAA/nWzcyrETbEfLvvDzZSX4oxHhdMuD34Ptrqyp10L +eX4Wc1LiXsn5m0cLmqfRDNH++Dn5Q76CvI2zlEY5yiM/xXGOQozoiuRQq1NTRDUmuBAVbpW nRZ4ODHBiqY+p/TUr1gbcr3eZ0c8h5Re8o1hoFIwQmpSeoIdYY5SLpCfCQ9RKeRxQvoJUZd4 UJeiEZPsyI+jlYvlm8aOxyn4I0KyGC+Kx0TdbFfCefjq0bb761KcOdcHi5/GdwVt8onJ3am3 eD+uDzKEh+tlkRmyisie5A5yLcu//lxzyKK84bVOWXdU06Et6q2Bu5O9YrdUMuNn8+t+1HSz e1dFR/kSLni0df+urnzPofuaEGHljowHrnPD/T0+88RtIYFhQtHJt8uN1Z4HLhicPkdCfbcv UdJJKiFoDaVLEv4BTbFk7DUDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA02SfUzMcRzHfX9P9+t0/FzJV9lsZ6SM0jx80Oif+M0wxjCz1eHHna7L7up2 sSwVpnVXyYxfl46USpydHq6WxpXUUFZK5SFKxmah0no4D13N9N9rn/f78/r882FJ+Vnal1Vr YwWdVqlRMFJKumND8vIYep0q+J55CVhsJQzcHjHCrfcOGizF5QiGRl9LYLDuCQN514dJsDSn UPDTNkZCX32PBLoLPlFQfb6ChJ70BgZMKeMkJDkKCajNaaThRbmZhktj+SRUJL6XQGuVhYF3 JX9o+OQ0UdAoFlHQbQ6DeqsPDD/9iqDOVkHAcFoOA1ktVgZ6U7oRtNT2UJB9xozAVtNBw/iI hQlT8KVFnQRfKb6V8FZ7HH+/MJBP7WgheXvxBYa3D1yU8G/aqxm+4co4xVc6BgnelNzP8D/6 uij+W00bw+d9/k7wttI2in9mrZPsnHNAGnpE0KgNgi5oY6RUVdusPeFYamx89JxMRF/8UpEH i7lV+Pm7B5SbKW4xzi+7LHEzw/njjo5R0s3eXAC+WZNJu5nkumncZDnuZi9uPy68U0+4Wcat xQPZjom+lJVzBQjbv4xJpoI5uPHqR2pq2R+7rrVMlNgJ9sO3frNT44U4uSx78pYHtwvbB9Im b83lFuGH5U+IDDRLnGYSp5nE/yZxmsmKqGLkrdYaopVqzeoV+ihVvFZtXHE4JtqOJp6jIMGV 6UBDrVuciGORwlMG4aCS00qDPj7aiTBLKrxlSZvWqOSyI8r4k4IuJkIXpxH0TuTHUop5sq37 hEg5d0wZK0QJwglB9y8lWA/fRBTR1Ls399eCsLejLmUT2fU4tdaQn5Lt5/XhXHq7cdjHvnQr cWNPVnjn7uvHjVnr++9eFtWmbbnkK2f9o7iEgwtjc+dXJfzyDNpzNNi8rqgrLvNQqKs4ZM3m GZtnhseHzY7WZVTKxaFFP1+K27FnW1p6yLaADZq1rlLDjGWnvKpMp6MUlF6lXBlI6vTKvx4l hL8YAwAA X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 709711C0005 X-Stat-Signature: do9k8qg7yuahyf41gxm9shtrsk8q1cqr X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1748567410-660244 X-HE-Meta: U2FsdGVkX19hCcBpE5SF8EGtYWN+5QyDhG8TwuBcBmhpXYps3kHqvoR6R/F0QRuk6nhKGG/fKO1Joi4SYH5bMOZs3trWXltzwiCDAGf8qp5l/A8tmi50KXsa0/8KsxQDvIvyOXslqaSqIct48HhDN9qugneBKzTNK9Xeyz6ucoU14V3uwSFOj4HbyVruXQ80inwcdZZaBmVZQNgyu+4DQEOsM+C23VcpyTVvvZuMRgt65ClF1la9OpQh4QZfxPK1rZc6kUV/LSpFMDf05NZY92kmmmQXhGw14PjS6d1eH5whW1VeimNfgRbS/oLGL374qJpnSjx6lSIGxlXoDUhZQ3dHJvNCPkDjuZbpb32AG6rPqYXbUWUdtsYC4TMod1wiQhSeKN8FxTu8FMco1IxN+Lpvf66GFsoaguCfWdJoN7m/aMBAFEHOTDMggNCPDA5wH/BbVMzNeOYk5BWCQ3QWfL00nMlOoLfkJBqyqNKkS5nJO11nhtyKnxVvezDUq1jJMn+pXqlFuWbH+Su5OoBjaox4Q/03ZMvUj+OVGJyjApGrnDL9jFXIVeVP4qGZnnET2ch7kEVYghH1VtCbSnrAy7zqTG5jmHB/vY5kHRpGj09aqBjUUoJmnFYeaO9G0xuQMtk4uuCTtV5HqPf1E0fSCwhSoiKVH8OwTzN7zKWoT+uxLRPvSHkuMx4w8jngV2tAC0/dSdxXAua3MKIuR+Bu4vvsyeR+MWIBz5rgSTtB5miz21zJ0h902bXSL5Kmcz+TQzHgos09EgfcUm7jJVYeVi1Yqy7ca3CAZwIHe0HGPVaOxOJ4Tx9VT9huiDHUW/wezVCX7S/Kkykow0jWqLIM2qjuWvy2pzFwVKCaEvTTEcrrWlYv0qIB9UH/oa6wF+tLOWJxxtnaSzGfeLIBDf7o81SkYTktiFajRF5XYVAJ0D96HKFXdfGpRMzBh1t/mcC2L++g7oazDLvyqxoAfC5 O6JTcoVH bWH9sbJWZ1qOHDY7u0DDJJ7dMjiQoZgc4I+ep2DEtmdJqHHv+Sdas9eOfPtgAX2WJDn7b06/l13VQGUb3mmhZ5bS/yTbYC5sNLcpQF3PguX8L6rboNu+T8vwBW8oLhYvR0QbN+22GGw50y41R1H3j5Y/G0Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, May 29, 2025 at 09:31:40AM -0700, Mina Almasry wrote: > On Wed, May 28, 2025 at 8:11 PM Byungchul Park wrote: > > struct net_iov { > > - enum net_iov_type type; > > - unsigned long pp_magic; > > - struct page_pool *pp; > > - struct net_iov_area *owner; > > - unsigned long dma_addr; > > - atomic_long_t pp_ref_count; > > + union { > > + struct netmem_desc desc; > > + > > + /* XXX: The following part should be removed once all > > + * the references to them are converted so as to be > > + * accessed via netmem_desc e.g. niov->desc.pp instead > > + * of niov->pp. > > + * > > + * Plus, once struct netmem_desc has it own instance > > + * from slab, network's fields of the following can be > > + * moved out of struct netmem_desc like: > > + * > > + * struct net_iov { > > + * struct netmem_desc desc; > > + * struct net_iov_area *owner; > > + * ... > > + * }; > > + */ > > We do not need to wait until netmem_desc has its own instance from > slab to move the net_iov-specific fields out of netmem_desc. We can do > that now, because there are no size restrictions on net_iov. Got it. Thanks for explanation. > So, I recommend change this to: > > struct net_iov { > /* Union for anonymous aliasing: */ > union { > struct netmem_desc desc; > struct { > unsigned long _flags; > unsigned long pp_magic; > struct page_pool *pp; > unsigned long _pp_mapping_pad; > unsigned long dma_addr; > atomic_long_t pp_ref_count; > }; > struct net_iov_area *owner; > enum net_iov_type type; > }; Do you mean? struct net_iov { /* Union for anonymous aliasing: */ union { struct netmem_desc desc; struct { unsigned long _flags; unsigned long pp_magic; struct page_pool *pp; unsigned long _pp_mapping_pad; unsigned long dma_addr; atomic_long_t pp_ref_count; }; }; struct net_iov_area *owner; enum net_iov_type type; }; Right? If so, I will. > > struct net_iov_area { > > @@ -48,27 +110,22 @@ struct net_iov_area { > > unsigned long base_virtual; > > }; > > > > -/* These fields in struct page are used by the page_pool and net stack: > > +/* net_iov is union'ed with struct netmem_desc mirroring struct page, so > > + * the page_pool can access these fields without worrying whether the > > + * underlying fields are accessed via netmem_desc or directly via > > + * net_iov, until all the references to them are converted so as to be > > + * accessed via netmem_desc e.g. niov->desc.pp instead of niov->pp. > > * > > - * struct { > > - * unsigned long pp_magic; > > - * struct page_pool *pp; > > - * unsigned long _pp_mapping_pad; > > - * unsigned long dma_addr; > > - * atomic_long_t pp_ref_count; > > - * }; > > - * > > - * We mirror the page_pool fields here so the page_pool can access these fields > > - * without worrying whether the underlying fields belong to a page or net_iov. > > - * > > - * The non-net stack fields of struct page are private to the mm stack and must > > - * never be mirrored to net_iov. > > + * The non-net stack fields of struct page are private to the mm stack > > + * and must never be mirrored to net_iov. > > */ > > -#define NET_IOV_ASSERT_OFFSET(pg, iov) \ > > - static_assert(offsetof(struct page, pg) == \ > > +#define NET_IOV_ASSERT_OFFSET(desc, iov) \ > > + static_assert(offsetof(struct netmem_desc, desc) == \ > > offsetof(struct net_iov, iov)) > > +NET_IOV_ASSERT_OFFSET(_flags, type); > > Remove this assertion. I will. > > > NET_IOV_ASSERT_OFFSET(pp_magic, pp_magic); > > NET_IOV_ASSERT_OFFSET(pp, pp); > > +NET_IOV_ASSERT_OFFSET(_pp_mapping_pad, owner); > > And this one. I will. > (_flags, type) and (_pp_mapping_pad, owner) have very different > semantics and usage, we should not assert they are not the same > offset. However (pp, pp) and (pp_magic,pp_magic) have the same > semantics and usage, so we do assert they are at the same offset. > > Code is allowed to access __netmem_clear_lsb(netmem)->pp or > __netmem_clear_lsb(netmem)->pp_magic without caring what's the > underlying memory type because both fields have the same semantics and > usage. > > Code should *not* assume it can access > __netmem_clear_lsb(netmem)->owner or __netmem_clear_lsb(netmem)->type > without doing a check whether the underlying memory is > page/netmem_desc or net_iov. These fields are only usable for net_iov, Sounds good. Thanks. Byungchul > so let's explicitly move them to a different place. > > -- > Thanks, > Mina