From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9DADC28B28 for ; Wed, 12 Mar 2025 12:27:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E2CD280002; Wed, 12 Mar 2025 08:27:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 19287280001; Wed, 12 Mar 2025 08:27:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05C0A280002; Wed, 12 Mar 2025 08:27:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D703F280001 for ; Wed, 12 Mar 2025 08:27:21 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id AB17680FC3 for ; Wed, 12 Mar 2025 12:27:23 +0000 (UTC) X-FDA: 83212824366.16.FEED81E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf16.hostedemail.com (Postfix) with ESMTP id 62524180011 for ; Wed, 12 Mar 2025 12:27:21 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IXiuzo2C; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf16.hostedemail.com: domain of toke@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=toke@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741782441; a=rsa-sha256; cv=none; b=Wcqr3G+9bWIeFZksvWY2fNxXVZujfvEe2y0ynfOLF8RdPL26v23YNX+PDr5aE+yBnSqRTx nnjGbXosGnaN1+suY51aF0mM/fLsxerXEmJJtH92Mz+uBU4MgdBrGovwaqG9UqlM6D40Zq BOzn1hXdlMtlDmHfMX26l+Ii7uXf15w= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IXiuzo2C; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf16.hostedemail.com: domain of toke@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=toke@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741782441; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3wRnOa+u5sRbwZ9+eMAIK9ISj9z0+KJn8ao3601bMZ0=; b=KWIXjtSuvL2SVXIGjZRPJndCXlQDZ4G1w3LV1f4Aq+h6KIp7CQJ0ZDOCscTF2+Hq9Bq8gi 9klMWNX5V7F8EPEK428q73dFaM8j1x2B5S06ZBw7qvhqEEyXHrQVKbFhWQSZMan6Xnr1Tx uSCmNuWoc3oDsNPgHCpU40XkDUBTcoQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741782440; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3wRnOa+u5sRbwZ9+eMAIK9ISj9z0+KJn8ao3601bMZ0=; b=IXiuzo2CBCYuSJiq1+7nf+O1V7kxbTVortoscKFyWnpkjGyAM4QKvM0j5Wr4wXCUYAr9Cy qLYSD364Buu/uBsUv7vW3XbrY5gbcCxsUnPpR2n+tFmjE5ZLZwkUoE493yJtMBQiuXw5e3 UwmzPU6La8DRYQeYovSK5B7ymhFbH9Q= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-680-Tc_B2vQUMX-ezI8BBqzqMQ-1; Wed, 12 Mar 2025 08:27:19 -0400 X-MC-Unique: Tc_B2vQUMX-ezI8BBqzqMQ-1 X-Mimecast-MFC-AGG-ID: Tc_B2vQUMX-ezI8BBqzqMQ_1741782438 Received: by mail-ed1-f72.google.com with SMTP id 4fb4d7f45d1cf-5e6eb748b17so4042376a12.3 for ; Wed, 12 Mar 2025 05:27:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741782438; x=1742387238; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3wRnOa+u5sRbwZ9+eMAIK9ISj9z0+KJn8ao3601bMZ0=; b=shBVz1RO9BeWlYC+IkDKappMJuSemCghi6yNAdnW+z8UeGLmOojJw1G2n7P5Mh07U4 TELa0/pshtjmzb1N8FAMBSGLCjr0MoWK+jNWkfuAQZEvrGimUsdK8YpAV8hMuoFkIJqF mCRs3RFBYl3Y64d/1WHCp/KQGbDCvL9n1v6JBt3vrDbzJyu2leTSwyl+JFTU/qCnJZdZ U1ugWMU4+tllL7YH/293ULO/isWcFUk5zUG9MJlG4gK8B491c0PVTEO4YUzYeABxGZj9 m6FVKXEJsZby+J9+q1faRFLUROJBn0YoLNef5oniwglJ+adkSqXKwtqumOs3bmHZIGwb 1UiA== X-Forwarded-Encrypted: i=1; AJvYcCWYWgokG6G8cdnomF0rpSPmfyJoJSFx8HJ9YzhozM9VYGdx2ka/vd5dBrCypjrzXGLJD5ECDb2AJw==@kvack.org X-Gm-Message-State: AOJu0Yzr394sn2WFSMH5SsKU25EsGIIv35Mu2XcK2A/+K+bD5zUw+3V5 YpU+sOgo3JkRaogVVvSF+lWifOfc2gUTnICmXP7cPuf4DS70TD7xCbU0mwZe+6iFmuMhSFin3Wt x/qHaMDr715VTgUtvNwuJX067yfHuIRH9b4yROA5QlVqmEm+A X-Gm-Gg: ASbGncsphdYrXMBXFt74aT3zeyxLyidorKhX79ybrJcmJgoam7zqLmzFEhweD4ms/zM XoyPgKYVEedZhnEvp6n1yGda9OuctXuTQi6hJ9ExG+PcvzDMvsYn4/4gzmyye1aHIUXbmRYQ9Bo QCywzkX0fMTBDYPKk01p65GLQHmjxFMGqxXdViE5byGuhgmlYIXc0H7SG9iHCkwUwHelkINWllV gHhY6ngaOPYQ+7jUR7BurpeikPjLvdl6VVnZtRrEax5jh0juldrNvS/kGDOurvDklPFjxyvxmUk pxdewHtpphMf X-Received: by 2002:a05:6402:43cd:b0:5e0:4276:c39e with SMTP id 4fb4d7f45d1cf-5e5e251190cmr29052220a12.30.1741782436426; Wed, 12 Mar 2025 05:27:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGVG8B9kssrNX/dypmUvhA8C/yTo7s4UTkpGrLBmy1jM7OUguW6oG4MjZt+HeexMG1YDArA4A== X-Received: by 2002:a05:6402:43cd:b0:5e0:4276:c39e with SMTP id 4fb4d7f45d1cf-5e5e251190cmr29052168a12.30.1741782435950; Wed, 12 Mar 2025 05:27:15 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5e5c74721c3sm9767369a12.22.2025.03.12.05.27.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Mar 2025 05:27:15 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 314FD18FA68A; Wed, 12 Mar 2025 13:27:14 +0100 (CET) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= To: Yunsheng Lin , Yunsheng Lin , Andrew Morton , Jesper Dangaard Brouer , Ilias Apalodimas , "David S. Miller" Cc: Yonglong Liu , Mina Almasry , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , linux-mm@kvack.org, netdev@vger.kernel.org Subject: Re: [RFC PATCH net-next] page_pool: Track DMA-mapped pages and unmap them when destroying the pool In-Reply-To: References: <20250308145500.14046-1-toke@redhat.com> <87cyepxn7n.fsf@toke.dk> <2c363f6a-f9e4-4dd2-941d-db446c501885@huawei.com> <875xkgykmi.fsf@toke.dk> <136f1d94-2cdd-43f6-a195-b87c55df2110@huawei.com> <87wmcvitq8.fsf@toke.dk> X-Clacks-Overhead: GNU Terry Pratchett Date: Wed, 12 Mar 2025 13:27:14 +0100 Message-ID: <871pv2igd9.fsf@toke.dk> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: jtOcVW0bwtN2KS0XK-EpZYR_SwMPhySCz-bDHWzyhWw_1741782438 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Queue-Id: 62524180011 X-Stat-Signature: s8usjkn4wxfa3cb5ju71jmfxchaicf5q X-Rspamd-Server: rspam06 X-HE-Tag: 1741782441-657969 X-HE-Meta: U2FsdGVkX1/hJMmthbPXVtYI7JbOaIWsQzo8n413s2hHqpN2C+Qqe1Mdf2pYADlMrPh9nOFp5+M+YRd0TdwCeGPqXoRIYT1b9SL+wmcHoWowUKvgo5tz5MoSsJqdCsqByRUie2LL/qEvPHWwAU0O7fD5ZheQn/hJ/O+7kkXjgGgGTnZfszVvrVLJxKlPMd287YrofpnjkTIdbKtFxErIscaKTupN40SXGqXubc8przDnT8lZT+RQXU5quAiWib/eftsKB0C8K9JhCPFndUiHnW0yG20E9fUxv0VQNHuYh3eZM+wUTxz6ajil2vS/jOpkKHpQNXC7G9uw010zlCMURgF+FsI/8SABBClbFwygQDR/xk20mExdTnjrC9S1H2DtYLZduoPa82pQAEKqmGQ9N+kU++T0Fe6Mth/YHxn1TsYiXwnT4UJdZ2kkgXn0TWCV7XMo4itQk9IlzAOLzoYVHMIS4vHzQCpghgKrHhEy9LKUOoO5ro4Ed9FaYYs8M1vjDJwWcWLp8vw0Fgs8ziZjBfo3/foieeW/UocNNmALk5C1Ae9Wht2GMpR/wBjH2oaFu2sM9ZXFpoIS5pPAb9RCeq6tZUphRmFSi+F9/I9d8feDFWGiAOY458uw5EwbaCPDac6NGTJYZRt0+qAefDSWrb57CJ9a/pXkGIDVm4ycJgnzaLZGjKY1nuT5FZX6B94QNshRP5TPDsal/R+qXQrVycqsbBqOAT3hUhybbv0focfql/qt+LafaPRkRZGwdcx+ltLL4nrDJTCfY8Ul0vBIhOihuS63gni1Igal7UqhXt+uv5l/j7Dette9k+IzCaO7q96DXx7X0QbKQBE9Pt/+8zyopV76Ieo4hb26av6uTWdqt7D8Q+HCyoalS5D0Nz2IXSRFLoKyuS+42jUWz/5y2GEd7npoPWFDNHLcSNpBEol4OPYx0mFOa0K3mGbYgb7fjWoMZAhqvZiRQw74eyR t8AwMZTS qYBCWwBITDGq2O7/c4GoCk4+rKSmlh6Fyw/PV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Yunsheng Lin writes: > On 2025/3/11 21:26, Toke H=C3=B8iland-J=C3=B8rgensen wrote: >> Yunsheng Lin writes: >>=20 >>> On 2025/3/10 23:24, Toke H=C3=B8iland-J=C3=B8rgensen wrote: >>> >>>>> >>>>> I guess that is one of the disadvantages that an advanced struct like >>>>> Xarray is used:( >>>> >>>> Sure, there will be some overhead from using xarray, but I think the >>>> simplicity makes up for it; especially since we can limit this to the >>> >>> As my understanding, it is more complicated, it is just that >>> complexity is hidden before xarray now. >>=20 >> Yes, which encapsulates the complexity into a shared abstraction that is >> widely used in the kernel, so it does not add new complexity to the >> kernel as a whole. Whereas your series adds a whole bunch of new >> complexity to the kernel in the form of a new slab allocator. >>=20 >>> Even if there is no space in 'struct page' to store the id, the >>> 'struct page' pointer itself can be used as id if the xarray can >>> use pointer as id. But it might mean the memory utilization might >>> not be as efficient as it should be, and performance hurts too if >>> there is more memory to be allocated and freed. >>=20 >> I don't think it can. But sure, there can be other ways around this. >>=20 >> FWIW, I don't think your idea of allocating page_pool_items to use as an >> indirection is totally crazy, but all the complexity around it (the >> custom slab allocator etc) is way too much. And if we can avoid the item >> indirection that is obviously better. >>=20 >>> It seems it is just a matter of choices between using tailor-made >>> page_pool specific optimization and using some generic advanced >>> struct like xarray. >>=20 >> Yup, basically. >>=20 >>> I chose the tailor-made one because it ensure least overhead as >>> much as possibe from performance and memory utilization perspective, >>> for example, the 'single producer, multiple consumer' guarantee >>> offered by NAPI context can avoid some lock and atomic operation. >>=20 >> Right, and my main point is that the complexity of this is not worth it = :) > > Without the complexity, there is about 400ns performance overhead > for Xarray compared to about 10ns performance overhead for the > tailor-made one, which means there is about more than 200% performance > degradation for page_pool03_slow test case: Great, thanks for testing! I will include this in the non-RFC posting of this series :) >>>> cases where it's absolutely needed. >>> >>> The above can also be done for using page_pool_item too as the >>> lower 2 bits can be used to indicate the pointer in 'struct page' >>> is 'page_pool_item' or 'page_pool', I just don't think it is >>> necessary yet as it might add more checking in the fast path. >>=20 >> Yup, did think about using the lower bits to distinguish if it does turn >> out that we can't avoid an indirection. See above; it's not actually the > > The 'memdesc' seems like an indirection to me when using that to shrink > 'struct page' to a smaller size. Yes, it does seem like we'll end up with an indirection of some kind eventually. But let's cross that bridge when we get to it... -Toke