From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35720C3600B for ; Thu, 27 Mar 2025 04:59:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0D2DC2800C4; Thu, 27 Mar 2025 00:59:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 080BE2800A5; Thu, 27 Mar 2025 00:59:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E8A782800C4; Thu, 27 Mar 2025 00:59:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CBA122800A5 for ; Thu, 27 Mar 2025 00:59:43 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 38F5C1C6F5F for ; Thu, 27 Mar 2025 04:59:44 +0000 (UTC) X-FDA: 83266128288.20.A0C7937 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf27.hostedemail.com (Postfix) with ESMTP id 4A1E64000A for ; Thu, 27 Mar 2025 04:59:42 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=jhYIQIEQ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of almasrymina@google.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=almasrymina@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743051582; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0xgWQBUgBmTnndS66WlYcMLJwKm8NVOlZEeL7gYQRGE=; b=M6GzDOWVktXyRniYJbwy2OdXiBuhFaHznLSWg3umsumoz6tKaeSKUd8PMbwGYzVTfcp96C Vg6QlDh9GDq2y8o9zi+TMx08I4rTLz5OQV6YF6NTkbgd2o6otmYBz+Y0SGxk20aAMa/lAr IVBxhHfpGWI86RWow0K4stSxuu+0Wjo= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=jhYIQIEQ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of almasrymina@google.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=almasrymina@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743051582; a=rsa-sha256; cv=none; b=BNS/d9d17rkNsc7QGmpE4xiGQ5/J6NKCJchAxHnwCipb+bT0BcFRWkibh8AzTSpHeL4nZN xQMSjM/RHI+tUaAraq8lMdWEf5OGZhL/XlmRK59yTPy2mbkQk4qG6qgbRzjogBjY35p6wZ OW+vCsrh1aAq2KkQps7D672YPXaiSSQ= Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2264c9d0295so126745ad.0 for ; Wed, 26 Mar 2025 21:59:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1743051581; x=1743656381; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=0xgWQBUgBmTnndS66WlYcMLJwKm8NVOlZEeL7gYQRGE=; b=jhYIQIEQNtKCDaPfejelYijAFUc1sy+h2ofwsoV3nPogvrYUgGssIIu1P3DIsGClvl hKZeXm9brNMYhOdhbxk/CQcTTEsZhtpOdgyht6ySp/4iqpGEZTBAs7fRQ7DQYg0uACuH 7Mw28xPWAjzGXaZUIy2OOjCc7PnTNls9/qcpLvOZTIQilGqkL06Taaa1GRAPr6XQZ2kW j4vw75lGd8nG1mNLTw27ICRa4wvIsdt1oFpqhsrO9C7MgI1ADpGA48iEFgtSPwAdUr0u rr8gpb5ssX8WIdC9GEmYpQjIXrbjzhKTeoz4Qjg3p0wLdYytPbBrrTJyADP0nW7Dc6my ++Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743051581; x=1743656381; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0xgWQBUgBmTnndS66WlYcMLJwKm8NVOlZEeL7gYQRGE=; b=mj5UnEFnhnS8LnhZ64Iz62fYD45/zQfEerHdPkhqHb6dnHyZvwR9/OXzxseHHjV7GR SIibq2GXmWqZOrZEuvDZi+5bMOkiQ9UG4p2b5iW9yk3O1KkGrUuda8zyJh6rD7nI1yJ2 Y2/VoomK2coysFBKdWDxTapIHj2I1i9SHvEwUthLX7UTciA+Q3bKD+djuqhdlGGr9KWs knmZRKtMeyEnCp+3Q5sKfiOTEI4BnhPHQNH5NFCYeD6bej+CIOYNe6KCmQ4fTWucwzd4 2hCRnfSHpZQZqUbcxydqVK6D1dKdcFBPekC6/VlBW/dlwFzQn5/PlqgV4Sd9s28X8I8S bLHw== X-Forwarded-Encrypted: i=1; AJvYcCXFZ0pyANocsNrhOQJxMfH3xeI7ZijRbg0cv1bq/hi4IXboAwY5q3CxSlWtwgQRl43J4+NrJqmpMQ==@kvack.org X-Gm-Message-State: AOJu0YxzQqxrS/EuwIRV/dYgboeR43BkdvsHUwv/6+M2VWcyZgQjawza jTkJJTkqrEGMkkMzSbzSUPiiORb10XOhk9xTWa2ysIdxZGzKKWpUPgrd+IkK8AOG+ButKuT7DsO q33rj/1JLQYWdC6NUtfX6S17HSeI0xMmsJTg+ X-Gm-Gg: ASbGncufFs7X4nHLKPsENRty/N3ssKJo0aH+f+QukkFvD/UDtHI899JZijEOpl2J747 mNDts2Q0rWbtFNZ8H34vt+8SJRuKqUlaI9zbzVKxde/PTkyjCO+ZheqeQsxkboC9FtmJN0L2Ara 0+b3ZKt0TEdsztPCVXEon3zQyqVhM= X-Google-Smtp-Source: AGHT+IF0s/GL6MGZcOQirhOil9Z2LYOTGQkcFqDLrlKFFE7owCzr/ZTwokplwNC7ei/E8n2Q7ElvPfvNpzACxJtepBE= X-Received: by 2002:a17:902:d68b:b0:215:f0c6:4dbf with SMTP id d9443c01a7336-22806bdf5demr1524155ad.14.1743051580749; Wed, 26 Mar 2025 21:59:40 -0700 (PDT) MIME-Version: 1.0 References: <20250325-page-pool-track-dma-v2-0-113ebc1946f3@redhat.com> <20250325-page-pool-track-dma-v2-3-113ebc1946f3@redhat.com> In-Reply-To: From: Mina Almasry Date: Wed, 26 Mar 2025 21:59:27 -0700 X-Gm-Features: AQ5f1Jo9fgAhoERMV5502HOGec-jhjpS6HzgrC5Fcd3htnByWUcvPEsIqXM0Lu4 Message-ID: Subject: Re: [PATCH net-next v2 3/3] page_pool: Track DMA-mapped pages and unmap them when destroying the pool To: Yunsheng Lin Cc: Saeed Mahameed , =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , Leon Romanovsky , Tariq Toukan , Andrew Lunn , Eric Dumazet , Paolo Abeni , Ilias Apalodimas , Simon Horman , Andrew Morton , Yonglong Liu , Pavel Begunkov , Matthew Wilcox , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org, Qiuling Ren , Yuying Ma Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam01 X-Stat-Signature: twox6iwnnnecgocff9a3wz1g71mfzp3g X-Rspam-User: X-Rspamd-Queue-Id: 4A1E64000A X-HE-Tag: 1743051582-490081 X-HE-Meta: U2FsdGVkX1/dVCwgshywpVWU6j8malYMyPnvEN2qZdZAluBgw+WFyPw4XoDrfAw5yLnhER3zt2pAsl3bwYtUHlhynvEBPvDdWyqz9TxB66SrD+pQQfw1zyvXkoqzdrMU8XCHGCm0V/s3fgopRIe1xhvWYk5phXJG4jhyF9ofal9Ab+If6lJSTlIWzQxi7EmnB7AwZvMs3jwfB5jqF5TXqnrQd0tUPjMkq4vnpnFtOVqoFah+pKULanOsdxdCiP3oUq0VNq//royRfCuPyEIeT278LjE6wxb4OxrlVbBcf0KXVn2ZMjsnXoQyY13UvHfRTQNujEOBll464zUzr4/H20XpmlBvuLgsmhagMcdLNXtBBTqm+n7FhJuwyDlMZdIyFdMYsMS47A1ME+pfTturQON5SFmfFkbz1BJS6uj/b65KhchmO0yG54uNZyA9rrL+KCOWgF/l8wE1F50OoOetY57Lo6SR/mY+vkxa2py/t6hzElX1HF2uoPLBADiI8SiHCCngyJNFICFgOvOc7FbEwz1eaCLOVBYX+HsJKqDyQq2P/b3hVK/k2LeqLWz+fXcbqFWh9V3IiGmAj0sLuCD/Pc2MgUE5MsLaxroThBmykCSjaYPM4mW8jkmxcLiiVdpMWzfBXI0bLcwVkBoSHfPtn3V+rm/rpGp0wfHg9yxXh88CBSuYqQWGLrJZMPR1Qwmn1JIA8jSfWVCwpVo5VaApDhAZRa8sm4O34IIQCKIgL5KNaeyABfhvUKnEYqqyG4hVWoiMnWhvjKtmnYedwd9/2Hjv1v/3yAMdKUrcg1glu8PDNS/mkffuLIpaSSUnRD+bcXzvPfN2vfMemDF0ZSg8JaSNEwYSTEElpAyS0Xs3hh4Mkqiy6KKiuz7WXxaHMmwWNgkJ2GfP4KOQkuLj5XU2YkMh0enJyLGoNgZx0ILde35zVeO31NFPRcDvdb8ywWDbTUJ/JBo4vhkfmihFMJ1 anfnX2aH 0tTtldDXBshm4Vc1cMv9E4KfRu6McDAJfZfit5QCSh24yaWnOuQ+3XZW3LwG6953bruhrbL8Ph2bbp/vhBlNCFPo8ZEZzPJeGXDbg7v8GF2o3MMcPiGilZx5oI82qPpWiNfk8bM+GVKCfxGSoS2rDqrasKrjNILlGj0dFByUdaeHx6U7lqvm85kedbXjThcZiEE7L4E1glKj/+XzlGgfveYyp9fsbHxfbzbqjIr+Eb9szJ/i7Uj5eMRtQBFJeKt9eNHBRCY9fyg40zhxVDGrkVXdULPSBnH0FloenK6QyBZPlqd5x5sClbEnGYInoBRENnIH1FQbWTP5Vwm8rW/zS+mT/E0RZDV9sfqJW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Mar 26, 2025 at 8:54=E2=80=AFPM Yunsheng Lin wrote: > >> > >> Since all the tracking added in this patch is performed on DMA > >> map/unmap, no additional code is needed in the fast path, meaning the > >> performance overhead of this tracking is negligible there. A > >> micro-benchmark shows that the total overhead of the tracking itself i= s > >> about 400 ns (39 cycles(tsc) 395.218 ns; sum for both map and unmap[2]= ). > >> Since this cost is only paid on DMA map and unmap, it seems like an > >> acceptable cost to fix the late unmap issue. Further optimisation can > >> narrow the cases where this cost is paid (for instance by eliding the > >> tracking when DMA map/unmap is a no-op). > >> > > What I am missing here, what is the added cost of those extra operation= s on > > the slow path compared to before this patch? Total overhead being > > acceptable doesn't justify the change, we need diff before and after. > > Toke used my data in [2] below: > The above 400ns is the added cost of those extra operations on the slow p= ath, > before this patch the slow path only cost about 170ns, so there is more t= han > 200% performance degradation for the page tracking in this patch, which I > failed to see why it is acceptable:( > You may be correct about the absolute value of the overhead added (400ns), I'm not sure it's a 200% regression though. what time_bench_page_pool03_slow actually does each iteration: - Allocates a page *from the fast path* - Frees a page to through the slow path (recycling disabled). Notably it doesn't do anything in the slow path that I imagine is actually expensive: alloc_page, dma_map_page, & dma_unmap_page. We do not have an existing benchmark case that actually tests the full cost of the slow path (i.e full cost of page_pool_alloc from slow path with dma-mapping and page_pool_put_page to the slow path with dma-unmapping). That test case would have given us the full picture in terms of % regression. This is partly why I want to upstream the benchmark. Such cases can be added after it is upstreamed. -- Thanks, Mina