From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78673CF6491 for ; Sun, 29 Sep 2024 02:45:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 026436B010F; Sat, 28 Sep 2024 22:45:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EEEC86B018B; Sat, 28 Sep 2024 22:45:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D690D6B018C; Sat, 28 Sep 2024 22:45:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id AD0FC6B010F for ; Sat, 28 Sep 2024 22:45:02 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 27CEFA10A4 for ; Sun, 29 Sep 2024 02:45:02 +0000 (UTC) X-FDA: 82616233644.12.FE164CE Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf19.hostedemail.com (Postfix) with ESMTP id 67F1A1A0006 for ; Sun, 29 Sep 2024 02:44:58 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf19.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727577862; a=rsa-sha256; cv=none; b=rtuWeeocUlL4s+51F1MLQeNYJ67VzkQjimbwRxh9+n0Ady6/cixSCEa0EKvo6Ne7cBykU1 H0AXLgtAmVY0r8SKqZLegAXTGlutG7Up1hKZuMypJPys1tA94Q2czSm5EQFA17/5qVqBpZ Cm4/EGk/QrIFyncV51Zb50hZcOEYqjc= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf19.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727577862; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2q/9Xq3iyIg0BIaKZgbLzqV+w3imPdbeRr3Vi8phnk0=; b=48MXiW6W3SUWgrDEJHW7SdyUDdY2xKBNEGDIAzG129NVuQNWz2qlZDr6g2on4sFnw3hRuY 7DI5p4ZVF5iyNZavRQyhx0IoVCmceXdIKlLR7okrUoUROohmOszogXvHD9pTITwT+6fc6s M9JyQyDzioaEkj2515CyTb265nz90i0= Received: from mail.maildlp.com (unknown [172.19.163.48]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4XGT571jzCzfbjm; Sun, 29 Sep 2024 10:42:35 +0800 (CST) Received: from dggpemf200006.china.huawei.com (unknown [7.185.36.61]) by mail.maildlp.com (Postfix) with ESMTPS id 219E918007C; Sun, 29 Sep 2024 10:44:54 +0800 (CST) Received: from [10.67.120.129] (10.67.120.129) by dggpemf200006.china.huawei.com (7.185.36.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Sun, 29 Sep 2024 10:44:53 +0800 Message-ID: Date: Sun, 29 Sep 2024 10:44:53 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH net v2 2/2] page_pool: fix IOMMU crash when driver has already unbound To: Ilias Apalodimas CC: Mina Almasry , , , , , , , Robin Murphy , Alexander Duyck , IOMMU , Wei Fang , Shenwei Wang , Clark Wang , Eric Dumazet , Tony Nguyen , Przemek Kitszel , Alexander Lobakin , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , Felix Fietkau , Lorenzo Bianconi , Ryder Lee , Shayne Chen , Sean Wang , Kalle Valo , Matthias Brugger , AngeloGioacchino Del Regno , Andrew Morton , , , , , , , , , , References: <20240925075707.3970187-1-linyunsheng@huawei.com> <20240925075707.3970187-3-linyunsheng@huawei.com> <842c8cc6-f716-437a-bc98-70bc26d6fd38@huawei.com> <0ef315df-e8e9-41e8-9ba8-dcb69492c616@huawei.com> <934d601f-be43-4e04-b126-dc86890a4bfa@huawei.com> Content-Language: en-US From: Yunsheng Lin In-Reply-To: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.67.120.129] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemf200006.china.huawei.com (7.185.36.61) X-Rspam-User: X-Rspamd-Queue-Id: 67F1A1A0006 X-Rspamd-Server: rspam01 X-Stat-Signature: xdrzjqy8gnzyhyzw3yfunnua5cx5r9qt X-HE-Tag: 1727577898-207977 X-HE-Meta: U2FsdGVkX1+7aqUt4oTxkQcEqXveiIQNDijRR+Tu2sBGYpBzEiMLDw4NL9UxjQWl7P3upgHFLGkDbYHgBRYd29nGsVeuNHt6A5X+8bqU71HUM2tJqhg5mwSBhLH5z3Aj+k72z3qvQ8Sd30ATeiDQ4sbnRii20IpejGwfbLUDbU5decsnSo/he+JKkvo+1vb3/Xg+dpy8mn5rFvJmkPyUGgqslrgWcft1isf8ZsyavlmvxNcliCdpquOM3B+3AD/S3yaOexh7sLhUz1gCK/VKRNG6wSsds3xTYObD6GfxqvHDLYqpqikjlUJAYZe/e/8+LJhT2ICa3mmNA2/XZVBR7z/VqXU+e3GIWGEj4Ct4cwei+ftsjcGgac0qJ5jwrJX+oowOsTQEsdGubgAooIUHnnv2jERgFNoyKmo28az09l1Jc52Yf3LhprPXOiAhjZmFXyTnbJ479c5lA/5qYu/lBFn43FCsAq7hnMTAZmwO7nXh4Mt30ZVeKSQ+EEfYjEdOQt3YLP+liZscmls/+14zqUzzv0Loc54Mhk/mSMRV/ZTjjZPk/wq3GetyZYHn0v5ESZjKOxRYLV/o2VzG9QfsG92waB7GNvL5yoOKDAtP8rgRkPLLCRpd/5yP61aXmE20q2FS5ezp+63d8EwU4vX9V9cRdpIRi3LHR2IU9mSoEIXdZ6EeVLrmIgc7HRF1nKrXh5E/hG4pDn4upIGgk8ZZKBnh95F0uuupWpi+HmrMim5RWpcErROz61jQ5KVsYZOyI2l20N3Hy3mTu5ldtFLyeXuOe7vU0e77tNgz3FFCIxBo7IB1ldd/iO2OzeFisGe55QrLQrb5KKO9go5lgfJuUhIPZ602iXdJgcghTof8p7HQBwxWoodIQYp3qk6Bo2Q88nPrEV/RDYMUFLGOHgLP/tGAS4UrD4dQmYgLhcD9XFD/kHzsvp5gMB/1DlF7BzedhHWr6FX2vTTXrzScV9n GZJduG7l VdfUReTEcdlX9tO15Rm/AUTmMwcQgnWIpnMqe0vTg5/GuHEWMlr8+O/ebY688T4FniOGUI1eZFST1NrV3SnUfbYgVuySSpn1Y5vwkMQDtWQBbjxJ/UgC8OIbmVDZwRWNmHklDQYNUb4gq4pzIW1rfqZmuiIh/45ATHIq6n+qgPx/ATKny4+px2p1ejolFlrvSqUJrRJWWisaojtxHaXS9DuR1/yz7tZcXENoJt8X8zgN1OM0pZP4t27jGEoSLSmrQ+YTyqbThtxtDlNl9IC/1jshqQjQ5C8y6MJiYLl/VbUtMHMr4j5nwDTv78uU4M7VlucQ+APiXqEfPf79RAXAcMPt6PR9UT281Qy4z X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/9/28 15:34, Ilias Apalodimas wrote: ... > > Yes, that wasn't very clear indeed, apologies for any confusion. I was > trying to ask on a linked list that only lives in struct page_pool. > But I now realize this was a bad idea since the lookup would be way > slower. > >> If I understand question correctly, the single/doubly linked list >> is more costly than array as the page_pool case as my understanding. >> >> For single linked list, it doesn't allow deleting a specific entry but >> only support deleting the first entry and all the entries. It does support >> lockless operation using llist, but have limitation as below: >> https://elixir.bootlin.com/linux/v6.7-rc8/source/include/linux/llist.h#L13 >> >> For doubly linked list, it needs two pointer to support deleting a specific >> entry and it does not support lockless operation. > > I didn't look at the patch too carefully at first. Looking a bit > closer now, the array is indeed better, since the lookup is faster. > You just need the stored index in struct page to find the page we need > to unmap. Do you remember if we can reduce the atomic pp_ref_count to > 32bits? If so we can reuse that space for the index. Looking at it For 64 bits system, yes, we can reuse that. But for 32 bits system, we may have only 16 bits for each of them, and it seems that there is no atomic operation for variable that is less than 32 bits. > requires a bit more work in netmem, but that's mostly swapping all the > atomic64 calls to atomic ones. > >> >> For pool->items, as the alloc side is protected by NAPI context, and the >> free side use item->pp_idx to ensure there is only one producer for each >> item, which means for each item in pool->items, there is only one consumer >> and one producer, which seems much like the case when the page is not >> recyclable in __page_pool_put_page, we don't need a lock protection when >> calling page_pool_return_page(), the 'struct page' is also one consumer >> and one producer as the pool->items[item->pp_idx] does: >> https://elixir.bootlin.com/linux/v6.7-rc8/source/net/core/page_pool.c#L645 >> >> We only need a lock protection when page_pool_destroy() is called to >> check if there is inflight page to be unmapped as a consumer, and the >> __page_pool_put_page() may also called to unmapped the inflight page as >> another consumer, > > Thanks for the explanation. On the locking side, page_pool_destroy is > called once from the driver and then it's either the workqueue for > inflight packets or an SKB that got freed and tried to recycle right? > But do we still need to do all the unmapping etc from the delayed > work? Since the new function will unmap all packets in > page_pool_destroy, we can just skip unmapping when the delayed work > runs Yes, the pool->dma_map is clear in page_pool_item_uninit() after it does the unmapping for all inflight pages with the protection of pool->destroy_lock, so that the unmapping is skipped in page_pool_return_page() when those inflight pages are returned back to page_pool. >