From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F856CF58CD for ; Fri, 20 Sep 2024 05:29:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 869216B0082; Fri, 20 Sep 2024 01:29:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7F1756B0083; Fri, 20 Sep 2024 01:29:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 66B1E6B0085; Fri, 20 Sep 2024 01:29:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 48D366B0082 for ; Fri, 20 Sep 2024 01:29:58 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id BBC0EABA77 for ; Fri, 20 Sep 2024 05:29:57 +0000 (UTC) X-FDA: 82583990034.25.45A80A6 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf05.hostedemail.com (Postfix) with ESMTP id 037E3100002 for ; Fri, 20 Sep 2024 05:29:55 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b=IPfFivX8; spf=pass (imf05.hostedemail.com: domain of ilias.apalodimas@linaro.org designates 209.85.214.174 as permitted sender) smtp.mailfrom=ilias.apalodimas@linaro.org; dmarc=pass (policy=none) header.from=linaro.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726810138; a=rsa-sha256; cv=none; b=eCRZproggKbYgp9v3n7Ba157rYodFBxyGfo1F2zB9pI11pj5zN1CH9kAfzx4kf7XpYomhZ 7EjD01L2QDXLgiun980l6PJ0ZPdzMNfdqipxNjcI8/dfBMDHos7kZXvGimrVepjiK8cYLc u9NHDXB3K5E1c89Ys3jEU0+st9aFsB0= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b=IPfFivX8; spf=pass (imf05.hostedemail.com: domain of ilias.apalodimas@linaro.org designates 209.85.214.174 as permitted sender) smtp.mailfrom=ilias.apalodimas@linaro.org; dmarc=pass (policy=none) header.from=linaro.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726810138; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KlbRqa7nDF2on3envLfRNKR76Ma17jJTd9UlvPEe+/8=; b=bnqqcQYj5VTBXKKLyORbk/QYeV1PHK+wwsXx9LcQT8H8dNJDUJ+ncu9wtg+pZMebUl+eoR 9jd1fUdStUUIqSvpbFmsROsWY4qyLKHGC01i77tGHLJOCSkOgjm7k6QytJPjdBpB7w70Lk ga2pCcKBEXTSiO46ox5fsbmJpJPImJU= Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-20551eeba95so16305265ad.2 for ; Thu, 19 Sep 2024 22:29:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1726810194; x=1727414994; darn=kvack.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=KlbRqa7nDF2on3envLfRNKR76Ma17jJTd9UlvPEe+/8=; b=IPfFivX8RPgwARcEfMVZMZt5FjP+TfWds1wdk183DyKKsR7jB3SINqt2/OAkWm4uXv o/Rjh0A8b9o8dv2kePsY32vF7mh11FT4P7mIc45MeHFZ0ed/IJ+Cv32iaOfovo5XdGjm mVXHmtLRkg1fTv5cfyTi6bQEsDDjna6XqtjCWAgfmw17FwxSIffFXq+5lPn02yrNzWAj OdvZQ/LukxC5CcSVult0QEzSel/dL+xWjmCS9GwVUXrdfeotq/Si04mA9XUtSRNy3UfO Z/GiUQ3uPfClZw6N3zuUV70T7GM1WAmwDVeBGrkbkO8nSKt9GLyO45btd/WUGYGE6zb3 qDPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726810194; x=1727414994; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KlbRqa7nDF2on3envLfRNKR76Ma17jJTd9UlvPEe+/8=; b=PxfHS4M/QCuaZ8tJenZgAHlghk7BvU+R91gtxpBBnc3brh9QHiSUyM1T/3Tnmf5J3C JOuqfiKJW7U7hCEyWMcrKBwe2SWV/twNQ/D2hcbsFonsRyn3OPrSZW7LUGp9+NVd7jW2 4FeU72sZycRS4f/g/lozzt2kwHWggQx1Qt0emfNYDV7eN9TW+pXdfur44yuBb/kOuONg g/BBd1KFS2OeVzO2yJ3w3+ukrqCQ63YQS/sXaaml+p0bkONFUGDE0FH/05Ni1vcv57aU 0fa/BndKgiUtfebxiAG0FJON7OBrKDcsS8nT76UVaadkkFfU+vCWjLX7TmaAke+t0WQl uUqw== X-Forwarded-Encrypted: i=1; AJvYcCXUhQ0RF/GJLdy7xCI0nGe5jEbThCnLXF+geE4O29rjLQ1GzZ/DcdF98uApysYEnp0ToeDPmQ2IyQ==@kvack.org X-Gm-Message-State: AOJu0Yzzu9TjX2BZmfroZ/wDulfjPFhlmW/Tbb2GKEkv3iAJJLFJiCBa ibr0hh1aT/uToIvWX3gCqKVvAJuV6fll3r5RZoECRrvhLjqIS0GzNqe0Dd0Hax+JtO19HVo1SuG y+sY7Rnf8oyIgjLTkC7SPeDI6+7vVAVaynGSNQA== X-Google-Smtp-Source: AGHT+IFXWjjSwUV/swiBcaoOWRQSWx9kHyL3ewiTd5ev59pob+f5VLbFm74ieex5x8O25zo1tfmYvjNJafp2P2Ogezc= X-Received: by 2002:a17:903:32c7:b0:206:ba7c:9f2e with SMTP id d9443c01a7336-208d83b69f0mr25703685ad.25.1726810194586; Thu, 19 Sep 2024 22:29:54 -0700 (PDT) MIME-Version: 1.0 References: <20240918111826.863596-1-linyunsheng@huawei.com> <20240918111826.863596-3-linyunsheng@huawei.com> <894a3c2c-22f9-45b9-a82b-de7320066b42@kernel.org> <0e8c7a7a-0e2a-42ec-adbc-b29f6a514517@kernel.org> In-Reply-To: <0e8c7a7a-0e2a-42ec-adbc-b29f6a514517@kernel.org> From: Ilias Apalodimas Date: Fri, 20 Sep 2024 08:29:18 +0300 Message-ID: Subject: Re: [PATCH net 2/2] page_pool: fix IOMMU crash when driver has already unbound To: Jesper Dangaard Brouer Cc: Yunsheng Lin , davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, liuyonglong@huawei.com, fanghaiqing@huawei.com, zhangkun09@huawei.com, Robin Murphy , Alexander Duyck , IOMMU , Wei Fang , Shenwei Wang , Clark Wang , Eric Dumazet , Tony Nguyen , Przemek Kitszel , Alexander Lobakin , Alexei Starovoitov , Daniel Borkmann , John Fastabend , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , Felix Fietkau , Lorenzo Bianconi , Ryder Lee , Shayne Chen , Sean Wang , Kalle Valo , Matthias Brugger , AngeloGioacchino Del Regno , Andrew Morton , imx@lists.linux.dev, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, intel-wired-lan@lists.osuosl.org, bpf@vger.kernel.org, linux-rdma@vger.kernel.org, linux-wireless@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: ad7nps4wukn5up7ybe8mjwhdwzokf3ko X-Rspamd-Queue-Id: 037E3100002 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1726810195-743775 X-HE-Meta: U2FsdGVkX1+aCw3VwqlJKJjJNwdZgSk8sqOIazV+DIX0WNDGcVRtejiwU3MjQ4DyuYx5W5+z9Kw+UMzCglUoL0PpAA1e9acoRYR50DSEWInueEYyBaEVoK/ANiZ+tkHrfZTYXcEl4W4Pw2g+vZvJFcr8w9zpHwBWc+bjj7BjbFZ4hq0XXa34nMxMhgj+z5YKlN/omKMgYj021TxBp6kxK/4waTYLEjZxaZ5JFZaSYpeV9jlCDAHp5ZZwwe6GtYo8b5VWqYvkLyNjdVahQY7+eT3pAOQPh1KKUuK6ytjOK1JRwvdv4wBafkEc9coYTh96kGSC2SXqkg7VCRZfCgJoy2E/1VUwEA6Aj5TS+5W4bhTIR2vL/WXz4vw/OUc29XepB0UPIzMbMxmc963zpY3b/5dUpi4a9kK3eiDf+SHSlce0kKKYFrbK13rdXb68lYh+JPqjAjC8yoPb7Y/K6NC89b5fuw8kYTzRZFwbTdobzmjSUj6F2Ln7h8oiG5B6ySKEDdRU5wxxSvBpoKngM1Cntzp7RpNy+GvWdk34VQkyCPXSHWv5O/Jnp3SK5sHmFZOwtKttNAQfroNphXv/7CLVF/vQeqvflSME+phYovZFjK1MOjMdKOpAcKbT+P3I8TBYAdRfbSolhuxo/qcssYZsxFEmv+MoryJt4KTXNa+S9VEQOI8MqlINKhGyihG0YmO4voXv9WdmLRc8Sdjw7wzDC8WYE7d1u03XGJ5OLMEvNtNSJS4AH/s3kLIgwEcxZMu7Ck3ANQb3XBRzBwOEFNav4t79K1zuv4SOqAowinEEtc3d1kKTVyivOZBU1Zj9O5XbHtwLybCecZHRHZfU1mnHJkPd0PXRQdJ8YM5CCnpaRQ6yMlGfFluqZY3+jBz3tsDtBCMTS+NX6LL2dsDt1wzfNiJDGnCu6Q6pRNrI4RAhFYH3PID5sXiogeKs7D18phtIYAsLnUs6dIhuzVfg2Eh jiW50lJS FrwitjGvFuUXBhmdnNugW6lC1PiYwrVQugnMytdCovSHGuxd/YiT0zGSCRhUQWTjlFxylUxOplv6WaRfD/xRwhRWs45HAGZ5AdAx7LXy0hmZFHuTi8i+e4u5tLnsrg4QYwKp4dhC7iU/7bL/pa2J2DBi1hibz2OXEWxCHVT0sKZl+G0SlShhfh8WgGRHxqgmi+BPexVjLmwAlAMVbRmI05EQ1Wpfi16D0BPl7KN+H51PHjWAoyUFyiAoTkkel+6ptPgZ5EYN58W9PtJOkeilF39siiC/6Ts8TyV2sObsXBcPa7j2a9npLH5ySBEKpgQa4CAvKRUf8VL0QO4Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Jesper, On Fri, 20 Sept 2024 at 00:04, Jesper Dangaard Brouer wrote: > > > > On 19/09/2024 13.15, Yunsheng Lin wrote: > > On 2024/9/19 17:42, Jesper Dangaard Brouer wrote: > >> > >> On 18/09/2024 19.06, Ilias Apalodimas wrote: > >>>> In order not to do the dma unmmapping after driver has already > >>>> unbound and stall the unloading of the networking driver, add > >>>> the pool->items array to record all the pages including the ones > >>>> which are handed over to network stack, so the page_pool can > >>>> do the dma unmmapping for those pages when page_pool_destroy() > >>>> is called. > >>> > >>> So, I was thinking of a very similar idea. But what do you mean by > >>> "all"? The pages that are still in caches (slow or fast) of the pool > >>> will be unmapped during page_pool_destroy(). > >> > >> I really dislike this idea of having to keep track of all outstanding pages. > >> > >> I liked Jakub's idea of keeping the netdev around for longer. > >> > >> This is all related to destroying the struct device that have points to > >> the DMA engine, right? > > > > Yes, the problem seems to be that when device_del() is called, there is > > no guarantee hw behind the 'struct device ' will be usable even if we > > call get_device() on it. > > > >> > >> Why don't we add an API that allow netdev to "give" struct device to > >> page_pool. And then the page_poll will take over when we can safely > >> free the stuct device? > > > > By 'allow netdev to "give" struct device to page_pool', does it mean > > page_pool become the driver for the device? > > If yes, it seems that is similar to jakub's idea, as both seems to stall > > the calling of device_del() by not returning when the driver unloading. > > Yes, this is what I mean. (That is why I mentioned Jakub's idea). Keeping track of inflight packets that need to be unmapped is certainly more complex. Delaying the netdevice destruction certainly solves the problem but there's a huge cost IMHO. Those devices might stay there forever and we have zero guarantees that the network stack will eventually release (and unmap) those packets. What happens in that case? The user basically has to reboot the entire machine, just because he tries to bring an interface down and up again. Thanks /Ilias > > > > If no, it seems that the problem is still existed when the driver for > > the device has unbound after device_del() is called.