From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14A39D2AB3F for ; Tue, 29 Oct 2024 13:58:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8C6F56B008A; Tue, 29 Oct 2024 09:58:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 876646B008C; Tue, 29 Oct 2024 09:58:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F0526B0092; Tue, 29 Oct 2024 09:58:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 4E5886B008A for ; Tue, 29 Oct 2024 09:58:41 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id F195841305 for ; Tue, 29 Oct 2024 13:58:40 +0000 (UTC) X-FDA: 82726793520.17.91062B6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf07.hostedemail.com (Postfix) with ESMTP id 8E6E04001E for ; Tue, 29 Oct 2024 13:58:04 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=c8xe9A3w; spf=pass (imf07.hostedemail.com: domain of toke@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=toke@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730210239; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=G1jV1/Yzm8C7w0dHBYs7ihPXUdLAMbLSEy37RaAfnIE=; b=rfd7sLw1b+SLkl0UvPMuFqJhQRo4OGjzwNy7oBMgLpzhqM9wh6qVQ2GMf72mTokyJTH5G8 cjd+QpiHbAnWu7q8SY498CLki4vNF1ubPFSr/6efnfqOXaEfQXpyhwLFxJv6g7duVscaGQ WEaZ/CM0C2hOMAwoj84lzliJJIlrWb4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=c8xe9A3w; spf=pass (imf07.hostedemail.com: domain of toke@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=toke@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730210239; a=rsa-sha256; cv=none; b=qLE47Wa1GWWV0Tuu5PJnDKzLzFXdjRcQ3qg/CpF9SK7S5wQYYu1aNnjdfwqmIrC431aSkW eWCQ1F85e3xI65KNCvBAJdm4lIKh5LSJLQpMTCHRh6gJjlPRGlf/viaEZEYzQ2trlq2glX e0FUYEhxGJfmZFm+eUe8v44d85QCRGM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1730210318; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=G1jV1/Yzm8C7w0dHBYs7ihPXUdLAMbLSEy37RaAfnIE=; b=c8xe9A3wnTrsxsOJ2fD5w9sOX4yzjaUVm0UqgpCeJLK3L60IQ1pKkpAIacECRLye6us3NX K6wB/yEFmh7PuuGWmbjjOSM8rVWI2kgH9PwU28RYu+3Cf/b+Nl4Xo4H9UwqpBBLB0eVhGw oejcl496O6RQD6hmTARvE3RO4R76GEQ= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-43-Kvjyhc2sNH6atd9Ly-n1ng-1; Tue, 29 Oct 2024 09:58:37 -0400 X-MC-Unique: Kvjyhc2sNH6atd9Ly-n1ng-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-4315ad4938fso39114835e9.0 for ; Tue, 29 Oct 2024 06:58:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730210315; x=1730815115; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=G1jV1/Yzm8C7w0dHBYs7ihPXUdLAMbLSEy37RaAfnIE=; b=DxGRVjN99q78vSmNXX+BtA6lUp7sxnGMuNJaUpF+WCBqUq8lJb6Pc8dIGCKxO9PUwH 1HAqYp/xJ5TP54vvKLiMWloMtw4CR3G1vXMA5VKIg/838FLDtjtmtEW1zvGGEZzthMkm BLuFTetE655cn5/kFnmBxMKufFgD4GarRNiodd3lA5B1q6qdw0GvSxYVbQAS+Xh/NlzS D/tfbgK+BDex8lArrQ8B6qkxH0zA5gJtTm4shu3RV7cJBgq/t2t1YIX1ZuQ/3UKQ+W08 N4hYe8Zdbhzg4sjfWRgYuh90KEhmyyz9e3rWFeY8p/EBoScsfk60mddTfQEXkDP8c+9k 4zAw== X-Forwarded-Encrypted: i=1; AJvYcCUCiI+5VBe6i5HC7idhJ69wyeJs878QEpLPdxPBSkLu+Zlcq9r20vwEZkoqJt6DXSCh1puelWJxGw==@kvack.org X-Gm-Message-State: AOJu0YxR8duGsr2+LCbO3dnTQbTmvGh+ZXK+Xws4ee1fIFTm18idXpuF uYU3wHgwaePDAUpqYdmCrX6nU+2UVZuvSJwaDk7KFgM5uH+VmfO+dFCXdNvOl98iT7UjQ3BG4Iz 0bxWl+vP9IDMHW8KKxk9dA1jqPEdIbIgWfe00sIU16W3hUXSl X-Received: by 2002:a05:600c:4fcb:b0:425:7bbf:fd07 with SMTP id 5b1f17b1804b1-4319ac6fb93mr108811305e9.5.1730210314921; Tue, 29 Oct 2024 06:58:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGmjNhzuq00pUPaKiJbsibQab44XUqMBzjhRGWk84vFKvwkmvCnitvIW369yK4nhBxXErWTvQ== X-Received: by 2002:a05:600c:4fcb:b0:425:7bbf:fd07 with SMTP id 5b1f17b1804b1-4319ac6fb93mr108811095e9.5.1730210314561; Tue, 29 Oct 2024 06:58:34 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4318b567e18sm173445365e9.26.2024.10.29.06.58.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Oct 2024 06:58:34 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 1A9D9164B204; Tue, 29 Oct 2024 14:58:33 +0100 (CET) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= To: Yunsheng Lin , Jesper Dangaard Brouer , davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com Cc: zhangkun09@huawei.com, fanghaiqing@huawei.com, liuyonglong@huawei.com, Robin Murphy , Alexander Duyck , IOMMU , Andrew Morton , Eric Dumazet , Ilias Apalodimas , linux-mm@kvack.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, kernel-team Subject: Re: [PATCH net-next v3 3/3] page_pool: fix IOMMU crash when driver has already unbound In-Reply-To: References: <20241022032214.3915232-1-linyunsheng@huawei.com> <20241022032214.3915232-4-linyunsheng@huawei.com> <113c9835-f170-46cf-92ba-df4ca5dfab3d@huawei.com> <878qudftsn.fsf@toke.dk> <87r084e8lc.fsf@toke.dk> X-Clacks-Overhead: GNU Terry Pratchett Date: Tue, 29 Oct 2024 14:58:33 +0100 Message-ID: <878qu7c8om.fsf@toke.dk> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 8E6E04001E X-Stat-Signature: xeh84a1wtxwy3qutpn8o67aa6bngs1tg X-HE-Tag: 1730210284-129464 X-HE-Meta: U2FsdGVkX18GvIUWGkvPlvVTdbUTTHC+0AackUr4Mb2aQuO+RhZmMz8XnFRqBfR0YVIKuNK4rmQS+mxoQBE4ZtE+Ncsc6gRdITLvNo83Z4OYEC+h0WRH6jH7rSFJOdw0eaRLI1B5xJYpsNkk/NFjlKLVntoABK8daRAwnLgtp8eaEXg1K1Dl+etE9iFFBNJMVvmanF6WoWfBw5AhYJewt28C2y47ne/QdbiPdrrt7caVsNE64ElMlzd+Etlxaja2G3/vn9qKFjqjuSwUg4OQXLasC+u759wO2sD9Alg9JHdgCLvoWlFpkONMawAOJEJ3e91sHa6WmjkgKPpNozkkJfpcPn6rfpXD4hFCy7KXzOwlhZBPkKu+4xxzIkK9LygRtz0dwDCVqXtynmX4LylcYPzIjNq1nu7IBmDRhKC0A8LDGgwRuwbjn74P4swBWfnHlvlwLpJWmfqYclKBODUwE3S2vFNNM7IVDXsfpq8vsapcuXtcUxnGqb75qsC0ra3aagsL6RyP4VWotpuoq62fuvBCIoQCoJAkvg3ZzOU2mtot8wgzRfWcdP5UzvaATN/yt8lqqjoxn9EhZ5vpEeFFPzAI79vhH5WbZViWuzs8aS66Jbk5QJbaONHSdcQi2d61GX/RJcVwkaAGezM5Nd4H6TbeQuSGvLWvz/yd7NKxzOBX15wV6UDL11mBimOp7n/LbWAiGIumBHMIfz94W1VirTLd60jKanTrGRPv4wn7GJ3yJzmBAYCecR8RuH4kluobh5ee9IMvq4jeMZMSu5uOnFa7pXBGKGLb/4ht5dUZflOqj3mIFwjC5f2ALXe7ARgCkcs75mZEXSySBpya+ApmgTJgF0618kngNGaj1QgsvbwO25sb9Jj8WQ3hsrGh4d9H8C0RFZ8KpcYZ8bK5Z7IzlVQczIMA9m2ztlnZgjzu0TtDwzhZidLbrf0GPnlBHphOfhlhQ4lyu291TbdJCZF E3Kv7m15 kIbvHCdnPwiZqGXJM3KKhQCXFpCk6/ZIZcotm6jCEzBTG9Extpx2pbRNPt+OkEjNMbEUqj3Ms8z8gO6N0wmZ/46Wogj3/vPwgpq4rpWjP0SGQAiAc1BIJoOC397ApP20TlvRnp8ayUajsWf7VS6PqfddGKX5HJM8EDPngkesW17wGPsXKZDje52ipNB0zdGSiqSMTInMQKFxgLkVZdcm3MqpPtyCWwZyu0J8TUe41roAJaVq0G6ECOzjud/UVQam3E6xbqiwcCUfn03O10bL45APsaF8aUeKn0VwkfU6ZeboFeLvQZdIoNfKOvwXrcGQs2iQw4swd7x+LPkMNGw/c4Wz9ILnMqPwaa4MVX++1r0w2qgWBlF/kPcKGg4yioOiBEuyZTzXWFFIm70x1SZYoKi2RlVDfzo8VXU79E+elkqPoY0mkvN4DRx4+tZI+aSiD0V3dOCY1LUHwdL8VoFy07KkWW/gfnliqjiAq0RvnlwcO1A4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Yunsheng Lin writes: >>> I would prefer the waiting too if simple waiting fixed the test cases that >>> Youglong and Haiqing were reporting and I did not look into the rabbit hole >>> of possible caching in networking. >>> >>> As mentioned in commit log and [1]: >>> 1. ipv4 packet defragmentation timeout: this seems to cause delay up to 30 >>> secs, which was reported by Haiqing. >>> 2. skb_defer_free_flush(): this may cause infinite delay if there is no >>> triggering for net_rx_action(), which was reported by Yonglong. >>> >>> For case 1, is it really ok to stall the driver unbound up to 30 secs for the >>> default setting of defragmentation timeout? >>> >>> For case 2, it is possible to add timeout for those kind of caching like the >>> defragmentation timeout too, but as mentioned in [2], it seems to be a normal >>> thing for this kind of caching in networking: >> >> Both 1 and 2 seem to be cases where the netdev teardown code can just >> make sure to kick the respective queues and make sure there's nothing >> outstanding (for (1), walk the defrag cache and clear out anything >> related to the netdev going away, for (2) make sure to kick >> net_rx_action() as part of the teardown). > > It would be good to be more specific about the 'kick' here, does it mean > taking the lock and doing one of below action for each cache instance? > 1. flush all the cache of each cache instance. > 2. scan for the page_pool owned page and do the finegrained flushing. Depends on the context. The page pool is attached to a device, so it should be possible to walk the skb frags queue and just remove any skbs that refer to that netdevice, or something like that. As for the lack of net_rx_action(), this is related to the deferred freeing of skbs, so it seems like just calling skb_defer_free_flush() on teardown could be an option. >>> "Eric pointed out/predicted there's no guarantee that applications will >>> read / close their sockets so a page pool page may be stuck in a socket >>> (but not leaked) forever." >> >> As for this one, I would put that in the "well, let's see if this >> becomes a problem in practice" bucket. > > As the commit log in [2], it seems it is already happening. > > Those cache are mostly per-cpu and per-socket, and there may be hundreds of > CPUs and thousands of sockets in one system, are you really sure we need > to take the lock of each cache instance, which may be thousands of them, > and do the flushing/scaning of memory used in networking, which may be as > large as '24 GiB' mentioned by Jesper? Well, as above, the two issues you mentioned are per-netns (or possibly per-CPU), so those seem to be manageable to do on device teardown if the wait is really a problem. But, well, I'm not sure it is? You seem to be taking it as axiomatic that the wait in itself is bad. Why? It's just a bit memory being held on to while it is still in use, and so what? -Toke