From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC19FC43460 for ; Mon, 19 Apr 2021 15:43:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4CD0361284 for ; Mon, 19 Apr 2021 15:43:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4CD0361284 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B60EA6B006C; Mon, 19 Apr 2021 11:43:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B0FF26B006E; Mon, 19 Apr 2021 11:43:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 914736B0070; Mon, 19 Apr 2021 11:43:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0047.hostedemail.com [216.40.44.47]) by kanga.kvack.org (Postfix) with ESMTP id 6DF386B006C for ; Mon, 19 Apr 2021 11:43:25 -0400 (EDT) Received: from smtpin32.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 1A7E68249980 for ; Mon, 19 Apr 2021 15:43:25 +0000 (UTC) X-FDA: 78049535970.32.AFEA449 Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) by imf25.hostedemail.com (Postfix) with ESMTP id C94476000123 for ; Mon, 19 Apr 2021 15:43:21 +0000 (UTC) Received: by mail-ed1-f44.google.com with SMTP id h8so1142874edb.2 for ; Mon, 19 Apr 2021 08:43:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=wf0QWwNwYK6Jhss3gCic3ehYYejoO/XLDc6PTFbc/rU=; b=AcV8eFNybarUUIKChCGLHFNWKTXeCXUS2RwBuMjj5z9IPjcJJzr+bKeR2lL61o4jJJ sRwQo3IzAcvyzF4JFtxWbtvpUxNLNtl+iUWDSbH8GdI2gQSryXkcPO2EgjqBsZxNL/Dp oEVDryNxsWLjXMkNbGCx5oX+lFL+nn6Okl0BYTH1LHz5mYShDONiNnVrbqfgCiPz4xQG h+xu8ewH/NOn5+DsVldXLFK4y0RknlSRIbJpnzFt+Me2/3yAJBF8qKjo7Wwfn1w/uHs6 mb2HggcdKRRt97ivNvwAFF9kD37O4OBBnD8HNyLoFsDdEp6+/21qeeklsXWgJzvdIn5X hmRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=wf0QWwNwYK6Jhss3gCic3ehYYejoO/XLDc6PTFbc/rU=; b=VYIqkuIZTpfNsJugBRPTopMFJx8K1mt1l9T33mudjEe0652juGfFqBV2SK6HvLoxbp gGHCnuiZom/sUItutBgSOWMLTG/Db1jIrfEutGN/2Sr70ktGActg5sctRfhCsZgqDpJi AxhTGzRtKlwaZnu66nqQ+4inEjIpsE3b8r3nmtrBqCiEBhJOwQzqAbiHb7/HhH0pucXm fRSdK6yTqSOuq7aOqJPeIgSnhMbqUQi4kK/EkgprnFHwP0RrM9cOBh4YIazbO7AyUtdN pp+QMWTzyhP3bcci70ToCfee4xcaA3xjo6Yd6K05snvNAzpbmOGlLfWyI1MSRGljomF4 eqcA== X-Gm-Message-State: AOAM530fE807hxQJo0X4EMhTG7ik45O+oEv3OCrYZT25o50WuPKbPns1 FcxAPmfXDhEj3Ho6FhWwhqY+cQ== X-Google-Smtp-Source: ABdhPJwF0JLAYSLI4B5eBDz2Xkr4BA5xRzg20NOS5BQtJee0K4m+lYUS7uBNOMRCPGktz319AfgEZw== X-Received: by 2002:aa7:d3c6:: with SMTP id o6mr19000389edr.359.1618847003325; Mon, 19 Apr 2021 08:43:23 -0700 (PDT) Received: from apalos.home (ppp-94-65-92-88.home.otenet.gr. [94.65.92.88]) by smtp.gmail.com with ESMTPSA id s11sm13468946edt.27.2021.04.19.08.43.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Apr 2021 08:43:22 -0700 (PDT) Date: Mon, 19 Apr 2021 18:43:17 +0300 From: Ilias Apalodimas To: Shakeel Butt Cc: Jesper Dangaard Brouer , Matthew Wilcox , Matteo Croce , netdev , Linux MM , Ayush Sawal , Vinay Kumar Yadav , Rohit Maheshwari , "David S. Miller" , Jakub Kicinski , Thomas Petazzoni , Marcin Wojtas , Russell King , Mirko Lindner , Stephen Hemminger , Tariq Toukan , Jesper Dangaard Brouer , Alexei Starovoitov , Daniel Borkmann , John Fastabend , Boris Pismenny , Arnd Bergmann , Andrew Morton , "Peter Zijlstra (Intel)" , Vlastimil Babka , Yu Zhao , Will Deacon , Michel Lespinasse , Fenghua Yu , Roman Gushchin , Hugh Dickins , Peter Xu , Jason Gunthorpe , Guoqing Jiang , Jonathan Lemon , Alexander Lobakin , Cong Wang , wenxu , Kevin Hao , Aleksandr Nogikh , Jakub Sitnicki , Marco Elver , Willem de Bruijn , Miaohe Lin , Yunsheng Lin , Guillaume Nault , LKML , linux-rdma@vger.kernel.org, bpf , Eric Dumazet , David Ahern , Lorenzo Bianconi , Saeed Mahameed , Andrew Lunn , Paolo Abeni Subject: Re: [PATCH net-next v3 2/5] mm: add a signature in struct page Message-ID: References: <20210409223801.104657-1-mcroce@linux.microsoft.com> <20210409223801.104657-3-mcroce@linux.microsoft.com> <20210410154824.GZ2531743@casper.infradead.org> <20210414214132.74f721dd@carbon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: C94476000123 X-Stat-Signature: 5bc4k64r1id3c7rk37woa7b6t7qt7koc Received-SPF: none (linaro.org>: No applicable sender policy available) receiver=imf25; identity=mailfrom; envelope-from=""; helo=mail-ed1-f44.google.com; client-ip=209.85.208.44 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618847001-776563 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Shakeel, On Mon, Apr 19, 2021 at 07:57:03AM -0700, Shakeel Butt wrote: > On Sun, Apr 18, 2021 at 10:12 PM Ilias Apalodimas > wrote: > > > > On Wed, Apr 14, 2021 at 01:09:47PM -0700, Shakeel Butt wrote: > > > On Wed, Apr 14, 2021 at 12:42 PM Jesper Dangaard Brouer > > > wrote: > > > > > > > [...] > > > > > > > > > > > > Can this page_pool be used for TCP RX zerocopy? If yes then PageType > > > > > > can not be used. > > > > > > > > > > Yes it can, since it's going to be used as your default allocator for > > > > > payloads, which might end up on an SKB. > > > > > > > > I'm not sure we want or should "allow" page_pool be used for TCP RX > > > > zerocopy. > > > > For several reasons. > > > > > > > > (1) This implies mapping these pages page to userspace, which AFAIK > > > > means using page->mapping and page->index members (right?). > > > > > > > > > > No, only page->_mapcount is used. > > > > > > > I am not sure I like leaving out TCP RX zerocopy. Since we want driver to > > adopt the recycling mechanism we should try preserving the current > > functionality of the network stack. > > > > The question is how does it work with the current drivers that already have an > > internal page recycling mechanism. > > > > I think the current drivers check page_ref_count(page) to decide to > reuse (or not) the already allocated pages. > > Some examples from the drivers: > drivers/net/ethernet/intel/ixgbe/ixgbe_main.c:ixgbe_can_reuse_rx_page() > drivers/net/ethernet/intel/igb/igb_main.c:igb_can_reuse_rx_page() > drivers/net/ethernet/mellanox/mlx5/core/en_rx.c:mlx5e_rx_cache_get() > Yes, that's how internal recycling is done in drivers. As Jesper mentioned the refcnt of the page is 1 for the page_pool owned pages and that's how we decide what to do with the page. > > > > (2) It feels wrong (security wise) to keep the DMA-mapping (for the > > > > device) and also map this page into userspace. > > > > > > > > > > I think this is already the case i.e pages still DMA-mapped and also > > > mapped into userspace. > > > > > > > (3) The page_pool is optimized for refcnt==1 case, and AFAIK TCP-RX > > > > zerocopy will bump the refcnt, which means the page_pool will not > > > > recycle the page when it see the elevated refcnt (it will instead > > > > release its DMA-mapping). > > > > > > Yes this is right but the userspace might have already consumed and > > > unmapped the page before the driver considers to recycle the page. > > > > Same question here. I'll have a closer look in a few days and make sure we are > > not breaking anything wrt zerocopy. > > > > Pages mapped into the userspace have their refcnt elevated, so the > page_ref_count() check by the drivers indicates to not reuse such > pages. > When tcp_zerocopy_receive() is invoked it will call tcp_zerocopy_vm_insert_batch() which will end up doing a get_page(). What you are saying is that once the zerocopy is done though, skb_release_data() won't be called, but instead put_page() will be? If that's the case then we are indeed leaking DMA mappings and memory. That sounds weird though, since the refcnt will be one in that case (zerocopy will do +1/-1 once it's done), so who eventually frees the page? If kfree_skb() (or any wrapper that calls skb_release_data()) is called eventually, we'll end up properly recycling the page into our pool. > > > > > > > > > > > (4) I remember vaguely that this code path for (TCP RX zerocopy) uses > > > > page->private for tricks. And our patch [3/5] use page->private for > > > > storing xdp_mem_info. > > > > > > > > IMHO when the SKB travel into this TCP RX zerocopy code path, we should > > > > call page_pool_release_page() to release its DMA-mapping. > > > > > > > > > > I will let TCP RX zerocopy experts respond to this but from my high > > > level code inspection, I didn't see page->private usage. > > > > Shakeel are you aware of any 'easy' way I can have rx zerocopy running? > > > > I would recommend tools/testing/selftests/net/tcp_mmap.c. Ok, thanks I'll have a look. Cheers /Ilias