From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76DF2C433C1 for ; Wed, 24 Mar 2021 22:21:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D5FA561A1B for ; Wed, 24 Mar 2021 22:21:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D5FA561A1B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4EEB56B032E; Wed, 24 Mar 2021 18:21:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 49E5F6B032F; Wed, 24 Mar 2021 18:21:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33F646B0330; Wed, 24 Mar 2021 18:21:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0163.hostedemail.com [216.40.44.163]) by kanga.kvack.org (Postfix) with ESMTP id 1B2906B032E for ; Wed, 24 Mar 2021 18:21:38 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id D3155183C781C for ; Wed, 24 Mar 2021 22:21:37 +0000 (UTC) X-FDA: 77956190634.16.86546E3 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf01.hostedemail.com (Postfix) with ESMTP id DC7B75001522 for ; Wed, 24 Mar 2021 22:21:36 +0000 (UTC) Received: by mail-pj1-f53.google.com with SMTP id s21so218572pjq.1 for ; Wed, 24 Mar 2021 15:21:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=jIDPF7ri84OLFsNd4OhxmJPSCi5FNfpE4FjtwEiJamI=; b=RiBXhaenM9VBbNf7Ikof8uRUzpei4NDWIP7Ov/61XaLR+CqRNKdKd7fevMR7JsPQSn dak0k7Il33xba+i95mj9167kOtqh9O/gBnyrBpD3mYXz+NZec2sWUmXXD+jKJMkdYHmZ S+kX+o7ONkaASTwWepkBXvtypIYfNzOTkqHPjiHglI7PlYC22lTTYnLEDwcMwTCgkxec wRO9mcct/h+YU/KkpwyhIU78PUQXyAKvNNIsc77BBuYWU+MYo1EK/fOQTtdF7aj/WVKb qWktnXp87q3dMVTR2qYDoRBCoPIB64pUP9Gyq3wDM/GVZaCK+gv6QQIbZe8N95o3IkGQ IQ+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=jIDPF7ri84OLFsNd4OhxmJPSCi5FNfpE4FjtwEiJamI=; b=mpOaWliz3XfBX3V33N0dfCgbQX7+sAuR18+amEY8HR/xzDR7PodZ/kIVkjjm8to4dI ESBu8JUWN9FgQqnmC8Sx4OwqN7yeeuVb5lYC4H31njkSMUnS1AZyeiutH/4yj7RqSPiT 3I/I2c/rNHhGZ7Bds6LaRB7Co+RF2vvkZ3UVRzuDXSrfFEahWrIyMybSeouhhUZ8fYFs 89shZiO4CnPPAnAEbUpFtYGQqvoNIWP6NjN+9lPtZ980IKLjcnTlYGYkTXCdoUfMGxlc HyEK1gD25f/fIF4/BMjapV2JQxEXM6wkrWM0kUd0La6Wzc2zGCsP0JxH5JwtvlQAsGYu 54LQ== X-Gm-Message-State: AOAM532fqo3c60ECL8NLOz/XCE2xGyTx+CL4dZBaOgmffdYNRfITJg8a lM5puOzNlIo8xjcD91z6TzDk5cgXt/G+q78ve5QZMA== X-Google-Smtp-Source: ABdhPJxO78gga38eOSSGkRb7JR3LPL4ca3GN3Kg63kSbnkYrOAUmk+vpnbWzVH0WjGPGdqNPQJZhFiJatpAuCeF1xyw= X-Received: by 2002:a17:90b:947:: with SMTP id dw7mr5682193pjb.178.1616624496196; Wed, 24 Mar 2021 15:21:36 -0700 (PDT) MIME-Version: 1.0 References: <20210316041645.144249-1-arjunroy.kdev@gmail.com> In-Reply-To: From: Arjun Roy Date: Wed, 24 Mar 2021 15:21:25 -0700 Message-ID: Subject: Re: [mm, net-next v2] mm: net: memcg accounting for TCP rx zerocopy To: Shakeel Butt Cc: Johannes Weiner , Arjun Roy , Andrew Morton , David Miller , netdev , Linux Kernel Mailing List , Cgroups , Linux MM , Eric Dumazet , Soheil Hassas Yeganeh , Jakub Kicinski , Michal Hocko , Yang Shi , Roman Gushchin Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: ecpxmdns17pizya3d5wewoggmm54mwg7 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: DC7B75001522 Received-SPF: none (google.com>: No applicable sender policy available) receiver=imf01; identity=mailfrom; envelope-from=""; helo=mail-pj1-f53.google.com; client-ip=209.85.216.53 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616624496-708342 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Mar 24, 2021 at 11:26 AM Shakeel Butt wrote: > > On Tue, Mar 23, 2021 at 11:42 AM Arjun Roy wrote: > > > [...] > > > > To summarize then, it seems to me that we're on the same page now. > > I'll put together a tentative v3 such that: > > 1. It uses pre-charging, as previously discussed. > > 2. It uses a page flag to delineate pages of a certain networking sort > > (ie. this mechanism). > > 3. It avails itself of up to 4 words of data inside struct page, > > inside the networking specific struct. > > 4. And it sets up this opt-in lifecycle notification for drivers that > > choose to use it, falling back to existing behaviour without. > > > > Arjun, if you don't mind, can you explain how the lifetime of such a > page will look like? > > For example: > > Driver: > page = dev_alloc_page() > /* page has 1 ref */ Yes, this is the case. > dev_map_page(page) > /* I don't think dev_map_page() takes a ref on page, so the ref remains 1. */ > To be clear, do you mean things like DMA setup here? Or specifically what do you mean by dev_map_page? > On incoming traffic the page goes to skb and which then gets assigned > to a struct sock. Does the kernel increase refcnt of the page on these > operations? > Adding a page to an skb will mean that, when the skb is cleaned up, a page ref is dropped: https://github.com/torvalds/linux/blob/master/net/core/skbuff.c#L666 So a driver may bump the refcount for the page, before adding it to the skb: https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c#L442 > The page gets mapped into user space which increments its refcnt. > Yes. > After processing the data, the application unmaps the page and its > refcnt will be decremented. > Yes. > __put_page() will be called when refcnt reaches 0, so, the initial > refcnt which the driver has acquired, has to be transferred to the > next layer. So, I am trying to understand how that will work? Ah, I see - there was a miscommunication. Johannes mentioned __put_page() but I read put_page(). That is where I was planning on adding the interposition for these network pages. So in put_page(), if it turns out it's a network page, we do our handling then as I described in prior emails. Sorry for the confusion. Thanks, -Arjun