From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 191E2C433DB for ; Wed, 24 Mar 2021 20:39:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A536E619E5 for ; Wed, 24 Mar 2021 20:39:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A536E619E5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0CDF26B0318; Wed, 24 Mar 2021 16:39:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 07EDD6B031A; Wed, 24 Mar 2021 16:39:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E3A916B031B; Wed, 24 Mar 2021 16:39:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0154.hostedemail.com [216.40.44.154]) by kanga.kvack.org (Postfix) with ESMTP id CB7E56B0318 for ; Wed, 24 Mar 2021 16:39:21 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 771DF181CC1D9 for ; Wed, 24 Mar 2021 20:39:21 +0000 (UTC) X-FDA: 77955932922.07.75E9F3C Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) by imf15.hostedemail.com (Postfix) with ESMTP id 38CFFA000389 for ; Wed, 24 Mar 2021 20:39:17 +0000 (UTC) Received: by mail-pg1-f171.google.com with SMTP id v2so15526984pgk.11 for ; Wed, 24 Mar 2021 13:39:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=qcDyxyk5bvCS0WEAV1+mJY8jO/8xTEDVqNFuTGwD3vw=; b=LQv/zun+j1/d9r4zt9gm0IaWVEHFp8r2eRBRjyUN6GFZODOU1mbnZt2VglPEMqNrMk 5XO3EJkRspqLHmcUlkVAoJFRaMzGO65rMOxOVb89V6LEr3WYzYSl+7LKLIW3XD5V/y+J CjJh7fXJoQ5eSHjuGEm0atA60APspXDZ2TJJlFC34cmQda/5bX9QW968cxxC00GNf9yj 73+uXyYSMPdhj/hgFSsp4QabE3/KzaQ+YoUte/HNOtLyEX5iyrUeQb786gluhJc4wQ2/ U2hJkhVbGvJWYbEF8SsnOsjUTOxexa8kSlA64w/vw5ylM6SNWnXSPaYLHextD3GJ82Ir WeCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qcDyxyk5bvCS0WEAV1+mJY8jO/8xTEDVqNFuTGwD3vw=; b=JYTwzEwgZAbDiITpSPt+HMTIuVsTD+zGbPTeqm7+sJuTLc4deCFpYqOhYGHSynTWwe C0pAdX7B4jAGNuinzVtZzCRhjkm0FrZzWB7zu3/D0qm0FXNQQcx/ahlDAo/YOEw20AJg fSL4ellEmLBBKeo6lRX2EjEbtcQEP4LyzXYJU2q9z5cfVyj6ukgQrtdc6ZFLxwHjkZia zDW8YjPmltbKSIni7vC+pQvbT92DVqUKTBee/YV26zaKp3JCEgAwJk686ArCByHnm4ds DKkWDtyt1ugkEheQTIKPJx3s5WXUSDwv6Bl4d/+5CPks0QlHulA0XvVDC7oZdyYGeMAd +FuQ== X-Gm-Message-State: AOAM53371sRTCUtlhivMjfutqdD1tERp8MhSjj/v7WtEd2aLXNzdbOpO fWgcZjb4QmXSpUQf3MR80MX18tPB1SYfwkvTSnt0OQ== X-Google-Smtp-Source: ABdhPJxqyWyGQerrdRG4/ZwAhh29b/XqSh9nGCbJv8jstAuKiAFa1OCgZB4UOXLlZza+isSu0CEK1/GgUeeZe+kuhYU= X-Received: by 2002:aa7:881a:0:b029:1f1:6148:15c3 with SMTP id c26-20020aa7881a0000b02901f1614815c3mr4760683pfo.30.1616618357193; Wed, 24 Mar 2021 13:39:17 -0700 (PDT) MIME-Version: 1.0 References: <20210316041645.144249-1-arjunroy.kdev@gmail.com> In-Reply-To: From: Arjun Roy Date: Wed, 24 Mar 2021 13:39:06 -0700 Message-ID: Subject: Re: [mm, net-next v2] mm: net: memcg accounting for TCP rx zerocopy To: Michal Hocko Cc: Johannes Weiner , Arjun Roy , Andrew Morton , David Miller , netdev , Linux Kernel Mailing List , Cgroups , Linux MM , Shakeel Butt , Eric Dumazet , Soheil Hassas Yeganeh , Jakub Kicinski , Yang Shi , Roman Gushchin Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 38CFFA000389 X-Stat-Signature: hwykzhumtpgk8z9posnmg7pgcnpuqpmm Received-SPF: none (google.com>: No applicable sender policy available) receiver=imf15; identity=mailfrom; envelope-from=""; helo=mail-pg1-f171.google.com; client-ip=209.85.215.171 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616618357-111051 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Mar 24, 2021 at 2:12 AM Michal Hocko wrote: > > On Tue 23-03-21 11:47:54, Arjun Roy wrote: > > On Tue, Mar 23, 2021 at 7:34 AM Michal Hocko wrote: > > > > > > On Wed 17-03-21 18:12:55, Johannes Weiner wrote: > > > [...] > > > > Here is an idea of how it could work: > > > > > > > > struct page already has > > > > > > > > struct { /* page_pool used by netstack */ > > > > /** > > > > * @dma_addr: might require a 64-bit value even on > > > > * 32-bit architectures. > > > > */ > > > > dma_addr_t dma_addr; > > > > }; > > > > > > > > and as you can see from its union neighbors, there is quite a bit more > > > > room to store private data necessary for the page pool. > > > > > > > > When a page's refcount hits zero and it's a networking page, we can > > > > feed it back to the page pool instead of the page allocator. > > > > > > > > From a first look, we should be able to use the PG_owner_priv_1 page > > > > flag for network pages (see how this flag is overloaded, we can add a > > > > PG_network alias). With this, we can identify the page in __put_page() > > > > and __release_page(). These functions are already aware of different > > > > types of pages and do their respective cleanup handling. We can > > > > similarly make network a first-class citizen and hand pages back to > > > > the network allocator from in there. > > > > > > For compound pages we have a concept of destructors. Maybe we can extend > > > that for order-0 pages as well. The struct page is heavily packed and > > > compound_dtor shares the storage without other metadata > > > int pages; /* 16 4 */ > > > unsigned char compound_dtor; /* 16 1 */ > > > atomic_t hpage_pinned_refcount; /* 16 4 */ > > > pgtable_t pmd_huge_pte; /* 16 8 */ > > > void * zone_device_data; /* 16 8 */ > > > > > > But none of those should really require to be valid when a page is freed > > > unless I am missing something. It would really require to check their > > > users whether they can leave the state behind. But if we can establish a > > > contract that compound_dtor can be always valid when a page is freed > > > this would be really a nice and useful abstraction because you wouldn't > > > have to care about the specific type of page. > > > > > > But maybe I am just overlooking the real complexity there. > > > -- > > > > For now probably the easiest way is to have network pages be first > > class with a specific flag as previously discussed and have concrete > > handling for it, rather than trying to establish the contract across > > page types. > > If you are going to claim a page flag then it would be much better to > have it more generic. Flags are really scarce and if all you care about > is PageHasDestructor() and provide one via page->dtor then the similar > mechanism can be reused by somebody else. Or does anything prevent that? The way I see it - the fundamental want here is, for some arbitrary page that we are dropping a reference on, to be able to tell that the provenance of the page is some network driver's page pool. If we added an enum target to compound_dtor, if we examine that offset in the page and look at that value, what guarantee do we have that the page isn't instead some other kind of page, and the byte value there was just coincidentally the one we were looking for (but it wasn't a network driver pool page)? Existing users of compound_dtor seem to check first that a PageCompound() or PageHead() return true - the specific scenario here, of receiving network packets, those pages will tend to not be compound (and more specifically, compound pages are explicitly disallowed for TCP receive zerocopy). Given that's the case, the options seem to be: 1) Use a page flag - with the downside that they are a severely limited resource, 2) Use some bits inside page->memcg_data - this I believe Johannes had reasons against, and it isn't always the case that MEMCG support is enabled. 3) Use compound_dtor - but I think this would have problems for the prior reasons. Thanks, -Arjun > -- > Michal Hocko > SUSE Labs