From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5567DC433DB for ; Thu, 25 Mar 2021 17:50:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D028161A2E for ; Thu, 25 Mar 2021 17:50:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D028161A2E Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1A2F26B006C; Thu, 25 Mar 2021 13:50:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 17A136B006E; Thu, 25 Mar 2021 13:50:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 041F96B0070; Thu, 25 Mar 2021 13:50:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0026.hostedemail.com [216.40.44.26]) by kanga.kvack.org (Postfix) with ESMTP id DD3FA6B006C for ; Thu, 25 Mar 2021 13:50:40 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9E9F21E16 for ; Thu, 25 Mar 2021 17:50:40 +0000 (UTC) X-FDA: 77959136640.04.99884F1 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf26.hostedemail.com (Postfix) with ESMTP id 783A540002E1 for ; Thu, 25 Mar 2021 17:50:38 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1616694638; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=1FeCnC7WnMeWEj1gEjR7GjHOI7D/MTEeLRIo3/QZ87I=; b=bZAJb6mLyiAMxcYv0r40lqxfJfIz0ihLWV4bk7vzIJNh5hPJ+eEuKDstI9YikR9cKGp6Ll TcgCo2d5VBwmadG4q4RyCi0yXJ+VP4ewWHYbqcT0TYOGWe0SgOquLhuLOnytimqlhZVJUk 2M+e8raAOYjJaSFiw4KyiNuQ+mb6z9I= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 52C86AC16; Thu, 25 Mar 2021 17:50:38 +0000 (UTC) Date: Thu, 25 Mar 2021 18:50:30 +0100 From: Michal Hocko To: Johannes Weiner Cc: Arjun Roy , Arjun Roy , Andrew Morton , David Miller , netdev , Linux Kernel Mailing List , Cgroups , Linux MM , Shakeel Butt , Eric Dumazet , Soheil Hassas Yeganeh , Jakub Kicinski , Yang Shi , Roman Gushchin Subject: Re: [mm, net-next v2] mm: net: memcg accounting for TCP rx zerocopy Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: xctfw3tws1ah89eomkdot3ecfcjphgkk X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 783A540002E1 Received-SPF: none (suse.com>: No applicable sender policy available) receiver=imf26; identity=mailfrom; envelope-from=""; helo=mx2.suse.de; client-ip=195.135.220.15 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616694638-196074 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu 25-03-21 12:47:04, Johannes Weiner wrote: > On Thu, Mar 25, 2021 at 10:02:28AM +0100, Michal Hocko wrote: > > On Wed 24-03-21 15:49:15, Arjun Roy wrote: > > > On Wed, Mar 24, 2021 at 2:24 PM Johannes Weiner wrote: > > > > > > > > On Wed, Mar 24, 2021 at 10:12:46AM +0100, Michal Hocko wrote: > > > > > On Tue 23-03-21 11:47:54, Arjun Roy wrote: > > > > > > On Tue, Mar 23, 2021 at 7:34 AM Michal Hocko wrote: > > > > > > > > > > > > > > On Wed 17-03-21 18:12:55, Johannes Weiner wrote: > > > > > > > [...] > > > > > > > > Here is an idea of how it could work: > > > > > > > > > > > > > > > > struct page already has > > > > > > > > > > > > > > > > struct { /* page_pool used by netstack */ > > > > > > > > /** > > > > > > > > * @dma_addr: might require a 64-bit value even on > > > > > > > > * 32-bit architectures. > > > > > > > > */ > > > > > > > > dma_addr_t dma_addr; > > > > > > > > }; > > > > > > > > > > > > > > > > and as you can see from its union neighbors, there is quite a bit more > > > > > > > > room to store private data necessary for the page pool. > > > > > > > > > > > > > > > > When a page's refcount hits zero and it's a networking page, we can > > > > > > > > feed it back to the page pool instead of the page allocator. > > > > > > > > > > > > > > > > From a first look, we should be able to use the PG_owner_priv_1 page > > > > > > > > flag for network pages (see how this flag is overloaded, we can add a > > > > > > > > PG_network alias). With this, we can identify the page in __put_page() > > > > > > > > and __release_page(). These functions are already aware of different > > > > > > > > types of pages and do their respective cleanup handling. We can > > > > > > > > similarly make network a first-class citizen and hand pages back to > > > > > > > > the network allocator from in there. > > > > > > > > > > > > > > For compound pages we have a concept of destructors. Maybe we can extend > > > > > > > that for order-0 pages as well. The struct page is heavily packed and > > > > > > > compound_dtor shares the storage without other metadata > > > > > > > int pages; /* 16 4 */ > > > > > > > unsigned char compound_dtor; /* 16 1 */ > > > > > > > atomic_t hpage_pinned_refcount; /* 16 4 */ > > > > > > > pgtable_t pmd_huge_pte; /* 16 8 */ > > > > > > > void * zone_device_data; /* 16 8 */ > > > > > > > > > > > > > > But none of those should really require to be valid when a page is freed > > > > > > > unless I am missing something. It would really require to check their > > > > > > > users whether they can leave the state behind. But if we can establish a > > > > > > > contract that compound_dtor can be always valid when a page is freed > > > > > > > this would be really a nice and useful abstraction because you wouldn't > > > > > > > have to care about the specific type of page. > > > > > > > > Yeah technically nobody should leave these fields behind, but it > > > > sounds pretty awkward to manage an overloaded destructor with a > > > > refcounted object: > > > > > > > > Either every put would have to check ref==1 before to see if it will > > > > be the one to free the page, and then set up the destructor before > > > > putting the final ref. But that means we can't support lockless > > > > tryget() schemes like we have in the page cache with a destructor. > > > > I do not follow the ref==1 part. I mean to use the hugetlb model where > > the destructore is configured for the whole lifetime until the page is > > freed back to the allocator (see below). > > That only works if the destructor field doesn't overlap with a member > the page type itself doesn't want to use. Page types that do want to > use it would need to keep that field exclusive. Right. > We couldn't use it for LRU pages e.g. because it overlaps with the > lru.next pointer. Dang, I have completely missed this. I was looking at pahole because struct page is unreadable in the C code but I tricked myself to only look at offset 16. The initial set of candidate looked really promissing. But overlapping with list_head is a deal breaker. This makes use of dtor for most order-0 pages indeed unfeasible. Maybe dtor can be rellocated but that is certain a rabbit hole people (rightfully) avoid as much as possible. So you are right and going with networking specific way is more reasonable. [...] > So again, yes it would be nice to have generic destructors, but I just > don't see how it's practical. just to clarify on this. I didn't really mean to use this mechanism to all/most pages I just wanted to have PageHasDestructor rather than PageNetwork because both would express a special nead for freeing but that would require that the dtor would be outside of lru. Thanks! -- Michal Hocko SUSE Labs