From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BEBCC433B4 for ; Fri, 16 Apr 2021 21:09:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B1343613C3 for ; Fri, 16 Apr 2021 21:09:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B1343613C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4F85C8D0001; Fri, 16 Apr 2021 17:09:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4A8496B0070; Fri, 16 Apr 2021 17:09:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 370338D0001; Fri, 16 Apr 2021 17:09:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0163.hostedemail.com [216.40.44.163]) by kanga.kvack.org (Postfix) with ESMTP id 1A6AA6B006E for ; Fri, 16 Apr 2021 17:09:21 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id D1EE65DD8 for ; Fri, 16 Apr 2021 21:09:20 +0000 (UTC) X-FDA: 78039470880.20.F517B9D Received: from mail-il1-f176.google.com (mail-il1-f176.google.com [209.85.166.176]) by imf21.hostedemail.com (Postfix) with ESMTP id 597A1E000113 for ; Fri, 16 Apr 2021 21:09:18 +0000 (UTC) Received: by mail-il1-f176.google.com with SMTP id r5so16050271ilb.2 for ; Fri, 16 Apr 2021 14:09:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=Ku325xR6gcu6Ab0160Ej5sx3fi9r5K88bMC216XFIYo=; b=aF3FqbqNEYZEuN0ip83URgZDR/Uk5SWI4meS4WGuJOS7Yt72fYMg8ZmgCLd148ML18 R7ozEe6p2bPleQRzZGQ1vjD9rICsAYoVt+7c27wr5Go2FO9UjCe2EQxzaIXtZIKxIdOm qZRSA26tiIk7N8XJHCuveJwBkokWmqqDWUH0MCdE1brJlVOa4YRYhLKSAUaXu4wme6Je zx5PiFTbDtuNt9NHrD5yszeh1O09o++KGqYOPruf72aqcBXij8lb7d3LPwdk7DtCQbzc aavel/nIpRELONiW1EKow/rAuvsN+uNPPRCeJaDkFDIGnnaupCOzlrPDw7IW80HL+09b v3cA== X-Gm-Message-State: AOAM5307WARpjkwAhw2Q2NWyFOqQ2hhHuviHSntFpNqv4rFZOA+aZrr3 2MWUPw82ZwCjIv/GrGZjWss= X-Google-Smtp-Source: ABdhPJySvdl8wEWlZ8KAwhuM6eRd0zx+FsVY7Ptfz+lSoY969IE0mg5VyOC8981H+HI2mL37KZvM9A== X-Received: by 2002:a05:6e02:10cf:: with SMTP id s15mr8848576ilj.302.1618607359891; Fri, 16 Apr 2021 14:09:19 -0700 (PDT) Received: from google.com (243.199.238.35.bc.googleusercontent.com. [35.238.199.243]) by smtp.gmail.com with ESMTPSA id j2sm3003494ila.2.2021.04.16.14.09.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:09:19 -0700 (PDT) Date: Fri, 16 Apr 2021 21:09:18 +0000 From: Dennis Zhou To: Roman Gushchin Cc: Tejun Heo , Christoph Lameter , Andrew Morton , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 4/6] percpu: generalize pcpu_balance_populated() Message-ID: References: <20210408035736.883861-1-guro@fb.com> <20210408035736.883861-5-guro@fb.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210408035736.883861-5-guro@fb.com> X-Stat-Signature: bagssrtt4rain5u39t8dciaxgq5d4rn8 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 597A1E000113 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf21; identity=mailfrom; envelope-from=""; helo=mail-il1-f176.google.com; client-ip=209.85.166.176 X-HE-DKIM-Result: none/none X-HE-Tag: 1618607358-658696 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hello, On Wed, Apr 07, 2021 at 08:57:34PM -0700, Roman Gushchin wrote: > To prepare for the depopulation of percpu chunks, split out the > populating part of the pcpu_balance_populated() into the new > pcpu_grow_populated() (with an intention to add > pcpu_shrink_populated() in the next commit). > > The goal of pcpu_balance_populated() is to determine whether > there is a shortage or an excessive amount of empty percpu pages > and call into the corresponding function. > > pcpu_grow_populated() takes a desired number of pages as an argument > (nr_to_pop). If it creates a new chunk, nr_to_pop should be updated > to reflect that the new chunk could be created already populated. > Otherwise an infinite loop might appear. > > Signed-off-by: Roman Gushchin > --- > mm/percpu.c | 63 +++++++++++++++++++++++++++++++++-------------------- > 1 file changed, 39 insertions(+), 24 deletions(-) > > diff --git a/mm/percpu.c b/mm/percpu.c > index 61339b3d9337..e20119668c42 100644 > --- a/mm/percpu.c > +++ b/mm/percpu.c > @@ -1979,7 +1979,7 @@ static void pcpu_balance_free(enum pcpu_chunk_type type) > } > > /** > - * pcpu_balance_populated - manage the amount of populated pages > + * pcpu_grow_populated - populate chunk(s) to satisfy atomic allocations > * @type: chunk type > * > * Maintain a certain amount of populated pages to satisfy atomic allocations. > @@ -1988,35 +1988,15 @@ static void pcpu_balance_free(enum pcpu_chunk_type type) > * allocation causes the failure as it is possible that requests can be > * serviced from already backed regions. > */ > -static void pcpu_balance_populated(enum pcpu_chunk_type type) > +static void pcpu_grow_populated(enum pcpu_chunk_type type, int nr_to_pop) > { > /* gfp flags passed to underlying allocators */ > const gfp_t gfp = GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN; > struct list_head *pcpu_slot = pcpu_chunk_list(type); > struct pcpu_chunk *chunk; > - int slot, nr_to_pop, ret; > + int slot, ret; > > - /* > - * Ensure there are certain number of free populated pages for > - * atomic allocs. Fill up from the most packed so that atomic > - * allocs don't increase fragmentation. If atomic allocation > - * failed previously, always populate the maximum amount. This > - * should prevent atomic allocs larger than PAGE_SIZE from keeping > - * failing indefinitely; however, large atomic allocs are not > - * something we support properly and can be highly unreliable and > - * inefficient. > - */ > retry_pop: > - if (pcpu_atomic_alloc_failed) { > - nr_to_pop = PCPU_EMPTY_POP_PAGES_HIGH; > - /* best effort anyway, don't worry about synchronization */ > - pcpu_atomic_alloc_failed = false; > - } else { > - nr_to_pop = clamp(PCPU_EMPTY_POP_PAGES_HIGH - > - pcpu_nr_empty_pop_pages[type], > - 0, PCPU_EMPTY_POP_PAGES_HIGH); > - } > - > for (slot = pcpu_size_to_slot(PAGE_SIZE); slot < pcpu_nr_slots; slot++) { > unsigned int nr_unpop = 0, rs, re; > > @@ -2060,12 +2040,47 @@ static void pcpu_balance_populated(enum pcpu_chunk_type type) > if (chunk) { > spin_lock_irq(&pcpu_lock); > pcpu_chunk_relocate(chunk, -1); > + nr_to_pop = max_t(int, 0, nr_to_pop - chunk->nr_populated); > spin_unlock_irq(&pcpu_lock); > - goto retry_pop; > + if (nr_to_pop) > + goto retry_pop; > } > } > } > > +/** > + * pcpu_balance_populated - manage the amount of populated pages > + * @type: chunk type > + * > + * Populate or depopulate chunks to maintain a certain amount > + * of free pages to satisfy atomic allocations, but not waste > + * large amounts of memory. > + */ > +static void pcpu_balance_populated(enum pcpu_chunk_type type) > +{ > + int nr_to_pop; > + > + /* > + * Ensure there are certain number of free populated pages for > + * atomic allocs. Fill up from the most packed so that atomic > + * allocs don't increase fragmentation. If atomic allocation > + * failed previously, always populate the maximum amount. This > + * should prevent atomic allocs larger than PAGE_SIZE from keeping > + * failing indefinitely; however, large atomic allocs are not > + * something we support properly and can be highly unreliable and > + * inefficient. > + */ > + if (pcpu_atomic_alloc_failed) { > + nr_to_pop = PCPU_EMPTY_POP_PAGES_HIGH; > + /* best effort anyway, don't worry about synchronization */ > + pcpu_atomic_alloc_failed = false; > + pcpu_grow_populated(type, nr_to_pop); > + } else if (pcpu_nr_empty_pop_pages[type] < PCPU_EMPTY_POP_PAGES_HIGH) { > + nr_to_pop = PCPU_EMPTY_POP_PAGES_HIGH - pcpu_nr_empty_pop_pages[type]; > + pcpu_grow_populated(type, nr_to_pop); > + } > +} > + > /** > * pcpu_balance_workfn - manage the amount of free chunks and populated pages > * @work: unused > -- > 2.30.2 > I've applied this for-5.14. Thanks, Dennis