From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2103BC433E0 for ; Fri, 15 May 2020 07:41:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A615C2065C for ; Fri, 15 May 2020 07:41:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A615C2065C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1AA4B90000C; Fri, 15 May 2020 03:41:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 180A38E0005; Fri, 15 May 2020 03:41:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0967290000C; Fri, 15 May 2020 03:41:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0083.hostedemail.com [216.40.44.83]) by kanga.kvack.org (Postfix) with ESMTP id E0C338E0005 for ; Fri, 15 May 2020 03:41:29 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9406D82499A8 for ; Fri, 15 May 2020 07:41:29 +0000 (UTC) X-FDA: 76818158298.09.steam79_472f73cf59c51 X-HE-Tag: steam79_472f73cf59c51 X-Filterd-Recvd-Size: 8284 Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Fri, 15 May 2020 07:41:29 +0000 (UTC) Received: by mail-wr1-f68.google.com with SMTP id h17so2301508wrc.8 for ; Fri, 15 May 2020 00:41:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=kns14HbaP0c1lE2MwJ7v5s2pJzjyzLnxLUIpMzdDsM4=; b=CWcxxw+F2SwNzyjYv+CFJp7eGixV9/dzGAg0tSnEKqQhjEa9coFFtm1I1Ubl/U1mx6 8WslBbWZnoOTP/x8eKkjDxVTG5qd3qaTwIspQnKICDLg98KODo1GmjF2FBXn2a0jXtKJ kxEzZ94mONEM/C4Y6kBxhzX8Z2F2isLN28GxTAMeF3XRR1x4Wy4iijp+v05pqinqKNlI RiH+gYDdnxpOEHzVrg/tbTOH0sQvPVZASarwXLc9waemJ22Jz4OOoMmK7uQCbe1jILl8 GtY/Dtv2YSC142yKanxX2+9dbYuGBBTdtpN0PYVflumylVl5lNiZ+JrckXB6r61Ikbev Vu0w== X-Gm-Message-State: AOAM530WbyFRtjEGirIDvzxzfJmtWHSiBFPCG2vCoyX8tjWbnGxoVr2l jW2vrDdqKg+vO3FFgYv+PB8= X-Google-Smtp-Source: ABdhPJxkQm4wcb2otWahvuSXI6T8PGegoI+XGijl8LbVKi7oXi9T6Mpv1K0OLZpEZXlo126o5SxfeA== X-Received: by 2002:adf:fa92:: with SMTP id h18mr2800656wrr.260.1589528487966; Fri, 15 May 2020 00:41:27 -0700 (PDT) Received: from localhost (ip-37-188-249-36.eurotel.cz. [37.188.249.36]) by smtp.gmail.com with ESMTPSA id t22sm2216596wmj.37.2020.05.15.00.41.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 May 2020 00:41:27 -0700 (PDT) Date: Fri, 15 May 2020 09:41:25 +0200 From: Michal Hocko To: Feng Tang Cc: Andrew Morton , Johannes Weiner , Matthew Wilcox , Mel Gorman , Kees Cook , Luis Chamberlain , Iurii Zaikin , andi.kleen@intel.com, tim.c.chen@intel.com, dave.hansen@intel.com, ying.huang@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 3/3] mm: adjust vm_committed_as_batch according to vm overcommit policy Message-ID: <20200515074125.GH29153@dhcp22.suse.cz> References: <1588922717-63697-1-git-send-email-feng.tang@intel.com> <1588922717-63697-4-git-send-email-feng.tang@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1588922717-63697-4-git-send-email-feng.tang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri 08-05-20 15:25:17, Feng Tang wrote: > When checking a performance change for will-it-scale scalability > mmap test [1], we found very high lock contention for spinlock of > percpu counter 'vm_committed_as': > > 94.14% 0.35% [kernel.kallsyms] [k] _raw_spin_lock_irqsave > 48.21% _raw_spin_lock_irqsave;percpu_counter_add_batch;__vm_enough_memory;mmap_region;do_mmap; > 45.91% _raw_spin_lock_irqsave;percpu_counter_add_batch;__do_munmap; > > Actually this heavy lock contention is not always necessary. The > 'vm_committed_as' needs to be very precise when the strict > OVERCOMMIT_NEVER policy is set, which requires a rather small batch > number for the percpu counter. > > So lift the batch number to 16X for OVERCOMMIT_ALWAYS and > OVERCOMMIT_GUESS policies, and add a sysctl handler to adjust it > when the policy is reconfigured. Increasing the batch size for weaker overcommit modes makes sense. But your patch is also tuning OVERCOMMIT_NEVER without any explanation why that is still "small enough to be precise". > Benchmark with the same testcase in [1] shows 53% improvement on a > 8C/16T desktop, and 2097%(20X) on a 4S/72C/144T server. And no change > for some platforms, due to the test mmap size of the case is bigger > than the batch number computed, though the patch will help mmap/munmap > generally. > > [1] https://lkml.org/lkml/2020/3/5/57 Please do not use lkml.org links in the changelog. Use http://lkml.kernel.org/r/$msg instead. > Signed-off-by: Feng Tang > --- > include/linux/mm.h | 2 ++ > include/linux/mman.h | 4 ++++ > kernel/sysctl.c | 2 +- > mm/mm_init.c | 19 +++++++++++++++---- > mm/util.c | 13 +++++++++++++ > 5 files changed, 35 insertions(+), 5 deletions(-) > > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 5a32342..bc3722f 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -205,6 +205,8 @@ extern int overcommit_ratio_handler(struct ctl_table *, int, void __user *, > size_t *, loff_t *); > extern int overcommit_kbytes_handler(struct ctl_table *, int, void __user *, > size_t *, loff_t *); > +extern int overcommit_policy_handler(struct ctl_table *, int, void __user *, > + size_t *, loff_t *); > > #define nth_page(page,n) pfn_to_page(page_to_pfn((page)) + (n)) > > diff --git a/include/linux/mman.h b/include/linux/mman.h > index 4b08e9c..91c93c1 100644 > --- a/include/linux/mman.h > +++ b/include/linux/mman.h > @@ -57,8 +57,12 @@ extern struct percpu_counter vm_committed_as; > > #ifdef CONFIG_SMP > extern s32 vm_committed_as_batch; > +extern void mm_compute_batch(void); > #else > #define vm_committed_as_batch 0 > +static inline void mm_compute_batch(void) > +{ > +} > #endif > > unsigned long vm_memory_committed(void); > diff --git a/kernel/sysctl.c b/kernel/sysctl.c > index 8a176d8..6fa552d 100644 > --- a/kernel/sysctl.c > +++ b/kernel/sysctl.c > @@ -1278,7 +1278,7 @@ static struct ctl_table vm_table[] = { > .data = &sysctl_overcommit_memory, > .maxlen = sizeof(sysctl_overcommit_memory), > .mode = 0644, > - .proc_handler = proc_dointvec_minmax, > + .proc_handler = overcommit_policy_handler, > .extra1 = SYSCTL_ZERO, > .extra2 = &two, > }, > diff --git a/mm/mm_init.c b/mm/mm_init.c > index 7da6991..1654358 100644 > --- a/mm/mm_init.c > +++ b/mm/mm_init.c > @@ -13,6 +13,7 @@ > #include > #include > #include > +#include > #include "internal.h" > > #ifdef CONFIG_DEBUG_MEMORY_INIT > @@ -140,16 +141,26 @@ EXPORT_SYMBOL_GPL(mm_kobj); > #ifdef CONFIG_SMP > s32 vm_committed_as_batch = 32; > > -static void __meminit mm_compute_batch(void) > +void mm_compute_batch(void) > { > u64 memsized_batch; > s32 nr = num_present_cpus(); > s32 batch = max_t(s32, nr*2, 32); > - > - /* batch size set to 0.4% of (total memory/#cpus), or max int32 */ > - memsized_batch = min_t(u64, (totalram_pages()/nr)/256, 0x7fffffff); > + unsigned long ram_pages = totalram_pages(); > + > + /* > + * For policy of OVERCOMMIT_NEVER, set batch size to 0.4% > + * of (total memory/#cpus), and lift it to 6.25% for other > + * policies to easy the possible lock contention for percpu_counter > + * vm_committed_as, while the max limit is INT_MAX > + */ > + if (sysctl_overcommit_memory == OVERCOMMIT_NEVER) > + memsized_batch = min_t(u64, ram_pages/nr/256, INT_MAX); > + else > + memsized_batch = min_t(u64, ram_pages/nr/16, INT_MAX); > > vm_committed_as_batch = max_t(s32, memsized_batch, batch); > + printk("vm_committed_as_batch = %d\n", vm_committed_as_batch); > } > > static int __meminit mm_compute_batch_notifier(struct notifier_block *self, > diff --git a/mm/util.c b/mm/util.c > index 3de78e9..99936d3 100644 > --- a/mm/util.c > +++ b/mm/util.c > @@ -729,6 +729,19 @@ int overcommit_ratio_handler(struct ctl_table *table, int write, > return ret; > } > > +int overcommit_policy_handler(struct ctl_table *table, int write, > + void __user *buffer, size_t *lenp, > + loff_t *ppos) > +{ > + int ret; > + > + ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos); > + if (ret == 0 && write) > + mm_compute_batch(); > + > + return ret; > +} > + > int overcommit_kbytes_handler(struct ctl_table *table, int write, > void __user *buffer, size_t *lenp, > loff_t *ppos) > -- > 2.7.4 -- Michal Hocko SUSE Labs