From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AECACC43460 for ; Mon, 17 May 2021 07:46:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 327B061209 for ; Mon, 17 May 2021 07:46:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 327B061209 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 61F5E6B0036; Mon, 17 May 2021 03:46:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5CF546B006E; Mon, 17 May 2021 03:46:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 448F26B0070; Mon, 17 May 2021 03:46:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0162.hostedemail.com [216.40.44.162]) by kanga.kvack.org (Postfix) with ESMTP id 102986B0036 for ; Mon, 17 May 2021 03:46:21 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 8DC558249980 for ; Mon, 17 May 2021 07:46:20 +0000 (UTC) X-FDA: 78149940120.20.7FC45B2 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf24.hostedemail.com (Postfix) with ESMTP id 374AFA000384 for ; Mon, 17 May 2021 07:46:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621237579; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oW3LzEcy6xhREj6wjyfZMKHY6gt96o4WsFjzrVstiLo=; b=NC3lCaM/WB347Ie9WWVZ3r1BWKOEb6N2vV+TXWT97ndrvdYRoewC5ROjnAT9rN6C8RCveo SKEjPXfYZ0OTKLknAMGxGxgRpAjmBNmclo0Jhtd8Xhv/XeSOcvZHaviSCaG9qLYDw+OKgY +gBzd3rwaUtoT5jHfszEbMrvJPQNCTY= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-536-WgE-qH-6OSOGCjtrPBNw-g-1; Mon, 17 May 2021 03:46:15 -0400 X-MC-Unique: WgE-qH-6OSOGCjtrPBNw-g-1 Received: by mail-wm1-f72.google.com with SMTP id g206-20020a1c39d70000b029016ac627fbe9so1094080wma.9 for ; Mon, 17 May 2021 00:46:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=oW3LzEcy6xhREj6wjyfZMKHY6gt96o4WsFjzrVstiLo=; b=FIIR4LSU5S1jXpAH/TkxbikUpBlZkE0gNnflxPU2SPM1mxdMJuxz6LdQxXmO9yyEZs g4MU1ZU5b/3U3VDgK1kDKiXnLWU5dZ2HMfKMy56ZtKE8pWiZpH0MDawxf+R2FbBQiAMH 64adeKavLJA5BCu6b7Sn06AOh6mFGkwBLXOnI/wHgL35cyICo4pjdOLuQmD+jHK3n5gU tSMoqDEehF8BX2qqHtbA+CrqXMGkesLk9csRJNsGgGFr8ewtoXtpleWQUufQLdyxh6bz K+dff3/ZoCdRjQ19xB3KM63StJktT/q4ieCLGT7NfEnqsjeNtizub46UJ0BUrbti0Zi2 lB9g== X-Gm-Message-State: AOAM530NX2Uv5BpaH1HF47gN5AO+D+BzKsmK9Y2odZtq0MnBN5uQhP3b Opl0UA+bN+jGJJn6nBE0ZdEN64bxFCXPpqdjh1WL7qHjqdTV1VCmCvRUH4AQ+j6hrnPRTuubpdK 7ou+540NGCAjQZxuZpNLlErEnKkn3EZPsdJzm9ud1Jaxckq265z87DWq0HTE= X-Received: by 2002:a5d:438c:: with SMTP id i12mr26903818wrq.44.1621237574562; Mon, 17 May 2021 00:46:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwROdnuXYbqFLPiHMKE8n+P0duVXjs1e5WKix8FiCyLOl1Qa/HiqWLT3lnItQNHmk8XZbmiTQ== X-Received: by 2002:a5d:438c:: with SMTP id i12mr26903768wrq.44.1621237574201; Mon, 17 May 2021 00:46:14 -0700 (PDT) Received: from [192.168.3.132] (p5b0c6833.dip0.t-ipconnect.de. [91.12.104.51]) by smtp.gmail.com with ESMTPSA id f4sm16730658wrz.33.2021.05.17.00.46.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 17 May 2021 00:46:13 -0700 (PDT) Subject: Re: alloc_contig_range() with MIGRATE_MOVABLE performance regression since 4.9 To: Florian Fainelli , Michal Hocko Cc: Vlastimil Babka , Mel Gorman , Minchan Kim , Johannes Weiner , l.stach@pengutronix.de, LKML , Jaewon Kim , Michal Nazarewicz , Joonsoo Kim , Oscar Salvador , "linux-mm@kvack.org" References: <8919b724-ce5b-a80f-bbea-98b99af97357@redhat.com> <58726a6b-5468-a6b4-7c26-371ef5d71ee2@gmail.com> <9df905cf-cc4f-c739-26cb-c2e5c6e5a234@redhat.com> From: David Hildenbrand Organization: Red Hat Message-ID: Date: Mon, 17 May 2021 09:46:12 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="NC3lCaM/"; spf=none (imf24.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 374AFA000384 X-Stat-Signature: sm556b5tf7cchp4akusaj41cnqboa5c6 X-HE-Tag: 1621237578-34596 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 16.05.21 18:13, Florian Fainelli wrote: >=20 >=20 > On 4/22/2021 12:31 PM, Florian Fainelli wrote: >>> For >>> >>> https://lkml.kernel.org/r/20210121175502.274391-3-minchan@kernel.org >>> >>> to do its work you'll have to pass=C2=A0 __GFP_NORETRY to >>> alloc_contig_range(). This requires CMA adaptions, from where we call >>> alloc_contig_range(). >> >> Yes, I did modify the alloc_contig_range() caller to pass GFP_KERNEL | >> __GFP_NORETRY. I did run for a more iterations (1000) and the results >> are not very conclusive as with __GFP_NORETRY the allocation time per >> allocation was not significantly better, in fact it was slightly worse >> by 100us than without. >> >> My x86 VM with 1GB of DRAM including 512MB being in ZONE_MOVABLE does >> shows identical numbers for both 4.9 and 5.4 so this must be something >> specific to ARM64 and/or the code we added to create a ZONE_MOVABLE on >> that architecture since movablecore does not appear to have any effect >> unlike x86. >=20 > We tracked down the slowdowns to be caused by two major contributors: >=20 > - for a reason that we do not fully understand yet the same cpufreq > governor (conservative) did not cause alloc_contig_range() to be slowed > down on 4.9 as much as it it with 5.4, running tests with the > performance cpufreq governor works a tad better and the results are mor= e > consistent from run to run with a smaller variation. Interesting! So your CPU is down-clocking while performing (heavy)=20 kernel work? Is that expected or are we mis-accounting kernel cpu time=20 somehow when it comes to determining the CPU target frequency? >=20 > - another large contributor to the slowdown was having enabled > CONFIG_IRQSOFF_TRACER. After c3bc8fd637a9623f5c507bd18f9677effbddf584 > ("tracing: Centralize preemptirq tracepoints and unify their usage") we > now prepare arguments for tracing even if we end-up not using them sinc= e > tracing is not enabled at runtime. Getting the caller function's return > address is cheap on arm64 for level =3D=3D 0, but getting the preceding > caller involves doing a backtrace walk which is expensive (see > arch/arm64/kernel/return_address.c). Again, very interesting finding. >=20 > So with these two variables eliminated we are only about x2 slower on > 5.4 than we were on 4.9 and this is acceptable for our use case. I woul= d > not say the case is closed but at least we understand it better. We now > have 5.10 brought up to speed so any new investigation will be focused > on that kernel. >=20 Thanks for the insight, please do let me know when you learn more. x2=20 slowdown still is quite a lot. --=20 Thanks, David / dhildenb