From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7609C433B4 for ; Sun, 16 May 2021 16:13:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 58CEC6115C for ; Sun, 16 May 2021 16:13:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 58CEC6115C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AD72F6B0070; Sun, 16 May 2021 12:13:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A883E6B0071; Sun, 16 May 2021 12:13:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 901696B0072; Sun, 16 May 2021 12:13:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0131.hostedemail.com [216.40.44.131]) by kanga.kvack.org (Postfix) with ESMTP id 5BF596B0070 for ; Sun, 16 May 2021 12:13:54 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id E46118249980 for ; Sun, 16 May 2021 16:13:53 +0000 (UTC) X-FDA: 78147590346.03.8E3C418 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf27.hostedemail.com (Postfix) with ESMTP id EB08980192E6 for ; Sun, 16 May 2021 16:13:51 +0000 (UTC) Received: by mail-pl1-f174.google.com with SMTP id 69so1865915plc.5 for ; Sun, 16 May 2021 09:13:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=dlfMqv9wfrGJ82dGdPYDeZ8xiu5IJiVrMvefg8fg1h0=; b=OG/wbGXafii2uuL2ViZem+b+tSNCCwrA42+zMs+79hdTr6lyCSx4vySbm+hwv5JHN3 Uoo4zpZZHFspu77OhK35a2pDrSkC5pPo3dfo/KdrTwNagH3WoWjH6vuHH16a0YjbJ4ID Wq7G+7TLzl66GT7xIn/lMChmkNIzuY8ZScEP940+tGAO6pSWYsrDY8wDLVuulpXLy04b xfb+BYk/5v/8LZvHamrAchsEvF4LPlzDpeDCfrCn/JCodU1cXs1AQ8xr2QkmNHWMKRNp 4qdyp8nMc3LzEa+PGrPjH7bymK+Oap+k+NO418r0HrDavO+jFFq+3VQkSmGxKpv/cHoc UyCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=dlfMqv9wfrGJ82dGdPYDeZ8xiu5IJiVrMvefg8fg1h0=; b=scq38el62rYzBzIEJfZDJ3l18oxTvq5J0tiU97dng6PLn4h9X3w8xCLu/eoPdJMrsg CuEHFOSmL2ZeJoauyypTyVCDzxnQdAdf1VRS/oYj2PkvIYNDzwzhGIuOj1+8ilR4GRCw c0MX40JzTT++RDqNccQdDfAbd9g/BQjg3rUOtcSFStLNFJ4IiKDmrRqCjMTzXIgYtXfa 6vWQjV14E1/y9bWzsszKdNhADREpgJgUYI2IDD4yS2Zwcb+0VSwZWzn5SXlrC/i0nMud JXuAmTAnm5HFDoGgESjXwF3dQnSX0wypwtMPF5rAcf9uMJY+uGz4qgJ0VAq2+Fv8M64w evpA== X-Gm-Message-State: AOAM533dnw8hZnIMbqVWxg0m4i/gH909fAjR38px83UAzGH/J+71EfCF /C7K16x/Qlo4C23hoMOS8M8F8jCUqHw= X-Google-Smtp-Source: ABdhPJxze5aQgQw4bgIhABio1KoTULA2u8EBkiqDYbtNPv9gs5mrJigXuIFtBcDE0zXmbkpeNEBPCA== X-Received: by 2002:a17:902:b18f:b029:ec:7ac0:fd1a with SMTP id s15-20020a170902b18fb02900ec7ac0fd1amr56232059plr.84.1621181632031; Sun, 16 May 2021 09:13:52 -0700 (PDT) Received: from [10.230.29.202] ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id k15sm8258105pfi.0.2021.05.16.09.13.44 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 16 May 2021 09:13:51 -0700 (PDT) Subject: Re: alloc_contig_range() with MIGRATE_MOVABLE performance regression since 4.9 From: Florian Fainelli To: David Hildenbrand , Michal Hocko Cc: Vlastimil Babka , Mel Gorman , Minchan Kim , Johannes Weiner , l.stach@pengutronix.de, LKML , Jaewon Kim , Michal Nazarewicz , Joonsoo Kim , Oscar Salvador , "linux-mm@kvack.org" References: <8919b724-ce5b-a80f-bbea-98b99af97357@redhat.com> <58726a6b-5468-a6b4-7c26-371ef5d71ee2@gmail.com> <9df905cf-cc4f-c739-26cb-c2e5c6e5a234@redhat.com> Message-ID: Date: Sun, 16 May 2021 09:13:41 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b="OG/wbGXa"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of ffainelli@gmail.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=ffainelli@gmail.com X-Stat-Signature: cdihn355hmz49who9tdz9msb8kkuxeyg X-Rspamd-Queue-Id: EB08980192E6 X-Rspamd-Server: rspam02 X-HE-Tag: 1621181631-270165 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 4/22/2021 12:31 PM, Florian Fainelli wrote: >> For >> >> https://lkml.kernel.org/r/20210121175502.274391-3-minchan@kernel.org >> >> to do its work you'll have to pass=C2=A0 __GFP_NORETRY to >> alloc_contig_range(). This requires CMA adaptions, from where we call >> alloc_contig_range(). >=20 > Yes, I did modify the alloc_contig_range() caller to pass GFP_KERNEL | > __GFP_NORETRY. I did run for a more iterations (1000) and the results > are not very conclusive as with __GFP_NORETRY the allocation time per > allocation was not significantly better, in fact it was slightly worse > by 100us than without. >=20 > My x86 VM with 1GB of DRAM including 512MB being in ZONE_MOVABLE does > shows identical numbers for both 4.9 and 5.4 so this must be something > specific to ARM64 and/or the code we added to create a ZONE_MOVABLE on > that architecture since movablecore does not appear to have any effect > unlike x86. We tracked down the slowdowns to be caused by two major contributors: - for a reason that we do not fully understand yet the same cpufreq governor (conservative) did not cause alloc_contig_range() to be slowed down on 4.9 as much as it it with 5.4, running tests with the performance cpufreq governor works a tad better and the results are more consistent from run to run with a smaller variation. - another large contributor to the slowdown was having enabled CONFIG_IRQSOFF_TRACER. After c3bc8fd637a9623f5c507bd18f9677effbddf584 ("tracing: Centralize preemptirq tracepoints and unify their usage") we now prepare arguments for tracing even if we end-up not using them since tracing is not enabled at runtime. Getting the caller function's return address is cheap on arm64 for level =3D=3D 0, but getting the preceding caller involves doing a backtrace walk which is expensive (see arch/arm64/kernel/return_address.c). So with these two variables eliminated we are only about x2 slower on 5.4 than we were on 4.9 and this is acceptable for our use case. I would not say the case is closed but at least we understand it better. We now have 5.10 brought up to speed so any new investigation will be focused on that kernel. Thanks a lot for your help David! --=20 Florian