From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EDE1C433F5 for ; Sun, 13 Mar 2022 21:10:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8C0696B0072; Sun, 13 Mar 2022 17:10:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 86F5C6B0073; Sun, 13 Mar 2022 17:10:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 736356B0074; Sun, 13 Mar 2022 17:10:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id 61E076B0072 for ; Sun, 13 Mar 2022 17:10:25 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 3209C6034E for ; Sun, 13 Mar 2022 21:10:25 +0000 (UTC) X-FDA: 79240606410.02.DA6976A Received: from mail-yw1-f171.google.com (mail-yw1-f171.google.com [209.85.128.171]) by imf07.hostedemail.com (Postfix) with ESMTP id 9EBC940028 for ; Sun, 13 Mar 2022 21:10:24 +0000 (UTC) Received: by mail-yw1-f171.google.com with SMTP id 00721157ae682-2dc28791ecbso143511247b3.4 for ; Sun, 13 Mar 2022 14:10:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=7BcrhjiAUP2vD6MkcaltfQ6gzbMYeWrrnPqDFWPSb7o=; b=HyMlGeHVYZt5dRa71d44WLx/Uiv+k8VOexRZ6IXTBqjbcYXOqER6FVM/s85ZEBzLqE GdC2amBwxzgLU+6rdnKiS87gem+827830aRBqCk60Ae+YPwjfjCM8IgPc8FNGDsTVdPM oBQm2BvYNIrjsIhjNemkmBddnFR/rZCH1hXlGoXAiZ5mH824z+lWvRFtFgKQuynC4tkC Hcb1xSDKI3JP3+57bP7YVLDjA+7Oar9sq+PFTtyBA9fPmhIVrk4Xzs1PtOJXeZdgpwkc DJDBGylaNIusfZK3npj7WOf/LDPNzsilZ/WlcZRGir1SQ8Sw+qyoMFkZk6XEpzpoViwL D4Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7BcrhjiAUP2vD6MkcaltfQ6gzbMYeWrrnPqDFWPSb7o=; b=j8RNbrf7n/IHgh/sk/DDYxfvsqv2FunTKGIrkv6fIgV8e4Wi698/Y1StFze1/Susd6 wAI9fF8i+/ZVy3H64SC+idWTUl5SX/b1mOwlaSOAD+c/4dD7F09AjaQBC9jp9qFpMxiA yQ8zAOmUCSNJHd4GNVibm2NFryjNUMiJnLoWHwXc4kfUWpuThKQhWCLVPKuK7Il4VkdR hj830aNt8DjO6qP+v/tAxvVNKC3B/gZqQw6UhpDGmGbPiDa5hd/ApxadZGkkKiNjftbS 24Xn6/U5m4X7Vzh10rOTNhCEq8k5RCtk14B7b3r+/7b2CLZ9OjzEBXuS+J/XGNbccnPO STOg== X-Gm-Message-State: AOAM5325tWrXGLGHEQ597kLcuS67uw1PtQN0GeawgTRgqhuTZQKQTTrN JV6NyG8W9XVlOa9X9c/E9os1VAJx6DsSt3Zg1QHhZQ== X-Google-Smtp-Source: ABdhPJxOyrsjv4TuZBvhsFdZvWfIQV7f1ePeaVk/xFg+n8dE+Ox8SomDLoLm3ByGnMEHOlZnf6dy0GO5ep3B6k2/4lo= X-Received: by 2002:a81:af57:0:b0:2dc:40d0:1380 with SMTP id x23-20020a81af57000000b002dc40d01380mr15998588ywj.255.1647205823423; Sun, 13 Mar 2022 14:10:23 -0700 (PDT) MIME-Version: 1.0 References: <20220312154321.GC1189@xsang-OptiPlex-9020> <15307f8a-c202-75d8-1361-dae0146df734@suse.cz> <8f499c76-68cb-a2c3-01fd-c8759e2fd317@suse.cz> In-Reply-To: <8f499c76-68cb-a2c3-01fd-c8759e2fd317@suse.cz> From: Eric Dumazet Date: Sun, 13 Mar 2022 14:10:12 -0700 Message-ID: Subject: Re: [mm/page_alloc] 8212a964ee: vm-scalability.throughput 30.5% improvement To: Vlastimil Babka Cc: kernel test robot , Mel Gorman , 0day robot , Michal Hocko , Shakeel Butt , Wei Xu , Greg Thelen , Hugh Dickins , David Rientjes , LKML , lkp@lists.01.org, "Huang, Ying" , "Tang, Feng" , zhengjun.xing@linux.intel.com, fengwei.yin@intel.com, Eric Dumazet , Andrew Morton , linux-mm Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 9EBC940028 X-Stat-Signature: euznku4xjuckcb5imbo4e1dixsw4jx7q Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=HyMlGeHV; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of edumazet@google.com designates 209.85.128.171 as permitted sender) smtp.mailfrom=edumazet@google.com X-Rspam-User: X-HE-Tag: 1647205824-373516 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, Mar 13, 2022 at 1:29 AM Vlastimil Babka wrote: > > On 3/13/22 00:26, Eric Dumazet wrote: > > On Sat, Mar 12, 2022 at 10:59 AM Vlastimil Babka wrote: > >> > >> On 3/12/22 16:43, kernel test robot wrote: > >>> > >>> > >>> Greeting, > >>> > >>> FYI, we noticed a 30.5% improvement of vm-scalability.throughput due to commit: > >>> > >>> > >>> commit: 8212a964ee020471104e34dce7029dec33c218a9 ("Re: [PATCH v2] mm/page_alloc: call check_new_pages() while zone spinlock is not held") > >>> url: https://github.com/0day-ci/linux/commits/Mel-Gorman/Re-PATCH-v2-mm-page_alloc-call-check_new_pages-while-zone-spinlock-is-not-held/20220309-203504 > >>> patch link: https://lore.kernel.org/lkml/20220309123245.GI15701@techsingularity.net > >> > >> Heh, that's weird. I would expect some improvement from Eric's patch, > >> but this seems to be actually about Mel's "mm/page_alloc: check > >> high-order pages for corruption during PCP operations" applied directly > >> on 5.17-rc7 per the github url above. This was rather expected to make > >> performance worse if anything, so maybe the improvement is due to some > >> unexpected side-effect of different inlining decisions or cache alignment... > >> > > > > I doubt this has anything to do with inlining or cache alignment. > > > > I am not familiar with the benchmark, but its name > > (anon-w-rand-hugetlb) hints at hugetlb ? > > > > After Mel fix, we go over 512 'struct page' to perform sanity checks, > > thus loading into cpu caches the 512 cache lines. > > Ah, that's true. > > > This caching is done while no lock is held. > > But I don't think this is. The test was AFAICS done without your patch, > so the lock is still held in rmqueue(). And it's also held in > rmqueue_bulk() -> check_pcp_refill(). Note that Mel patch touches both check_pcp_refill() and check_new_pcp() __rmqueue_pcplist() definitely calls check_new_pcp() while the zone spinlock is _not_ held. Note that it is possible to defer calls to check_pcp_refill after the spinlock is released. Untested patch: diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1804287c1b792b8aa0e964b17eb002b6b1115258..3c504b4c068a5dbeeaf8f386bb09b673236f7a11 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3024,6 +3024,7 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order, unsigned long count, struct list_head *list, int migratetype, unsigned int alloc_flags) { + struct page *page, *tmp; int i, allocated = 0; /* @@ -3032,14 +3033,10 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order, */ spin_lock(&zone->lock); for (i = 0; i < count; ++i) { - struct page *page = __rmqueue(zone, order, migratetype, - alloc_flags); + page = __rmqueue(zone, order, migratetype, alloc_flags); if (unlikely(page == NULL)) break; - if (unlikely(check_pcp_refill(page))) - continue; - /* * Split buddy pages returned by expand() are received here in * physical page order. The page is added to the tail of @@ -3065,6 +3062,12 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order, */ __mod_zone_page_state(zone, NR_FREE_PAGES, -(i << order)); spin_unlock(&zone->lock); + list_for_each_entry_safe(page, tmp, list, lru) { + if (unlikely(check_pcp_refill(page))) { + list_del(&page->lru); + allocated--; + } + } return allocated; }