From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE101C433E2 for ; Sun, 19 Jul 2020 09:12:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6FE212073A for ; Sun, 19 Jul 2020 09:12:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6FE212073A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9EC616B0003; Sun, 19 Jul 2020 05:12:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 99E206B0005; Sun, 19 Jul 2020 05:12:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B3906B0006; Sun, 19 Jul 2020 05:12:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0138.hostedemail.com [216.40.44.138]) by kanga.kvack.org (Postfix) with ESMTP id 758336B0003 for ; Sun, 19 Jul 2020 05:12:47 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id F019F181DB395 for ; Sun, 19 Jul 2020 09:12:46 +0000 (UTC) X-FDA: 77054260332.16.news88_340202526f1a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id B9974100692F5 for ; Sun, 19 Jul 2020 09:12:46 +0000 (UTC) X-HE-Tag: news88_340202526f1a X-Filterd-Recvd-Size: 5046 Received: from out30-43.freemail.mail.aliyun.com (out30-43.freemail.mail.aliyun.com [115.124.30.43]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Sun, 19 Jul 2020 09:12:44 +0000 (UTC) X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04407;MF=alex.shi@linux.alibaba.com;NM=1;PH=DS;RN=21;SR=0;TI=SMTPD_---0U37udgA_1595149946; Received: from IT-FVFX43SYHV2H.local(mailfrom:alex.shi@linux.alibaba.com fp:SMTPD_---0U37udgA_1595149946) by smtp.aliyun-inc.com(127.0.0.1); Sun, 19 Jul 2020 17:12:27 +0800 Subject: Re: [PATCH v16 18/22] mm/lru: replace pgdat lru_lock with lruvec lock From: Alex Shi To: Alexander Duyck Cc: Andrew Morton , Mel Gorman , Tejun Heo , Hugh Dickins , Konstantin Khlebnikov , Daniel Jordan , Yang Shi , Matthew Wilcox , Johannes Weiner , kbuild test robot , linux-mm , LKML , cgroups@vger.kernel.org, Shakeel Butt , Joonsoo Kim , Wei Yang , "Kirill A. Shutemov" , Michal Hocko , Vladimir Davydov , Rong Chen References: <1594429136-20002-1-git-send-email-alex.shi@linux.alibaba.com> <1594429136-20002-19-git-send-email-alex.shi@linux.alibaba.com> <62dfd262-a7ac-d18e-216a-2988c690b256@linux.alibaba.com> Message-ID: Date: Sun, 19 Jul 2020 17:12:23 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <62dfd262-a7ac-d18e-216a-2988c690b256@linux.alibaba.com> Content-Type: text/plain; charset=utf-8 X-Rspamd-Queue-Id: B9974100692F5 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: =E5=9C=A8 2020/7/18 =E4=B8=8B=E5=8D=8810:15, Alex Shi =E5=86=99=E9=81=93: >>> >>> struct wb_domain *mem_cgroup_wb_domain(struct bdi_writeback *wb); >>> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h >>> index 14c668b7e793..36c1680efd90 100644 >>> --- a/include/linux/mmzone.h >>> +++ b/include/linux/mmzone.h >>> @@ -261,6 +261,8 @@ struct lruvec { >>> atomic_long_t nonresident_age; >>> /* Refaults at the time of last reclaim cycle */ >>> unsigned long refaults; >>> + /* per lruvec lru_lock for memcg */ >>> + spinlock_t lru_lock; >>> /* Various lruvec state flags (enum lruvec_flags) */ >>> unsigned long flags; >> Any reason for placing this here instead of at the end of the >> structure? From what I can tell it looks like lruvec is already 128B >> long so placing the lock on the end would put it into the next >> cacheline which may provide some performance benefit since it is >> likely to be bounced quite a bit. > Rong Chen(Cced) once reported a performance regression when the lock at > the end of struct, and move here could remove it. > Although I can't not reproduce. But I trust his report. >=20 Oops, Rong's report is on another member which is different with current struct.=20 Compare to move to tail, how about to move it to head of struct, which is close to lru list? Did you have some data of the place change? Thanks Alex =20 > ... >=20 >>> putback: >>> - spin_unlock_irq(&zone->zone_pgdat->lru_lock); >>> pagevec_add(&pvec_putback, pvec->pages[i]); >>> pvec->pages[i] =3D NULL; >>> } >>> - /* tempary disable irq, will remove later */ >>> - local_irq_disable(); >>> __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); >>> - local_irq_enable(); >>> + if (lruvec) >>> + unlock_page_lruvec_irq(lruvec); >> So I am not a fan of this change. You went to all the trouble of >> reducing the lock scope just to bring it back out here again. In >> addition it implies there is a path where you might try to update the >> page state without disabling interrupts. > Right. but any idea to avoid this except a extra local_irq_disable? >=20 The following changes would resolve the problem. Is this ok? @@ -324,7 +322,8 @@ static void __munlock_pagevec(struct pagevec *pvec, s= truct zone *zone) pagevec_add(&pvec_putback, pvec->pages[i]); pvec->pages[i] =3D NULL; } - __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); + if (delta_munlocked) + __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); if (lruvec) unlock_page_lruvec_irq(lruvec);