From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCEA0C433E1 for ; Wed, 26 Aug 2020 01:11:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8E388206E3 for ; Wed, 26 Aug 2020 01:11:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="jQNvNVHY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E388206E3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1DA9E6B0003; Tue, 25 Aug 2020 21:11:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 18D716B0005; Tue, 25 Aug 2020 21:11:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 054B36B0006; Tue, 25 Aug 2020 21:11:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0242.hostedemail.com [216.40.44.242]) by kanga.kvack.org (Postfix) with ESMTP id E4A9D6B0003 for ; Tue, 25 Aug 2020 21:11:10 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id AD98A180AD817 for ; Wed, 26 Aug 2020 01:11:10 +0000 (UTC) X-FDA: 77190941100.05.rule65_0e1659d27060 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 827D6180275B8 for ; Wed, 26 Aug 2020 01:11:10 +0000 (UTC) X-HE-Tag: rule65_0e1659d27060 X-Filterd-Recvd-Size: 6143 Received: from userp2120.oracle.com (userp2120.oracle.com [156.151.31.85]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Wed, 26 Aug 2020 01:11:09 +0000 (UTC) Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 07Q1A691044031; Wed, 26 Aug 2020 01:10:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : content-transfer-encoding : in-reply-to; s=corp-2020-01-29; bh=VWKaRWmTmrlTrbnIky0PuQ3aZ0bL/HrXLsBsMQQqZo4=; b=jQNvNVHYVl0Oq89XEgsmuaLNiJpGZJT9brk12EKPFxStCdsik5I4IBEl6Bg/H3NtYf2Z cmzX1sVKVAwMR/9dtkl5+JbbWT934jfcpvNxDywlgqCg51YQs0e0n9qAHlHsLQ5ihWaP jfDlzlh4U6ZT4v89eaYGfzPPP/w9t5HO2yAgiNyJ1PgteJXcHlLGZW6e7NrjGNIVpSeB lU7mwgsqKocBkjf4K5fjzhl9uxqclIjlOPIoYhiqdyWwa04NYvZW7c+F0nPNNYYZ4/rV FMmXYE4rJEyLxS6Vur9iT/YFuDO8s0NbT0Q+rWL/Ni1XlHheonNF35vprX/YllYb3zk6 rA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2120.oracle.com with ESMTP id 333w6tv73f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 26 Aug 2020 01:10:55 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 07Q1AC7X034251; Wed, 26 Aug 2020 01:10:54 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserp3020.oracle.com with ESMTP id 333ru8x57n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 26 Aug 2020 01:10:54 +0000 Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 07Q1AnBB002426; Wed, 26 Aug 2020 01:10:49 GMT Received: from ca-dmjordan1.us.oracle.com (/10.211.9.48) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 25 Aug 2020 18:10:48 -0700 Date: Tue, 25 Aug 2020 21:19:46 -0400 From: Daniel Jordan To: Alex Shi Cc: Daniel Jordan , Hugh Dickins , Andrew Morton , mgorman@techsingularity.net, tj@kernel.org, khlebnikov@yandex-team.ru, willy@infradead.org, hannes@cmpxchg.org, lkp@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, shakeelb@google.com, iamjoonsoo.kim@lge.com, richard.weiyang@gmail.com, kirill@shutemov.name, alexander.duyck@gmail.com, rong.a.chen@intel.com, mhocko@suse.com, vdavydov.dev@gmail.com, shy828301@gmail.com Subject: Re: [PATCH v18 00/32] per memcg lru_lock Message-ID: <20200826011946.spknwjt44d2szrdo@ca-dmjordan1.us.oracle.com> References: <1598273705-69124-1-git-send-email-alex.shi@linux.alibaba.com> <20200824114204.cc796ca182db95809dd70a47@linux-foundation.org> <20200825015627.3c3pnwauqznnp3gc@ca-dmjordan1.us.oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20180716 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9724 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 phishscore=0 bulkscore=0 suspectscore=0 spamscore=0 mlxscore=0 adultscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2008260007 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9724 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 impostorscore=0 mlxlogscore=999 suspectscore=0 phishscore=0 malwarescore=0 spamscore=0 priorityscore=1501 clxscore=1015 mlxscore=0 lowpriorityscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2008260006 X-Rspamd-Queue-Id: 827D6180275B8 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Aug 25, 2020 at 11:26:58AM +0800, Alex Shi wrote: > =E5=9C=A8 2020/8/25 =E4=B8=8A=E5=8D=889:56, Daniel Jordan =E5=86=99=E9=81= =93: > > Alex, do you have a pointer to the modified readtwice case? >=20 > Sorry, no. my developer machine crashed, so I lost case my container an= d modified > case. I am struggling to get my container back from a account problemat= ic repository.=20 >=20 > But some testing scripts is here, generally, the original readtwice cas= e will > run each of threads on each of cpus. The new case will run one containe= r on each cpus, > and just run one readtwice thead in each of containers. Ok, what you've sent so far gives me an idea of what you did. My readtwi= ce changes were similar, except I used the cgroup interface directly instead= of docker and shared a filesystem between all the cgroups whereas it looks l= ike you had one per memcg. 30 second runs on 5.9-rc2 and v18 gave 11% more d= ata read with v18. This was using 16 cgroups (32 dd tasks) on a 40 CPU, 2 so= cket machine. > > Even better would be a description of the problem you're having in pr= oduction > > with lru_lock. We might be able to create at least a simulation of i= t to show > > what the expected improvement of your real workload is. >=20 > we are using thousands memcgs in a machine, but as a simulation, I gues= s above case > could be helpful to show the problem. Using thousands of memcgs to do what? Any particulars about the type of workload? Surely it's more complicated than page cache reads :) > > I ran a few benchmarks on v17 last week (sysbench oltp readonly, kern= devel from > > mmtests, a memcg-ized version of the readtwice case I cooked up) and = then today > > discovered there's a chance I wasn't running the right kernels, so I'= m redoing > > them on v18. Neither kernel compile nor git checkout in the root cgroup changed much, = just 0.31% slower on elapsed time for the compile, so no significant regressio= ns there. Now for sysbench again.