From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3EA1C636CC for ; Wed, 15 Feb 2023 17:33:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 68BEA6B0074; Wed, 15 Feb 2023 12:33:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 614536B0075; Wed, 15 Feb 2023 12:33:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 48F2B6B0078; Wed, 15 Feb 2023 12:33:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 3674B6B0074 for ; Wed, 15 Feb 2023 12:33:06 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 03834A7489 for ; Wed, 15 Feb 2023 17:33:05 +0000 (UTC) X-FDA: 80470221972.17.7A9760F Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) by imf19.hostedemail.com (Postfix) with ESMTP id 024661A0002 for ; Wed, 15 Feb 2023 17:33:01 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=wTYzp+HL; spf=pass (imf19.hostedemail.com: domain of punit.agrawal@bytedance.com designates 209.85.221.54 as permitted sender) smtp.mailfrom=punit.agrawal@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676482383; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0t94xwbIE7oAhUJFQ2BQHFhw0oZE0qnNECyAb2ZFKHQ=; b=F4dFsWZFAOTLnpOUzobeX8Kp+9koFTLXf+ogjVZW7fDujdW8V8uHWecfYcItbYZL5eyWQR CB6HwSvGZf3fB37ebEf6xcglwU3TtYyTn3wM3ycOLNsI+1EiOmlthMW8Bgv380Hj8z81Nr 3sKh3WvXG/jFGxECso+rjk3+bsdJMTU= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=wTYzp+HL; spf=pass (imf19.hostedemail.com: domain of punit.agrawal@bytedance.com designates 209.85.221.54 as permitted sender) smtp.mailfrom=punit.agrawal@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676482383; a=rsa-sha256; cv=none; b=gOjSVR/M3DyW7WXpDT+N6JkETNsw9MP16aaVO3tZTyWf4i8Vma3MUJVczhS2TEZdbLtQQp OX9j/SCzUsBtZWPw88A3HyJ6NP8KZALio1kUeo9pro1JBdjQ4T9z5gDVEdDko07bFBDSIq fxIj+yHXn1TOklEuzxM+FaVL5vB0YII= Received: by mail-wr1-f54.google.com with SMTP id l2so3312714wry.0 for ; Wed, 15 Feb 2023 09:33:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=0t94xwbIE7oAhUJFQ2BQHFhw0oZE0qnNECyAb2ZFKHQ=; b=wTYzp+HLVi+lx88ricSM/4zA8JGJ0VYAUK8sP1YwmHt5MGHWabBceqwlwxcEPxzMvc magXOb6suv1DXlOStB5PZvc2zBV9hlnvnH4H50+kYlzyuV0Uu06GW3ZzJsNjP1wjmEm/ /g6mWpsZrj7fTLUZSrzvfBri7ZHgBO27wAq3dbmuY1x4ZZ0PkgpMBAjk0VwNa34aASAE A6rGzpy27hnSGLZSLkcS6pi1DiFjFqnQuJ3Rc47jhXlM2nYrZ2UnOXk5csTd2qgzNdtr 43SrIB3xEnsd/MLqrCKN/npLj7g40d4R1m+2AC2+7QkNOt17ZlxEe1FozUu7SHAncT2A rrYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=0t94xwbIE7oAhUJFQ2BQHFhw0oZE0qnNECyAb2ZFKHQ=; b=Eu8NvPTlU8RZunFA+UlpwsoyWLDCGgXic94yXG86mIZd6rJnWcX3X3bUonoLRaX3rE 5jm7j2r+RVsy3SBtN3kKpz1FYUX/E2tOIgfsBmwglOO2fhRQeNfM7PwiZt56pLXUyzTG Kr0tr41JxKwMIw9aG2vJc5BpmeXdwBLZD2m6O1R/h+uFV61hY0G22AUNW6n3jBDrP3Dl B6y6xtFWsVyfFB88m1iC0acKiP0b8NPCPsDJ8IMQ6yC0G8zIXn7fzFI3ms2zrvXI+6qu ssIfpGtnUVZrvTMWJ1ITMfilQYBsZK/C0mBBfwghcIEEfmV8afBqrc/71GqY8XKlRayH VQUw== X-Gm-Message-State: AO0yUKXM2TJa2KSQQ783y7x8FxArQOvfjLMNZRV/Y0Mr5Z8lHpHpuCON jjc0o1GGh854rVRuQUpshnI2LQ== X-Google-Smtp-Source: AK7set9kT9H6NMj2/S3WBxDWHmAmNxq0wUR9ec7mDgnR1zjNhxV91JrWyeNEVDzBJm76EQ32XvPpCg== X-Received: by 2002:a5d:4b03:0:b0:2c5:644c:3e89 with SMTP id v3-20020a5d4b03000000b002c5644c3e89mr2357955wrq.67.1676482380110; Wed, 15 Feb 2023 09:33:00 -0800 (PST) Received: from localhost ([95.148.15.48]) by smtp.gmail.com with ESMTPSA id f26-20020a5d58fa000000b002c5526680cbsm10288128wrd.92.2023.02.15.09.32.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Feb 2023 09:32:59 -0800 (PST) From: Punit Agrawal To: Suren Baghdasaryan Cc: akpm@linux-foundation.org, michel@lespinasse.org, jglisse@google.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, peterz@infradead.org, ldufour@linux.ibm.com, paulmck@kernel.org, mingo@redhat.com, will@kernel.org, luto@kernel.org, songliubraving@fb.com, peterx@redhat.com, david@redhat.com, dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de, kent.overstreet@linux.dev, punit.agrawal@bytedance.com, lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com, axelrasmussen@google.com, joelaf@google.com, minchan@google.com, rppt@kernel.org, jannh@google.com, shakeelb@google.com, tatashin@google.com, edumazet@google.com, gthelen@google.com, gurua@google.com, arjunroy@google.com, soheil@google.com, leewalsh@google.com, posk@google.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com Subject: Re: [External] [PATCH v2 00/33] Per-VMA locks References: <20230127194110.533103-1-surenb@google.com> Date: Wed, 15 Feb 2023 17:32:58 +0000 In-Reply-To: <20230127194110.533103-1-surenb@google.com> (Suren Baghdasaryan's message of "Fri, 27 Jan 2023 11:40:37 -0800") Message-ID: <87r0uqq1f9.fsf@stealth> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 024661A0002 X-Rspam-User: X-Stat-Signature: 1mdm3x71z3bu9fz6d3u14jh9pb7k768u X-HE-Tag: 1676482381-374033 X-HE-Meta: U2FsdGVkX1+4PnMVIYhfMw/AImUxcGSsx2dKqsYOeeHp1zC/K46ETtcLWn9PXdm5ptOhpjEpX4UiakW+WFWCqMc6rBa1x8fvQW+a7jE4zfd0ggw6KNhMa6kUmwkFFrSgg8WSXBcrVGTe5U4al5fwkPYTgHSWunLei8BlF0sOtMkX1pxuIwz0HaJdmYp8KqnuM2fJ+qYgHR1wa96bvxhHlydlPPiaIkwc+z3rPGGyXVDpH6XwDJcCG2EsnRqh8j6ZImorK5NXKJCZIAXDcRn59yZ+5uq+HgiuKPu52Ij2EFI1JSwGCFR59MVVUNH99fJ2l1LpKJVGyv0flnFSjbn2DpEAYMz8MYGFN3vKNK393OKE1o7JL8epZl95X/kKe5WvSBdNNRuHYVh239GMYt6FyXhGyvtc8PnrRyc7VEYli76YBeEck3ZsrIaN6PsJXcKmTj+fUVikFJRwcRwfQHjL8gkBVYnKoRWocdfYe0xbgZcbsmcorqOZu1C1QkqGiY006G48SjdqIbc7CST6jTKaag8cGsc4TNPHibor0pNOv4mcnJA2kIk8zkWvWDjOpIc33nJNDLrIQ5oPGs9hsRdVwtQH1GSLZOK25po/RZDidKiZQDlNsYwK5EKvo7lhg+R4BB15+VyT6b7HTcB0qtLnGRnO0qw5rH2n0krsy9ciWJ/BVfoyPBa7/pCC8TigqUglepgk2ydlLbajv8dr2l4SZnLsSOA5GpQtKcQlK+W6jNaiHii83zcwWsG1ijdZ/Dh2T4zDJiLvzEjNmuFM0y4/hF9zOz7rM5SPcE8wY6vIrI+TnUJH0gxmhiRnPeCqR/Gs1cljLl1c4h66V2/QHMnqrvqABGpsQ0YrnCgYgD4rV6hrsHb2KbBWNCkCoQ8VV0E5F5WbCGuTZ9lD00LL+1zQ3o5WaKxMj4ekTdq1uwKLC3NqusWamBu1Y2OiCRiHBkn5ZdfpCmbtve0KtfSANxT Dk4qd/84 4k2io90J7XBGU/uwETSyBvcnIbEC62tZyzM/Qy1k/RaESwehSOynILkDXqBMZlya5898Cy7VzwHIauPlF0yaXsOBZrmMbbmHdB56qqAzK41DHoHXyJvnqLsdlK9jeI7u4ailCUgNURakK0qRXtoOtxX41tkHvSPVGXCbZ9otn04wx4BwomIjqmuPTaTTCsgyEP8EdKw6EqjyU78jLT61DzktxJydKDNN5f9TJHzE3BW0TJ5gbMgPoT32l+/rQdPT6yPZiCJpbjmfu3bUpzsa7hQ18RfKXEBMYBV3t/z1E4XSiXnaZFRWHW6n1hiNbNvykT9N+8/edVHw4JdRH6dWMleD1eWkXvGvxhmYlh15tW93G++yFEt7G+OeHphnIj7GUDUM5Ac+nzF+9OVSIpkUyQ0M7o/kf0gd7i/ZyhhiyTleT0VVWF6bHXnq+AsGltUGDPVDIPMdlnsePKnG7ZmHlGicsu0zN/Ntz0TJIgkd5xhzVzGptDmMMP5u9khuvdm9wN7ZfjQ4AQQFVQDsTBQ1pRXCbwOAVHywgcKPBhrc1KHJapndln0cUt1xEGcyMnYnR1ZGCZ2zRblyfI6FfUNxvt1AoCnaTTL44lc0B X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Suren Baghdasaryan writes: > Previous version: > v1: https://lore.kernel.org/all/20230109205336.3665937-1-surenb@google.co= m/ > RFC: https://lore.kernel.org/all/20220901173516.702122-1-surenb@google.co= m/ > > LWN article describing the feature: > https://lwn.net/Articles/906852/ > > Per-vma locks idea that was discussed during SPF [1] discussion at LSF/MM > last year [2], which concluded with suggestion that =E2=80=9Ca reader/wri= ter > semaphore could be put into the VMA itself; that would have the effect of > using the VMA as a sort of range lock. There would still be contention at > the VMA level, but it would be an improvement.=E2=80=9D This patchset imp= lements > this suggested approach. I took the patches for a spin on a 2-socket 32 core (64 threads) system with Intel 8336C (Ice Lake) and 512GB of RAM. For the initial testing, "pft-threads" from the mm-tests suite[0] was used. The test mmaps a memory region (~100GB on the test system) and triggers access by a number of threads executing in parallel. For each degree of parallelism, the test is repeated 10 times to get a better feel for the behaviour. Below is an excerpt of the harmonic mean reported by 'compare_kernel' script[1] included with mm-tests. The first column is results for mm-unstable as of 2023-02-10, the second column is the patches posted here while the third column includes optimizations to reclaim some of the observed regression. >From the results, there is a drop in page fault/second for low number of CPUs but good improvement with higher CPUs. 6.2.0-rc4 6.2.0-rc4 = 6.2.0-rc4 mm-unstable-20230210 pvl-v2 = pvl-v2+opt Hmean faults/cpu-1 898792.9338 ( 0.00%) 894597.0474 * -0.47%* = 895933.2782 * -0.32%* Hmean faults/cpu-4 751903.9803 ( 0.00%) 677764.2975 * -9.86%* = 688643.8163 * -8.41%* Hmean faults/cpu-7 612275.5663 ( 0.00%) 565363.4137 * -7.66%* = 597538.9396 * -2.41%* Hmean faults/cpu-12 434460.9074 ( 0.00%) 410974.2708 * -5.41%* = 452501.4290 * 4.15%* Hmean faults/cpu-21 291475.5165 ( 0.00%) 293936.8460 ( 0.84%) = 308712.2434 * 5.91%* Hmean faults/cpu-30 218021.3980 ( 0.00%) 228265.0559 * 4.70%* = 241897.5225 * 10.95%* Hmean faults/cpu-48 141798.5030 ( 0.00%) 162322.5972 * 14.47%* = 166081.9459 * 17.13%* Hmean faults/cpu-79 90060.9577 ( 0.00%) 107028.7779 * 18.84%* = 109810.4488 * 21.93%* Hmean faults/cpu-110 64729.3561 ( 0.00%) 80597.7246 * 24.51%* = 83134.0679 * 28.43%* Hmean faults/cpu-128 55740.1334 ( 0.00%) 68395.4426 * 22.70%* = 69248.2836 * 24.23%* Hmean faults/sec-1 898781.7694 ( 0.00%) 894247.3174 * -0.50%* = 894440.3118 * -0.48%* Hmean faults/sec-4 2965588.9697 ( 0.00%) 2683651.5664 * -9.51%* = 2726450.9710 * -8.06%* Hmean faults/sec-7 4144512.3996 ( 0.00%) 3891644.2128 * -6.10%* = 4099918.8601 ( -1.08%) Hmean faults/sec-12 4969513.6934 ( 0.00%) 4829731.4355 * -2.81%* = 5264682.7371 * 5.94%* Hmean faults/sec-21 5814379.4789 ( 0.00%) 5941405.3116 * 2.18%* = 6263716.3903 * 7.73%* Hmean faults/sec-30 6153685.3709 ( 0.00%) 6489311.6634 * 5.45%* = 6910843.5858 * 12.30%* Hmean faults/sec-48 6197953.1327 ( 0.00%) 7216320.7727 * 16.43%* = 7412782.2927 * 19.60%* Hmean faults/sec-79 6167135.3738 ( 0.00%) 7425927.1022 * 20.41%* = 7637042.2198 * 23.83%* Hmean faults/sec-110 6264768.2247 ( 0.00%) 7813329.3863 * 24.72%* = 7984344.4005 * 27.45%* Hmean faults/sec-128 6460727.8216 ( 0.00%) 7875664.8999 * 21.90%* = 8049910.3601 * 24.60%* [0] https://github.com/gormanm/mmtests [1] https://github.com/gormanm/mmtests/blob/master/compare-kernels.sh