From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BB54C4345F for ; Fri, 19 Apr 2024 06:22:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E49916B0083; Fri, 19 Apr 2024 02:22:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DFA206B0085; Fri, 19 Apr 2024 02:22:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC0FA6B0087; Fri, 19 Apr 2024 02:22:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AD7D86B0083 for ; Fri, 19 Apr 2024 02:22:08 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2D8FAA0305 for ; Fri, 19 Apr 2024 06:22:08 +0000 (UTC) X-FDA: 82025286336.11.53F9A29 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf25.hostedemail.com (Postfix) with ESMTP id 164F7A0002 for ; Fri, 19 Apr 2024 06:22:02 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=temperror reason="server fail" header.from=sk.com (policy=temperror); spf=temperror (imf25.hostedemail.com: error in processing during lookup of byungchul@sk.com: DNS error) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713507726; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=28tw5W7Kj7h/D9Fs6+B39u4mEpQQhNAM/15+PmYeAYY=; b=gPXdJfSg5hQnDKQpegbpU4wuXY5p+Bv+1Qgx0DW7pIUHLBMMcwnJ7sTK57jum8VJfE2atB xuyplmkqVVQQCjC7TlwSR3T3BTKKscaOXRq6/9PfwGOy0mqEim8JUFDlCIPK0TlQ63oO9s eG9prC8/07p32X3SQCK85KiuoRujNXM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713507726; a=rsa-sha256; cv=none; b=tGbupux7Cgi8nG7nu/Gh/pgrFXR0iRQH9duZYU7rIImU9dbtmimxIZuApXmV2fzB70gLGw ltYPn+sMuJzz9GRGOq3uiaYIWq+BtfBcfUQlzA1KHib7GqIdjc3jG2YyJosH3tMLVhFu0L F+4ilF6WmReIprLN4zAxx01XdgKdnd0= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; dmarc=temperror reason="server fail" header.from=sk.com (policy=temperror); spf=temperror (imf25.hostedemail.com: error in processing during lookup of byungchul@sk.com: DNS error) smtp.mailfrom=byungchul@sk.com X-AuditID: a67dfc5b-d6dff70000001748-de-66220d871b94 Date: Fri, 19 Apr 2024 15:21:54 +0900 From: Byungchul Park To: "Huang, Ying" Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, akpm@linux-foundation.org, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: Re: [PATCH v9 rebase on mm-unstable 0/8] Reduce tlb and interrupt numbers over 90% by improving folio migration Message-ID: <20240419062154.GA26669@system.software.com> References: <20240418061536.11645-1-byungchul@sk.com> <87cyqlyjh5.fsf@yhuang6-desk2.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87cyqlyjh5.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrNIsWRmVeSWpSXmKPExsXC9ZZnoW4Hr1KawZI5LBZz1q9hs/i84R+b xYsN7YwWX9f/YrZ4+qmPxeLyrjlsFvfW/Ge1OL9rLavFjqX7mCwuHVjAZHG89wCTxfx7n9ks Nm+aymxxfMpURovfP4CKT86azOIg4PG9tY/FY+esu+weCzaVemxeoeWxeM9LJo9NqzrZPDZ9 msTu8e7cOXaPEzN+s3jMOxno8X7fVTaPrb/sPBqnXmPz+LxJLoAvissmJTUnsyy1SN8ugSvj 6fEXzAUPdSqWbE9tYNyo0MXIwSEhYCLR3B7VxcgJZn7r+swKYrMIqEos7WoAs9kE1CVu3PjJ DGKLCGhIfFq4nL2LkYuDWeA5k8TvN7/ZQRLCAlUS627fZQOxeQUsJA4fXATWICSQKXH3/gYm iLigxMmZT1hAbGYBLYkb/14ygdzALCAtsfwfB0iYU8BO4vGXr2DlogLKEge2HWcC2SUhsI5d Yu6hbkaIQyUlDq64wTKBUWAWkrGzkIydhTB2ASPzKkahzLyy3MTMHBO9jMq8zAq95PzcTYzA KFxW+yd6B+OnC8GHGAU4GJV4eBfEK6YJsSaWFVfmHmKU4GBWEuE14wAK8aYkVlalFuXHF5Xm pBYfYpTmYFES5zX6Vp4iJJCeWJKanZpakFoEk2Xi4JRqYAzzPnkuQUDGd87LaZ/nHgu+ccVh wbxn5lpie0Wl/158Ffd9ofOX+dKVT2bKruPL+/H1ZL5b/6M3O2KWrOxTLdgVJDDVrmY32/xY CQG1eX31ASvuLS9/M3/ZxbmnMo3rUnZP/KHK1ibF0Vbs0dNYMN9YzKNgx5TbQVdC82NcN4ay sBe2fqsOFVFiKc5INNRiLipOBABquc+XvgIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprMIsWRmVeSWpSXmKPExsXC5WfdrNvOq5RmcPaSnsWc9WvYLD5v+Mdm 8WJDO6PF1/W/mC2efupjsTg89ySrxeVdc9gs7q35z2pxftdaVosdS/cxWVw6sIDJ4njvASaL +fc+s1ls3jSV2eL4lKmMFr9/ABWfnDWZxUHQ43trH4vHzll32T0WbCr12LxCy2PxnpdMHptW dbJ5bPo0id3j3blz7B4nZvxm8Zh3MtDj/b6rbB6LX3xg8tj6y86jceo1No/Pm+QC+KO4bFJS czLLUov07RK4Mp4ef8Fc8FCnYsn21AbGjQpdjJwcEgImEt+6PrOC2CwCqhJLuxrAbDYBdYkb N34yg9giAhoSnxYuZ+9i5OJgFnjOJPH7zW92kISwQJXEutt32UBsXgELicMHF4E1CAlkSty9 v4EJIi4ocXLmExYQm1lAS+LGv5dAcQ4gW1pi+T8OkDCngJ3E4y9fwcpFBZQlDmw7zjSBkXcW ku5ZSLpnIXQvYGRexSiSmVeWm5iZY6pXnJ1RmZdZoZecn7uJERhVy2r/TNzB+OWy+yFGAQ5G JR7eBfGKaUKsiWXFlbmHGCU4mJVEeM04gEK8KYmVValF+fFFpTmpxYcYpTlYlMR5vcJTE4QE 0hNLUrNTUwtSi2CyTBycUg2MGb+cN/2/f3PBZJPn828+8X124UX1vAO7fnP/WDNfRuTJObNo 5WTPa67x9Yas7CuW/5y8fxJrVfvqBc1+r5fukJm0ZeKsJ5zRgSvu5clczWj84xH1dP/9EI7g vxdWHzy6y31LdUUaW7c994SUeK8FH4Tyj37MO1QmYrWgb+np91vr91tVcHxSU1NiKc5INNRi LipOBACRLonapgIAAA== X-CFilter-Loop: Reflected X-Stat-Signature: kbka5hhkmq685p7sqpczbatf3arrknaw X-Rspamd-Queue-Id: 164F7A0002 X-Rspamd-Server: rspam06 X-Rspam-User: X-HE-Tag: 1713507722-928892 X-HE-Meta: U2FsdGVkX1+q+0HX+byv6VyXZg0xZ2l7KjFqkxkWrGn5bZf0EyZsNI6SaVJ1k4+8hNVjRFVea2tA24ZhIq03s/As+oPdJiFIaxHoIZOfnpvk8HBaCnTMD5JEhIYM2YxiNMy8pKcXrGqRq6oOqArlQviwLN5nanyMhWvFQn+lr+RfVBMoxOXpNgZyz5uIWX/qntpGie6hKBFS1Y+x8Ksa8wR3wXO2QmFrpNoU0NAhPlAqrhzYDgzizjlBqTMYpErnY9dX7zuWHJmD6d3BoA3Rig1SDGgWiwHghcjhwpjGksoQvDcaWan+47ZpnBW87SCfOccghM4lf0Tg+mwGUfhHkJM7c5chJ5W6mbYPzvC3LWSxjDwKosxrl3roQlO1SuXSAXeZPDEUfLTIxI09mRLTVpFkyGUL97kKtaHyWzqsQrIfFhVe50iJoHvu8X6/GbHCwN4O9fvijR4iOCWTw5EbKgw/5GInq117kl9hi19PDLnSm+fA5d4UArzTJ5OZaCzUp+LYnGAoEloQibKkmKzmIvx16W+xJ5ojszk+9LBb/mRCSJEEcSuKkrQYw/3DSU9Z9qk08FjK5EqB/RxcW58lQ+kt6uu9PwaEU/3pzl2344lkFF5p64wQSDuE+MwiTqJLM4r2vQpxGnyekistJRRKGVvUdYfiTlWgA8S2TdrBQ5IygxGG0vh1Rg0XuXwoWDQb4TjcUprgPcc7/owOqzIAuHi69ubhe3uTAxpFTqGlSwmg50fyBziYOC2/oAjqy2o+YZgDybWlAwhc8FtsE5GoxKAzcmhh1FrfukzudwPUuAYM4b1jrHXY0uyPGJ6mzdYAapD2bqeBGiZrCjhjIiDes+Yh1gnUhCOwT56ZjSiPjYr/W3JxgWMOyBdkrsVxvHG+d4K/NacdTluY1TVMqRmGurUTcU+AlzYQYh/A4TNqTB3l5jMjxKcveC4UuIOR5QqNo+mM4Zw74pFUujIUxbI arHvA14x c/TfwOmsBVdsgklJdMTdoD+1oxtzSrkl6DFuMbTnUcuAERHTI9OF2PfA44dO7goHp2kldLrFUXq0kNLYeeMA/pwB+vcDEG7G/GpbZJvOD1jcfj6ERTEM4dlNCyGnqHC2eKxlk890fOh48z4t1pBeLmi4oJKWGHVgnY6S/NOGK2KLhBuY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Apr 19, 2024 at 02:06:30PM +0800, Huang, Ying wrote: > Byungchul Park writes: > > > Hi everyone, > > > > While I'm working with a tiered memory system e.g. CXL memory, I have > > been facing migration overhead esp. tlb shootdown on promotion or > > demotion between different tiers. Yeah.. most tlb shootdowns on > > migration through hinting fault can be avoided thanks to Huang Ying's > > work, commit 4d4b6d66db ("mm,unmap: avoid flushing tlb in batch if PTE > > is inaccessible"). See the following link for more information: > > > > https://lore.kernel.org/lkml/20231115025755.GA29979@system.software.com/ > > > > However, it's only for ones using hinting fault. I thought it'd be much > > better if we have a general mechanism to reduce all tlb numbers that we > > can ultimately apply to any type of migration. > > > > I'm suggesting a mechanism called MIGRC that stands for 'Migration Read > > Copy', to reduce tlb numbers by deferring tlb flush until the source > > folios at migration actually become used, of course, only if the target > > PTE don't have write permission. > > > > To achieve that: > > > > 1. For the folios that map only to non-writable tlb entries, prevent > > tlb flush during migration but perform it just before the source > > folios actually become used out of buddy or pcp. > > > > 2. When any non-writable tlb entry changes to writable e.g. through > > fault handler, give up migrc mechanism and perform tlb flush > > required right away. > > > > No matter what type of workload is used for performance evaluation, the > > result would be positive thanks to the unconditional reduction of tlb > > flushes, tlb misses and interrupts. For the test, I picked up XSBench > > that is widely used for performance analysis on high performance > > computing architectures - https://github.com/ANL-CESAR/XSBench. > > > > The result would depend on memory latency and how often reclaim runs, > > which implies tlb miss overhead and how many times migration happens. > > The slower the memory is and the more reclaim runs, the better migrc > > works so as to obtain the better result. In my system, the result > > shows: > > > > 1. itlb flushes are reduced over 90%. > > 2. itlb misses are reduced over 30%. > > 3. All the other tlb numbers also get enhanced. > > 4. tlb shootdown interrupts are reduced over 90%. > > 5. The test program runtime is reduced over 5%. > > > > The test envitonment: > > > > Architecture - x86_64 > > QEMU - kvm enabled, host cpu > > The test is run in VM? Do you have test results in bare metal > environment? I will test in a bare metal environment and share the result. > > Numa - 2 nodes (16 CPUs 1GB, no CPUs 99GB) > > The configuration looks quite abnormal. Have you tested with other > configuration, such 1:4 or 1:8? Okay I will test with the configurations. > > Linux Kernel - v6.9-rc4, numa balancing tiering on, demotion enabled > > > > < measurement: raw data - tlb and interrupt numbers > > > > > $ perf stat -a \ > > -e itlb.itlb_flush \ > > -e tlb_flush.dtlb_thread \ > > -e tlb_flush.stlb_any \ > > -e dtlb-load-misses \ > > -e dtlb-store-misses \ > > -e itlb-load-misses \ > > XSBench -t 16 -p 50000000 > > > > $ grep "TLB shootdowns" /proc/interrupts > > > > BEFORE > > ------ > > 40417078 itlb.itlb_flush > > 234852566 tlb_flush.dtlb_thread > > 153192357 tlb_flush.stlb_any > > 119001107892 dTLB-load-misses > > 307921167 dTLB-store-misses > > 1355272118 iTLB-load-misses > > > > TLB: 1364803 1303670 1333921 1349607 > > 1356934 1354216 1332972 1342842 > > 1350265 1316443 1355928 1360793 > > 1298239 1326358 1343006 1340971 > > TLB shootdowns > > > > AFTER > > ----- > > 3316495 itlb.itlb_flush > > 138912511 tlb_flush.dtlb_thread > > 115199341 tlb_flush.stlb_any > > 117610390021 dTLB-load-misses > > 198042233 dTLB-store-misses > > 840066984 iTLB-load-misses > > > > TLB: 117257 119219 117178 115737 > > 117967 118948 117508 116079 > > 116962 117266 117320 117215 > > 105808 103934 115672 117610 > > TLB shootdowns > > > > < measurement: user experience - runtime > > > > > $ time XSBench -t 16 -p 50000000 > > > > BEFORE > > ------ > > Threads: 16 > > Runtime: 968.783 seconds > > Lookups: 1,700,000,000 > > Lookups/s: 1,754,778 > > > > 15208.91s user 141.44s system 1564% cpu 16:20.98 total > > > > AFTER > > ----- > > Threads: 16 > > Runtime: 913.210 seconds > > Lookups: 1,700,000,000 > > Lookups/s: 1,861,565 > > > > 14351.69s user 138.23s system 1565% cpu 15:25.47 total > > IIUC, the memory footprint will be larger with the patchset. Do you > have data? No. The footprint is, I expect, same as vanilla with this patchset. I will share the data. Last time, since you pointed out that the footprint seemed to be larger with the previous patchset becasue it worked anyway based on deferring freeing folios. Which made me rework on it so as to avoid tweaking the original behavior of mm. Instead, the current version of migrc let it go exactly same as it is with vanilla until the interesting folios exit from pcp or buddy, and do tlb flush if needed. Byungchul > -- > Best Regards, > Huang, Ying