From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3B55C4332F for ; Mon, 30 Oct 2023 22:55:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 49D5D6B0295; Mon, 30 Oct 2023 18:55:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 44D426B0296; Mon, 30 Oct 2023 18:55:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 315926B0297; Mon, 30 Oct 2023 18:55:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 211606B0295 for ; Mon, 30 Oct 2023 18:55:59 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id DF031160317 for ; Mon, 30 Oct 2023 22:55:58 +0000 (UTC) X-FDA: 81403637196.07.A152AC0 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf22.hostedemail.com (Postfix) with ESMTP id 24285C0005 for ; Mon, 30 Oct 2023 22:55:55 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698706557; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8SC4KqwkBKXYWEMzT9ecbst7jHDUDYIv9zgc2NQp0cI=; b=uf70D3LNQn5sxVwzm4+TNSiNpOAxtwCWMy1Hw9aXzg4x6Ql6ZxlUaQMgS6cH0pPrvDcX7F DST2xJzpRmomhQgmAZK34GCJz/Qqmi3ChS89m+oqwrNfULJ1CvFaXkJlHZHTYTeKKICyy5 zv6WJWzVpcM03q9Oj4kEC1s+hjZqvsA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698706557; a=rsa-sha256; cv=none; b=3LOKfI/C5pJy0NBccT2Zb1lahHyzx2T/8138LV5aiyujrxTG2CKuNMIclwilHWDXFZi66A yAJnAWxRwsdDFy6dubl2aqNje78813PrEqvnLDU8SLr50BOpKZ8EBvxWhn536YDLcpXBGq uCrJF2MJCgRsqg9PyKkbri8hLlOH1ec= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none X-AuditID: a67dfc5b-d6dff70000001748-81-654034790898 Date: Tue, 31 Oct 2023 07:55:48 +0900 From: Byungchul Park To: Dave Hansen Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, namit@vmware.com, xhao@linux.alibaba.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com Subject: Re: [v3 0/3] Reduce TLB flushes under some specific conditions Message-ID: <20231030225548.GB900@system.software.com> References: <20231030072540.38631-1-byungchul@sk.com> <08c82a91-87d1-42c7-93c4-4028f3725340@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <08c82a91-87d1-42c7-93c4-4028f3725340@intel.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrAIsWRmVeSWpSXmKPExsXC9ZZnoW6liUOqwZ42S4s569ewWXze8I/N 4tPLB4wWLza0M1p8Xf+L2eLppz4Wi8u75rBZ3Fvzn9Xi/K61rBY7lu5jsrh0YAGTxfVdDxkt jvceYLLYvGkqs8XvH0B1c6ZYWZycNZnFQdDje2sfi8eCTaUem1doeSze85LJY9OqTjaPTZ8m sXu8O3eO3ePEjN8sHjsfWnrMOxno8X7fVTaPrb/sPD5vkvN4N/8tWwBfFJdNSmpOZllqkb5d AlfGztd9TAX/uCuWzLrP3sC4mrOLkZNDQsBEYuHkG6ww9o3Zt1lAbBYBVYlbu+6C2WwC6hI3 bvxkBrFFgOxTK5ezdzFycTALNDFLXFu/GaxZWMBd4mXHFkYQm1fATOLJq5tgDUICGRJfmjuZ IOKCEidnPgEbyiygJXHj30ugOAeQLS2x/B8HSJhTwFbiYe9vsBJRAWWJA9uOM4HskhBYxy5x 4/5ZJohDJSUOrrjBMoFRYBaSsbOQjJ2FMHYBI/MqRqHMvLLcxMwcE72MyrzMCr3k/NxNjMB4 XFb7J3oH46cLwYcYBTgYlXh4N/TYpwqxJpYVV+YeYpTgYFYS4WV2tEkV4k1JrKxKLcqPLyrN SS0+xCjNwaIkzmv0rTxFSCA9sSQ1OzW1ILUIJsvEwSnVwFjxvXxb0+d7WwSnCO0Tddv+63HP KlujO8b3PqgXrPpbs06YZaOvrVrqbb/+5n9nD21sWSC44brFfMvXtotvPZ1W9/H1zNQzdmzf d6bwOtyZ+0leq+66320dESfXZuFJGnbGCQuPb/P6u+7e4h8CjWJ5h78dT4x1/RjLpxb2Oipq y4d2ng2ul82UWIozEg21mIuKEwFJU9JxwwIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprOIsWRmVeSWpSXmKPExsXC5WfdrFtp4pBqsH+PisWc9WvYLD5v+Mdm 8enlA0aLFxvaGS2+rv/FbPH0Ux+LxeG5J1ktLu+aw2Zxb81/Vovzu9ayWuxYuo/J4tKBBUwW 13c9ZLQ43nuAyWLzpqnMFr9/ANXNmWJlcXLWZBYHIY/vrX0sHgs2lXpsXqHlsXjPSyaPTas6 2Tw2fZrE7vHu3Dl2jxMzfrN47Hxo6THvZKDH+31X2TwWv/jA5LH1l53H501yHu/mv2UL4I/i sklJzcksSy3St0vgytj5uo+p4B93xZJZ99kbGFdzdjFyckgImEjcmH2bBcRmEVCVuLXrLpjN JqAucePGT2YQWwTIPrVyOXsXIxcHs0ATs8S19ZtZQRLCAu4SLzu2MILYvAJmEk9e3QRrEBLI kPjS3MkEEReUODnzCdhQZgEtiRv/XgLFOYBsaYnl/zhAwpwCthIPe3+DlYgKKEsc2HacaQIj 7ywk3bOQdM9C6F7AyLyKUSQzryw3MTPHVK84O6MyL7NCLzk/dxMjMLqW1f6ZuIPxy2X3Q4wC HIxKPLwbeuxThVgTy4orcw8xSnAwK4nwMjvapArxpiRWVqUW5ccXleakFh9ilOZgURLn9QpP TRASSE8sSc1OTS1ILYLJMnFwSjUwZj9WLH/JlG787BR7fO7J8JldizYUdk19YrJe6FaczJ6N lxkvPvjqbqhQHrIq/8cKwQL/67yr3bo0z4Xp6jSoLMjYU2WxPEKq9meOsPfMu1lbpp56dOn4 /TCb/U6VxkW6RQvMTmwMV87p+/5y+e/ZT7OveR+I+Mgr1Hdnhdnfs0HlTwpU7BxKlViKMxIN tZiLihMB1bFmnqoCAAA= X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 24285C0005 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 6ypayqggdtkmg4qisdgorijm6f1eitdo X-HE-Tag: 1698706555-477099 X-HE-Meta: U2FsdGVkX1/FGoyQ6FONyU8b+0rKxYaWUBef7Nnwb6xd4pzLHZJGDtgBRw79ykwdkLWVfl/645kJG6MYAGG5nVi+XDYZH8HTPIvh6cILGU0CzcQ4lfRsTL7GPU0IF0h9e9/6JJyTqcVcFvNPfFLt0OYsm9Xdd6gP2Rk1srXhynkoggRDT2oc728Nvm8VjUVRiRXbqjVhzisZdsbp80uD91gpE6aOULwlbp2A2c08Lqu3gUlUNvjQhlXl2eU2WurTzuNfIBWr5YBThowOfjO+698qR0wEWi7J7+LyyCrBGkvIkWTX9rAcizu1U9lhPbuFkt+1rMKYa4HFiSIa6x2EJP1FNQ4cChJjNha37oCzdKQiwagPhg4g/naXAW1Hvbx6RcG7zfz5Gg+YgGo8v7MerRkG6L6VkqT67d6+NKzABQ3EMsUWuFZb3537OCHK3d5R2raYMIJoQTawTFsMombPxwnWfCzs+ZqM/UUDB7faYXGoYbqny/rmBdqLjwwxKI2CzQGBd2ZOhkIwbwcfPbU4eEd+/EZfvlHZuz7iyNBRKZ5/7743j+zRBx84tpDx88CEWa/DKbjKyiA7ngBOAwIOISoIb0JL5HlLADnOVMjT01jw5YpCrzy1+l6HkLvAAZxPz5A3yuCM7291Gvy8WplUvaLyJVmQOqbM8eyA3ED/jsej5GUbUgYt5qyETSx73TQWW0SMkcs3V/RFR25egcRkjD6Yttdt2luXrErBr69NtDy8I98Nkryj65faNz1dLRmxco0VxcAGu/xQaW730cpHVwKgNdNbglcIKLY9sikc4yLTz1WZHar6x3auJdEgu9ktFvKleEKvE40ludYAUPn/T4lm1V4wm3mdxm/Bk13jhuAnFlK/XcJKJ+34/jErMcndDpafVLQ5PEQpY/OTaTtJVP5388Eft+Wa X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Oct 30, 2023 at 10:55:07AM -0700, Dave Hansen wrote: > On 10/30/23 00:25, Byungchul Park wrote: > > I'm suggesting a mechanism to reduce TLB flushes by keeping source and > > destination of folios participated in the migrations until all TLB > > flushes required are done, only if those folios are not mapped with > > write permission PTE entries at all. I worked Based on v6.6-rc5. > > There's a lot of common overhead here, on top of the complexity in general: > > * A new page flag > * A new cpumask_t in task_struct > * A new zone list > * Extra (temporary) memory consumption > > and the benefits are ... "performance improved a little bit" on one > workload. That doesn't seem like a good overall tradeoff to me. > > There will certainly be workloads that, before this patch, would have > little or no memory pressure and after this patch would need to do reclaim. 'if (gain - cost) > 0 ?'" is a difficult problem. I think the followings are already big benefit in general: 1. big reduction of IPIs # 2. big reduction of TLB flushes # 3. big reduction of TLB misses # Of course, I or we need to keep trying to see a better number in end-to-end performance. > Also, looking with my arch/x86 hat on, there's really nothing > arch-specific here. Please try to keep stuff out of arch/x86 unless > it's very much arch-specific. Okay. I will try to keep it out of arch code. I should give up an optimization that can be achieved by working on arch code tho. Byungchul