From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61560C07548 for ; Wed, 15 Nov 2023 06:43:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 812306B0322; Wed, 15 Nov 2023 01:43:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C3D76B0324; Wed, 15 Nov 2023 01:43:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68A116B032A; Wed, 15 Nov 2023 01:43:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 588ED6B0322 for ; Wed, 15 Nov 2023 01:43:22 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 32D33B5D68 for ; Wed, 15 Nov 2023 06:43:22 +0000 (UTC) X-FDA: 81459247044.11.FA77A10 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf04.hostedemail.com (Postfix) with ESMTP id 6D5D740008 for ; Wed, 15 Nov 2023 06:43:19 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf04.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700030600; a=rsa-sha256; cv=none; b=gNvrp8cGhkoHjWNJ5UkG/o8bWELlrRll4N5310cuQJ/Hd16e9lxU/OrhejcRlxL4vZB1UU zL4x7Nm57hNXfzmH0cw/MSt8H4INe1J9n3eW4YBTl4sXVv3W8x8ps36ZA4/R67jqkQaFc3 aa0xr3vokLGgkBNK97kF2LnrlRmcRNk= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf04.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700030600; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NgPBUtrXUl754KuyTIpOJ3IRFCFgaBW+KV3QR5Uy0ZI=; b=q+cuzAzkoZYbLxTGY3yqsaXErlcDGaXboMEl/n4w8VVT5OEh1NqtKEosJC9DF6Bh3tY7KN bhGRkXYnowy9LyRRiY1mGOZx0pr9nxPAEmYkf5648vuBBQbDrs5oyXpQDW5oGpT4c0KG90 LpM216jm+broKbihvkkgdlEAbOQS+do= X-AuditID: a67dfc5b-d6dff70000001748-52-65546885ec9d Date: Wed, 15 Nov 2023 15:43:11 +0900 From: Byungchul Park To: Dave Hansen Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, namit@vmware.com, xhao@linux.alibaba.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com Subject: Re: [v4 0/3] Reduce TLB flushes under some specific conditions Message-ID: <20231115064311.GA41022@system.software.com> References: <20231109045908.54996-1-byungchul@sk.com> <64cb078b-d2e7-417f-8125-b38d423163ce@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <64cb078b-d2e7-417f-8125-b38d423163ce@intel.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrLIsWRmVeSWpSXmKPExsXC9ZZnkW5rRkiqwcaFIhZz1q9hs/i84R+b xaeXDxgtXmxoZ7T4uv4Xs8XTT30sFpd3zWGzuLfmP6vF+V1rWS12LN3HZHHpwAImi+u7HjJa HO89wGSxedNUZovfP4Dq5kyxsjg5azKLg6DH99Y+Fo8Fm0o9Nq/Q8li85yWTx6ZVnWwemz5N Yvd4d+4cu8eJGb9ZPHY+tPSYdzLQ4/2+q2weW3/ZeXzeJOfxbv5btgC+KC6blNSczLLUIn27 BK6MjdeCC5r4Kl5MPs/cwNjP3cXIySEhYCLx+HAjUxcjB5g9Z6IwiMkioCpx6JkiSAWbgLrE jRs/mUFsESD71Mrl7F2MXBzMAk3MEtfWb2YFSQgLuEvsOnUHrIhXwEKi9VsnE4gtJJAhcfz1 HHaIuKDEyZlPWEBsZgEtiRv/XoKtZRaQllj+jwMkzClgK7G/4RkbiC0qoCxxYNtxJpBdEgKb 2CVWTLjIBHGypMTBFTdYJjAKzEIydhaSsbMQxi5gZF7FKJSZV5abmJljopdRmZdZoZecn7uJ ERiJy2r/RO9g/HQh+BCjAAejEg9vw8zgVCHWxLLiytxDjBIczEoivOZyIalCvCmJlVWpRfnx RaU5qcWHGKU5WJTEeY2+lacICaQnlqRmp6YWpBbBZJk4OKUaGHnNvSceYXvpabTD3WTfemVF v8/bC+pqJQUezq4OUVBb82pmVM07a/391yIdD/z1utcZolB+dZ/166QLn2zanx7iWdOXJzph 9aRYowuTyo0LWzgS5jl+/7QwMFSi8NW3xrzqJB8l5xDOl/cyHn3fG7/pxpHUt7c8YgSnXgqe /l/Y2Wa5cucvNyWW4oxEQy3mouJEAL5nD2zAAgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprCIsWRmVeSWpSXmKPExsXC5WfdrNuSEZJqsHyjlcWc9WvYLD5v+Mdm 8enlA0aLFxvaGS2+rv/FbPH0Ux+LxeG5J1ktLu+aw2Zxb81/Vovzu9ayWuxYuo/J4tKBBUwW 13c9ZLQ43nuAyWLzpqnMFr9/ANXNmWJlcXLWZBYHIY/vrX0sHgs2lXpsXqHlsXjPSyaPTas6 2Tw2fZrE7vHu3Dl2jxMzfrN47Hxo6THvZKDH+31X2TwWv/jA5LH1l53H501yHu/mv2UL4I/i sklJzcksSy3St0vgyth4Lbigia/ixeTzzA2M/dxdjBwcEgImEnMmCoOYLAKqEoeeKXYxcnKw CahL3LjxkxnEFgGyT61czt7FyMXBLNDELHFt/WZWkISwgLvErlN3wIp4BSwkWr91MoHYQgIZ Esdfz2GHiAtKnJz5hAXEZhbQkrjx7yUTyC5mAWmJ5f84QMKcArYS+xuesYHYogLKEge2HWea wMg7C0n3LCTdsxC6FzAyr2IUycwry03MzDHVK87OqMzLrNBLzs/dxAiMq2W1fybuYPxy2f0Q owAHoxIPb8PM4FQh1sSy4srcQ4wSHMxKIrzmciGpQrwpiZVVqUX58UWlOanFhxilOViUxHm9 wlMThATSE0tSs1NTC1KLYLJMHJxSDYwh8ZqOhb1X+eabrPsx5Zj8Mcu3+UKzUvbNvuTtnrZ9 dcy+pdMuFJ+tnrdrq7Uq646VnSw3/Fdy8r7ImNzGu8RGIni61aRyV+/L29+5HzRkW691pt+c NV8j2PLcuoMOG2xO6Pr2/Pzozvy96gQTi/O/FPGPGlNv7u9YJDQxRHTKw82PO7kEF05WYinO SDTUYi4qTgQA1c1wOacCAAA= X-CFilter-Loop: Reflected X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 6D5D740008 X-Stat-Signature: aca6brj8i559u4m9jtic14d6f9kkrjf8 X-Rspam-User: X-HE-Tag: 1700030599-750665 X-HE-Meta: U2FsdGVkX1+ecMPABZ1pO0H/j7XfarYY3li0NSEJmTHzeWJP7/fSXfm3qOHgiZD6kypY1vdnoEv1WQy/7fsgh+Dv3LKdznCbW+DZPPem3taiD/7B29RAG16XewxybB92yzcYE5i57QbUo1RsJ9L1dgi101nZMQSTqqjeUp9j/uUjPx4WJu2Ol8bA+u3ffRrNTRMve+wXR0cgzEgPinJAZWeReyOdVjFYo0fOCLqwi7MbgVvxvMt5lryvN2QGr+8Blin553tPHTGhpeWs+Fz1jbxtliuW7T1EgeJcNdmvYxwPoe3P74b1wB372M56R1BckGkptbfvx1q82LRQxtJ9XVnu+7Ap2mQrAWvSJEckh40wwSMO0pPg6Pl/oa+Iz2b0tI7/w1qFTFIZAnYTGIgHi++uTj+Kx+UlG9DcZZ01lfcBcI+jnC/x0eHwHzG6gbSdh3yQ9/QG3N+S1MTavIxvuwR/6jh7FVoOn1BRINODp3LI6XeaxSmvSIdNCci4Fo5srWajGoEA99iqJmZjqUGxFRmOK65kYrOlIn5DwDQCYw/DARG3zBa/8ajjA2NWIMYcV9NVt0Kfi003MfX6S3PzbuMToImiNhcsWSEsub/ADH3Jx2reZy1cjusdhd2D73t8B4UrCPkgpLqrgsiwa3ZC7PSA1mkFm+eZfKmp19Sh9dSvvtt5kY4FVCqa2Xv4lW4gSRjFK7wmFfE+Bh7TGj9G13apSRqZ3Jmm7lnIZRf3K0/gACzK6w2HWC9fpVw4MzL0Lydjn8R3PQrl8zJ3AbhOwLYwCTSvK/zmYlqX/9gdSHWCgIqdScwY9PdIdI+mJd7vD9XVKKtf4+nGeWDKla2OXTJfon+MhnWGoYDCu9tB+cEbBuTMGfLpCpSUZE9h/XWpWXZr4AhszYbMmOU0xdNk7+4AzCoLlZBGltyLEUKabwaRQ80+JLowU4LtOqyMO0DixOV2QaVTRhjPXho8AMS SUFwA68Y KF48/RxSehAurg5MY3B+xi4S4nWRlk4spnmQxgGBMQg1uDCL/XNWNYk1RVe30riJWCyDc X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Nov 09, 2023 at 06:26:08AM -0800, Dave Hansen wrote: > On 11/8/23 20:59, Byungchul Park wrote: > > Can you believe it? I saw the number of TLB full flush reduced about > > 80% and iTLB miss reduced about 50%, and the time wise performance > > always shows at least 1% stable improvement with the workload I tested > > with, XSBench. However, I believe that it would help more with other > > ones or any real ones. It'd be appreciated to let me know if I'm missing > > something. > > I see that you've moved a substantial amount of code out of arch/x86. > That's great. > > But there doesn't appear to be any improvement in the justification or > performance data. The page flag is also here, which is horribly frowned > upon. It's an absolute no-go with this level of justification. > > I'd really suggest not sending any more of these out until those issues > are rectified. I know I definitely won't be reviewing them in this state. As I expected, I got a fair better result when I tested migrc with a system with a slower DRAM to make TLB miss overhead stand out. 1. XSBench execution time was reduced about 7%. 2. iTLB flush # was reduced stably about 90% while running XSBench. 3. iTLB miss # was reduced stably about 50% while running XSBench. https://lore.kernel.org/lkml/20231115025755.GA29979@system.software.com/ Of course, I can reimplement migrc to replace PG_migrc with another thing like hash table but, IMHO, it's worth having the page flag if it gives such a good performance. Lemme know if not so that I'll change the way to implement. I'd like to note that no doubt migrc significantly reduces TLB miss and the impact depends on TLB miss overhead that varies according to the system configuration. Byungchul