From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 336AEC4167D for ; Mon, 30 Oct 2023 01:55:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 66AA86B0168; Sun, 29 Oct 2023 21:55:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 61AE86B0169; Sun, 29 Oct 2023 21:55:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BB936B016B; Sun, 29 Oct 2023 21:55:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 383A56B0168 for ; Sun, 29 Oct 2023 21:55:20 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 01F7B80538 for ; Mon, 30 Oct 2023 01:55:19 +0000 (UTC) X-FDA: 81400460400.22.396021B Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by imf10.hostedemail.com (Postfix) with ESMTP id 69AA6C0009 for ; Mon, 30 Oct 2023 01:55:16 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=FMbWx16Y; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf10.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698630918; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/ZOhQkhSb9mtEzPBRTLdPtqRJ6zBjawZu30hGole3M0=; b=aBKdOYPsGEOX8k5Ojgrm2H+W2nJgDLKu1QG9O2uHYenN9EK4ZvjveEQecyzkjOmwV94YXq +tvpVYrOeR3o/iEPzs2Cx7iYZGap+zCAMiBzfIAm2VD/jKp8fYjFVKrx9+jFokvbMyCKZX GJ9Tbms4ljJNah+uVSOkdOEJiKmSgnc= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=FMbWx16Y; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf10.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698630918; a=rsa-sha256; cv=none; b=KWxQFuydE+/fZ71Rjhfg2MQjqZVCCmrxuKfCYawmQ/Saf26/bEO+DwUJMoKwMJmw6/o+8X hMLBGmO8FloseFlqgAisYdZENsmEqdU4wkxDolH1fXgCpZbnHHBdam7FmMueWhfZ+8t6K6 ddT6fZcmv/nRHBRDSWQBVnJhTY2aCrU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1698630916; x=1730166916; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=4z9cc4o/HwXQhG6Oi7EH4mIi1M0DulyFnn9b0G5P/MQ=; b=FMbWx16YZaw6WngEnmo11iVuPiaBQjB181IqPcVVKITowEfIIfu+jfcO qAsEVrrFxcVi/9YmIGARVlVKRipLSTCsLfc6hgt62rNCpf2M/8oIzMd5X VGid378yBrcns2nwfhooF+a2mbC1xGS/jhLY0fTYCT1CXf2uca9oELTVu czNp3ka+XWkAs0f1HJcKXPbVcJH84+KQRoh0F89rHIbLPb2IeheKNhM4y y1LcCQDWpzy22gLgjFIPXXnje/LS7oTL5TV/wfJBNSgcU3wD0DbNREake caF+NAvIEo9jRfdxMdTIWVdo1nZXqWmyknn705RTvhRaRJMFrYLpD2d/s Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10878"; a="390854921" X-IronPort-AV: E=Sophos;i="6.03,262,1694761200"; d="scan'208";a="390854921" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Oct 2023 18:55:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10878"; a="789317352" X-IronPort-AV: E=Sophos;i="6.03,262,1694761200"; d="scan'208";a="789317352" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Oct 2023 18:55:12 -0700 From: "Huang, Ying" To: Baolin Wang Cc: , , , Subject: Re: [PATCH] mm: huge_memory: batch tlb flush when splitting a pte-mapped THP In-Reply-To: <431d9fb6823036369dcb1d3b2f63732f01df21a7.1698488264.git.baolin.wang@linux.alibaba.com> (Baolin Wang's message of "Mon, 30 Oct 2023 09:11:47 +0800") References: <431d9fb6823036369dcb1d3b2f63732f01df21a7.1698488264.git.baolin.wang@linux.alibaba.com> Date: Mon, 30 Oct 2023 09:53:10 +0800 Message-ID: <87bkcgev09.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Queue-Id: 69AA6C0009 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: n7miwkmdriax4h5kga15maojjkewxi1x X-HE-Tag: 1698630916-870744 X-HE-Meta: U2FsdGVkX18VH8HzHu7gc5SKmkXi0sit7PBUR15JZJBBtP9ALgj4ZzHoaSDyq1baEwO4yWGqJLPlEe18SssKiZ2KIAavkzYfC2tIUnHU29DPcEQiKQOapHa+476p+l4DbKuGbjfYa7bUPbYDlkmW0AhY3bporJOKrGKtU9TTnN1xHIxyr5kPYHuPwTaT9c8FaCaJUPn8QmjGGAZ76VsIovSjW8NT/WNSSNFQNn3HQnj9imlVFSIbQGvf7vaY/WO9LiIvChgxpxSjzF0P2qhusJNgCK9fuzNJAY2r7CpUXfiW8i66W4iAGtiptFnqUisoJXjzW0+m6cezWhpkDwbP/B/MZ59o3RkTT+lBVLT/KHNisSiU9NBDPx5/ZJc/h5YrTESJW5oA22w3MNqqylM1kH9tUrBhwneaXWjVosOIlUs0z0SeW2fdmN3pBzdN+8azz0TXtppEQZjD4EQyPkQfx/JXCzvYmL8Xz+SKAzne2m0UwSeFt763GLVbcwQm1Z0e/C6VLs1jJ7eYrevW0nKi44+NdqdghVZUItEKJe3Yej7oFNv/iIf8AvQL7UXnuqocJlhMnweuCQRZ/CRsjAEjHBSE2xHVv63uMrEVjK9WfkDK8g4FSJuc4ehRNiV3OGKzAf0xNVf2f7LKyMegxRcRTOUzF1NwGoy1gbyWJEL5x9aHfulk+K+EOVMkF+xJo5bQVCregBtn71EBoDfDhb1gdv6wOSB/6+SZQHh7UGpSJNrZoOM/gCssM3GZiNEO6yxfKXMO6S2wJ1maBugiHT5iO1HOQPgWoPqfmjXYyNyyrFWpIvcuOE23L84JsIk2pUVE43Ady6y5DkeJMSbDTMRMAeHXD17zrSVVR8g8CsssERJVxsSKeqKBkWDlpQbY5qVqTMJcs7Uz3JGQhtew2jTbkHBqmcJbPGyY4i0R+xt6keeqh4Oip7ymW/quDhuVTwlevtVnTbfgxI0Azx9Rw/Z nLEE06AF /GOrs0glcd6keBJBDL5wdG/i8wdjHDrHgZKXyYIvOyfmHpIRrjms0FGzSULlwZxrEvddBejkz9njthnbK6wBxQLsb2VowkmtesrrVmXBmskZjuMiOB6HGdd4xoGPcHGD4kthUTnrCMCEtObKaOPCE/Y43mbub9cDfJiCjsWMa91m8FDFaHlou4wDFiyHo2jrIam/L69lcyMf0AByTs+LpkeocC4S+ij8ATEM1eRkqjg0322E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Baolin Wang writes: > I can observe an obvious tlb flush hotpot when splitting a pte-mapped THP on > my ARM64 server, and the distribution of this hotspot is as follows: > > - 16.85% split_huge_page_to_list > + 7.80% down_write > - 7.49% try_to_migrate > - 7.48% rmap_walk_anon > 7.23% ptep_clear_flush > + 1.52% __split_huge_page > > The reason is that the split_huge_page_to_list() will build migration entries > for each subpage of a pte-mapped Anon THP by try_to_migrate(), or unmap for > file THP, and it will clear and tlb flush for each subpage's pte. Moreover, > the split_huge_page_to_list() will set TTU_SPLIT_HUGE_PMD flag to ensure > the THP is already a pte-mapped THP before splitting it to some normal pages. > > Actually, there is no need to flush tlb for each subpage immediately, instead > we can batch tlb flush for the pte-mapped THP to improve the performance. > > After this patch, we can see the batch tlb flush can improve the latency > obviously when running thpscale. > k6.5-base patched > Amean fault-both-1 1071.17 ( 0.00%) 901.83 * 15.81%* > Amean fault-both-3 2386.08 ( 0.00%) 1865.32 * 21.82%* > Amean fault-both-5 2851.10 ( 0.00%) 2273.84 * 20.25%* > Amean fault-both-7 3679.91 ( 0.00%) 2881.66 * 21.69%* > Amean fault-both-12 5916.66 ( 0.00%) 4369.55 * 26.15%* > Amean fault-both-18 7981.36 ( 0.00%) 6303.57 * 21.02%* > Amean fault-both-24 10950.79 ( 0.00%) 8752.56 * 20.07%* > Amean fault-both-30 14077.35 ( 0.00%) 10170.01 * 27.76%* > Amean fault-both-32 13061.57 ( 0.00%) 11630.08 * 10.96%* > > Signed-off-by: Baolin Wang LGTM, Thanks! Reviewed-by: "Huang, Ying" > --- > mm/huge_memory.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > index f31f02472396..0e4c14bf6872 100644 > --- a/mm/huge_memory.c > +++ b/mm/huge_memory.c > @@ -2379,7 +2379,7 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma, > static void unmap_folio(struct folio *folio) > { > enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD | > - TTU_SYNC; > + TTU_SYNC | TTU_BATCH_FLUSH; > > VM_BUG_ON_FOLIO(!folio_test_large(folio), folio); > > @@ -2392,6 +2392,8 @@ static void unmap_folio(struct folio *folio) > try_to_migrate(folio, ttu_flags); > else > try_to_unmap(folio, ttu_flags | TTU_IGNORE_MLOCK); > + > + try_to_unmap_flush(); > } > > static void remap_page(struct folio *folio, unsigned long nr)