From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F32DC4167D for ; Tue, 31 Oct 2023 18:27:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0418A6B02B5; Tue, 31 Oct 2023 14:27:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F33F06B02B7; Tue, 31 Oct 2023 14:27:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFB776B02B8; Tue, 31 Oct 2023 14:27:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id CD34E6B02B5 for ; Tue, 31 Oct 2023 14:27:12 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 9BD2B1605D0 for ; Tue, 31 Oct 2023 18:27:12 +0000 (UTC) X-FDA: 81406588704.21.1EAC092 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) by imf08.hostedemail.com (Postfix) with ESMTP id C44D516001D for ; Tue, 31 Oct 2023 18:27:10 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="KJh7/SU4"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.51 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698776830; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=U69Eg1i5YMr1sJwmANMppaJJXceNvx8HFkukPz27Hbo=; b=jz8v3+jcvGT2/r8zBQLOIh0BU1WApKy9IkESV3tJrrEkzJP+zwIEM85BO/KBcg9xqadiSF UQOB2q8QVXvOPDmGOlePNUykW/h8FB17MwYgiVQ6nCfkolI0tgGNY6xRADoQH1Fxmxd9sP p1hHAS3VWtkR2kZPv4U3zS788a4RPhk= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="KJh7/SU4"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.51 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698776830; a=rsa-sha256; cv=none; b=KE4sJ33W+Elug4JzR3wPaclMJWjN/XP3v5NzO9ZxC3smDCPzo5Qe0KEQtlcQv1gBXbBrzi TouJqwNQmKgwb9vmV41tiNwXFHlFEj75Vh8SjK2sBx+s5hbvgr8V9XxLxiwEX4A/469YHv QSkOpiMUoQjgc3QuDsE2MYejN8Fq9RU= Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-27d0e3d823fso4320834a91.1 for ; Tue, 31 Oct 2023 11:27:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698776829; x=1699381629; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=U69Eg1i5YMr1sJwmANMppaJJXceNvx8HFkukPz27Hbo=; b=KJh7/SU4JD1V7PXiuJhNrTK2Ipet1EHpinAY33OklGnM0MhF4NWqS9tmh/UBzxDRY7 kWu+UVzZXKmTola4IvrG1yHp/znGpepZF4fffmWeiU8hJiZChQwEy6EBlbvMphyP49Ff kwprvjtNaTayX+jRdSRcbckSJZGhJpeqUieHsZQ19eCxD254OF/55iGFdVdbVkufBgg1 GSRajud0gjyD/J3RyaQVwrZ9bQtGZ2aAc14r4MLvRPYi6ISsK2w/AB0hjGC8uzLDLAsd aDNw3Nj7omfOLoTxi5P2CfLWEuH9grugyuey+9Shfaf0IHHNbGgDyyj47jPTOTIy/nc+ v5Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698776829; x=1699381629; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U69Eg1i5YMr1sJwmANMppaJJXceNvx8HFkukPz27Hbo=; b=Gr+Lcn/LBpkPCGwRvmyrY8HwCJCDRjQ0y3guKtfARJWxTgnFaXOJnwPzPTjeBV4vsM qVSuiGMwqsdUJrPEO3AQwNoSqXAAgcQ0ast1IRbR2A6G+uUwVaQY9wV/ZDZHn52nE1Zu LiWChye/OM0FtgUSuAcQMMi8yDgzym2z5uWR17v/RnL3tn/bjYYvuuaVZVHp5Iwvq72P UgXkCkQSEwlYKyaXvuahPzspLKQUqMxItllaN0+Cu7Q5H3n5F6o1P1MZaaO95ylRjBXN 6XA7XwbngAOq30XPYVTwZqJu3cTmywSH9Q08pRYRW7ZyZII80SpG+feV237w744v857k 45CQ== X-Gm-Message-State: AOJu0YyZVU3E1rmen/UiZIfVyrp574G1CsE14DVZPAwSlFMGr+I4jvKz s5W0lTZEWnrVXeR1LLI5Q285OTCuT3vZUgJn/3LAPAFZ X-Google-Smtp-Source: AGHT+IFkKwl6RxjQ7PqjHXLWiBJqz4KC39TuYkrflzvEHUC0SdM4frt8XxboY7J8UTrC1rh8ZpnM3YCub5G/WKOmOqQ= X-Received: by 2002:a17:90a:31ca:b0:27d:c36:e12c with SMTP id j10-20020a17090a31ca00b0027d0c36e12cmr10811012pjf.9.1698776829643; Tue, 31 Oct 2023 11:27:09 -0700 (PDT) MIME-Version: 1.0 References: <431d9fb6823036369dcb1d3b2f63732f01df21a7.1698488264.git.baolin.wang@linux.alibaba.com> In-Reply-To: <431d9fb6823036369dcb1d3b2f63732f01df21a7.1698488264.git.baolin.wang@linux.alibaba.com> From: Yang Shi Date: Tue, 31 Oct 2023 11:26:58 -0700 Message-ID: Subject: Re: [PATCH] mm: huge_memory: batch tlb flush when splitting a pte-mapped THP To: Baolin Wang Cc: akpm@linux-foundation.org, ying.huang@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Stat-Signature: i9eu9rsehwq5zuk6y76beb8jd6gjh1s6 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C44D516001D X-HE-Tag: 1698776830-906760 X-HE-Meta: U2FsdGVkX18FU+ih5ANvuQme1dqKn8G25vGzH51kWXoK58v78DBkt3Z+C2QjONrHToFVkojwUzghOOF5xQW94u+2cjQBO/gfDovEgATntDIpS9MO7G7xfM/GbsxPDqYRbVDX9kzHeAu9zOiygFqT3F3jOInerP3WKuabhkoXJrKYnzYs0NrKblsQAZEaQ+JUoWofrnUVlVTfi3h5QQ7jhMfGl6hsklnrBHqYI3uDRmLyCuT8ltikvosFKMy3I6W2kRjCBVXS3Dok6L9JYwFB8N1s4vSdduT2ytHNb91+KtXDcB3QhNcmrF+Z2eS/KIYiQMguXsHf/BnCRErDWwiRu+4pqwCiHtuuvkJ2NQp93H6QkAioKMaKMYrxBMIlXFAHLccYmk9bmcQW1Ejumv4o631wy3f4j+wFRC1ERcBuUN8UW1LHEAQC0LgiC7TCTWeFkSoeVqx9pFiRXo2DCAXfTAs3oU8LtTQ4Kkt5JXPgn9ZCw+m/21gcN1WLSB6eSSnifRaIh1Zp5xQ2itOiDXWXnBHua2kT2lCqdWRDEsG6OZoaW4RyG4re5inS31iG1ff01tbtk6s/x8y5tzmT6yCuJopJtTJo35gBzAk55szyGpvc7gYR2DzJBke+40MwAYhnRQHHKCn7/3d3FL/VIiiNajY6MAN+6YdU8YinuwDx1eJEfHcKiDptKkzsmJYb5oq9b+eU+GeNFq89cw53yeBHLOCAqwkEqQAa3hAZSSBx1O+R4OjuM4Cnt6MPv8IJX7oyw5JQAZUf1H+WXXgPmIxAs17ze0N4GnHNGmGNP+ximwQKxzKOF0Dp9HE4EWDrMMoZXuk7Ttfd1EPwdsUoAv95J1HuYNMM8MyA7eYgo3mGfyCZjVmCQ+gyvJ0UVKksH1DwYoJFbS9qiwbHCnuEuy84JxFo/SiQyiBlrzV/X4zRtpFSV99E2P27v8WSK6K03Wke0qXqZ/tG6zlTR1LngXz ggT6uAlC c4QkBKEmqgaBE8aQc8EkcWpy/RIrTKb+sxCHBr0dCI46sMzw1raROG1jnowViYvruiQ6g1EX1azY2fMx23h8O3G3oUcmGkyeIeVZ5J075EzYHr6M7K5+SEKcaWN7fyOCimZ1TasBfb6G86h1A4zXX1GYye+HOEX0CfHQL1btOaYyznLWHGXb6QPgFz7MgdijB5VY1fUYbJ3j9eg+celNZpPppsawLN1WVpZVjvoNFfW2akzNAqjjDSxndc/GWEqvFDX/+OXqu8q3bhUsKYSHPDLk75JC67tZmfUQg8V1eBHFtMK21zZ8+rQB5PBMxQuEC74RLb7ne+YSYo72ZkwFGRrSTRycgAH2wOvLspLVDe0JRuE88Psjzt7eu/twczcpEFcq0lPPFdIPxpBWay6iqS5EGmZsuOBG4BRBgK6W68lPH95aeS2+DoOKITA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sun, Oct 29, 2023 at 6:12=E2=80=AFPM Baolin Wang wrote: > > I can observe an obvious tlb flush hotpot when splitting a pte-mapped THP= on > my ARM64 server, and the distribution of this hotspot is as follows: > > - 16.85% split_huge_page_to_list > + 7.80% down_write > - 7.49% try_to_migrate > - 7.48% rmap_walk_anon > 7.23% ptep_clear_flush > + 1.52% __split_huge_page > > The reason is that the split_huge_page_to_list() will build migration ent= ries > for each subpage of a pte-mapped Anon THP by try_to_migrate(), or unmap f= or > file THP, and it will clear and tlb flush for each subpage's pte. Moreove= r, > the split_huge_page_to_list() will set TTU_SPLIT_HUGE_PMD flag to ensure > the THP is already a pte-mapped THP before splitting it to some normal pa= ges. > > Actually, there is no need to flush tlb for each subpage immediately, ins= tead > we can batch tlb flush for the pte-mapped THP to improve the performance. > > After this patch, we can see the batch tlb flush can improve the latency > obviously when running thpscale. > k6.5-base patched > Amean fault-both-1 1071.17 ( 0.00%) 901.83 * 15.81%* > Amean fault-both-3 2386.08 ( 0.00%) 1865.32 * 21.82%* > Amean fault-both-5 2851.10 ( 0.00%) 2273.84 * 20.25%* > Amean fault-both-7 3679.91 ( 0.00%) 2881.66 * 21.69%* > Amean fault-both-12 5916.66 ( 0.00%) 4369.55 * 26.15%* > Amean fault-both-18 7981.36 ( 0.00%) 6303.57 * 21.02%* > Amean fault-both-24 10950.79 ( 0.00%) 8752.56 * 20.07%* > Amean fault-both-30 14077.35 ( 0.00%) 10170.01 * 27.76%* > Amean fault-both-32 13061.57 ( 0.00%) 11630.08 * 10.96%* > > Signed-off-by: Baolin Wang Reviewed-by: Yang Shi > --- > mm/huge_memory.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > index f31f02472396..0e4c14bf6872 100644 > --- a/mm/huge_memory.c > +++ b/mm/huge_memory.c > @@ -2379,7 +2379,7 @@ void vma_adjust_trans_huge(struct vm_area_struct *v= ma, > static void unmap_folio(struct folio *folio) > { > enum ttu_flags ttu_flags =3D TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD= | > - TTU_SYNC; > + TTU_SYNC | TTU_BATCH_FLUSH; > > VM_BUG_ON_FOLIO(!folio_test_large(folio), folio); > > @@ -2392,6 +2392,8 @@ static void unmap_folio(struct folio *folio) > try_to_migrate(folio, ttu_flags); > else > try_to_unmap(folio, ttu_flags | TTU_IGNORE_MLOCK); > + > + try_to_unmap_flush(); > } > > static void remap_page(struct folio *folio, unsigned long nr) > -- > 2.39.3 >