From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDBABE7734F for ; Sat, 30 Sep 2023 04:28:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 413826B01A3; Sat, 30 Sep 2023 00:28:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 39A726B01A4; Sat, 30 Sep 2023 00:28:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2140A6B01A5; Sat, 30 Sep 2023 00:28:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 08D136B01A3 for ; Sat, 30 Sep 2023 00:28:18 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id D76BDA0561 for ; Sat, 30 Sep 2023 04:28:17 +0000 (UTC) X-FDA: 81291981834.09.B654AD8 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by imf15.hostedemail.com (Postfix) with ESMTP id EFFD0A002C for ; Sat, 30 Sep 2023 04:28:14 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=PrfvxiKh; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf15.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696048096; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KCfWLUOk3e1JYc4W76Rfp1gmEnzrFY9/V52vbHqCqSQ=; b=E7TNo2fQ9sYOAW4Bm8bwLG1HWQ1Fl1frJ75Y/y+qMHt3JL87oIWgT8DtD8l2Zqli0nxgZV Qi6h+FFU+3BJn6iLnfr7QYgEpIbjJ4D5Yc3LQwkLuQ6BcaS3J+znRqT0smWAeVhPMKdhUP hG4gS3fl5oMwlp90kBOuJ4VhND0X840= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=PrfvxiKh; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf15.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696048096; a=rsa-sha256; cv=none; b=nb6j52JqGDytkB/wx1MrbUFT2dDoWz1uevJaqZnNA69rrSNDJgeeLmHhWTyrw8iU9rn+V+ kg3PyMEa5ipCzZnEHy4h+VdI2lEYJC+pirZPWM4QeDX9gAdiDgsERo6UiqUzWmPN9/r4pe i6YjE4tVWnTjhG+FFnNNFKG9o9YwUUA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696048095; x=1727584095; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=rVpj1lqLtJb8TCKjOHawgxXj1vC2ae/09G5U9A5u7Xo=; b=PrfvxiKhjovxPVXm6s6J+57HrdW5+H6j+NwrccCTfD5WGP6ZQheSZtzR AhUwtwqq1l6m2893FRCeQlFsVelo/gHzWqXR6T4qFUS6uObEzlkidbVhQ Ob7Jl9OKdcqf7Ma0A+s9ttlK2ryvmkD+MJ0zusBOl3UpR6uDD3WrWl458 UNQRGreGzS03Z9nHv0s5ykfVUaAqwCKmI3Z6uvHE1I6kD6O2MmBNfGNAL Rx0tL3FNibFTJpLt7ar00ug75zpHREoqkpFYqbYuk4rYjHgGxF3UV4qlg D2NCVqLYsdHkQY6KKWxFxF6S8Ljssrq+5kcSyRC6WqVCxivubR+TpOfz6 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10848"; a="381301880" X-IronPort-AV: E=Sophos;i="6.03,189,1694761200"; d="scan'208";a="381301880" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2023 21:28:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10848"; a="750110687" X-IronPort-AV: E=Sophos;i="6.03,189,1694761200"; d="scan'208";a="750110687" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2023 21:28:10 -0700 From: "Huang, Ying" To: Johannes Weiner Cc: Andrew Morton , Vlastimil Babka , Mel Gorman , Miaohe Lin , Kefeng Wang , Zi Yan , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/6] mm: page_alloc: remove pcppage migratetype caching References: <20230911195023.247694-1-hannes@cmpxchg.org> <20230911195023.247694-2-hannes@cmpxchg.org> <87y1gsrx32.fsf@yhuang6-desk2.ccr.corp.intel.com> <20230927145115.GA365513@cmpxchg.org> Date: Sat, 30 Sep 2023 12:26:01 +0800 In-Reply-To: <20230927145115.GA365513@cmpxchg.org> (Johannes Weiner's message of "Wed, 27 Sep 2023 10:51:15 -0400") Message-ID: <87pm20p9ra.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspam-User: X-Stat-Signature: 7f41h5dd6ui4crns8n7xyn1fxmgdkg36 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: EFFD0A002C X-HE-Tag: 1696048094-398084 X-HE-Meta: U2FsdGVkX1+UWcWEoXYwDhthCBXVeolnsEs+FCuGh5TfolTDWSoMnOiUJbVHixS1AhAUqoKUOseTpSrhlWTr26+mJnlEzN/fFyuhFCrRO/qPlYr9GWduh8ZTb1Amhgyd98GfdkALICnRSzKFdHGKPwb5ZxYVhPOm6rb8tr+4z6yqywnEj+tdMsC7A9B3XoR10kWzpOv7T/ZVI3MRJFsmHYpPrq9hqOgy5RFGv0bH3GMOm4QQpg4X4Qg9GrGM0011V5+NlLvo5uHMVzN+jaxuOX/koriV8eyyU+QF6q7GcSZcEipAoDE0iKkXZJIBPDZvegx3yatD0/YRAkZhdBgWgAeGCNqzHtPuYOVmIR2b0dFPDs4Fwu0i7ff3uhqlq5Slge0vpi6n8JTOkf+Ullbbx18aILW5ckLnZdyJaVHPmhFbFXHOMSerxr6zNW3HdS8kwGTStdAx03NTx15nqYwMgR84a++f5NzKmBh3s9XeoPtqYNmBCEObgjo0nssGjsaOXDXe+xQJ+y9byo+7VdNSdh7KDcJstH1FiveBWro5DS74afwEprftiGznwMD5fS40Y/dHd0CT7zT/XuEGjX/GjN1wItN7SDgsup4XZwEZdRPiCXTOZjSWt+9K2Tsgx4pZwVuYGSwWqs0/ADEYkNOouDGQqNJ6xtxIUguV3fVVJxbqOifWpsMhTfwd+1Hx9aRcNkyvfS9WxLeMsN949XwzBV5W8DZtk5Nw7tNoZkfmoXI+uRIQvXtyB4NC3ehGRXAmhUCXHvFHKgGY1o7ZRACT7q01Ekjn3/SbaGZSkA9oYueqeHAet6nPJxkf/yvv/Q5akRhSQRO6wByIqohzQGUbg+IK77Kjk1NGsLUMCcf73uZO+tjc82pbACLmEoJSRDeL+MVAuTdwYJAhV6Ck86+LKrWL3tvlZyfNF63Yu0IwJjdG7SDZgplTU/sFVBPl82aeCn6Obfd9lLrYKwKeOKv 4jxdDfA/ 64eZKhBNhD9eGwpoosSOKATnmG4Ax0e+jxA3l81AnxAYJt5Y0x6HRuEsYNSgvwfOGE6y+ZRUPbR6apnEUkkMIsLlMDyJl2A9QMKZ1FLnl7yPLzmTKXB9UQnS5sW/6GRp9R028q8blhzYZ1drnhxXenus4Pj6yJ1n+95A7Ydj4+oQyHCCn4GgopX0NBvFGuGXVyqFAB9S+Pjr6yfl2m5c7pPDxeQsAhMWjFDcTEKMKUm33gzq7y3B9Bx/U3iaycYIVvYED X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Johannes Weiner writes: > On Wed, Sep 27, 2023 at 01:42:25PM +0800, Huang, Ying wrote: >> Johannes Weiner writes: >> >> > The idea behind the cache is to save get_pageblock_migratetype() >> > lookups during bulk freeing. A microbenchmark suggests this isn't >> > helping, though. The pcp migratetype can get stale, which means that >> > bulk freeing has an extra branch to check if the pageblock was >> > isolated while on the pcp. >> > >> > While the variance overlaps, the cache write and the branch seem to >> > make this a net negative. The following test allocates and frees >> > batches of 10,000 pages (~3x the pcp high marks to trigger flushing): >> > >> > Before: >> > 8,668.48 msec task-clock # 99.735 CPUs utilized ( +- 2.90% ) >> > 19 context-switches # 4.341 /sec ( +- 3.24% ) >> > 0 cpu-migrations # 0.000 /sec >> > 17,440 page-faults # 3.984 K/sec ( +- 2.90% ) >> > 41,758,692,473 cycles # 9.541 GHz ( +- 2.90% ) >> > 126,201,294,231 instructions # 5.98 insn per cycle ( +- 2.90% ) >> > 25,348,098,335 branches # 5.791 G/sec ( +- 2.90% ) >> > 33,436,921 branch-misses # 0.26% of all branches ( +- 2.90% ) >> > >> > 0.0869148 +- 0.0000302 seconds time elapsed ( +- 0.03% ) >> > >> > After: >> > 8,444.81 msec task-clock # 99.726 CPUs utilized ( +- 2.90% ) >> > 22 context-switches # 5.160 /sec ( +- 3.23% ) >> > 0 cpu-migrations # 0.000 /sec >> > 17,443 page-faults # 4.091 K/sec ( +- 2.90% ) >> > 40,616,738,355 cycles # 9.527 GHz ( +- 2.90% ) >> > 126,383,351,792 instructions # 6.16 insn per cycle ( +- 2.90% ) >> > 25,224,985,153 branches # 5.917 G/sec ( +- 2.90% ) >> > 32,236,793 branch-misses # 0.25% of all branches ( +- 2.90% ) >> > >> > 0.0846799 +- 0.0000412 seconds time elapsed ( +- 0.05% ) >> > >> > A side effect is that this also ensures that pages whose pageblock >> > gets stolen while on the pcplist end up on the right freelist and we >> > don't perform potentially type-incompatible buddy merges (or skip >> > merges when we shouldn't), whis is likely beneficial to long-term >> > fragmentation management, although the effects would be harder to >> > measure. Settle for simpler and faster code as justification here. >> >> I suspected the PCP allocating/freeing path may be influenced (that is, >> allocating/freeing batch is less than PCP high). So I tested >> one-process will-it-scale/page_fault1 with sysctl >> percpu_pagelist_high_fraction=8. So pages will be allocated/freed >> from/to PCP only. The test results are as follows, >> >> Before: >> will-it-scale.1.processes 618364.3 (+- 0.075%) >> perf-profile.children.get_pfnblock_flags_mask 0.13 (+- 9.350%) >> >> After: >> will-it-scale.1.processes 616512.0 (+- 0.057%) >> perf-profile.children.get_pfnblock_flags_mask 0.41 (+- 22.44%) >> >> The change isn't large: -0.3%. Perf profiling shows the cycles% of >> get_pfnblock_flags_mask() increases. > > Ah, this is going through the free_unref_page_list() path that > Vlastimil had pointed out as well. I made another change on top that > eliminates the second lookup. After that, both pcp fast paths have the > same number of lookups as before: 1. This fixes the regression for me. > > Would you mind confirming this as well? I have done more test for the series and addon patches. The test results are as follows, base perf-profile.children.get_pfnblock_flags_mask 0.15 (+- 32.62%) will-it-scale.1.processes 618621.7 (+- 0.18%) mm: page_alloc: remove pcppage migratetype caching perf-profile.children.get_pfnblock_flags_mask 0.40 (+- 21.55%) will-it-scale.1.processes 616350.3 (+- 0.27%) mm: page_alloc: fix up block types when merging compatible blocks perf-profile.children.get_pfnblock_flags_mask 0.36 (+- 8.36%) will-it-scale.1.processes 617121.0 (+- 0.17%) mm: page_alloc: move free pages when converting block during isolation perf-profile.children.get_pfnblock_flags_mask 0.36 (+- 15.10%) will-it-scale.1.processes 615578.0 (+- 0.18%) mm: page_alloc: fix move_freepages_block() range error perf-profile.children.get_pfnblock_flags_mask 0.36 (+- 12.78%) will-it-scale.1.processes 615364.7 (+- 0.27%) mm: page_alloc: fix freelist movement during block conversion perf-profile.children.get_pfnblock_flags_mask 0.36 (+- 10.52%) will-it-scale.1.processes 617834.8 (+- 0.52%) mm: page_alloc: consolidate free page accounting perf-profile.children.get_pfnblock_flags_mask 0.39 (+- 8.27%) will-it-scale.1.processes 621000.0 (+- 0.13%) mm: page_alloc: close migratetype race between freeing and stealing perf-profile.children.get_pfnblock_flags_mask 0.37 (+- 5.87%) will-it-scale.1.processes 618378.8 (+- 0.17%) mm: page_alloc: optimize free_unref_page_list() perf-profile.children.get_pfnblock_flags_mask 0.20 (+- 14.96%) will-it-scale.1.processes 618136.3 (+- 0.16%) It seems that the will-it-scale score is influenced by some other factors too. But anyway, the series + addon patches restores the score of will-it-scale. And the cycles% of get_pfnblock_flags_mask() is almost restored by the final patch (mm: page_alloc: optimize free_unref_page_list()). Feel free to add my "Tested-by" for these patches. -- Best Regards, Huang, Ying