From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D623CDB46E for ; Thu, 12 Oct 2023 12:13:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 11C3F8D0123; Thu, 12 Oct 2023 08:13:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0CC9F8D0002; Thu, 12 Oct 2023 08:13:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EB02A8D0123; Thu, 12 Oct 2023 08:13:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D71EE8D0002 for ; Thu, 12 Oct 2023 08:13:30 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A54DBA060B for ; Thu, 12 Oct 2023 12:13:30 +0000 (UTC) X-FDA: 81336699780.22.8843A1F Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.43]) by imf09.hostedemail.com (Postfix) with ESMTP id 7684D14001F for ; Thu, 12 Oct 2023 12:13:28 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=AgImKoe1; spf=pass (imf09.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697112808; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FDSWCRwXqsnp5hycKErcp9tddRy7oS50ZF4M12lFBXA=; b=zlIoe3/f/UnBxYlcr5ysfohTazCkh46nkVuDupRbnugh4AaJY914OEq470BEGb1hUqevgq krngM+XfA1xDhLd4Cpt0UX4XeD92MZCRiJ4ijCqB0omIgczO1Q1yskSBhedtW8pVoU/PWH 6qVALehFJGdyfq+ZQeHV6LD28kQm6p4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697112808; a=rsa-sha256; cv=none; b=hWQ7udOqHtfRMheviq2jTSUgGPVPsVx2fhyLF32NlKqw6ICFApBZgn7YxKWeCKUf36jgoY Ogw1KclLIimcYY/RGuAii2F/7Fye334mT+b97MFZN4KhEj60SGMBfr8xGePEcr1KtxYb86 p5hn1opbHqPbC7Z3b1qR/H61QGDlceM= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=AgImKoe1; spf=pass (imf09.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697112808; x=1728648808; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=w8yQt9oo86/nZjDSgadz2LDv3J3dqgnu9r6s3Un8WxE=; b=AgImKoe1/JipFQeRQL4HgLEAm70pLfn2/ObldG7PIUBP/fc56BUMMLD6 MOCI1oqubpKrNHsLn1ZnkYd0a57vnd1bwhDzSJ/UV7xYp5a2RFRDOCaTe lLPoXVO+RfsgH8ahly3GJJ73Z0YBWlm7UCsafXWvcJnHQXdQObe9E3ZNd PpvgAsOSr1zx5q1TWRB+HBJXZIu7Bk5yWSPvncUtlZDFzTlZfIDYxyz+3 PQwUGrL1RDazS/EVBKzmhqYtLgVFOgwDu5HJFtIoWQlpw4aluDKCJAeVZ 8KTH549zSu+9dBjUHNw8gKzLYfmAy07hlFSVpmsW53Ggdc9JZZbq+FmTQ A==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="471162906" X-IronPort-AV: E=Sophos;i="6.03,218,1694761200"; d="scan'208";a="471162906" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 05:13:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="789374334" X-IronPort-AV: E=Sophos;i="6.03,218,1694761200"; d="scan'208";a="789374334" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 05:13:23 -0700 From: "Huang, Ying" To: Mel Gorman Cc: , , Arjan Van De Ven , Andrew Morton , Vlastimil Babka , David Hildenbrand , Johannes Weiner , Dave Hansen , Michal Hocko , Pavel Tatashin , Matthew Wilcox , "Christoph Lameter" Subject: Re: [PATCH 03/10] mm, pcp: reduce lock contention for draining high-order pages References: <20230920061856.257597-1-ying.huang@intel.com> <20230920061856.257597-4-ying.huang@intel.com> <20231011124900.sp22hoxoitrslbia@techsingularity.net> Date: Thu, 12 Oct 2023 20:11:19 +0800 In-Reply-To: <20231011124900.sp22hoxoitrslbia@techsingularity.net> (Mel Gorman's message of "Wed, 11 Oct 2023 13:49:00 +0100") Message-ID: <87cyxkf3bs.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Queue-Id: 7684D14001F X-Rspam-User: X-Stat-Signature: feanzkftxqcacukaqz8jficj16z1yoqb X-Rspamd-Server: rspam03 X-HE-Tag: 1697112808-744971 X-HE-Meta: U2FsdGVkX18qbU72IKl55Q3n9R5c/h0elRld8WFH/VsgwwXhrMQ4KW0Nf98kYSiyD0eZ40HuKMjMbUz8ln23fbta5c3mSES+G0gQcJuqB8NFJVlz474Cqtty3ShjSm03t274evpGep18W+UqHqu3IJ/G0LMiBfoq9fgwXzTfhd89Togv7cKeL5tv1dq+IEq+RCHt0nFZzyEIr1IiZuDjVUXB9yVVLQnmv6urEtMWTzC9I3DWdDvq4XAkM5PPQGLQu3bzwoD41cynxVJiVFf0ZEAx40EX6eqJEkBfoUZulILUrqaFmQ0r0UegVCKH8Tt7DBP+gbiTA/MprCzlOulYHmVQEHabysCmJkMhwxoQmwEYv17D+DSVpUr0kxRjWHHDgUqEDeJS/wH705btnmT+QCOUKPRES3F2+oRZywPCozjbs9QnbzMaDl6XPM3bIxKFvn+jH5mhDLnZeUvBTMat9F+QTWP0eu43jAWiMk6gb8WBXtYRkE3b2rpeiUEkgZUKfrNKRS01uff1LpHHr9H7MMN/YSbjbRSSr85UDQ3yS4v/Ay69rEYalOKhadrGDBcAdxyjuZoy0PVL/SEvsbikt7UZ/0JtKUZZd6qF3Q9X7SpvddZdLcWC1T7WDDQQCw++Jy/xNmcAeW7k+mIdaiG9pI3lK1EefH1Li8AcipXrsTwS8P2Hgmq97pdPD8cTRBhiRniK3da03cpX60gEGSnTjT+7FS7wO5Md85knR0FjsmDrYaWVQ+GeRM9DPYkOZx+GKLlBvHeI30U17E1X6smRkh3B3bP485yjS6PFWI/QyuymYry6cEZgopWsSyYZKojINZ5MDhb5H8GQ71BGCiYoDGCWTAJWGwmvpzzixx3BPNU8QTjAHTUnwfC6u0JGa/aUOuY3oRg4TRU2Q7vBNwWeF0NCkY5CGlbkchfGa4WfiiVMM4Pzf2YinIJmNbkXPIvKgQOgkO5RAIlzHZVWVCq dYbJmt4h JrKtNkuRLt35985T7rebhv2DbKkADqPh4uCXfrAIjVyWLWi1gZdiHzOS2CUwvXovqYHjJyYJOTS+0CcjUub4WpLB5mieK2nCg5z5U7uxIfzGyNQLY64XMMOvH4XQ4gW3whc95qOIrvOW/Dy3zPiFGCN2oYhr5HdBVJP3Q2PUXXC2wiEBq6ULbo0YOEsKofMMUuByHSEgu4zWmcy0sdjLbWgXpi1zGmAxapI5S X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Mel Gorman writes: > On Wed, Sep 20, 2023 at 02:18:49PM +0800, Huang Ying wrote: >> In commit f26b3fa04611 ("mm/page_alloc: limit number of high-order >> pages on PCP during bulk free"), the PCP (Per-CPU Pageset) will be >> drained when PCP is mostly used for high-order pages freeing to >> improve the cache-hot pages reusing between page allocating and >> freeing CPUs. >> >> On system with small per-CPU data cache, pages shouldn't be cached >> before draining to guarantee cache-hot. But on a system with large >> per-CPU data cache, more pages can be cached before draining to reduce >> zone lock contention. >> >> So, in this patch, instead of draining without any caching, "batch" >> pages will be cached in PCP before draining if the per-CPU data cache >> size is more than "4 * batch". >> >> On a 2-socket Intel server with 128 logical CPU, with the patch, the >> network bandwidth of the UNIX (AF_UNIX) test case of lmbench test >> suite with 16-pair processes increase 72.2%. The cycles% of the >> spinlock contention (mostly for zone lock) decreases from 45.8% to >> 21.2%. The number of PCP draining for high order pages >> freeing (free_high) decreases 89.8%. The cache miss rate keeps 0.3%. >> >> Signed-off-by: "Huang, Ying" > > Acked-by: Mel Gorman > > However, the flag should also have been documented to make it clear that > it preserves some pages on the PCP if the cache is large enough. Sure. Will do this. > Similar > to the previous patch, it would have been easier to reason about in the > general case if the decision had only been based on the LLC without > having to worry if any intermediate layer has a meaningful impact that > varies across CPU implementations. Sure. Will do this. -- Best Regards, Huang, Ying