From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F0F5CD6E64 for ; Wed, 11 Oct 2023 12:49:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 11D6A8D0105; Wed, 11 Oct 2023 08:49:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A69C8D0002; Wed, 11 Oct 2023 08:49:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED7618D0105; Wed, 11 Oct 2023 08:49:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id DADB18D0002 for ; Wed, 11 Oct 2023 08:49:05 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id ABB5480143 for ; Wed, 11 Oct 2023 12:49:05 +0000 (UTC) X-FDA: 81333160650.18.24454C5 Received: from outbound-smtp18.blacknight.com (outbound-smtp18.blacknight.com [46.22.139.245]) by imf02.hostedemail.com (Postfix) with ESMTP id E5B0380018 for ; Wed, 11 Oct 2023 12:49:03 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.139.245 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697028544; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GySnxjPXLl2D/+TYnAt/qOPXeURf1oxRhxBX9ZQKXvw=; b=3PnaCzxoWC97x1kuTDpwHBwRle/DKXrSPeforD7w2ZivoMptvrA6DOJDWUfIHSPwW0hLSh GLgzYMX5PafZ/WishWgmxPkiV1uYp7XDOzYHDz3RWXBC+dTVCKFk7I5UIScD2+sP2YMx1p muRefbaZ6c0zyMLM5IHaQycUZaS6nwM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697028544; a=rsa-sha256; cv=none; b=4f5Hr9hM8Ivrpa48aOeky/Cr06FGUsrayFSiZDV9UyaA7z/zjeHH0dQnRrKIuM2xxjXFTA Z8yfGR4ZkWSc1V6lWKZY0z4KiRGbD4mgeiZJ57QkwX1mgTfCcwufVfYYu6TCtYMLlV0sNj l/SxR1Hf72Q9CFYLhcxXBwEcjF1FPzs= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.139.245 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none Received: from mail.blacknight.com (pemlinmail01.blacknight.ie [81.17.254.10]) by outbound-smtp18.blacknight.com (Postfix) with ESMTPS id 69E821C48B2 for ; Wed, 11 Oct 2023 13:49:02 +0100 (IST) Received: (qmail 12957 invoked from network); 11 Oct 2023 12:49:02 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[84.203.197.19]) by 81.17.254.9 with ESMTPSA (AES256-SHA encrypted, authenticated); 11 Oct 2023 12:49:02 -0000 Date: Wed, 11 Oct 2023 13:49:00 +0100 From: Mel Gorman To: Huang Ying Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Arjan Van De Ven , Andrew Morton , Vlastimil Babka , David Hildenbrand , Johannes Weiner , Dave Hansen , Michal Hocko , Pavel Tatashin , Matthew Wilcox , Christoph Lameter Subject: Re: [PATCH 03/10] mm, pcp: reduce lock contention for draining high-order pages Message-ID: <20231011124900.sp22hoxoitrslbia@techsingularity.net> References: <20230920061856.257597-1-ying.huang@intel.com> <20230920061856.257597-4-ying.huang@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: <20230920061856.257597-4-ying.huang@intel.com> X-Stat-Signature: qjkuprxq1cd7dhcqy9kf59pxu9ijb1ep X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: E5B0380018 X-Rspam-User: X-HE-Tag: 1697028543-807298 X-HE-Meta: U2FsdGVkX18adJWiEeqjFmoWH4hZkzey7TasBRmpnuF1pBcCxRWCuL1nPxQmF1r/QHjuypLloWobHLZfoSsM9Mdyf3OgBOhiX14dyZKtlrjQnR9aFtXTC85Pb9+t77cgeyIK4cJMm67AFahVCEJt95GfU/EfP57NnjU1/QGXTgzD326JJiA5Bvd6iTBdMzxl9EOfnGO7uBv9a/eCu4ndutDVz7zLwmODibM+LhjWzo8JS9TB5xLDVt/W7nQ1iV01Ap3Wh/QwU5D3eMmNN+a2xfSwd/M33V6xsbnzhNn5hOXyD6EpRcDaJMtcfJ+QFm6gPkSZvJbZduY5AAnSjW3aKoYwOENlHfDBXa5+oDLam6NnvfsJIjybM+kazFvFmQif00PzwXJ5Bgo/ZvMVz+GAUrbi8alEKv6OaEdmnO7NhYcrgz4+FSyh/7Fbu1X8o0NKTeDF5EnJOao8CrEMceOxd3tY0TuhxhEdqXkcyEl6x9yF2XZ3JrujMzTcQDLgmrtFqG5t4AG9tSCkG/m4pzJPemHc/ZZWixwiWo8TL+EkzMztgXiLQDJpCvmCNG/ahoiZDTeMZKSPPbbJ27HiYTVgUztiYRGRu45qPyouMDvuhZA/WCypHBKx2uYJuncR2TmoLhKe5I6ketKLa0oDKjq+1oZZqD0csW9Ptv3sFBQG7N7k3X106QMQizzS8nkPCRBKREcjRwkZPs7zbIMjBb46BKO5EP2/rQ9DRDF6SbmEYxMEq+sfO/xbEA+eUl67hSz4LUMHB+b7EUeAVrLrz4TzLd5z33r96zV0WnJRm1qms6bUDPFkMfPJhMP2VnokjP8PR9tl2gif5290j80yC701DkUu9gwC5KePg2ZG7yzcMW2K5MOyUEdFAs4s4CuEtqMPdez75Lmqm5wXLt4sfL+PYRej2FFtuL/1/Tuez2yZl52TPBOJn7NfldvdZZXQcjHVPcSw/S6bXSrq29tgrfS IK5Acq5r nNkW9/bUIhfUVJxVwqKEdvbOSkr6G5IWY6dJtZA2SK4C/HH4m70zlf7gC1A89dSIeaCElHqNQpmLrj4b+UFTevgJ1n5/XrsBPuUh5hgpP46l79jEp5Cr/cghx2AIc8xIK8SOIp/Mqm//xYeWWsACCvtg3BWRvQBiopors5DoxS7jol02dYftnCc/qY9qQTQarBpeD8KKArYkCZPRUrIyp1Ghlvbqv3QN25k73zlf6Ferm6OP44Ir9/CG2jWBepPxgbnEU X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Sep 20, 2023 at 02:18:49PM +0800, Huang Ying wrote: > In commit f26b3fa04611 ("mm/page_alloc: limit number of high-order > pages on PCP during bulk free"), the PCP (Per-CPU Pageset) will be > drained when PCP is mostly used for high-order pages freeing to > improve the cache-hot pages reusing between page allocating and > freeing CPUs. > > On system with small per-CPU data cache, pages shouldn't be cached > before draining to guarantee cache-hot. But on a system with large > per-CPU data cache, more pages can be cached before draining to reduce > zone lock contention. > > So, in this patch, instead of draining without any caching, "batch" > pages will be cached in PCP before draining if the per-CPU data cache > size is more than "4 * batch". > > On a 2-socket Intel server with 128 logical CPU, with the patch, the > network bandwidth of the UNIX (AF_UNIX) test case of lmbench test > suite with 16-pair processes increase 72.2%. The cycles% of the > spinlock contention (mostly for zone lock) decreases from 45.8% to > 21.2%. The number of PCP draining for high order pages > freeing (free_high) decreases 89.8%. The cache miss rate keeps 0.3%. > > Signed-off-by: "Huang, Ying" Acked-by: Mel Gorman However, the flag should also have been documented to make it clear that it preserves some pages on the PCP if the cache is large enough. Similar to the previous patch, it would have been easier to reason about in the general case if the decision had only been based on the LLC without having to worry if any intermediate layer has a meaningful impact that varies across CPU implementations. -- Mel Gorman SUSE Labs