From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEA75C4345F for ; Tue, 16 Apr 2024 00:21:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 12B176B0087; Mon, 15 Apr 2024 20:21:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D9D36B0088; Mon, 15 Apr 2024 20:21:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EE36C6B0089; Mon, 15 Apr 2024 20:21:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D24496B0087 for ; Mon, 15 Apr 2024 20:21:30 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6D0EF1C04CF for ; Tue, 16 Apr 2024 00:21:30 +0000 (UTC) X-FDA: 82013491140.07.D2BF85F Received: from mail-vk1-f174.google.com (mail-vk1-f174.google.com [209.85.221.174]) by imf04.hostedemail.com (Postfix) with ESMTP id AF7C240003 for ; Tue, 16 Apr 2024 00:21:28 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=jLINRIji; spf=pass (imf04.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.221.174 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713226888; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Nb+P7n9dUZe78brfegE+89GndH/vFNJwg++3mhisACs=; b=HunThxVPfWoHm2P+xYyKJo7ki7IlBQXcG1ZQShfmfO3NjgLtE+5VUAtqT2vJOwf1qkJHvm qCNedC+LRmUmgk3BrhYdWs5zNPoCX3q4i4AlWWji/fm11zWJGQ/wnCJ6FiHARs6Yxob1mS F8+PoVg9QV196mMmLuz8oZqCZEf8gUU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713226888; a=rsa-sha256; cv=none; b=TX6VQUTzY9r24HLcDy956WZTTjNv4qv7OuUnTLgcQTr5Y4HGf1Ofi+0ImZdRLM9fKY64jP f5f4MHJdy04gRpx/4bR82JXeGuHagIiw3RsZPXDWnO4Ca9KxGQY2Yh45bdXakjTe20NnTm FW83m+m83nPTQKDyv5wJUbBC5g20Hpw= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=jLINRIji; spf=pass (imf04.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.221.174 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-vk1-f174.google.com with SMTP id 71dfb90a1353d-4dcc7c1055bso511703e0c.0 for ; Mon, 15 Apr 2024 17:21:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713226886; x=1713831686; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Nb+P7n9dUZe78brfegE+89GndH/vFNJwg++3mhisACs=; b=jLINRIjiV9fjRsfdtnNSWUuMhpYF1PiMrN/aDfX97FD8zysNPLP68eJH5HfDg3F1JK lxw1VqfqO12FhHDAdo8oaE5pZ3efdrmh19n41VBrvU9XKfnnH81vOA1wnx02rhwF92fR 1S1gVRWElQpdMH850p/5NpCIKhW4oO1p0BV0PtnoNSZYK4vnXFktXABABujDcO6xc/ve gcsztWOoVu16/pw65HBvqYRHc7vRSbFrLbUfOfJ4DmQKxTO/n1O1PqahvkkRSDMNPVdz JPEEThGsbH9UOsnOAVPtFZlrlcBxObzOIqzLdOGKVs+ycwkAJ7tCfT8QbyiFrmjlBBze BJ+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713226886; x=1713831686; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Nb+P7n9dUZe78brfegE+89GndH/vFNJwg++3mhisACs=; b=q48F/SWY3Kl1cAdFQ39MC4gqef58+2OY7D39X8cgqIEtF+fYKH5lh9VikEn7MaD0mf QAj/dSdDd19Rwy1oh5eFf5glMDVxmhJPmxmQ+epCoy0e8SSP0ZXFzlmUW3Bo4G/mO21O AAPMHiU33wP+ClEQ776bt9PH96F2TfwP6/DEOcUwgaqxV1aqlOxZYwxbEY67zckfPKTn RZCPNMj1LQXBpIfR87GPwwY9sN5CQmhb5QmWBLo7EaDuXDtHRfInCiZph/MASs1TWXdE KVFqWAUFmhCqHJa/03RyxAZYkgBhgxiPkYTOWM120SaLW9jAbvOapFGlXdbGzVlycLOt MmIg== X-Forwarded-Encrypted: i=1; AJvYcCXi3UK56jDDgeokj+GSIBytsP4t51AUVoMwXi5Jpf39BrRoWgGDXBKwGRv5I/FSWnv463TgScgm89p6v1/onlvnLm0= X-Gm-Message-State: AOJu0YwvC4h+GzHpKzV26jb1VlF/lGQPs/zD44x2PLzvJeSC3etTA8Q9 KH9cg8AwcSQFgy3V5deofoMQIVG2IWpooUfcUqPAOCmClTmoNr7gQJWHa33gUX6+xYZrxGPCARP mWRctjja4vh7gkBpdv+zrFgXcdmc= X-Google-Smtp-Source: AGHT+IG2VdOm+kHoxLkclirscQOxK+7+43Y0fMF4w7W+ZuDxcpta6JhTIuLH6vWGbHiHyBU1GnqPwYVC3D0XA4QIAaA= X-Received: by 2002:a05:6122:3c83:b0:4c8:ee1:5a0b with SMTP id fy3-20020a0561223c8300b004c80ee15a0bmr9544657vkb.15.1713226886269; Mon, 15 Apr 2024 17:21:26 -0700 (PDT) MIME-Version: 1.0 References: <20240415081220.3246839-1-wangkefeng.wang@huawei.com> <54623c8c-a94f-4f88-bf53-5f92c634f78a@huawei.com> <3b931621-7cd1-4df8-9070-535ecaee970e@redhat.com> <90501d59-e3f2-4ac4-9e42-4eca3bb0a91b@huawei.com> In-Reply-To: <90501d59-e3f2-4ac4-9e42-4eca3bb0a91b@huawei.com> From: Barry Song <21cnbao@gmail.com> Date: Tue, 16 Apr 2024 12:21:15 +1200 Message-ID: Subject: Re: [PATCH rfc 0/3] mm: allow more high-order pages stored on PCP lists To: Kefeng Wang Cc: David Hildenbrand , Andrew Morton , Huang Ying , Mel Gorman , Ryan Roberts , Barry Song , Vlastimil Babka , Zi Yan , "Matthew Wilcox (Oracle)" , Jonathan Corbet , Yang Shi , Yu Zhao , linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: AF7C240003 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: t5x9pxrzn5m55ozzd8j4zjnzh9dyay6z X-HE-Tag: 1713226888-627978 X-HE-Meta: U2FsdGVkX1/fZZ/e0wtu123iOVk6TuQclfR+difPu8/ncLr1+6fOkJHtxOaRl1zYhmLZ+NXuwKtVKJ4kMaBIGbVbRGGHv+Ie62B7bcxRsqtycR0ELt7qTvxTJj4iNWsBAfWPbEsDzgG083kIRaeOy0NA2pAxE/Mw+gn4mm9XgQke3XmH9xKW4j7SJu4t0RY32/o+yDQvQFoTIz/vh7iaRK5i5EJb8x23MORghZ0gIwEKCDx+ztMjKJMvb+TFH8FlZ1qtDsWYzIB7qPvk/CjRQ9f6q9PIu+t/JaNnnwXX58KvXSHudh9DptYBAhCAr2ZExkVbuwBwgekPYnuE5JFuL4VRnPZefBFdxd8oxhZ+usr1BftKoEpp73zoDRLw7CNIxgV6IxKtqT+Icb2ds443PASvZdbXKXyr2ai/hi+RDug/1qJuvt3jkB5sJ1NrdQ2ebhNv/zh/xIspXCMj8BQ7PCTMrC2XBkOmPPxuYrSw4UYt7LplAjUii9m8OnRGjImCXHtAPSCQmhXmr0f5ylRLi8l67x9rCevqz15BE/nfIiznwfNJKUspmB+Gwxz0VyMsE1OSMv7OZPHGRb7D6EwVleE5ujmJyDhcvpSgSE2wC7YaXit6z5jW2/fROhyNWcpJsrRn8GcssjPTubCtcwvvaZ0F/PyAMlqvRy46LXRz/4/D6y1NwR4nlPdPAZ1iYaUkL86Bz8Wfl9zsb3kFF2xX1qW9S1AKOGbI3LBST+ObGLK706vJRtElpWJjlZenozWm+UN2SEp2Z0lVliRxwOO9h4T7OxgxIzkWdt9I1CKfNrLuJt12csUbiUe8aYjmJ46VR3w2x+Q1r6Iuk1a74jXpHiuLs+p5Dg8uNw6FOBXvAW/7jqmBAQo6ZU1ewNdCPrs/g04POWUd+sKTWssb29QgWbUoW9EkW8eUdTngO2fF6iS0x2t2FTOUNeIfBgHdo8YWv1Q3yvaUIcyWYJeuQ1P t6B6SZgi krr8OCVa3YbWwosSEKeP++7yxYfxs/E4G8XC0rvz3y9iZLFDXzvZUyFFzD1nmNEQJrOZiNwhyizFDy4b2BbirO2AQ1i01vs3nzRhdugUH12Fu+mG9LFVh8k1TsQlWY0pnbzx2SctKFaky8CwnXSJN/dZ+sp/7F2VDZnCbty+CB29RNK2RdZi5Bxs5TpRmstaBYQhBk5zL7ddSNnI4XkxxIb1reHIh67EgrX3qyqXPny51qQGC9E6MvYQDjqst5mOjNZSybr7JVWNs8j2BcfrjJFfatODNC6xgtbG6v4WCJPzZ8/DrmBE9Ub2vt4UHzT4/u6r6nrbKltCaIssKaG+v56PQe9hExgC9SxMA3zj2uUloTdQQoh/X/EQxgpY7WiBHGYyc X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Apr 16, 2024 at 12:18=E2=80=AFAM Kefeng Wang wrote: > > > > On 2024/4/15 18:52, David Hildenbrand wrote: > > On 15.04.24 10:59, Kefeng Wang wrote: > >> > >> > >> On 2024/4/15 16:18, Barry Song wrote: > >>> On Mon, Apr 15, 2024 at 8:12=E2=80=AFPM Kefeng Wang > >>> wrote: > >>>> > >>>> Both the file pages and anonymous pages support large folio, high-or= der > >>>> pages except PMD_ORDER will also be allocated frequently which could > >>>> increase the zone lock contention, allow high-order pages on pcp lis= ts > >>>> could reduce the big zone lock contention, but as commit 44042b44987= 2 > >>>> ("mm/page_alloc: allow high-order pages to be stored on the per-cpu > >>>> lists") > >>>> pointed, it may not win in all the scenes, add a new control sysfs t= o > >>>> enable or disable specified high-order pages stored on PCP lists, > >>>> the order > >>>> (PAGE_ALLOC_COSTLY_ORDER, PMD_ORDER) won't be stored on PCP list by > >>>> default. > >>> > >>> This is precisely something Baolin and I have discussed and intended > >>> to implement[1], > >>> but unfortunately, we haven't had the time to do so. > >> > >> Indeed, same thing. Recently, we are working on unixbench/lmbench > >> optimization, I tested Multi-size THP for anonymous memory by hard-cor= d > >> PAGE_ALLOC_COSTLY_ORDER from 3 to 4[1], it shows some improvement but > >> not for all cases and not very stable, so re-implemented it by accordi= ng > >> to the user requirement and enable it dynamically. > > > > I'm wondering, though, if this is really a suitable candidate for a > > sysctl toggle. Can anybody really come up with an educated guess for > > these values? > > Not sure this is suitable in sysctl, but mTHP anon is enabled in sysctl, > we could trace __alloc_pages() and do order statistic to decide to > choose the high-order to be enabled on PCP. > > > > > Especially reading "Benchmarks Score shows a little improvoment(0.28%)" > > and "it may not win in all the scenes", to me it mostly sounds like > > "minimal impact" -- so who cares? > > Even though lock conflicts are eliminated, there is very limited > performance improvement(even maybe fluctuation), it is not a good > testcase to show improvement, just show the zone-lock issue, we need to > find other better testcase, maybe some test on Andriod(heavy use 64K, no > PMD THP), or LKP maybe give some help? > > I will try to find other testcase to show the benefit. Hi Kefeng, I wonder if you will see some major improvements on mTHP 64KiB using the below microbench I wrote just now, for example perf and time to finish the program #define DATA_SIZE (2UL * 1024 * 1024) int main(int argc, char **argv) { /* make 32 concurrent alloc and free of mTHP */ fork(); fork(); fork(); fork(); fork(); for (int i =3D 0; i < 100000; i++) { void *addr =3D mmap(NULL, DATA_SIZE, PROT_READ | PROT_WRITE= , MAP_ANONYMOUS | MAP_PRIVATE, -1, 0); if (addr =3D=3D MAP_FAILED) { perror("fail to malloc"); return -1; } memset(addr, 0x11, DATA_SIZE); munmap(addr, DATA_SIZE); } return 0; } > > > > > How much is the cost vs. benefit of just having one sane system > > configuration? > > > > For arm64 with 4k, five more high-orders(4~8), five more pcplists, > and for high-orders, we assumes most of them are moveable, but maybe > not, so enable it by default maybe more fragmentization, see > 5d0a661d808f ("mm/page_alloc: use only one PCP list for THP-sized > allocations"). >