From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBB96C369A2 for ; Thu, 10 Apr 2025 07:03:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2EA326B0102; Thu, 10 Apr 2025 03:03:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 29BC16B0103; Thu, 10 Apr 2025 03:03:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0EBF46B02DD; Thu, 10 Apr 2025 03:03:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DEFD06B0102 for ; Thu, 10 Apr 2025 03:03:07 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7596012035E for ; Thu, 10 Apr 2025 07:03:09 +0000 (UTC) X-FDA: 83317242498.01.27000AE Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2083.outbound.protection.outlook.com [40.107.22.83]) by imf11.hostedemail.com (Postfix) with ESMTP id 3FEE540002 for ; Thu, 10 Apr 2025 07:03:06 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=nxp.com header.s=selector1 header.b=HILKqu47; spf=pass (imf11.hostedemail.com: domain of carlos.song@nxp.com designates 40.107.22.83 as permitted sender) smtp.mailfrom=carlos.song@nxp.com; dmarc=pass (policy=none) header.from=nxp.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744268586; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=d7inkriHw8VD10qn0nfRiw6rOzEYpsSe2q/6ndYk0sA=; b=gqVV32ylokMCI6D60g3vJs9JAwiybNjpKkX7ylMxEbW7WLzIw08hwDV6fdThrRWez5DWLD ML3ABNtXGI6baa1jc/IYRPyQMkw1dFXQrwc8qpDGRcz1kyet15I9yiGz/cfxvWjNIS5BEl vjyaAHP5pYVGK/GnUTZ53mVcuOoxANg= ARC-Authentication-Results: i=2; imf11.hostedemail.com; dkim=pass header.d=nxp.com header.s=selector1 header.b=HILKqu47; spf=pass (imf11.hostedemail.com: domain of carlos.song@nxp.com designates 40.107.22.83 as permitted sender) smtp.mailfrom=carlos.song@nxp.com; dmarc=pass (policy=none) header.from=nxp.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1744268586; a=rsa-sha256; cv=pass; b=NBdD88dJAf7Ug3ky41TyXLoGuSohSydc4Q4q4r+Llfs/RYC5juZOgZuxv4wgZAf3l14Q04 IwDcHG0gGE+xmRcYpSflEcQxHXcdzBurkMWWt59vZO2J4qwnTJncXz4AwpXFqytYUN9hka os9igrwSkhKZ5yoDRVJecRI3rusupaw= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=RmjZS70iebfJQwa2Cj3Y/eZpdpwD/a9jhyKahVe2QjyTwBHo8tp4f34DfM0pjKcTGXnceDtASaRgC7xX/0YQpDasnLMwgyP9+b59mmMxZ02evuJmCJMV+zpZm25qO0G1FlMJ2M76s56jpKrWS65bRHGQWLbZeS95ApreTY1MY/2p6PtcTbvcnU3BSQmBxn9UTc0WZK53bZq8whOhiXgmhdy50SOEIib9RxjtfqKTTLqH4+OCn0KAEpYp/K01jKldXzZSY2w+t6j+wFJ+fDicZi+RmmZaF+bSEqYHB73EOsJ2Bp+Qu7Acplg7VHxpjSoohKSmKEoEc9/frIwZirwc7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=d7inkriHw8VD10qn0nfRiw6rOzEYpsSe2q/6ndYk0sA=; b=XIMzKuRZQUjceW0jBgrfAGoAG3EsM1JlBseh72dtfEBtAxcGL/FELYTj7H7GwnwbmCp952x+DTYgfkihFitvYiV/U873vrZbpUpE42pE2frd7nsYSS+y4z40zUqsIZwql/8e0vnaHJu1BYSwXK3pJZY3dp6dC52kIgp74Dq0c7SzAkAVXNEpRIw/D61LjPgni6wPfDdgifk8cQPwH9aD8dwUZDv5vZO7NLR7pDz2GOMUgWUgzdDsQRCedhdYfteDnnDxJxT2kDGaP3inGm9Wk+KIoesXl0ru73gcXSysa+hNn/8FIhTtGNJqJSk/6QW7I2sGF6lzNCyTNim/rj2OiQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=d7inkriHw8VD10qn0nfRiw6rOzEYpsSe2q/6ndYk0sA=; b=HILKqu4715fECWxo68oIJ0gjXEHX0w5H64TeZe/MZ3EACgrse+gFJv1hrUV3ISuD/8eX2/8bdFFHTdZVr52hXKnL1W2NMEgHAiSgT6/tdtGgc9g+/yhtc8fBATiWAdkx7YgvrVvdJPCqq5dnsiqFtoLqmmm2ZqP6APbbeeq/35AWmIZtEtDxk+EBx+GmDC23mnaMj1jOPk6q8qDFFdkj6kZGObaiGjMQrPJmSpdCAg5KLoVCcDNMAKo4pzuPneJwXu7pcj0magWI2CLNYOSywSc9LCvSPypv2kgdmQvr6YHe/Qv5a6rQtXj8WM45pXNr+93bFsjv6dgowwmVFUNKrA== Received: from VI2PR04MB11147.eurprd04.prod.outlook.com (2603:10a6:800:293::14) by PAXPR04MB8958.eurprd04.prod.outlook.com (2603:10a6:102:20d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.23; Thu, 10 Apr 2025 07:03:03 +0000 Received: from VI2PR04MB11147.eurprd04.prod.outlook.com ([fe80::75ad:fac7:cfe7:b687]) by VI2PR04MB11147.eurprd04.prod.outlook.com ([fe80::75ad:fac7:cfe7:b687%7]) with mapi id 15.20.8606.029; Thu, 10 Apr 2025 07:03:02 +0000 From: Carlos Song To: Johannes Weiner , Andrew Morton CC: Vlastimil Babka , Brendan Jackman , Mel Gorman , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , kernel test robot , "stable@vger.kernel.org" Subject: RE: [EXT] [PATCH 1/2] mm: page_alloc: speed up fallbacks in rmqueue_bulk() Thread-Topic: [EXT] [PATCH 1/2] mm: page_alloc: speed up fallbacks in rmqueue_bulk() Thread-Index: AQHbp+cqJgEnSiMjBU6DO4eJQEZtrrOcfLkw Date: Thu, 10 Apr 2025 07:03:02 +0000 Message-ID: References: <20250407180154.63348-1-hannes@cmpxchg.org> In-Reply-To: <20250407180154.63348-1-hannes@cmpxchg.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI2PR04MB11147:EE_|PAXPR04MB8958:EE_ x-ms-office365-filtering-correlation-id: e386f8e6-275f-4aa0-6b33-08dd77fdbc4f x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230040|366016|376014|1800799024|38070700018; x-microsoft-antispam-message-info: =?iso-8859-1?Q?ImxpqBsW8+PyNtVdsOXfexMw7yDQbtsQUSo5xvxDgnN8CkkYvlej7EFKPX?= =?iso-8859-1?Q?S3GLkU9ZeC6W89QcNTBuIHQxfxoSwQTmkx4XhKIEr5CIG60R+4Hilq21ew?= =?iso-8859-1?Q?9xQpUSa/uFmzO8i14k+ue12Q3UWM1i+xf05gVNA0NojpS2Op45kllW21ib?= =?iso-8859-1?Q?sJp4CRfkCbIeOTPP8jtiWpsbKGQg1b1k0Sgs1Z+LzU7HyHXcaNPxNxDgD8?= =?iso-8859-1?Q?EwNXFhsn1fQUUNz1Lbok1vBxlbdY88BQeQ4r3i4I9gKH5Zc9PgOAmvIelc?= =?iso-8859-1?Q?KY7e9ZXv9/FGeTdazEfM+sXeSb1AJYTEyCm3BUiIF2Ryb1n/Ucttbq3p1b?= =?iso-8859-1?Q?meqziLKDp9eOLi3N4D7yqa2hayzaSC532TKvSecZOb2d1LO50EnblCimb/?= =?iso-8859-1?Q?bC8gx2Ayk61/hbyRuJUKsLC4/Xx9tDAjVlmNjMX6Kg+H93pZEKQDGYDGPb?= =?iso-8859-1?Q?J3xZt+ynZCndtzRliYQFuOQ4JJsqrINE65Yf4uap6EzOcdzBTMAzrI/PAi?= =?iso-8859-1?Q?WEE8XElvtvtP+PN+GIN8K2zDHkOcgrFlwYhfnfmTK0vnXELdeEoGfgVd33?= =?iso-8859-1?Q?5/IHMXJFgtcAWDysCjPwxDaL/s/VwGP7HjUEVlOIY1FwwMEMizoMBbbCeu?= =?iso-8859-1?Q?GdOejc+KXLUmWT2Kyfdm5uRSK5bZXYBsOjS+7ZyPfwo+droZ3IL5Ez597c?= =?iso-8859-1?Q?fG7cn4veFD0jUc69AfjV880SLIkvba2zT44Kq2mYnvsJngghmIGYv+9h1j?= =?iso-8859-1?Q?p1pMo/cLbtwqC/5WNywg20Vi3gPsJYFfBO+vlOxJnsKuGxxYgKk9w85crk?= =?iso-8859-1?Q?nRPN0QwXcMpnZYDvhhzt94IKEIquSU7aV417YCFB2MxgAC5XKtQNkYRwoZ?= =?iso-8859-1?Q?uYzMitSeK/7spPUfVuF+/nFRIcSy5KAyDxJ2MypqYSvCmroCKocy5l6SBF?= =?iso-8859-1?Q?5Xigo9wge0hZCHUADVASorsGguU83f4++MS5vv/Jpx2cdXnm5YVAt5tjGq?= =?iso-8859-1?Q?gjKBkZW8eHfXjDfDdJhGBG9k3EimJd4mTyTq94yDzM/NYe1hPgDdL2eK7T?= =?iso-8859-1?Q?iON3PD+NiuVPmopd9Vbyvbksk8aOHX1CkALOuwxNaUldrqYCMP7XYw0MT8?= =?iso-8859-1?Q?Qlo7GFcljI2PvQ4cTZEZiLFRltziLf9BpmdN832f+IluTJtaQDeW+PFk+O?= =?iso-8859-1?Q?l5zBFckou4miYPFc0WK5oZOF6hnubhHnnhvN1MpilCRRjadhv3eQtJhRF/?= =?iso-8859-1?Q?DuKnQMbBT8wCBOXZa1TImViTDOwVwFuk3zJO7YmObp8Lv0SKVi1A19whnP?= =?iso-8859-1?Q?COurnpgvoxcBEcqRVN8NFhr+LGMbCKnkLbH9R4ev6HTL60tNNYHV2rRSLB?= =?iso-8859-1?Q?jtu8ZcBVc2qik+Ud7wzZVsG6O2Hyb2aimJGsGNzp/wkR0WsKA6YYQgCkqf?= =?iso-8859-1?Q?6Cu+F50Ypl3pgxDaO33CVgPdeqfzG1lvpjOgBKhDfUuFWBPD0OGCecgzVG?= =?iso-8859-1?Q?8=3D?= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI2PR04MB11147.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024)(38070700018);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?VtfZweFAUwMw0Z87lIMDVGVP+uMLKb9W0TCm86exA00nq+AYwZoz5QhZWg?= =?iso-8859-1?Q?TDYz5+By08a0DeAqCqtt4qU49IE5V/yUEO5g8EVaZhE4zXddXWxqu3O8Gp?= =?iso-8859-1?Q?K79ry2iWJvo6llo8aYtMvGEmhm44qL1+MWr8hwKw9JeIzy0w31T05lm7Ae?= =?iso-8859-1?Q?GifvenhpqI+hAhzthXyfJIaVKqk8KtbewvPVG4UNJAZysVPmm8c98KCvsl?= =?iso-8859-1?Q?w4e2k3YAcVLRDVSTez9QMAiBv6cRnKCrjNSagTZcTGFlADE1d30Qlhm3s2?= =?iso-8859-1?Q?x1NDMMjAJ8/70ecWeQcT4U/Jh585qfCBT/gH0EjUMHESmuBjaHODZfLNRA?= =?iso-8859-1?Q?F/3Easn+TLAZJ0puZKbvpt3MaNAtrZKcBgxmFj9lvmR/8TuPOjCTLDhjeW?= =?iso-8859-1?Q?jtgQQhWyHQsxdJTIvsXA3xqw+oqyx419RGYDBjaCPg6JMYDi82JlhAbrnE?= =?iso-8859-1?Q?zKyTtB7xzqKyWnj+jtObp/FC+zKhv78zZIpuNZUTaMJ3KFkhvd4vi+5+HR?= =?iso-8859-1?Q?Ve44af2WOO8NF/I2DpkKW0/vcO5A0XkdRKfaypxmG6QlXCCxB1Qw0u29ia?= =?iso-8859-1?Q?hEIDuRgQLBbL8ja2r7xc/hk8qQXRg5BYOUAnNnmB59AfQAMBSn0a5t5ONe?= =?iso-8859-1?Q?hltzCUaXvuXya2vnbUvIy3s8m+C56oZn3sBEgTcT00/3eT+EA2JVvLWelJ?= =?iso-8859-1?Q?ZBvPelEY31r1WRgJs6U7qbgyqTqECYpTsFeaaKT5DgHaaHKA/Ha0Gz24Ot?= =?iso-8859-1?Q?hcjYtR+UjUyyy8Qv9rqMgmkO4GnZ6NxTEPPa5EKloPvBQcv7YjTsOhwQwa?= =?iso-8859-1?Q?p3FmGuXdkWSddL0HNvoTjjRgpeSvfAYB8fIDFI1q73xOlwQbOP9IbWRSbz?= =?iso-8859-1?Q?EzODHUoiKV7xta09m7lLHhfdDX+apBTYVttD1m7HNuLVeB7z20BAU7H+r+?= =?iso-8859-1?Q?SGNxU/3fwAXJgTn4Ox4Z/9bDHxOu1Bk0YJ8x3zdzAh/RsWpz+MA1BnS9CC?= =?iso-8859-1?Q?kCAjNMP5VUPqbcWqUhvHsziraZGuJCwJMavGkTK9vomIMJaiG3J856uUFX?= =?iso-8859-1?Q?qgmLH+UcIEe/4iXwGG+EXmP2dS2LeodNPwXbCdyUAEDQzEzXpmqeKdH/kN?= =?iso-8859-1?Q?iR9PkljbNhwC6hU25cvyThhir/jdX15M8jwZEcBA9Weazp3CiLHCdYlAWU?= =?iso-8859-1?Q?AmxRblHTsT2/ncX85VmFst3m7THV7/5DrDDPL5Ko1qtx/ScsJwTYjyO8qK?= =?iso-8859-1?Q?AMz5IFPK86LFSH/jsriJM0ULceMGBn2K0UOxEfBPTHGghvUPOkckqXk8wU?= =?iso-8859-1?Q?E3eUNvKxkX7fVJsonFFt8NE4hDIbWc1gDR3zRpqRpgrH9wCHBb/0hGk4Q4?= =?iso-8859-1?Q?q0cVfkBDKCeuyDianOFKoJqG20KfUA9+r2lyT1ElqixMcYXdGuUOQQShCx?= =?iso-8859-1?Q?uqC1tdi/taaWmMUA24yRGa2idmBJMq8YX79dv2OayHbWb7Y0nohCKsAItK?= =?iso-8859-1?Q?11bkL1gruaSHMiO0rtR/FFV5VRKGW9rPtpeF1pTsaYLK0ws+ICnR9a0vtE?= =?iso-8859-1?Q?ypD3FhG1p2eixrh60wy7IHQeI/xxVJ9DSUCvDUmL9xmTX4LhQtAhCtISSU?= =?iso-8859-1?Q?/RNwLOTtEgOrE=3D?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI2PR04MB11147.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: e386f8e6-275f-4aa0-6b33-08dd77fdbc4f X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Apr 2025 07:03:02.5770 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: j8OmaqMZilBePVwpecbnQkpxnox7DlhMGhMpD41znSlGd182yOMWQ6zNudLduNF6VCrWMFOwrWjUSKHL+wm+bQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR04MB8958 X-Rspamd-Queue-Id: 3FEE540002 X-Rspamd-Server: rspam05 X-Rspam-User: X-Stat-Signature: rm37ytixpa8difpzxhz8sagarg4r35p5 X-HE-Tag: 1744268586-346952 X-HE-Meta: U2FsdGVkX19jLd5EUZqoiVjMupdSlBGAS91wLdcb4QuE+O2w8uSTorG20hU6s04CBaLtpTBZ6JaDGJaT8C3qYMEL6IYTznKyiJMTQLaALPw64HRZe0Tq/vjOWls9JAIXVIu77dZ6Zl5trdJRsojKF6MuoeOD0IJbeQCKkjPaCom+FRiF8pA7aHefH0Mr1bFcE2Wg561Fmughcu769e+m3oTswVQ+v5sddmkTffEd5KplzIvpfMv9NmdyCrMP20jRFesETd+OkV8FNTYAhtvgW9t3kvwZGtzKB4d84L2mZM61yrkKOJ6hN7qS7eWTxL5ZAnUdvqiq0C9UlNpIV/Wuu5F7kDNsykygZN1G50PE7KFJddDY6caNBO58z47OCwDaa0IGkvUuPJSz2GOXEOqU0pNY5yTA9HWzQkTY5qD8bevtWDY65VbdIL6vpWos8wLRzgdpaN3SiVTsI5u6Uu6+eANEkGArz8xZ0NT0rVLIOtN8lXB6thCLV5P2DUzQDxQbT6De1hUWyqjzK55tMIMzQldD5yEQ5WU9oSj4x6zwWq/NGb9Hvaxl9IOGJQNh8ybBQZS2vvhdmxbfrlUwv8EHIxnztqFq2zsNLJP8k/m59oXOEvLdUcJOy7XoutdX/gxp1eMD/Bwn+0T5Z44MgvHQMfsazvLAFCZOZxjgNQdIBlNEJ4+Tzx4fkx9OCueac+f0ciAzb5UGPnRdvPb0a4zqPWYxPCTyWzQyacVseSaWxc7BohMwW+809luxsEBSTB/Jii6Bl+WaZmfE5EVeCm/wqQIZQKrPsHgbSs65hXOwBhV5uQaKyUZiA2LxYfWum8uYI+CLV+WDCHkLUERnTtnwxvS8gcJCZYh3ur1KI/rkFIxAryc9GfymTmTqRQSKWIVZmltaIjNPySDi4+m/8Xm5uPDbd02CkKp4NUHb/Dtq/+OvGTMXYwAdw1zqwEd4WnCmKOlr61KlppSD7Os6s9I iL1jjTpf HjgVGWonF6f6F3SKUAYamhz9EDYY12ZFZ0PrAJi8jXjhJu4ytE4fXvJJHeaLWiDkibxv2WWmNTGcb5ycoYmIQwVJyTm1xEtzwkUwlceGGM2k1JGl8zakd5xRN8uRsH1Wc4qnmgFoyHSoissD7nP11dEfri9hKWRIqwZb5rhV6CTscP9Q21Dvv2VQUOP9AvZzkV04vOaHkxhVOY/i1IaRZBJbuhRYlWvx5WSrWj5U1ijIt0FaHckBedM8ojcZ37B6FQ8wN0xj9CPKY+ISBBXvQ4vDUFo0PEESmPFGgaqGugW98zcxUp/ADgyllqm4HGcjP6ufNTdJukwJG6PZMh0KXtAzWqiEYgna6/CrIy8cOI2d+z6yyDxXRFH2m+9oyOKHaD1Dkhj7uDj1WYO/0UIXUf0eh4UkA0SeEbh8hhSyfETxr5Dh40iC8QDU2mnT1GkTC5g5lb/pY4Wx/INv+0XObC0AU2joYiJdDVgH1eLbJRlOUmWzsSaWalSzwerDqdI9O3sCUMjZ9Y7YxSGke5Br+GRzT1i24nS2UoVQERkumx246EnX+kcSNwPMN1YHO8DfYs/BvW1CBJhOigRnyo/nWrcN5GLKS+qWbGeOEHTFJ21SkSMolDy444uTeZ8zZRHYKevT8nD2PmM6kQHKKUOw6h2fGZTMKtznvCX1san73oeTxy2SxD8YeJTjcqk4qkE8Wy/k7pIAXLxE6vLQI+5/IEcEFZt4z7TjrDEEAyRYSZ3rN+UIskKcRRk6xGxE3VUUKk77+uXs8M3qGXYWKt3TEQVu3rHH7Z1Rxel5ca7Ig9+XnFgcWuwl8xDfYEkuktL0rhqv36HaqE73MU0t/Hwpdvv6n+Dp+FEeLuGkhmmO6+UO5hsfSupH4381GSUmTo7tjESIZMt2X/3QiV3UpFbo46r8j9DEH5k4doP8l//PfW4ZsZQnzXOrzZ/bIpVzJRd2CvJOx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi, That is a nice fix! I test it at imx7d. Tested-by: Carlos Song > -----Original Message----- > From: Johannes Weiner > Sent: Tuesday, April 8, 2025 2:02 AM > To: Andrew Morton > Cc: Vlastimil Babka ; Brendan Jackman > ; Mel Gorman ; > Carlos Song ; linux-mm@kvack.org; > linux-kernel@vger.kernel.org; kernel test robot ; > stable@vger.kernel.org > Subject: [EXT] [PATCH 1/2] mm: page_alloc: speed up fallbacks in > rmqueue_bulk() > > Caution: This is an external email. Please take care when clicking links = or > opening attachments. When in doubt, report the message using the 'Report = this > email' button > > > The test robot identified c2f6ea38fc1b ("mm: page_alloc: don't steal sing= le > pages from biggest buddy") as the root cause of a 56.4% regression in > vm-scalability::lru-file-mmap-read. > > Carlos reports an earlier patch, c0cd6f557b90 ("mm: page_alloc: fix freel= ist > movement during block conversion"), as the root cause for a regression in > worst-case zone->lock+irqoff hold times. > > Both of these patches modify the page allocator's fallback path to be les= s greedy > in an effort to stave off fragmentation. The flip side of this is that fa= llbacks are > also less productive each time around, which means the fallback search ca= n run > much more frequently. > > Carlos' traces point to rmqueue_bulk() specifically, which tries to refil= l the > percpu cache by allocating a large batch of pages in a loop. It highlight= s how > once the native freelists are exhausted, the fallback code first scans or= ders > top-down for whole blocks to claim, then falls back to a bottom-up search= for the > smallest buddy to steal. > For the next batch page, it goes through the same thing again. > > This can be made more efficient. Since rmqueue_bulk() holds the > zone->lock over the entire batch, the freelists are not subject to > outside changes; when the search for a block to claim has already failed,= there is > no point in trying again for the next page. > > Modify __rmqueue() to remember the last successful fallback mode, and res= tart > directly from there on the next rmqueue_bulk() iteration. > > Oliver confirms that this improves beyond the regression that the test ro= bot > reported against c2f6ea38fc1b: > > commit: > f3b92176f4 ("tools/selftests: add guard region test for /proc/$pid/page= map") > c2f6ea38fc ("mm: page_alloc: don't steal single pages from biggest budd= y") > acc4d5ff0b ("Merge tag 'net-6.15-rc0' of > git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net") > 2c847f27c3 ("mm: page_alloc: speed up fallbacks in rmqueue_bulk()") <= --- > your patch > > f3b92176f4f7100f c2f6ea38fc1b640aa7a2e155cc1 > acc4d5ff0b61eb1715c498b6536 2c847f27c37da65a93d23c237c5 > ---------------- --------------------------- --------------------------- = --------------------------- > %stddev %change %stddev %change > %stddev %change %stddev > \ | \ | > \ | \ > 25525364 =B1 3% -56.4% 11135467 -57.8% > 10779336 +31.6% 33581409 > vm-scalability.throughput > > Carlos confirms that worst-case times are almost fully recovered compared= to > before the earlier culprit patch: > > 2dd482ba627d (before freelist hygiene): 1ms > c0cd6f557b90 (after freelist hygiene): 90ms > next-20250319 (steal smallest buddy): 280ms > this patch : 8ms > > Reported-by: kernel test robot > Reported-by: Carlos Song > Tested-by: kernel test robot > Fixes: c0cd6f557b90 ("mm: page_alloc: fix freelist movement during block > conversion") > Fixes: c2f6ea38fc1b ("mm: page_alloc: don't steal single pages from bigge= st > buddy") > Closes: > https://lore.kern/ > el.org%2Foe-lkp%2F202503271547.fc08b188-lkp%40intel.com&data=3D05%7C02 > %7Ccarlos.song%40nxp.com%7C3325bfa02b7942cca36508dd75fe4a3d%7C686 > ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C638796457225141027%7CUn > known%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCI > sIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdat > a=3Dgl5xQ8OLJNIccLDsYgVejUC%2B9ZQrxmt%2FxkfQXsmDuNY%3D&reserved=3D0 > Cc: stable@vger.kernel.org # 6.10+ > Signed-off-by: Johannes Weiner > --- > mm/page_alloc.c | 100 +++++++++++++++++++++++++++++++++++------------- > 1 file changed, 74 insertions(+), 26 deletions(-) > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c index > f51aa6051a99..03b0d45ed45a 100644 > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -2194,11 +2194,11 @@ try_to_claim_block(struct zone *zone, struct page > *page, > * The use of signed ints for order and current_order is a deliberate > * deviation from the rest of this file, to make the for loop > * condition simpler. > - * > - * Return the stolen page, or NULL if none can be found. > */ > + > +/* Try to claim a whole foreign block, take a page, expand the > +remainder */ > static __always_inline struct page * > -__rmqueue_fallback(struct zone *zone, int order, int start_migratetype, > +__rmqueue_claim(struct zone *zone, int order, int start_migratetype, > unsigned int > alloc_flags) { > struct free_area *area; > @@ -2236,14 +2236,26 @@ __rmqueue_fallback(struct zone *zone, int order, > int start_migratetype, > page =3D try_to_claim_block(zone, page, current_order, or= der, > start_migratetype, > fallback_mt, > alloc_flags); > - if (page) > - goto got_one; > + if (page) { > + trace_mm_page_alloc_extfrag(page, order, > current_order, > + > start_migratetype, fallback_mt); > + return page; > + } > } > > - if (alloc_flags & ALLOC_NOFRAGMENT) > - return NULL; > + return NULL; > +} > + > +/* Try to steal a single page from a foreign block */ static > +__always_inline struct page * __rmqueue_steal(struct zone *zone, int > +order, int start_migratetype) { > + struct free_area *area; > + int current_order; > + struct page *page; > + int fallback_mt; > + bool claim_block; > > - /* No luck claiming pageblock. Find the smallest fallback page */ > for (current_order =3D order; current_order < NR_PAGE_ORDERS; > current_order++) { > area =3D &(zone->free_area[current_order]); > fallback_mt =3D find_suitable_fallback(area, current_orde= r, > @@ -2253,25 +2265,28 @@ __rmqueue_fallback(struct zone *zone, int order, > int start_migratetype, > > page =3D get_page_from_free_area(area, fallback_mt); > page_del_and_expand(zone, page, order, current_order, > fallback_mt); > - goto got_one; > + trace_mm_page_alloc_extfrag(page, order, current_order, > + start_migratetype, > fallback_mt); > + return page; > } > > return NULL; > - > -got_one: > - trace_mm_page_alloc_extfrag(page, order, current_order, > - start_migratetype, fallback_mt); > - > - return page; > } > > +enum rmqueue_mode { > + RMQUEUE_NORMAL, > + RMQUEUE_CMA, > + RMQUEUE_CLAIM, > + RMQUEUE_STEAL, > +}; > + > /* > * Do the hard work of removing an element from the buddy allocator. > * Call me with the zone->lock already held. > */ > static __always_inline struct page * > __rmqueue(struct zone *zone, unsigned int order, int migratetype, > - unsigned int > alloc_flags) > + unsigned int alloc_flags, enum rmqueue_mode *mode) > { > struct page *page; > > @@ -2290,16 +2305,47 @@ __rmqueue(struct zone *zone, unsigned int order, > int migratetype, > } > } > > - page =3D __rmqueue_smallest(zone, order, migratetype); > - if (unlikely(!page)) { > - if (alloc_flags & ALLOC_CMA) > + /* > + * Try the different freelists, native then foreign. > + * > + * The fallback logic is expensive and rmqueue_bulk() calls in > + * a loop with the zone->lock held, meaning the freelists are > + * not subject to any outside changes. Remember in *mode where > + * we found pay dirt, to save us the search on the next call. > + */ > + switch (*mode) { > + case RMQUEUE_NORMAL: > + page =3D __rmqueue_smallest(zone, order, migratetype); > + if (page) > + return page; > + fallthrough; > + case RMQUEUE_CMA: > + if (alloc_flags & ALLOC_CMA) { > page =3D __rmqueue_cma_fallback(zone, order); > - > - if (!page) > - page =3D __rmqueue_fallback(zone, order, > migratetype, > - alloc_flags); > + if (page) { > + *mode =3D RMQUEUE_CMA; > + return page; > + } > + } > + fallthrough; > + case RMQUEUE_CLAIM: > + page =3D __rmqueue_claim(zone, order, migratetype, > alloc_flags); > + if (page) { > + /* Replenished native freelist, back to normal > mode */ > + *mode =3D RMQUEUE_NORMAL; > + return page; > + } > + fallthrough; > + case RMQUEUE_STEAL: > + if (!(alloc_flags & ALLOC_NOFRAGMENT)) { > + page =3D __rmqueue_steal(zone, order, > migratetype); > + if (page) { > + *mode =3D RMQUEUE_STEAL; > + return page; > + } > + } > } > - return page; > + return NULL; > } > > /* > @@ -2311,6 +2357,7 @@ static int rmqueue_bulk(struct zone *zone, unsigned > int order, > unsigned long count, struct list_head *list, > int migratetype, unsigned int alloc_flags) { > + enum rmqueue_mode rmqm =3D RMQUEUE_NORMAL; > unsigned long flags; > int i; > > @@ -2321,7 +2368,7 @@ static int rmqueue_bulk(struct zone *zone, unsigned > int order, > } > for (i =3D 0; i < count; ++i) { > struct page *page =3D __rmqueue(zone, order, migratetype, > - > alloc_flags); > + alloc_flags, &rmqm); > if (unlikely(page =3D=3D NULL)) > break; > > @@ -2934,6 +2981,7 @@ struct page *rmqueue_buddy(struct zone > *preferred_zone, struct zone *zone, { > struct page *page; > unsigned long flags; > + enum rmqueue_mode rmqm =3D RMQUEUE_NORMAL; > > do { > page =3D NULL; > @@ -2945,7 +2993,7 @@ struct page *rmqueue_buddy(struct zone > *preferred_zone, struct zone *zone, > if (alloc_flags & ALLOC_HIGHATOMIC) > page =3D __rmqueue_smallest(zone, order, > MIGRATE_HIGHATOMIC); > if (!page) { > - page =3D __rmqueue(zone, order, migratetype, > alloc_flags); > + page =3D __rmqueue(zone, order, migratetype, > + alloc_flags, &rmqm); > > /* > * If the allocation fails, allow OOM handling an= d > -- > 2.49.0