From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADD4EC36002 for ; Wed, 9 Apr 2025 17:30:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 63574280099; Wed, 9 Apr 2025 13:30:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E4CF280092; Wed, 9 Apr 2025 13:30:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4ACA7280099; Wed, 9 Apr 2025 13:30:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2CFE5280092 for ; Wed, 9 Apr 2025 13:30:54 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8C117C03E2 for ; Wed, 9 Apr 2025 17:30:54 +0000 (UTC) X-FDA: 83315195628.06.A02217F Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf06.hostedemail.com (Postfix) with ESMTP id 53934180016 for ; Wed, 9 Apr 2025 17:30:52 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KcOZIYlJ; spf=pass (imf06.hostedemail.com: domain of 3yq72ZwgKCEkulnvxlymrzzrwp.nzxwty58-xxv6lnv.z2r@flex--jackmanb.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3yq72ZwgKCEkulnvxlymrzzrwp.nzxwty58-xxv6lnv.z2r@flex--jackmanb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744219852; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jFuCrVcielRSxkqN5LBqmHKeHllW5E8rT4ctitcywvo=; b=LYwxXqidGV9fupj2benpIRlANZcHaz6/u3JmeO61Yr/POhbmL3pikX0UQ4mL/ktwud5brd hjPiNiCuEjU48tyJHogiA7vV1957vyoKbneXJbLo8Ty7SPBCIG8H++so5XHg/ND0HH+pxD pq6iCfm3CQuCrgD2TvaJWkeTbpTQZFQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744219852; a=rsa-sha256; cv=none; b=XcgIrjSY2zcERbXbUqon3TJWJqy60Ey5SZdqjS6VuMUhfwzuCcnSn5q5OOrHmwG+1NRpRK ra9JjvEH4RCIw/UoJv3KvsY9pE3IqWMbdcCfE7miPcnvtKM2U0zVEtKk8rkk4AUam5dK3N Kzt0sCrXSa13OiXOQlRGsZOFPsFRWNk= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KcOZIYlJ; spf=pass (imf06.hostedemail.com: domain of 3yq72ZwgKCEkulnvxlymrzzrwp.nzxwty58-xxv6lnv.z2r@flex--jackmanb.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3yq72ZwgKCEkulnvxlymrzzrwp.nzxwty58-xxv6lnv.z2r@flex--jackmanb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43cf5196c25so41437235e9.0 for ; Wed, 09 Apr 2025 10:30:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744219851; x=1744824651; darn=kvack.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=jFuCrVcielRSxkqN5LBqmHKeHllW5E8rT4ctitcywvo=; b=KcOZIYlJ18KGUOPhV5RWNcWpLPS//nGHpEFnIaAxnTukbilBpUTdzswOg/xLR7a14m vU7QlOMRuBIPDARBv8p9RIFEc05YsDvjsNHPI7+jTtzoSy0Ip0E12NPhWbr8qAxq7oci nuv9VFkm7HS3DnyluxdnmPHw17V0VoeJLVWyW4JH9Hqbb8J6sg0bY4qdj0bisGPGAhSf uUzOeW/Alx1fF2S5cr7Syte7ThmUJpmd/5ou3p4pyjAhtqaST99MPFxm1xxAuU0OM+UQ JOdUDjYGpqSneDHN7J9fArwrXpdxSINSfqyellhkabvKzCv7Ht0VBWgUduQFmOYscFmC jm6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744219851; x=1744824651; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=jFuCrVcielRSxkqN5LBqmHKeHllW5E8rT4ctitcywvo=; b=wQ1bKwCHNmLRoF+0NYSsmr3xVYo1/jNoJLyFB7X6goPg2M47i21/1vhRsBRtWZzse6 0bRJOEjd6JwrTKGiZGKVseuAxodBbBQ9Fw0du5Li6L85gOSHdBf4FvVCGmIZcI2fcaPN gKjN2YkE+hnyA+YW/GH7dSqhHDMzsJ3MQwgP1VXqt1BMJvfPSrCOHJyqJAIMfJPx8+QA ghk5RuNk4CNwjzoxl7Cjjix+JqcDPTfw4Bax2MvbSv2ZOBGvwuSpbSVgw0Fs7tFS0fsp bkY2QXzLmPQSeRMxAUUKOSW6KwSZpsyNbwEwo7Oz+9QUOVQCXsF9h+/tdAvITOgPfNA/ IaTg== X-Forwarded-Encrypted: i=1; AJvYcCXoPm+L4m9L4kXSVQY/IKlPriKHg1N7+hHFVRjvjLtzbtbCQGNu198+W21mE6x5rSKWSrSj2c0y3w==@kvack.org X-Gm-Message-State: AOJu0YzNCyMcJkV/6aRtk5BeZ/Vh7jONXNQOPjRvXyNj/tV6U+IPq1C/ +Km+7zjBv7KCwJr5bnVX6+mp6PXx51SOtOZ7jcQROHQItJ0SxD5veKG7RD5VRXKGVkZOFeL6oIk 3JpVDMUtP0g== X-Google-Smtp-Source: AGHT+IGOHH9oihfNAW2IS+f2IzXxx5SZ5pL5GY7qNP9YIhIPG4cLX1oYAQZci0raRdZxy6hcKyk1fZ5uOCFrjw== X-Received: from wmbev15.prod.google.com ([2002:a05:600c:800f:b0:43d:7e5:30f0]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:40dc:b0:391:ba6:c066 with SMTP id ffacd0b85a97d-39d87ac7fc2mr3710134f8f.35.1744219850563; Wed, 09 Apr 2025 10:30:50 -0700 (PDT) Date: Wed, 09 Apr 2025 17:30:48 +0000 In-Reply-To: <20250408185009.GF816@cmpxchg.org> Mime-Version: 1.0 References: <20250407180154.63348-1-hannes@cmpxchg.org> <20250408185009.GF816@cmpxchg.org> X-Mailer: aerc 0.20.0 Message-ID: Subject: Re: [PATCH 1/2] mm: page_alloc: speed up fallbacks in rmqueue_bulk() From: Brendan Jackman To: Johannes Weiner Cc: Andrew Morton , Vlastimil Babka , Mel Gorman , Carlos Song , , , kernel test robot , Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 53934180016 X-Stat-Signature: nqpybng3azc4ki3xpcx93seuxshhsodq X-HE-Tag: 1744219852-669442 X-HE-Meta: U2FsdGVkX18WiH58kCInrLZ2cjfRSmpuo1BUfsLHU0Rz0GxcGBqtSF83Xif8DfXjEeQlFML10v91TdjkZ+Kx0riF+6SmPpRoXx48Urmmp7bEz59ey+1Fn+5U4fLPe9E/UZ+7M4aZDGg3xJF2ydbaxCuSQOaG81WZZkQjgX4lMQnw3FBNs4oZJEVdzUIF7CxMS3DXC9OYU7F8m/PiMA1KjO5BCaOQ0SPn7lxh7E05r3K2z3CjYaCS3jZPzzxrHW8GiJh0Q+cl/ByBjmHC07I2DBv9tloAwhOcJgf6uw0xOaeZmIrziKOceMIrTjBYYjeDkDfaGjHyXnGOdu/eq6co5zcojdIdOsEAfhVjRnS+MW1kkNnYxXQmLsdYqZiLHZ9+fnI1L07VwZgNHli0uQWQeF7gnH+pe4PvlyKW9PnkcfnY4dosHfkxt2br4x358sSom6QFzDPujt36LxTJSBgycbbvVW/ePX07YTujzzcrVblUYMKykbsuuAl6+tHUsx/0p4phZQHPyTDAfQQn7a0epIuHzcXMC3OT0HeRXkSDtC+W6nj2v3am84QHPFCYpxWJgnupqyIKjUzWjAE49jUMtEbCMUiOAiJws/7jNjZx5sS1CInoHgepbkvGEGagEGRqcXk9Gz8KZWEY9DQSbAAZwLlFExzYZu4I+iQoMFLmljw0OQlBxtAxgxGscF90k0+5mXKXCwUYAlTEDAZJx3h67RN+TXXmgLAAsQOtIHM9/D9eRGt9RmAR8ypbo5q2Hwwkny3Dfki1eSZH15/PPQKrdZbZsx8gvg4q6u+szw9ndDzuSkb9NDH4MCj9QDQ8xJsL6PwjNuyFOu5/RakHpRBEvS9VDMqTBAuHzlw2/zrpArll4l1tfyNyBQShp2N6L/Fz+/y6COXgnm5Q5UwClZVC2/GJmd1034m7sx54A194AvNY4+LY7AOQJaYi6wc6ffXsnUs0SnUXqEJd+9pgV1G JTn8o/Sg 98G448ieqU28jglOZt8GkVI9XYKXoCJBs9Fj3waIx3Z28OjF3NHi0QWRYJCUq4725D9vLr0Qq5PKzz295gTWoB8Zanzvg/Hhwts52GQ2NiyzxihbhKxgmA20S/ZZGGBady4kh7CjjzF4nDeGkj5PFyb+0TV7f4Ez7ahtNTOuwsTAmQuq8aDypILjkZdNLcPDWSmA68BvCbs7pOlfqNc6GsjKbVeprBgkjYw3ZBWETUKJfx0G3rSAYcPd+4TqDA9GE9GWSHQJ+DlZHzcrFoM0KK41pReSwgLktLQeDlounxh0icXu7mBz3FA6NELda2b3iiour2zcj9JLVe5Lj2zAqZEdMtE9tfnQTTdxWLWZpAkjf7G8yJoDtRg3ivq2VdwjFVFwFOH8YPEI0vpUg0o4U6Z+snwV2syQLZR893mCS/vAZ018y0BYDgf4qY2MtyFYJi7ztdRyuBv3ODhM4ibRqp7daDTyP84ECG+ZR5t/TXe7xX4z5kknpn/vv/HPXdojvR5E02WTl+G34rgo2z4nHdaJQ+1qhijZIYwUTdYrgolmmfoUqjkVKm9zn4ZyripPhf5Ba+XGCBHXravNAyeAeaZ8vMO7LbZ2HLeykkFnLnK86PMe77Pu8x6U0FRoiVETBcDoCoT3gJV8auZ6ClwAZSyYEROCovwCTNXrp X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue Apr 8, 2025 at 6:50 PM UTC, Johannes Weiner wrote: >> > /* >> > * Find the largest available free page in the other list. This roughl= y >> > * approximates finding the pageblock with the most free pages, which >> > * would be too costly to do exactly. >> > */ >> > for (current_order =3D MAX_PAGE_ORDER; current_order >=3D min_order; >> > --current_order) { >>=20 >> IIUC we could go one step further here and also avoid repeating this >> iteration? Maybe something for a separate patch though? > > That might be worth a test, but agree this should be a separate patch. > > AFAICS, in the most common configurations MAX_PAGE_ORDER is only one > step above pageblock_order or even the same. It might not be worth the > complication. Oh yeah, makes sense. >> /* >> - * Try the different freelists, native then foreign. >> + * First try the freelists of the requested migratetype, then tr= y >> + * fallbacks. Roughly, each fallback stage poses more of a fragm= entation >> + * risk. > > How about "then try fallback modes with increasing levels of > fragmentation risk." Yep, nice thanks. >> * The fallback logic is expensive and rmqueue_bulk() calls in >> * a loop with the zone->lock held, meaning the freelists are >> @@ -2332,7 +2329,7 @@ __rmqueue(struct zone *zone, unsigned int order, i= nt migratetype, >> case RMQUEUE_CLAIM: >> page =3D __rmqueue_claim(zone, order, migratetype, alloc= _flags); >> if (page) { >> - /* Replenished native freelist, back to normal m= ode */ >> + /* Replenished requested migratetype's freelist,= back to normal mode */ >> *mode =3D RMQUEUE_NORMAL; > > This line is kind of long now. How about: > > /* Replenished preferred freelist, back to normal mode */ Yep, sounds good - it's still 81 characters, the rest of this file sticks to 80 for comments, I guess I'll leave it to Andrew to decide if that is an issue? > But yeah, I like your proposed changes. Would you care to send a > proper patch? Sure, pasting below. Andrew, could you fold this in? Also, I haven't done this style of patch sending before, please let me know if I'm doing something to make your life difficult. > Acked-by: Johannes Weiner Aside from the commen stuff fixed by the patch below: Reviewed-by: Brendan Jackman --- >From 8ff20dbb52770d082e182482d2b47e521de028d1 Mon Sep 17 00:00:00 2001 = = = =20 From: Brendan Jackman Date: Wed, 9 Apr 2025 17:22:14 +000 Subject: [PATCH] page_alloc: speed up fallbacks in rmqueue_bulk() - comment= updates Tidy up some terminology and redistribute commentary. = = = = =20 Signed-off-by: Brendan Jackman --- mm/page_alloc.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index dfb2b3f508af4..220bd0bcc38c3 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2183,21 +2183,13 @@ try_to_claim_block(struct zone *zone, struct page *= page, } =20 /* - * Try finding a free buddy page on the fallback list. - * - * This will attempt to claim a whole pageblock for the requested type - * to ensure grouping of such requests in the future. - * - * If a whole block cannot be claimed, steal an individual page, regressin= g to - * __rmqueue_smallest() logic to at least break up as little contiguity as - * possible. + * Try to allocate from some fallback migratetype by claiming the entire b= lock, + * i.e. converting it to the allocation's start migratetype. * * The use of signed ints for order and current_order is a deliberate=20 * deviation from the rest of this file, to make the for loop * condition simpler. */ - -/* Try to claim a whole foreign block, take a page, expand the remainder *= / static __always_inline struct page * __rmqueue_claim(struct zone *zone, int order, int start_migratetype, unsigned int alloc_flags) @@ -2247,7 +2239,10 @@ __rmqueue_claim(struct zone *zone, int order, int st= art_migratetype, return NULL; } =20 -/* Try to steal a single page from a foreign block */ +/* + * Try to steal a single page from some fallback migratetype. Leave the re= st of + * the block as its current migratetype, potentially causing fragmentation= . + */ static __always_inline struct page * __rmqueue_steal(struct zone *zone, int order, int start_migratetype) { @@ -2307,7 +2302,8 @@ __rmqueue(struct zone *zone, unsigned int order, int = migratetype, } =20 /* - * Try the different freelists, native then foreign. + * First try the freelists of the requested migratetype, then try + * fallbacks modes with increasing levels of fragmentation risk. * * The fallback logic is expensive and rmqueue_bulk() calls in * a loop with the zone->lock held, meaning the freelists are @@ -2332,7 +2328,7 @@ __rmqueue(struct zone *zone, unsigned int order, int = migratetype, case RMQUEUE_CLAIM: page =3D __rmqueue_claim(zone, order, migratetype, alloc_fl= ags); if (page) { - /* Replenished native freelist, back to normal mode= */ + /* Replenished preferred freelist, back to normal m= ode. */ *mode =3D RMQUEUE_NORMAL; return page; } base-commit: aa42382db4e2a4ed1f4ba97ffc50e2ce45accb0c --=20 2.49.0.504.g3bcea36a83-goog