From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A012C369C2 for ; Wed, 16 Apr 2025 13:52:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2DBE9280011; Wed, 16 Apr 2025 09:51:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 20EB828000B; Wed, 16 Apr 2025 09:51:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 07C48280011; Wed, 16 Apr 2025 09:51:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D58A528000B for ; Wed, 16 Apr 2025 09:51:57 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 62FFB160855 for ; Wed, 16 Apr 2025 13:51:59 +0000 (UTC) X-FDA: 83340045558.02.588EC98 Received: from mail-qv1-f53.google.com (mail-qv1-f53.google.com [209.85.219.53]) by imf04.hostedemail.com (Postfix) with ESMTP id 8EEE340005 for ; Wed, 16 Apr 2025 13:51:57 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=qjBEyhc5; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf04.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.53 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744811517; a=rsa-sha256; cv=none; b=AshPt8SS2CvSTWadAOf++x+1CJ37D8ttaKWMjuB0e6YnnbETVUCzCTtbbke4xkgcSKH7iK CAD4DUDhAaXo44WSKcQXqkfhdFmIlg4Ljt7kzDG02G4DayK4K/MV+9CittEld2O/ormO6D m4H9d+GOWz/dhZsR4V5Mz8caL4HXqPU= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=qjBEyhc5; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf04.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.53 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744811517; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ygbKH3YK/kUuGcHybi94DI1iP78iOJL69yAdVHR5KY0=; b=II6EOs/D0Cn11sU8b9j2qfsaGth16jGijFbEeGmp5apiKeMsnrgTLOfBXqXNjKg30UlkPI zhbOTBWayHChZOg0rNthZy+Hr4trl8wXT2SVDX5eog2frlIYQ4P0W9FuZGY8RKXIqKofPC 07KqTDUNwVFRAg57LTqeglv4BYX76wk= Received: by mail-qv1-f53.google.com with SMTP id 6a1803df08f44-6e8fb83e137so51029476d6.0 for ; Wed, 16 Apr 2025 06:51:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1744811516; x=1745416316; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ygbKH3YK/kUuGcHybi94DI1iP78iOJL69yAdVHR5KY0=; b=qjBEyhc5Aw+ABdkO4VZJgKjLjgrSDKOGKu73RIQHzoV/xFKfZ7GhbPmS3Oo1CVoEvn D2T2ufIsSRE6SE352FKpNKhrdDhXwVqWVIP50zLj8ywApL8Xe+LDrsHQumhh9UFhfEuE G/2qQG07dBlmnkdc4oA5upBFeK36cVUMI3G8hRcT71OAolu9dSn+W9CVSn5+7zCTHRcY m0gOfUz6WQG3y/QIXxH7Xa017vsDdWz+eqBUtKKVy4NQhuA0j5NDP93C1msqWAh5ZR3j kDaEq6cXUKjixv56n61LfoM+yt+Njm59GxSJgh3cGYHKlQhbezDlRQuUAk2kYgnlt75y 9nmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744811516; x=1745416316; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ygbKH3YK/kUuGcHybi94DI1iP78iOJL69yAdVHR5KY0=; b=KLUOVeVU60+tIkqvZXf1qsZuxPfbn9DAc+yHBrpkwSPfis8FSubiNQWulD1a5CeOi0 74Eo50W79Kx2/fxPJa8lNLy1E/31QBFkUukTOr2Ojfh3jSI5LHWIBknPgmJUKA6f8GBZ 7HE5v4GmYHIAY8W8j7cKxAWPPvdRbwk0n6HKdm7sQVnYqmVMtutj7aT/tEseHO09vyOb +C7B8VGxTVbD1OOeXiX69KjA8KT4NaKCnwRhSHnDHBBhGjJpdHc4f6kgKv2UHnurKcqQ EEJb8GIg9fGwCuuAReY5muWsR4ZUc47dOAaXc7CF5iu7OxypQYNNIf6Iu2NiDTo7uKiX zJOg== X-Forwarded-Encrypted: i=1; AJvYcCXj9Pph0sUPlT6UmPNAqTXTZPt1rZ3T/jyp1eBfxgV+TdTeLnAGceOSG0gJAWsjmzMH4gIyl6Tm0Q==@kvack.org X-Gm-Message-State: AOJu0YxVDKq20i0fWWW4qzdTwgxwvOb6B0oi3bEq8OAdH/IkJosHf9E2 1zlRxWcOgjjMhNQMJt+bDP1GW169riePqunSBLgtuVRBKmgGZj/Znu3q5fxExek= X-Gm-Gg: ASbGncuB6I/FLsXKz5YSGmaLEHJPtuTvtyrL+obL0XDCMMZmCp+HM6FJt6IPT0ojQsl 1R1NpAdUukGnjFDMp7+zZvuijoRP8m0ZTdPY0CDzsISDi/xs/QlO8XSfA8xL1KQ7cxZ8HU+A9dY 9vUpRyghytFkT4TOmbllfHfcv7C8gykEpJL59ZnFo72JOqcisDr8hVMXeapXN1ourCgL7+0wj6O XyokXvj5768rLTwUNKx6QQRERtid1OzcJrY855nOBZ+6cxmWrcHbd80DTpKqmQBoWw/adbQCysB Wkt/oD59XeJL8TOoKEeaQKQNgesXFGpjuawUWR0OQ2sYJpFWXg== X-Google-Smtp-Source: AGHT+IHu8b/zN0Bedn9rskehIH1b24+7rPIphRwS3idv/xejtxQrBsiOY8GHnLRydQ1etfGbJCF9Lw== X-Received: by 2002:a05:6214:20a2:b0:6e6:646e:a0f0 with SMTP id 6a1803df08f44-6f2b2f42279mr25909756d6.12.1744811516531; Wed, 16 Apr 2025 06:51:56 -0700 (PDT) Received: from localhost ([2603:7000:c01:2716:365a:60ff:fe62:ff29]) by smtp.gmail.com with UTF8SMTPSA id 6a1803df08f44-6f2b04c4ff7sm10478806d6.88.2025.04.16.06.51.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Apr 2025 06:51:55 -0700 (PDT) From: Johannes Weiner To: Andrew Morton Cc: Vlastimil Babka , Brendan Jackman , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/2] mm: vmscan: fix kswapd exit condition in defrag_mode Date: Wed, 16 Apr 2025 09:45:40 -0400 Message-ID: <20250416135142.778933-3-hannes@cmpxchg.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250416135142.778933-1-hannes@cmpxchg.org> References: <20250416135142.778933-1-hannes@cmpxchg.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 8EEE340005 X-Stat-Signature: nx4ftzb5ph7t9ag18ap145m95qqcmbkg X-Rspam-User: X-HE-Tag: 1744811517-876035 X-HE-Meta: U2FsdGVkX1/wibh3G9zzLkAZ2bKCRFos8248KS2vyK6mC+UcozJUgeW6VtC3YOTv2Am4mbSye/E36qwWQAJ+ES36V5w630f+VVh/kUlXxUzhNz4WXGTJnqryq0Sx/E8SSUhPzllep9pbG1NAFt1tqQHuk13R3hvBFoYC+VovRggPiphW+IBXoFqaTbOzm2fUUsFVd8XCd5pFgtzIQoU50D+YrvPk2hwLDUqkrmsnmuSJ+ZZ6RTHz+Mte5EyitGAp1BZ5TGH+rbyZKwKgH404wXyMqigYiHTVRUstmxgKXWsL/pys64UIySpraSNwg+aOjLL8RM090noERP9UJhfqCP+zZSuztha0PSSvGViIuM9zR3YJKsYdBgtJtuJR9NOIy3JyyRu1fQltuy7HSnJ+IFJM5OyY44JK7uOUfnVgkhxIkg39ZGpdsGHXbQEWAbLtS4vnkH6zieLaNVi1z4Nu6T9xoJ9Aq7kuRWCLorMeON+vAr/9GUuK6cfHfveym3B1ttmJVZOh1ziCaCDGf1xFOUKH/pZRsi3nPZIXN+bPGgw9yJss5n71FjFNgoqX9krj3EYSJ4t9AqV0lx57cfTBB2+CMs6JrRwFwTyHzKXHP9XKotp387nSr9FY6gMyuHfHBjIRd40WhGL00HZB06iC0TwYd8XbRqSYZ1XiSskNuiOZrbsxnCKEuzrMbwm2nTeeQSyk2cPXh9k4CE044tlQApux0WdYE6MZMY63cHe/+TuYad8tXcCCP93KmQToCu/WBi4OgsS9QWgZHCjet8bOHtonYrADJ3ZrEKsYLv1EzDbZWC2sfQPbQXVpD8NTSuEpFvP6gcXaSscz3RSq1Nd2YG/+sbFhxRR32wDcFRWZhOgGbrG95fKk0JnKAG+/9gW5jyytsVMS4vKsyzBB0VIz6MUTMzZVh+e1DJzSaxBIn/h3aX3nTnPIWi35CrcZYMJNUyBJoX2xAr2MCleCo9c lIJLzwJR LF4gBPqtZ8FaPKmuja88FcUJFaZgnHssBp+8cj30mPCTjokA78JKrPiK8iCrhcphPr+fP5zU9fep2volslbd821NCCwse6XfZ7536tE5xatTX3NE4sW43rVwJDGIXyIC/hnfzyHnSDd6AvhS8XyDBeapy0VyqenQIdN6iKoVrvQFxVNhZCt4L9jv5Bg6r//vPqdWkI2fLD0mCTsXAY1eHcO8Fu2eGpifW8bg64XUh25ivbxsLVI3qrzsj92+FW4EtCVW2MElyXhOJyajH+4RfNHVDhq3aRqMgayXhvl1LT0DTrDUHFa92C1NDZ8G+h+87mUtjfS++nTo3BaGTKw17I2MwaJHOeNhWKBHma8lf4FFLJxHCgl5s2647jpA1qGOVehFk1PWo8+e7vY9wfpY30dhiO3mM+5lrr/hy4y3MrJuJ82dPAvCQlah4upC9X6gHiw4Pvf+5N2NtFnPddqyG7pLTV10IvHn2/OoGiALG6iYAqFoQLAOXyAYAOTeJazgvZ4dgqUNOZPCgCn49xeNDbTJjZA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Vlastimil points out an issue with kswapd in defrag_mode not waking up kcompactd reliably. Background: When kswapd is woken for any higher-order request, it initially checks those high-order watermarks to decide if work is necesary. However, it cannot (efficiently) meet the contiguity goal of such a request by itself. So once it has reclaimed a compaction gap, it adjusts the request down to check for free order-0 pages, then wakes kcompactd to coalesce them into larger blocks. In defrag_mode, the initial watermark check needs to be analogously against free pageblocks. However, once kswapd drops the high-order to hand off contiguity work, it also needs to fall back to base page watermarks - otherwise it'll keep reclaiming until blocks are freed. While it appears kcompactd is woken up frequently enough to do most of the compaction work, kswapd ends up overreclaiming by quite a bit: DEFRAGMODE DEFRAGMODE-thispatch Hugealloc Time mean 79381.34 ( +0.00%) 88126.12 ( +11.02%) Hugealloc Time stddev 85852.16 ( +0.00%) 135366.75 ( +57.67%) Kbuild Real time 249.35 ( +0.00%) 226.71 ( -9.04%) Kbuild User time 1249.16 ( +0.00%) 1249.37 ( +0.02%) Kbuild System time 171.76 ( +0.00%) 166.93 ( -2.79%) THP fault alloc 51666.87 ( +0.00%) 52685.60 ( +1.97%) THP fault fallback 16970.00 ( +0.00%) 15951.87 ( -6.00%) Direct compact fail 166.53 ( +0.00%) 178.93 ( +7.40%) Direct compact success 17.13 ( +0.00%) 4.13 ( -71.69%) Compact daemon scanned migrate 3095413.33 ( +0.00%) 9231239.53 ( +198.22%) Compact daemon scanned free 2155966.53 ( +0.00%) 7053692.87 ( +227.17%) Compact direct scanned migrate 265642.47 ( +0.00%) 68388.33 ( -74.26%) Compact direct scanned free 130252.60 ( +0.00%) 55634.87 ( -57.29%) Compact total migrate scanned 3361055.80 ( +0.00%) 9299627.87 ( +176.69%) Compact total free scanned 2286219.13 ( +0.00%) 7109327.73 ( +210.96%) Alloc stall 1890.80 ( +0.00%) 6297.60 ( +232.94%) Pages kswapd scanned 9043558.80 ( +0.00%) 5952576.73 ( -34.18%) Pages kswapd reclaimed 1891708.67 ( +0.00%) 1030645.00 ( -45.52%) Pages direct scanned 1017090.60 ( +0.00%) 2688047.60 ( +164.29%) Pages direct reclaimed 92682.60 ( +0.00%) 309770.53 ( +234.22%) Pages total scanned 10060649.40 ( +0.00%) 8640624.33 ( -14.11%) Pages total reclaimed 1984391.27 ( +0.00%) 1340415.53 ( -32.45%) Swap out 884585.73 ( +0.00%) 417781.93 ( -52.77%) Swap in 287106.27 ( +0.00%) 95589.73 ( -66.71%) File refaults 551697.60 ( +0.00%) 426474.80 ( -22.70%) Reported-by: Vlastimil Babka Fixes: a211c6550efc ("mm: page_alloc: defrag_mode kswapd/kcompactd watermarks") Signed-off-by: Johannes Weiner --- mm/vmscan.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index cc422ad830d6..3783e45bfc92 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -6747,8 +6747,14 @@ static bool pgdat_balanced(pg_data_t *pgdat, int order, int highest_zoneidx) /* * In defrag_mode, watermarks must be met in whole * blocks to avoid polluting allocator fallbacks. + * + * However, kswapd usually cannot accomplish this on + * its own and needs kcompactd support. Once it's + * reclaimed a compaction gap, and kswapd_shrink_node + * has dropped order, simply ensure there are enough + * base pages for compaction, wake kcompactd & sleep. */ - if (defrag_mode) + if (defrag_mode && order) item = NR_FREE_PAGES_BLOCKS; else item = NR_FREE_PAGES; -- 2.49.0