From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DAC1C4828E for ; Fri, 2 Feb 2024 22:10:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7EDB26B00A3; Fri, 2 Feb 2024 17:10:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 79DBB6B00AB; Fri, 2 Feb 2024 17:10:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6651F6B00AD; Fri, 2 Feb 2024 17:10:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 54F0C6B00A3 for ; Fri, 2 Feb 2024 17:10:48 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E63458070E for ; Fri, 2 Feb 2024 22:10:47 +0000 (UTC) X-FDA: 81748259334.17.CCE337E Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf25.hostedemail.com (Postfix) with ESMTP id 37864A0010 for ; Fri, 2 Feb 2024 22:10:46 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AJBOE3sl; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of 3ZGi9ZQkKCF8QGJBO9FBODLLDIB.9LJIFKRU-JJHS79H.LOD@flex--tjmercier.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3ZGi9ZQkKCF8QGJBO9FBODLLDIB.9LJIFKRU-JJHS79H.LOD@flex--tjmercier.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706911846; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=HHbC9+IlrYehd7NC2Y5ENY9x+M2AXax5bbOnGa1kNgU=; b=Vj6JNngWl/ywvuwrzl4Kp7a2WY4LcXBe2822JetVdDNYnqzfIyXhpkzDVAK7xek8JhD6Vq WbRog1+aMkBUSCRhMSD8UfHk6n4tCM5ToFbU8R4+P6dRaiUolSpdsQrksB2807kl++QrmZ aaIEfmeEnb0TrLJcObZ9+z17futTn+M= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AJBOE3sl; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of 3ZGi9ZQkKCF8QGJBO9FBODLLDIB.9LJIFKRU-JJHS79H.LOD@flex--tjmercier.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3ZGi9ZQkKCF8QGJBO9FBODLLDIB.9LJIFKRU-JJHS79H.LOD@flex--tjmercier.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706911846; a=rsa-sha256; cv=none; b=6yr+OBI5lBzj1fvOUHfT9pbxbZUezLjze54xGCMWaWa1Jbler5x3ARq8LWrukH6M6Kq7UE kjT28hq/Y6bvN0Gtuf356KePEYkSSJxzJappCwufFjJ7v5S62WJVHeX7o3gKBZhOEQOe2q SGioa5sFx8f9GY84fK2V6yzPzwvV50A= Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-5d8dbe37d56so2772914a12.0 for ; Fri, 02 Feb 2024 14:10:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706911845; x=1707516645; darn=kvack.org; h=content-transfer-encoding:cc:to:from:subject:message-id :mime-version:date:from:to:cc:subject:date:message-id:reply-to; bh=HHbC9+IlrYehd7NC2Y5ENY9x+M2AXax5bbOnGa1kNgU=; b=AJBOE3slUHZJ9hBJ4Y7tFwEYgptb9WY42XDSf//1a331WTWK3KeGmMEDIUOO9wAdNM TiGc9oXYD/IJyTSbiEakVS2Ptjd597hwxpjYwqvnIBzi/RsyFKmqvUrV45TaO/Lt4qKF 1nXmJ1IywYOfj2bIjVIRSwWd8pkkKwdO3MlGfcHgvq4PKShZvVJKVaOUPZdSXlV3+rUG oLFS1zy0BDq0xDCoU1PoTiH3iol95V0w5iQoCekWMnydR8sKGKrnwjc1u5qlU1U3czzw AiKPmDgD702Jev9u+EcLbvfFZxcYxtqSrnbVzWSbWHxfQMcCG2A40uZWP/IlUTMvN82Z O5OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706911845; x=1707516645; h=content-transfer-encoding:cc:to:from:subject:message-id :mime-version:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=HHbC9+IlrYehd7NC2Y5ENY9x+M2AXax5bbOnGa1kNgU=; b=VM2CIfn3HupmHQiA8h+JAptUccuDiETOBRxyuMerEMk4j1BHxfHo00ZqbOagbysraP HW3vz8GX5ehxTS5Aq9B+fPIxMmwXQgGbcMY9Y5JgHXLlikd+kZh9WMBnOacyjCWN11sz ZSyds4dCAoehopPgNTeZ3cWwJZCu6jFxMDHo5otL81Q2nqKZIoxg24ptnrv+LPEEePw4 p4EDFaBDWNaQG51CxTMJXtpjV5DUav6ZDmUCKij1jz9dHif2DbvMgODC5c5aFOrtRRq2 f11znDWf/UfkJhSKgVrquWgHpzDbjLnoOfVrjTLIvpr96L8pmXzNiT4e/uKxs4PS0+Ia VdVg== X-Gm-Message-State: AOJu0YxgjP2Ebtiphc5l1XzjgWsefaL1l8lPvVR5uqdrhx0fukq7WoQk /9i7wEiL+X+/HX7c2ISFicCAGh3NeFsW5/WQzQgYmybRs2XXZ7h3F7FaQE6xc9yeZAcBSH1Ma15 hEzYw6kYEz0EbkA== X-Google-Smtp-Source: AGHT+IGVF+lwoe9gVyRO4B2PQUZH6+TCE7BUAMk/NEzPe2/7u2DlnIgUMd3xxewG37f2opj4wMD2PgOdCJcxf1w= X-Received: from tj-virt.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5683]) (user=tjmercier job=sendgmr) by 2002:a65:624a:0:b0:5cf:c149:8dc with SMTP id q10-20020a65624a000000b005cfc14908dcmr17021pgv.11.1706911844971; Fri, 02 Feb 2024 14:10:44 -0800 (PST) Date: Fri, 2 Feb 2024 22:10:25 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.43.0.594.gd9cf4e227d-goog Message-ID: <20240202221026.1055122-1-tjmercier@google.com> Subject: [PATCH v2] mm: memcg: Use larger batches for proactive reclaim From: "T.J. Mercier" To: tjmercier@google.com, Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , Efly Young Cc: android-mm@google.com, yuzhao@google.com, mkoutny@suse.com, cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 37864A0010 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: i79mtxs3eai1c3axo79towxjqbsg6x6f X-HE-Tag: 1706911846-436443 X-HE-Meta: U2FsdGVkX1/LeN/uiZIAQTGiyMIXIxOnA2jc48sVT6ooA8gsM7vOvclWX8/bxm0cwcp/HahBk6M6Zpbbeo4gk/6iCjd/EsEn8awRM3kc8sye59g8zpctx0lgmbIptr4G+powpoEy9xcwv/3RlSc3kIbwB0FTgBD8jXEWGKyD6txlKRqWUiErQF8xdQkJG5WfdC7KBLUryC+ZO35qnlqQh/PDwpu2lPG5p/NWSnZDyVopAPj+wZ2NXktBBvgvOfX2SFUlMGxhGiZTiS0Bl6oOP1wXsuluN7AR/sk4sGpw4+AEPAB4dX+a5Ae/k/FOW3jjrvx5QNYtR+x+PW0cEidLRKSSBhQ1HhioczJJPqAj9YfwP8TaNQiRoK9YL+mD8X6U+uSXde1L2uQJRlIUfEF0b37zfmekz/idyqagJ4/OeszIjDbp4f2KXHcizX3rAxrcH8D1ghxuuyLbOjINXxCWe/o604kJ+YV/E3lLvYn5DOuHBWlXsBrRSr6M2k+9ALCct/cwzniDseAv9kTzAWDaa3+9Ih4Iyy/hmx+STbWVrYk0WriEYGB0ie/Xa+4WhcFnnAykewul0bpBD8wy6KJspMjq9vA+wZsC8n4mHsUkH9UVIeNYTYZAWexsTrTxKhSrLWOu1MHveKrQzbDOpCPhSvDuTNI5ULq8PjbegfkPwNZi6tbAQE51P3cR+pag4/b2iMbBYUwe9OZ1UcdXgHJfiJFdzKyDUXd8QxFjAsX0Nrk2cvn2YV1Ixbq6niWS2OJxXUlk9HX4QBTbWA7dyasjkmUlvMOOnmMKSAR36Wzob7jM33R26sMFX09gmmvW+2wf6iClmy7lPFyViDSlb2jyXnMMiM1X2h1UDKq7Bybr2Fhe2D3rV0e6F5mD/V5UOglEGWBm6gTkrdVdbxwJrgRSETSXv02R1HVDISuQA+2/v9bbVuVu2qFWomeDFrmreOi+p2hXJ7bq9B6/igz4CMd Fnu0+pSf j1PfIsoWCIaFZ72yKTd2YkUdj76zgAvSuXZVoe2jDrP4sn/y0lHAw/aualTSEAFljx6cQlW4XbnTpbAyYQLfB1YBvCiwfQvWS3gea4igt25t5tTruJSw+W5KkHqj+QviGYCSHrR513YHLuiDI4BVIChad7Jixe3ajMIFauok4nOQ+7gI4GxTUt7DnvRBzTl6c4v1YFkasYmEF0c72QxbgtmVzItbmc5+yeg6GEQa5OrgX/rhxou4aObB8WtgZGO7fS9KVoEgTNlHC67zet5uo3Rp0tcoc37Kqj51myzKSIBZ3/jhTLzzlk+KK2B9jL1i+zFYxKq1XiEO3vC1EJ84UadoH3K0id2eU+06SmAT5dQfJTTP2AuVLLImE+fSgJXOZBPFUEW42ZnABCbSkXlauPqEZ3p5qTVgLflyjpFj4Dqv7qEXnBueDMkpDdH+LdHyRcEBP+dp/tSO/SAeyib/kmhuUfPONvlUxki1ENRxf7533GQHeZOO1Njzem2DjTGw52HsX81oc8TZeufKBlPJoHm5iCm1TNJOPpm348urvzPbR1KCC0nxsNsgQK3MYE3IbUkvCNUf8cdBYl3ePTG9wj1AKliuVxBXbc+KWt3hduqwYNIGZLjuniYrm7BEchSFoH1lK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Before 388536ac291 ("mm:vmscan: fix inaccurate reclaim during proactive reclaim") we passed the number of pages for the reclaim request directly to try_to_free_mem_cgroup_pages, which could lead to significant overreclaim. After 0388536ac291 the number of pages was limited to a maximum 32 (SWAP_CLUSTER_MAX) to reduce the amount of overreclaim. However such a small batch size caused a regression in reclaim performance due to many more reclaim start/stop cycles inside memory_reclaim. Reclaim tries to balance nr_to_reclaim fidelity with fairness across nodes and cgroups over which the pages are spread. As such, the bigger the request, the bigger the absolute overreclaim error. Historic in-kernel users of reclaim have used fixed, small sized requests to approach an appropriate reclaim rate over time. When we reclaim a user request of arbitrary size, use decaying batch sizes to manage error while maintaining reasonable throughput. root - full reclaim pages/sec time (sec) pre-0388536ac291 : 68047 10.46 post-0388536ac291 : 13742 inf (reclaim-reclaimed)/4 : 67352 10.51 /uid_0 - 1G reclaim pages/sec time (sec) overreclaim (MiB) pre-0388536ac291 : 258822 1.12 107.8 post-0388536ac291 : 105174 2.49 3.5 (reclaim-reclaimed)/4 : 233396 1.12 -7.4 /uid_0 - full reclaim pages/sec time (sec) pre-0388536ac291 : 72334 7.09 post-0388536ac291 : 38105 14.45 (reclaim-reclaimed)/4 : 72914 6.96 Fixes: 0388536ac291 ("mm:vmscan: fix inaccurate reclaim during proactive re= claim") Signed-off-by: T.J. Mercier --- v2: Simplify the request size calculation per Johannes Weiner and Michal Ko= utn=C3=BD mm/memcontrol.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 46d8d02114cf..e6f921555e07 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6965,6 +6965,9 @@ static ssize_t memory_reclaim(struct kernfs_open_file= *of, char *buf, while (nr_reclaimed < nr_to_reclaim) { unsigned long reclaimed; =20 + /* Will converge on zero, but reclaim enforces a minimum */ + unsigned long batch_size =3D (nr_to_reclaim - nr_reclaimed) / 4; + if (signal_pending(current)) return -EINTR; =20 @@ -6977,7 +6980,7 @@ static ssize_t memory_reclaim(struct kernfs_open_file= *of, char *buf, lru_add_drain_all(); =20 reclaimed =3D try_to_free_mem_cgroup_pages(memcg, - min(nr_to_reclaim - nr_reclaimed, SWAP_CLUSTER_MAX), + batch_size, GFP_KERNEL, reclaim_options); =20 if (!reclaimed && !nr_retries--) --=20 2.43.0.594.gd9cf4e227d-goog