From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0AC6EF327A8 for ; Tue, 21 Apr 2026 06:17:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC56B6B0096; Tue, 21 Apr 2026 02:16:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C9BF56B0095; Tue, 21 Apr 2026 02:16:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA17C6B0098; Tue, 21 Apr 2026 02:16:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 92DE66B0095 for ; Tue, 21 Apr 2026 02:16:59 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 3663E8CFAD for ; Tue, 21 Apr 2026 06:16:59 +0000 (UTC) X-FDA: 84681554958.17.D535317 Received: from sea.source.kernel.org (sea.source.kernel.org [172.234.252.31]) by imf17.hostedemail.com (Postfix) with ESMTP id 8DD7B4000A for ; Tue, 21 Apr 2026 06:16:56 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=IyXGjPUq; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of devnull+kasong.tencent.com@kernel.org designates 172.234.252.31 as permitted sender) smtp.mailfrom=devnull+kasong.tencent.com@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1776752216; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ETyMUE/Z7b1xh/23Y0MRw57aN/qiR0riEoKevTUv3mQ=; b=ZboUfljT+GuJIAcCUCQHVLqAN8wOqGND/NkDKWdqAdD+gaceAddh0ZT2wZ25aGBZ/LrEX/ YGu/f2y/0+2RztonWQd87p1mmKzxu7JJEhVejhozMn09ynqQ42jRaq3qAPFFVRci3F/kny xYhWiOuYlLXKJteOKs/oNtqw5szcmDQ= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=IyXGjPUq; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of devnull+kasong.tencent.com@kernel.org designates 172.234.252.31 as permitted sender) smtp.mailfrom=devnull+kasong.tencent.com@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1776752216; a=rsa-sha256; cv=none; b=rD3GOCEpdCIFWdkySfDml+DZ1HBtUDAWccUMDQdrxmj4l/XWLkM6UNGoJyxWhVt8utQMvW XcqN0YMnPN/OpAWxRrC7xX+bjLAvJ8oagAXmnX0abIiUKGdrS1aR2efrECpCiDjHdSRLdK QwZrUDe0NwIVURmxipYQNSu/fR2MDXI= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sea.source.kernel.org (Postfix) with ESMTP id E8A104470F; Tue, 21 Apr 2026 06:16:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPS id 8C825C2BD05; Tue, 21 Apr 2026 06:16:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1776752214; bh=Apc9Gc6RbHUejQ0QZ/J+CpG/qcB05eyHenvm22YITWo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=IyXGjPUqYtVLWe317We7sO0uw7Qsc4bHLU9xAGMnrcS44N3v3747zDxWBOn/pcvvO QUFOmq7pWJkmk9fQOryOMMLvDMD00rr3o/hh16ZAxLBgnGuonB6tIB/a4jFo1hBZBJ jukE/Eoj/g4PgdHtKbJ/wv3ETVrXULb/BnZmpaQ19pi1jorrXf709LCTaAllsww4rF P03a4bjbHGAzilnhG+4YoEMtcJV3fz+bcqVhvxaJCBs6/0aM6yAXtVI8umXRk869yb 4paeWAhgYPYZqS2RS81izF8f14DFHdEkww6xBXlt+PTK6iecrukILldBRkJxCc7g0F kueQqftcERIGA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8295BF327A6; Tue, 21 Apr 2026 06:16:54 +0000 (UTC) From: Kairui Song via B4 Relay Date: Tue, 21 Apr 2026 14:16:51 +0800 Subject: [PATCH v3 07/12] mm, swap: support flexible batch freeing of slots in different memcgs MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260421-swap-table-p4-v3-7-2f23759a76bc@tencent.com> References: <20260421-swap-table-p4-v3-0-2f23759a76bc@tencent.com> In-Reply-To: <20260421-swap-table-p4-v3-0-2f23759a76bc@tencent.com> To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , Barry Song , Hugh Dickins , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Johannes Weiner , Youngjun Park , Chengming Zhou , Roman Gushchin , Shakeel Butt , Muchun Song , Qi Zheng , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Kairui Song , Yosry Ahmed , Lorenzo Stoakes , Dev Jain , Lance Yang , Michal Hocko , Michal Hocko , Suren Baghdasaryan , Axel Rasmussen , Lorenzo Stoakes , Yosry Ahmed X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1776752211; l=2431; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=TF+4v7N3Y0fZu1fBTRYUKMMGHog+xsunCT0x6/UHUK8=; b=jKF0ucmqDotJQCko85mDlllnmWv+qK2so8CaITDB3VOptToZ8ErKzv7GTKqrb6flmdwtc/NBf Yajyzomm9HYDlRgerJnnpKVIJoHb3izF6/uzm34zTZR18PyobcZPRSq X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 8DD7B4000A X-Stat-Signature: iadab19t3mb1k6bhg1uq5eyjjkfw36w3 X-HE-Tag: 1776752216-88341 X-HE-Meta: U2FsdGVkX1/s9QUP5GStoZluJsnbaVpJmYcH+HMFBH23Po6QSG8aBYZmVmqXjJ157B+TxmdX3mnOvC3rzz3mW2nMGqSz1i0Q27nQlsjv3Z5yBJQdxsgum0hYQzJrB5JvNfsLQDOus4EdKRibBoMZ1awSZbB1HJOVEsd8lQI/mjHJZ26XKDsQ9QIrkB8XhBjWWl3GKsTflLsOlEd6wGSbIWeXuXQ+wDEYA8LGz0ABNqc1qGM/RSIuqnlknVfJ/UunphWNcgfrs7jwyurii/0BedWGoMX7N7e3X+ioQ2lS0FmPsD1R7e4ygIZ9VP7Se0IpGhqui7NH2V/O8xXPSB3IYht+RK9+ZqU9bPLtt0MU/D8yH3KRVwvamym6Y12Gte3MTFMyEdTPSuHTBT9bkf+T0bRujwbhUNtOi8uVRdFaB0N7EeGVZX2in8Mul6L90p5k/GrvXdPx6VLr0E8aItPK7Po6o1P5vPnlggcr2R6QHoMQEEjguxE02O0IlDSwt8TqYdhF/bHv0yepnAt8U86RKEHZ9szj4vSJ/LKACO89z/lFWqhNij80q8LQeEgybKIHBIGdfPywWV2ha2LA8MllyRngouHEv4ISrzFdt4F0rnM5bSqqpImAcJNgGIdS/y0s+vNUVbWypyGA3IgzXwFJyNwpgKyKjFc8wXCEIxzb+KoctGpX+qj1IYXFnWb9yuXyYyke1DYVDUCKwL9rDcxzisN7ohYzC7B/YN6mbV7GpMZZ3lIp6C2zIhU49xsmW0WViieBiHrjzNOgyhl+jB51U/YHrU4xTScnzXS31ujL9lCx50UUeFsMh/ZcJznDnBCkYKVyKGCPkcXS0XvLeYb8r7zTlw+7AYvYN1t6vR/2/owvOFcgVSuh4Rg9W+dpgzBMVe4qRh0QRqe4A04F6JJ8btj7xrbAV2Q5TlkoKX0DL5nr3pIRvgJM8ne/UWGd5+1b2nAbnue5P6qnwiig07M heE2XzOJ YhZMLamBoiPRwMLbvGi2E6SzABmr5/s44QPqCPJ83Sp6tAZxHbTzGzFIO1jAX5t0xyzG/6j0LqbUDDFhtmWb4ZE9CYiqpaXGZRubJW9p5//NQf9WhYnrnqS1uqMWa746OL/MN5ChvhXnvLkPmhpzWHdVLSewaWMbvSCWYvHz1p9j9zwtPPO6zYpndxS1EmyIUEwsRuS6lWPfxc5DY/pyf77lUDzjapJYdvI2iAQISp6kU+3bpwvZOApq59zbw6CBrtJM497OdrNWSSofrIiOMbKyw8Y37O4YQIO8hWKpdJQ9Hpj3Xl451Yv0pUkdrcBX7fe6VLFnbOwPawny67SZf0W3G8qBEv0wGkpTXXz/y9Pv0BObQdLs6vIHeecQArgPR0Bf+Yj4QFOLf840bfCAIDwalVQ== Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Instead of requiring the caller to ensure all slots are in the same memcg, make the function handle different memcgs at once. This is both a micro optimization and required for removing the memcg lookup in the page table layer, so it can be unified at the swap layer. We are not removing the memcg lookup in the page table in this commit. It has to be done after the memcg lookup is deferred to the swap layer. Signed-off-by: Kairui Song --- mm/swapfile.c | 33 +++++++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index e1ad77a69e54..8d3d22c463f3 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1872,21 +1872,46 @@ void __swap_cluster_free_entries(struct swap_info_struct *si, unsigned int ci_start, unsigned int nr_pages) { unsigned long old_tb; + unsigned int type = si->type; + unsigned short id = 0, id_cur; unsigned int ci_off = ci_start, ci_end = ci_start + nr_pages; - unsigned long offset = cluster_offset(si, ci) + ci_start; + unsigned long offset = cluster_offset(si, ci); + unsigned int ci_batch = ci_off; + swp_entry_t entry; VM_WARN_ON(ci->count < nr_pages); ci->count -= nr_pages; do { old_tb = __swap_table_get(ci, ci_off); - /* Release the last ref, or after swap cache is dropped */ + /* + * Freeing is done after release of the last swap count + * ref, or after swap cache is dropped + */ VM_WARN_ON(!swp_tb_is_shadow(old_tb) || __swp_tb_get_count(old_tb) > 1); __swap_table_set(ci, ci_off, null_to_swp_tb()); + + /* + * Uncharge swap slots by memcg in batches. Consecutive + * slots with the same cgroup id are uncharged together. + */ + entry = swp_entry(type, offset + ci_off); + id_cur = lookup_swap_cgroup_id(entry); + if (id != id_cur) { + if (id) + mem_cgroup_uncharge_swap(swp_entry(type, offset + ci_batch), + ci_off - ci_batch); + id = id_cur; + ci_batch = ci_off; + } } while (++ci_off < ci_end); - mem_cgroup_uncharge_swap(swp_entry(si->type, offset), nr_pages); - swap_range_free(si, offset, nr_pages); + if (id) { + mem_cgroup_uncharge_swap(swp_entry(type, offset + ci_batch), + ci_off - ci_batch); + } + + swap_range_free(si, offset + ci_start, nr_pages); swap_cluster_assert_empty(ci, ci_start, nr_pages, false); if (!ci->count) -- 2.53.0