From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBEBDD10C15 for ; Sun, 27 Oct 2024 10:12:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E2EC56B007B; Sun, 27 Oct 2024 06:12:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DDE416B0082; Sun, 27 Oct 2024 06:12:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA5716B0085; Sun, 27 Oct 2024 06:12:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id ABD056B007B for ; Sun, 27 Oct 2024 06:12:05 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 284A7A18F4 for ; Sun, 27 Oct 2024 10:11:26 +0000 (UTC) X-FDA: 82718966148.16.B36B2C7 Received: from forward501b.mail.yandex.net (forward501b.mail.yandex.net [178.154.239.145]) by imf24.hostedemail.com (Postfix) with ESMTP id EB2F2180013 for ; Sun, 27 Oct 2024 10:11:59 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=yandex.ru header.s=mail header.b=nQf1MkGD; dmarc=pass (policy=none) header.from=yandex.ru; spf=pass (imf24.hostedemail.com: domain of Hi-Angel@yandex.ru designates 178.154.239.145 as permitted sender) smtp.mailfrom=Hi-Angel@yandex.ru ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730023808; a=rsa-sha256; cv=none; b=KPZvqg27f1MSE47cPFXouMOOi4Zt//VZ7SE/wMRxO1JDC0/dN48JwUxyc34HNXFQtjySnU nS1fgEA9UPN5NQTBpUef7YJKbrUTj6wFnn6K4c+dnc7HCi30CnXtgm5KmurmXa8QV0WHZr CQ10ecyEM0KG3u8iru5omIW/AVCRZEo= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=yandex.ru header.s=mail header.b=nQf1MkGD; dmarc=pass (policy=none) header.from=yandex.ru; spf=pass (imf24.hostedemail.com: domain of Hi-Angel@yandex.ru designates 178.154.239.145 as permitted sender) smtp.mailfrom=Hi-Angel@yandex.ru ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730023808; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PlCWbGf+DsTcw+oMV449hQH89VWBpXiupTFHDFqbn+k=; b=beH7eSXvs/EfkpgD0P+1a75c0dweLe/73Dd15z5hovIGvPnZR9jBpSytlFGx/pNySM+l8U eQQmf5GVziu7pvhYq9MDoEp+AWgVPlRJGgYYnmYd+6qvf/dU7jUO/OupwXJ5fKZkNTMOVQ 0hNPDc3+UuoF0B0xVQGxCkV7wqiXiCg= Received: from mail-nwsmtp-smtp-production-main-46.sas.yp-c.yandex.net (mail-nwsmtp-smtp-production-main-46.sas.yp-c.yandex.net [IPv6:2a02:6b8:c07:109:0:640:efe1:0]) by forward501b.mail.yandex.net (Yandex) with ESMTPS id 0E50860F0F; Sun, 27 Oct 2024 13:12:00 +0300 (MSK) Received: by mail-nwsmtp-smtp-production-main-46.sas.yp-c.yandex.net (smtp/Yandex) with ESMTPSA id wBU8fPJJbeA0-ARTkq17k; Sun, 27 Oct 2024 13:11:59 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1730023919; bh=PlCWbGf+DsTcw+oMV449hQH89VWBpXiupTFHDFqbn+k=; h=References:Date:In-Reply-To:Cc:To:From:Subject:Message-ID; b=nQf1MkGDi3ovM50FdF4B+lZUW+4CpJvbsLK7IrJ+ZcPwucUqqmN5YpEAbPa/d9lGw RjECq8pLKD5G83tgWOG6cx1tGAoEgeNR1kWVuK3Q1/LHF6e2LpOwFnlHN67UihwPGJ D9FuEVKwSzfCkbkE1EyUn2x42F0uYS9Mm3fPE7pA= Message-ID: <656f9dad3edba22b3b8ae16b97050d3f189ff1dc.camel@yandex.ru> Subject: Re: [BUG] ZSwap leaks memory upon being disabled From: Konstantin Kharlamov To: Yosry Ahmed , Nhat Pham Cc: linux-mm@kvack.org, Johannes Weiner , Chengming Zhou Date: Sun, 27 Oct 2024 13:11:58 +0300 In-Reply-To: References: <7d873b2682a7513a4d5aad5ac09f5c78851efbb6.camel@yandex.ru> <28352fb75268060f2e78a32325f02e46737a16ea.camel@yandex.ru> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.54.0 MIME-Version: 1.0 X-Rspamd-Queue-Id: EB2F2180013 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: yakk1zyh379e8tio7w7maai65qn6cp5p X-HE-Tag: 1730023919-682107 X-HE-Meta: U2FsdGVkX1/j+gjvYqpRDULJxbfBsn+wvv3A22mO/9cvx/vthaQDDzhrIRzu/t6mRHe4FV6fIvsInkxURRZAhquElR7oV2zFQuKq/DFe53GFdtQhZygKeeHHC3JNgcx0GUHtALnX7ESYKwtoeImcPNx5dNd9EVAreSbDNOoi6mQIEYO367RRKPGtkyoeTuEfZPbMm57ztuzNEjZjX04IQLDDAmyOB1of4fPHFNLEKjUjneXHI1AjLsV0U51lfs/OolsA9zbwwVgD06BJEZ9gISatgkYRLv8eGn1DZ9fCytg0l3TXev8VeYt8eTB80UOapTiQ/QhLHcNrOkcRjbfmr1eTMvKESm+wRFGp8oFdQ3BxQAmwqyFIMikgDcKOqNkBpcfRR2K3ps/wwDrkAb+0sHEb6d6ZmvMqXbqck7AYAh2NzZFPqEErZQTXNFDoIKovg87lZQH1QPECCOTrP1c0AVo9noGFwSHlcNt/pISzul3bTCiqZxDlTuKhYei/co3N82kA/JKY0D/hDCq6kSbEWtnIs7OhMLEr/563ztBwY+VF90Pkr/ERAlww/YcAEzO7NZxlzey9CNqmpDZd+x9ggqRBbtqlFlUx3b4Faaz1shjbJTKZvJvLd3aEJc1Y9L/m+vFFv+/6TUP57y5ER+ElM1OclNj2Q/kVFgOYLft8JtFRr+UXl5/8o1n0d8dcvz2wqObYn/FYqeH8nqARpp6WCkMdTCeUCCCst1GQ86muXq2H8fr0iqHnkfu+ZvKr0iNGmuH028yNASAVxahDgCqBpzGFnWxQaHlKYk3TBA57OVEcYrS0NsX4A2P6qb9aKUxKyI++wBvrAYagEEVio46VVExbZrQVgAa6q3oPL9NQFGZv22sT2EURCxdheNVgToNe+4ZOD1dvyI54fEoS9mjjyFIWnljlav7SHoGei0Jwm3+MOkBxpleYoBdQmOhhI9sqz/6QYp9TQ5jv4RR+dv4 0MHgif9B CzPP2LSZXRn10o1F9RnBjMVUTE6WoUnGHhriqKHOTGeI2rfIuOLOm+9up0g6VyG2LVtI+p20LRMaDe1Gx63l4ZTYEpqsBTL4USVICf0+OqzszAlgFYYzA9MUZa8B3rQGoaZAFwRsIeKqPnDo8MHzYgyMW4sgXEHaaqL8fbn9bJPAr/ImnmFQN3OCeKrMNTS/uKTly6hY8GR0t5otUYg7M8KHu3JBc/C/eNKTRwhraR1cTS/nXbOEUDEFNBrBleAJFsVOFITHmGK5WtlhRPo/MKxXi4Pj03Vy1b79Pd0TWG40Y4rCq0Ys3S9TQAJPF+Ubm2OLYoiCcKzvuR8vaihjicySlE6GIul2TK3PWmMwh6yuwn9dVgMbLhLo2gg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000078, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, 2024-10-26 at 23:46 -0700, Yosry Ahmed wrote: > On Sat, Oct 26, 2024 at 8:14=E2=80=AFPM Nhat Pham wro= te: > >=20 > > On Sat, Oct 26, 2024 at 5:29=E2=80=AFPM Konstantin Kharlamov > > wrote: > > >=20 > > > That was a good idea! The > > > `/sys/fs/cgroup/system.slice/memory.swap.current` seems to have > > > the > > > missing half of the SWAP memory. From my understanding of the > > > `systemctl status` graph `sytem.slice` and `user.slice` groups do > > > not > > > intersect, and by adding up `system.slice/=E2=80=A6` + `user.slice/= =E2=80=A6` I > > > get > > > around 8G. > > >=20 > > > However, I'm still unclear what does this memory belong to. > > > `system.slice/memory.swap.current` is 4.4G currently, that's a > > > lot and > > > I'm not seeing anything that could take so much memory. >=20 > I am not very familiar with what usually runs in system.slice. >=20 > >=20 > > I assume you do not have any proactive memory reclaimer? :) I > > believe > > the top utility can display swap usage by process. Have you tried > > that? > >=20 > > There are a couple of edge cases - for instance, if you disable > > zswap > > writeback and zswap at the same time. We will allocate slots on > > swapfile, and store it at the page table entry, but we cannot store > > the page's content in zswap or the swapfile, so the page remains in > > memory. You're occupying swap space, but are not really saving any > > memory usage. > >=20 > > IIRC, there is also an edge case where a page is faulted back into > > memory from swap, but the associated swap space cannot be > > immediately > > released. This should be temporary though - memory reclaimer will > > attempt to release these pages later on, or they can be released > > when > > we scan the swapfile for slots during swap out. >=20 > I don't think this is an edge case. I think when we swapin a page we > generally leave it in the swapcache if there is no pressure on swap > space. In that case the memory is not really swapped out, but because > it remains in the swapcache it is still reserving a swap slot, so it > shows up as swap usage. >=20 > Konstantin, could you check the amount of swapcache you have, whether > through /proc/vmstat or memory.stat on both user and system slices? Sure =CE=BB grep cache /sys/fs/cgroup/*/memory.stat =E2=80=A6 /sys/fs/cgroup/system.slice/memory.stat:swapcached 434917376 /sys/fs/cgroup/user.slice/memory.stat:swapcached 15478784 `434917376` is a 0.4G, not much. In comparison, `system.slice/memory.swap.current` is currently `4764139520 =3D 4.4G`.