From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF856C282EC for ; Tue, 11 Mar 2025 09:33:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B7CF2280003; Tue, 11 Mar 2025 05:33:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B2CBD280001; Tue, 11 Mar 2025 05:33:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A1BAA280003; Tue, 11 Mar 2025 05:33:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 84431280001 for ; Tue, 11 Mar 2025 05:33:17 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id BAB6CC1BE3 for ; Tue, 11 Mar 2025 09:33:18 +0000 (UTC) X-FDA: 83208756876.16.655AE7D Received: from mail-ua1-f49.google.com (mail-ua1-f49.google.com [209.85.222.49]) by imf21.hostedemail.com (Postfix) with ESMTP id D6C0B1C0005 for ; Tue, 11 Mar 2025 09:33:16 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Yp66IBJw; spf=pass (imf21.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.49 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741685596; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Oci9eEq+toTA4eT0HbbNQG595NfpjyU1+E2PAns3Nzg=; b=VdbBkJmGgdoHc441P2jFK1TDJ++1VgoEyvZazybz059xcyJ8vGtg9TLIyus3bahdyBA6Gh tGULqKNWdm7xJPQUf4opEFyqgKD4Nr/GKfDc8NAjdyWePQkImvPjflJX6iSctIYIIauZoF VHiQp5ApXSpMt+g2t/f1LA+npMPOfbE= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Yp66IBJw; spf=pass (imf21.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.49 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741685596; a=rsa-sha256; cv=none; b=0vP0vbkUwzuQ/gjLYlWbX12OzZeUKS6tzYWjsKmtTYtKJNcVHHtJqmXB1YwHN870xbJyDX 9TBPK+vPUv7qndbAE7WPCIuTJsjQYF64QhA8jMN+8vD2DurqheZQfPBr/fH2wHcwFktAHb MRNlxl9t5pdP0nATPDQzGOyZoYooKUY= Received: by mail-ua1-f49.google.com with SMTP id a1e0cc1a2514c-86d75f4e9a1so214510241.3 for ; Tue, 11 Mar 2025 02:33:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1741685596; x=1742290396; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Oci9eEq+toTA4eT0HbbNQG595NfpjyU1+E2PAns3Nzg=; b=Yp66IBJwb6NRlw+6b6EOCH8TjH3UuaqaOzP3QUszzAxthLacRT0CvRtgYk8YKdQ3D7 NkiTf0sm7ajnKF+CnYy1ly5bOC4RQY4gPk2IJwRu0XLKoMyPVRRTWo4Ub8NUqvWqK+XV smaI6ZJs4QF62M87FlR0Het1W99iyhBsd4lCbMQ+tjY378aEr8IEwcWMKmIq5w3OhEE2 +VrATpbzjOPJZmFZ9F1JCkdxsJFItrAGQs85ti80XLve74w/HevOJjPg2OyS1BuUQsau dr2+x2qw7IUHqY0I1+sUVCHotB9BRH08sT+g2fRQbwDWV7vMMkjh4zfrlpabQz/EoZsv 5V+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741685596; x=1742290396; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Oci9eEq+toTA4eT0HbbNQG595NfpjyU1+E2PAns3Nzg=; b=XbvX310Mn9YrGeUoXBk5qOD/uztG86KeB6C/7ZUIaNj3IbbYOwyG4VEH+KIdWqYLyI c2wZxFSQTT4xnVNU9NaDKeQT56IV4KRIF8d6hMZQiIjZJ3OAz3hgH8yJB/L+Lqi3Vhtu F6/se7O0T7GxLfdD6mqf4WpPWwRbn/NHxr24DTniMWNltpqEzS0yO+11wnzrcLhfT6yf K05o0T5rQxmgC1ERrh/MeBLihNlNTytENx4qZhJ0aPJPCn5izc3imqTGOLsSwrXhweGe wM0hOynIZv6gjeJ9sDxgx7OhoTgvB+Zxn+1RSaqi/nmcmzugwLIWiD3PWq/oZyHCLyVp 6fzA== X-Forwarded-Encrypted: i=1; AJvYcCX16Jbqd4YSOb0NL6tyBPBoxKtNvT0IozxTxokcfm1e0bEVp1Ohvee0qO6MtLQaiI6MzoX+10MeXw==@kvack.org X-Gm-Message-State: AOJu0YxvN8sVVhROZho8PQ3dIad9IH1sts27rZsn73Q4BKNPWhFOtapt I/J8009mvtdPBWNi2+dd79tOC3EbkJQr2yCrmHjmsR0uGmWVwsn8Qlo+sTdrKunVh397TIQnnFY MYcv78ndZaXbUsLJ8I2gE0XtByvw= X-Gm-Gg: ASbGncuz8BmTgWONTMvETkgT8MfnfA3yIRUuGTbjW2OXMq7fAz2iDpOFxPA8CBLQlMk RC2pilqYXLoYSGZpJr43nG0T7ZpE+W9LKuGOyh0ewmf/7Ik3tZlIzYgzVc+11n2SH7r7JuDNtGE OdUWyBXvCyaZcQKztWxelqypj7M8ULsF1mceUa X-Google-Smtp-Source: AGHT+IFjbF2BT/V8wuSBTDQZbf6t/hT/Znk/xR8O11b3YilMHbEGP/m6CvuSuPbsGl833OZDuyeAqkTBH4Ybr+MgCeE= X-Received: by 2002:a05:6102:509f:b0:4c1:9e65:f904 with SMTP id ada2fe7eead31-4c30a718b91mr8809313137.23.1741685595784; Tue, 11 Mar 2025 02:33:15 -0700 (PDT) MIME-Version: 1.0 References: <20250307120141.1566673-1-qun-wei.lin@mediatek.com> In-Reply-To: From: Barry Song <21cnbao@gmail.com> Date: Tue, 11 Mar 2025 22:33:04 +1300 X-Gm-Features: AQ5f1JpaKY-TJruiEfKLncijqOvg19zgNrZm_fVI7uamNPGD1sBBaPT6nlX2PNg Message-ID: Subject: Re: [PATCH 0/2] Improve Zram by separating compression context from kswapd To: Sergey Senozhatsky Cc: Qun-Wei Lin , Nhat Pham , Jens Axboe , Minchan Kim , Vishal Verma , Dan Williams , Dave Jiang , Ira Weiny , Andrew Morton , Matthias Brugger , AngeloGioacchino Del Regno , Chris Li , Ryan Roberts , "Huang, Ying" , Kairui Song , Dan Schatzberg , Al Viro , linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, nvdimm@lists.linux.dev, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org, Casper Li , Chinwen Chang , Andrew Yang , James Hsu Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: s6imehm9ywi3u8cox3x5pad6am1hh6jg X-Rspam-User: X-Rspamd-Queue-Id: D6C0B1C0005 X-Rspamd-Server: rspam01 X-HE-Tag: 1741685596-861724 X-HE-Meta: U2FsdGVkX1+F2fXBQrELzJ91RmSM6R9SqlTBK0lIKpL7cnWYB8hckiKc35uuHfe1ojjk8eHRM/4xqih3MhxTNVtRxvecbBseNE/koeMD62SKdBE1MrwJXIyBu4ZM7dNYCDrFYRf96AWWOTXGt8PAVrIx/c5Mcb5R1ophewRUWD3yUi0zVHs6F5Idciapsi2R7JOAaV8cqz06QGP9/+GM9iYz6lmq6q63sBtqvFFKBlF+1SL7ZRejrHpBLa4hyZJy6YX0AJABvP8m7z2Kv4qu/Zh+EQR3JvDITC9XSkq/NoT9E5zs7kMGHyzRVZk95hEBaoqnPiRhz6vA+OKkc7vw4zxOFnf85F3TwMMkJCpTGX5JFR3SwzXlSlpS1y2FgZyOWkY+9+vLsQpq/jk6ry60JOxQctxeiSDYX3CkdNZ3/cITmp0I9/ctKzTaQQBpOr7ifpg1Jlwaib3MuziLyJj59wSHcUILzHjnICVIuBsTMsLsdtXXcchc6qf2nji0bBBR0wPPqUDaTjz/PbpqarlShWK4efNJHg2sZCrDvW001v4lLGBds8L75iOwG79UR06m1wEwJ0Zgesji2kPmdqOAiFzNKHY7PWtbnNsgNvppq2nP541CcUxsJcvwFC7haIuxLh9x2fwZrLfhKtUV3R476J9apuY2UGI8KApEN4Nwo9oH1EGK/25IaJOmhb6d+pL0LLrIWJuvHnm0f84vEz90BjEPY4dOa7/Ga0zO5RRi9Vc5L102aGVLlMe137xYC9ep4+TDsAAHrJlDXoYqADsnXbWkqfTd92pS+vGrIhc9uW1GLARpN+F1nxsfd9SNuCjCrbUl6phub88R+AR0vBH9f/+W7GaE7iul0UuUOK90SaQi+7slUlsIpwUD0RcxQroOfMXHLn+Z0bNOvC6cMqs+aMZlCf1RmviPUj0vgdvDNAM8qf2Bt0rP95uLfeiMK19YLoAEOTepnD5oTQYQiIz rNqSVzMi +zsDfktenIOh+S0Ux3i5zDZlSShppy2mmJvvtc1WUil5xZMR+gNOgP20no0eOrMZkqjCQRUpFGuhnD7ehNZuZtrosAP/9Jqbhx+gGBnO12QjDX8GOm6jJjyDO3YTsM/HAIhh57Qpkj8+WErM+P0gJKsEXf63jmNZx0NzgTnEfxgZVhomyhXIM8/lUTfq8ODZ3ZZZLWwW9brGVwAG0z5cc2vGD3CnREh5ZHQ0p8+WH3GWhu+ogoGKSo1JQhNv/sCMJmeoWCWE5EAXZ5XizASt+ZkvWx8w4BlU7OGqv/uZluPuubl5RQfQFtiwHhm966UCtJRiUnE78JM3PtusJZbqeP7COmcFwpzXRUlrHjWbsE/uLUqBvgY5FSzwXbFQ4VArOIazB3aEkVySqyeYn5+vVA+/vb/Ye9Lk4GcZPvl0duQuKOenIE2anfG33G5b/iNYUHtbxSR0PPCAGNzrOlb9lz8068w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Mar 11, 2025 at 5:58=E2=80=AFPM Sergey Senozhatsky wrote: > > On (25/03/08 18:41), Barry Song wrote: > > On Sat, Mar 8, 2025 at 12:03=E2=80=AFPM Nhat Pham w= rote: > > > > > > On Fri, Mar 7, 2025 at 4:02=E2=80=AFAM Qun-Wei Lin wrote: > > > > > > > > This patch series introduces a new mechanism called kcompressd to > > > > improve the efficiency of memory reclaiming in the operating system= . The > > > > main goal is to separate the tasks of page scanning and page compre= ssion > > > > into distinct processes or threads, thereby reducing the load on th= e > > > > kswapd thread and enhancing overall system performance under high m= emory > > > > pressure conditions. > > > > > > Please excuse my ignorance, but from your cover letter I still don't > > > quite get what is the problem here? And how would decouple compressio= n > > > and scanning help? > > > > My understanding is as follows: > > > > When kswapd attempts to reclaim M anonymous folios and N file folios, > > the process involves the following steps: > > > > * t1: Time to scan and unmap anonymous folios > > * t2: Time to compress anonymous folios > > * t3: Time to reclaim file folios > > > > Currently, these steps are executed sequentially, meaning the total tim= e > > required to reclaim M + N folios is t1 + t2 + t3. > > > > However, Qun-Wei's patch enables t1 + t3 and t2 to run in parallel, > > reducing the total time to max(t1 + t3, t2). This likely improves the > > reclamation speed, potentially reducing allocation stalls. > > If compression kthread-s can run (have CPUs to be scheduled on). > This looks a bit like a bottleneck. Is there anything that > guarantees forward progress? Also, if compression kthreads > constantly preempt kswapd, then it might not be worth it to > have compression kthreads, I assume? Thanks for your critical insights, all of which are valuable. Qun-Wei is likely working on an Android case where the CPU is relatively idle in many scenarios (though there are certainly cases where all CPUs are busy), but free memory is quite limited. We may soon see benefits for these types of use cases. I expect Android might have the opportunity to adopt it before it's fully ready upstream. If the workload keeps all CPUs busy, I suppose this async thread won=E2=80=99t help, but at least we might find a way to mitigate regression= . We likely need to collect more data on various scenarios=E2=80=94when CPUs are relatively idle and when all CPUs are busy=E2=80=94and determine the proper approach based on the data, which we currently lack :-) > > If we have a pagefault and need to map a page that is still in > the compression queue (not compressed and stored in zram yet, e.g. > dut to scheduling latency + slow compression algorithm) then what > happens? This is happening now even without the patch? Right now we are having 4 steps: 1. add_to_swap: The folio is added to the swapcache. 2. try_to_unmap: PTEs are converted to swap entries. 3. pageout: The folio is written back. 4. Swapcache is cleared. If a swap-in occurs between 2 and 4, doesn't that mean we've already encountered the case where we hit the swapcache for a folio undergoing compression? It seems we might have an opportunity to terminate compression if the request is still in the queue and compression hasn=E2=80=99t started for a folio yet? seems quite difficult to do? Thanks Barry