From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE6DAC28B28 for ; Wed, 12 Mar 2025 05:19:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 21101280002; Wed, 12 Mar 2025 01:19:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1994B280001; Wed, 12 Mar 2025 01:19:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0117A280002; Wed, 12 Mar 2025 01:19:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D3D5A280001 for ; Wed, 12 Mar 2025 01:19:13 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B4161C09E0 for ; Wed, 12 Mar 2025 05:19:13 +0000 (UTC) X-FDA: 83211745386.24.1358F5A Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf01.hostedemail.com (Postfix) with ESMTP id 98E6940009 for ; Wed, 12 Mar 2025 05:19:11 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=JwnRTZfl; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf01.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741756751; a=rsa-sha256; cv=none; b=wqGMDBDz3j5oUGNSTatYjsMuIJQ9oqdoJWkxRXB7n8XuxlFwHveHhWovy17XdOQ++YKC99 T+JBNfpNcv1FCkmS+J/+6g/ij7Z7Cl624LGmwu6T4SE3kPM1qTsSdxgu9JCz9hM+1avO4A iOt1G5WSuG0fHN4+MuXG/Ln3/CbElVY= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=JwnRTZfl; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf01.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741756751; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MatbiHDO0DCUqMkCHmZ0elSs7NnL3O74U74cu1zQUug=; b=kADjGo4IBho/gGQAL7un3qK065OqwwMfl2T+Lc+6VTkJmeh+Dg9pbkUOnPzcfr24z+asQV 4QkFrpPBg57OZFZPeaA7QmY4EdTBX6cW8vgp3HvFZSpW9Y6jEQJJD6ByAdya9WqyCnf5zU ZkI6PpDRUAdxm/Y121FMFYwQarYJq0I= Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-22423adf751so92759205ad.2 for ; Tue, 11 Mar 2025 22:19:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1741756750; x=1742361550; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=MatbiHDO0DCUqMkCHmZ0elSs7NnL3O74U74cu1zQUug=; b=JwnRTZfl7/5yLUJwf5Ki72Gy4VNZVIQOKqbB+hSLw8jEaH2Ov2ED+g79nC58tnnG5+ Z5555gs6plnKcF1sKRAVhw2MIn9Cuh5sXYIwjpJhhEP0RYBnvmA2Pbkl+zksGDdnPQcj VNk1von1I/v157M0MXEzLjy2cB+GYjdHjw93I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741756750; x=1742361550; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=MatbiHDO0DCUqMkCHmZ0elSs7NnL3O74U74cu1zQUug=; b=cWEZiTaHKmkwySn1FhiJMQmVg5ESkcWtsN9bAVpwIaKe43pKwXFc5VutUck1TbYhTq /UKVniOQKGbd80PbzNSKoSS0kaaSYkAFOjZe1EP/ND+q3xe9HLqRtQ5Yjj9X1A/E0Tny QsYnHI7Fa0gOvDr5lUfIVpL1otdWObU9j6D3YTZascERwymAwaB0IdEBSsIZ7TDnd79p Gke1XUfzoL4s81ZX/ZEa2cNndKYxYOlJfy55CmhqXHNN7darKSXn30cLXGZ6r6j0RFHB sctVFce7Bz8uXwZX7ZjF+Lwngd2l/m7W22hcyBmpv5IqHkrkvljhVs41xq49QaCiZNJF k9sQ== X-Forwarded-Encrypted: i=1; AJvYcCVwwUrmKtZQVu1qRc9cIPs2cLygLhMcLqaW+z1TdeSuxqzlqlc1433p7t9I5KEo9E/EgTGYUmZ+8Q==@kvack.org X-Gm-Message-State: AOJu0YyE6786/ihlaLWx1j+qQePuZEwjNHEyxuyggIRj90GMRQKLs4mK xIiJpzL2UQ3M8Jv0A6laPpxg0uK+S5e67Qj/mGnZvyrZouHHjD1mh8VhLxuScg== X-Gm-Gg: ASbGncu3bwoNgqNdsFXVyldgs13uSSmYFP9RhIlUrcy1AGOePHCAGnLvIB7jno+Wiul mZrpcRWH1rYE/B2h2T63tc0L+f4GMP2BjsTz7zzn5PcCYK3WmNcXFSn1hWFUf2vtf152heE6J6X WrmiiDYLCjHM7JD83wlkJGvvfxEkDk3lIcu/sc87blLflG1lYYnVrxgXQdXytFLwg2JPaS6aFFx Otk2Luux3aKlhUeQ54qkYLDhMdFT8IkTvHgft79ZYm39DyhdcgI92DnXuhsNXMwcsvPo8caTiNx D6ryIxVXsZVx6KhRBCxuTj9bXp30jOuwtcEpeJ0L62v90x61J5Drb4y+fkQ= X-Google-Smtp-Source: AGHT+IFe2j1+GACtRUcv6p7Y8qMgiJICjpvUMqBu5x7qOSpIgXxXSdQ6F+GhVarFS/5mqmKsiN8Zkg== X-Received: by 2002:a17:903:46c8:b0:224:1c41:a4cd with SMTP id d9443c01a7336-22592e207damr74581435ad.3.1741756750550; Tue, 11 Mar 2025 22:19:10 -0700 (PDT) Received: from google.com ([2401:fa00:8f:203:713f:ff5a:f7a8:2aae]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-224109df232sm107260695ad.41.2025.03.11.22.19.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Mar 2025 22:19:10 -0700 (PDT) Date: Wed, 12 Mar 2025 14:19:02 +0900 From: Sergey Senozhatsky To: Qun-wei Lin =?utf-8?B?KOael+e+pOW0tCk=?= Cc: "21cnbao@gmail.com" <21cnbao@gmail.com>, "senozhatsky@chromium.org" , Chinwen Chang =?utf-8?B?KOW8temMpuaWhyk=?= , Andrew Yang =?utf-8?B?KOaliuaZuuW8tyk=?= , Casper Li =?utf-8?B?KOadjuS4reamrik=?= , "nphamcs@gmail.com" , "chrisl@kernel.org" , James Hsu =?utf-8?B?KOW+kOaFtuiWsCk=?= , AngeloGioacchino Del Regno , "akpm@linux-foundation.org" , "linux-kernel@vger.kernel.org" , "linux-mediatek@lists.infradead.org" , "ira.weiny@intel.com" , "linux-mm@kvack.org" , "dave.jiang@intel.com" , "vishal.l.verma@intel.com" , "schatzberg.dan@gmail.com" , "viro@zeniv.linux.org.uk" , "ryan.roberts@arm.com" , "minchan@kernel.org" , "axboe@kernel.dk" , "linux-block@vger.kernel.org" , "kasong@tencent.com" , "nvdimm@lists.linux.dev" , "linux-arm-kernel@lists.infradead.org" , "matthias.bgg@gmail.com" , "ying.huang@intel.com" , "dan.j.williams@intel.com" Subject: Re: [PATCH 0/2] Improve Zram by separating compression context from kswapd Message-ID: References: <20250307120141.1566673-1-qun-wei.lin@mediatek.com> <32d951629ab18bcb2cb59b0c0baab65de915dbea.camel@mediatek.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <32d951629ab18bcb2cb59b0c0baab65de915dbea.camel@mediatek.com> X-Rspamd-Server: rspam07 X-Rspam-User: X-Stat-Signature: 85f5ddd9h94oeqru9br86m9i8uan7487 X-Rspamd-Queue-Id: 98E6940009 X-HE-Tag: 1741756751-674488 X-HE-Meta: U2FsdGVkX1+aKGa+6YhbB4vo/6a7hcB2WbtWL2xnSjj9JrQ++fF3W2VTn3q9Yc+FARlSiLxcCg4sGfPQGfPPph+Gf/yOybkYwHMJ7WZh811+6LRbTCmLY3uQcyTZNLyngwsRPC1nPpqrGmJNjRiYwJOyNcwGLmeUx6HoPWi5jcsQPS20vYnrDxflJjGP6ARLKJufys22dNFBbriMu8bu4vQq2adGykBEf5smF8tGBitc1Ev1mJGcT1OS7Y4tcHXumW+CbPmzc5ugt2I204Tx87P1VX1OmZl1r2aU8xmkULq8ZOeE85xBtWqmEdHC+Gu96e6e2pC+/ljIgvigzfG80GdsbD+GdOGo7MI/2brAMihbpFB4Wl8EC4mcHHDIBMNp+FgaWlXSDPVIv0N1fNp2jKQsj+lzoFi9dTJg2kFZOzJBRAIosIduFvgp+667opKu6QvxRpCm/Wrg5wSu19Z9F949SM/Ls7cBsnt7BdYQQewQk7Y1frPMbCTss+jm+Bm3pcdO1KSfPpMBf23DKsooXquVl6znLCEsB2ndLyYoC3RZ6+33DPVhpYasHU9eLTzHCjnMuDSmFyNcxnXB6CRZwDozUYgWQ2P1ojEoHNrSo3LWWkkPGmH3ZPJfU0wLpm3VAGLyL79eOW6VSgTvFf85ZzKXA1r8EZOoYbFBKQrghs/DAwe8o3v4fYcP6EryJsI3Iqs7hcAD+zRjqitsuqZeflNsnJTl9YPo9iRwZ9pZ6l8liG87p6N6Egielj0c6rROEpQi5VJ+BhQjdmSWsqsyc3OoJGL1UNlZ/VeMBQ+Rfjq5QJQEG/uTo6j2HS/VLi8iaILha/fw6xXAG2vOCaBlwsBgsgxpnc+LJFe57cC5N1l4GvGr0a0fRv+w5dBve3G7GMsL16Hj9Zue1tt6oXMLKyhvtFXca/sKgtT3QuwOazAzYMXGzoSrpEDQiFgnzu2l/DcUiXx2VZ4fflj/ZFf C/GtKnfF FTgwoBn8P7b2yAdts7Q01fL+KKdmS/sJ1Jrws8865Z+6YOnJI2Lu6hi7Zjku56tgaPxS6gfT2ItXTxejJljuUwKcU6Ub+g5ObjN0jhn8aNrurYtGO9xfJJb1zWFN+9mtzxWlPSA8Zc+eoTVCsrMofe5iDYvf/sgbJF74a3cxfWBEW2VePmgVs3dDyzqydbws+gQ/HlSAV9V9F27jLFXUgtnObowmB2XOf4MBNP081SmA3HmokOAC2cEQneVzQHMZpwVI4UcnD4M0a6gNVNZJTUYa3R8+7AIHQngj0y1V1jgAG/kk6aUWPfPo1waDGFvSQ7hzAUQk9KUzYhlTNFapOM9WIMRc0K8d6Exd9dZiLBMKhEquczaAEZI72z5gI2y6414f+BT1nViGOOTSNcXmaVIirh9pDdK2oPQNdynV72mRE0SZqEcKLoyhwVjJg3/C6Jv8MoIKA9UrZ8I9inOy6Whe/SQm/7GvBxD7QbUHc2wURyqoRGyL+wLH7lw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On (25/03/11 14:12), Qun-wei Lin (林群崴) wrote: > > > If compression kthread-s can run (have CPUs to be scheduled on). > > > This looks a bit like a bottleneck. Is there anything that > > > guarantees forward progress? Also, if compression kthreads > > > constantly preempt kswapd, then it might not be worth it to > > > have compression kthreads, I assume? > > > > Thanks for your critical insights, all of which are valuable. > > > > Qun-Wei is likely working on an Android case where the CPU is > > relatively idle in many scenarios (though there are certainly cases > > where all CPUs are busy), but free memory is quite limited. > > We may soon see benefits for these types of use cases. I expect > > Android might have the opportunity to adopt it before it's fully > > ready upstream. > > > > If the workload keeps all CPUs busy, I suppose this async thread > > won’t help, but at least we might find a way to mitigate regression. > > > > We likely need to collect more data on various scenarios—when > > CPUs are relatively idle and when all CPUs are busy—and > > determine the proper approach based on the data, which we > > currently lack :-) Right. The scan/unmap can move very fast (a rabbit) while the compressor can move rather slow (a tortoise.) There is some benefit in the fact that kswap does compression directly, I'd presume. Another thing to consider, perhaps, is that not every page is necessarily required to go through the compressor queue and stay there until the woken-up compressor finally picks it up just to realize that the page is filled with 0xff (or any other pattern). At least on the zram side such pages are not compressed and stored as an 8-byte pattern in the zram meta table (w/o using any zsmalloc memory.) > > > If we have a pagefault and need to map a page that is still in > > > the compression queue (not compressed and stored in zram yet, e.g. > > > dut to scheduling latency + slow compression algorithm) then what > > > happens? > > > > This is happening now even without the patch? Right now we are > > having 4 steps: > > 1. add_to_swap: The folio is added to the swapcache. > > 2. try_to_unmap: PTEs are converted to swap entries. > > 3. pageout: The folio is written back. > > 4. Swapcache is cleared. > > > > If a swap-in occurs between 2 and 4, doesn't that mean > > we've already encountered the case where we hit > > the swapcache for a folio undergoing compression? > > > > It seems we might have an opportunity to terminate > > compression if the request is still in the queue and > > compression hasn’t started for a folio yet? seems > > quite difficult to do? > > As Barry explained, these folios that are being compressed are in the > swapcache. If a refault occurs during the compression process, its > correctness is already guaranteed by the swap subsystem (similar to > other asynchronous swap devices). Right. I just was thinking that now there is a wake_up between scan/unmap and compress. Not sure how much trouble this can make. > Indeed, terminating a folio that is already in the queue waiting for > compression is a challenging task. Will this require some modifications > to the current architecture of swap subsystem? Yeah, I'll leave it mm folks to decide :)