From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9FA6C27C53 for ; Fri, 7 Jun 2024 16:14:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 85E506B0098; Fri, 7 Jun 2024 12:14:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E8DD6B009A; Fri, 7 Jun 2024 12:14:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6612D6B009B; Fri, 7 Jun 2024 12:14:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 42E7E6B0098 for ; Fri, 7 Jun 2024 12:14:32 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BF96D1A0159 for ; Fri, 7 Jun 2024 16:14:31 +0000 (UTC) X-FDA: 82204590342.03.307969D Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by imf26.hostedemail.com (Postfix) with ESMTP id CD9B6140004 for ; Fri, 7 Jun 2024 16:14:29 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=V69luSUS; spf=pass (imf26.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717776869; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VZmv+QIj4qVktftXDjS1d3Nkl+uCdFMuajR2qDI1h+o=; b=A8eIJRr3C8cHNmdGNGIMiJR4tMdzNMiTxH+v9Q/Rnc/3+HIcnwCNb0ws/dCn9qsxPJAYn2 yriQ9dMFFHwtqiVjS5lZQpMRDBZJwRyIGst/YTrB2qhBw73dWI9wIy2WopJmyawjlIEpfW 6RaFmhzWA8xuwaN1G8ABF3r11nxwg2A= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=V69luSUS; spf=pass (imf26.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717776869; a=rsa-sha256; cv=none; b=DmBoNhphVuXm9z+N5VRRAV+KY37QgxZXvK9yOSE3II21vj1Rk/6mvME0QKyG4mQc6Rr9D3 MoYBsxTRdOaTKFMA+kHUsuYf1LlDlHdmrGJ9C4Yp9t9Z6CP68dKCP1Z9LcCrxcML7sfrsM aBwLcPHyx+WMHj05ChOcE3ZdVn4fyVA= Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-7025f4f4572so2586537b3a.1 for ; Fri, 07 Jun 2024 09:14:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717776868; x=1718381668; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :from:to:cc:subject:date:message-id:reply-to; bh=VZmv+QIj4qVktftXDjS1d3Nkl+uCdFMuajR2qDI1h+o=; b=V69luSUSUGJrR7zia/UZZKSvrA6nSARUNqwhUpYucWf5Cr8zVLiN+2qQJRY9jHHcGz Ia/Dc9g5ntHTpI1zd3pIAkEaeEv5pLhJXCY10s50XyP9oZTHJxIjj9YeMutc5NBSida7 9ykrpPCg0hxtWDOZ4R9x3IacELGz8pX1qVDvJqFw73I0RZOoXC/j6o+NyzQihkIPYn9w s+m82XWW6fN1qA7atHujz/UQEJqJnS8iMjdNcF+B2v2xxUKfpKbN8UST5ubom4DCJnPv JojxWNiTnCfs344BajBh0qmA5UlL1oczKwcVurlzzWKUbsGm0UM/6c0EhSwvHAazjC5B VwAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717776868; x=1718381668; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VZmv+QIj4qVktftXDjS1d3Nkl+uCdFMuajR2qDI1h+o=; b=wWq+237De3TPdeMltMdU2t8kYtR8bVO5WD4kFNvuCIyzr8iACbFI/Re1pcPTUlsVrp sMxFpUajQxp3lAJGUHA95ectszqiHmAk/soU/0YDWhfq/LbizkAscz6KJBJ4yeHK6EQj 9GwcMusaIPgH2FnYor/NEmayCASZ1B1sE2tkRuXNcHTq9hl/dARw0joY5TCz8gzQELop I0oxJe4DFjDK2ISPGQxRyfgfJNEQ90EiaCItNtAbDP5GQMx/3r7o/eUZVTm/8K9NDmYL IUoY7uwAmYQxIZoKKJnpteV0/tMK9be3+sMecAN7ZsQ9lK8wMMXj9OWsQFFdPzcBKtur Qp7A== X-Forwarded-Encrypted: i=1; AJvYcCVn9jGuUkRHjKL8A95KDYP2LbHxA1C6iVkEU9xvlx+x9IXgwInf8Jxj+MnHVfDLD2LuGqZ8DHRqhx6yn6dD+3a+Wxo= X-Gm-Message-State: AOJu0YywOKSCmK4x6MEX95pMO9E+7Hg7JdkcFKcyPadAh7kuYerOe7sF 05RPKltDjphimUjkBzJTzd1nh7Ba7gFYruuO0tU3nOEN4XI5UCAW X-Google-Smtp-Source: AGHT+IGcbY/L1VLhL5JeOdeeuFfaV/TM4GOpeK4LrneaBGGHtXwUkFpcuqqFHZOSAgOT1iP2pHl50Q== X-Received: by 2002:a05:6a20:244f:b0:1b2:cd79:f41b with SMTP id adf61e73a8af0-1b2e781a5b9mr4585109637.25.1717776868386; Fri, 07 Jun 2024 09:14:28 -0700 (PDT) Received: from google.com ([2620:0:1000:8411:1225:d298:7d81:144c]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-703fde38722sm2700932b3a.215.2024.06.07.09.14.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Jun 2024 09:14:27 -0700 (PDT) Date: Fri, 7 Jun 2024 09:14:25 -0700 From: Minchan Kim To: Yosry Ahmed Cc: Andrew Morton , Sergey Senozhatsky , Vlastimil Babka , David Rientjes , Christoph Lameter , Erhard Furtner , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao Subject: Re: [PATCH] mm: zsmalloc: share slab caches for all zsmalloc zpools Message-ID: References: <20240604175340.218175-1-yosryahmed@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: CD9B6140004 X-Stat-Signature: bjx6f5c1tqmo38zesuzcrdaidzgx38sm X-Rspam-User: X-HE-Tag: 1717776869-939306 X-HE-Meta: U2FsdGVkX18ByWGkgaTfTok0u0/PDgvVHNAtfrfKnw3rF1pG9UmNvI2AG1Z9x+Azr8bNtJB1SNVLSA6G6RsmUD7bCPdYWmyzVTtJFCHC/TsvVj5XSZukqQIruuIqsRu/5LNyQmS3eeHSq4a9u/XG1WUBPQm89J42PkxHroDiKROkBwsvqth8IPlYuMzMgp9rPcb7mSPvrNMiEIMPAlkKeWAChrwIgCcPAORGMXz4wA0dVqbKB5OX2rvCCNiiBDAJiPZRcUVa7Csxn7oYMmfz8BL/UmWUyQ6oJ3R04xp96wzT9/6kxi7il82HS0esWao1kvY7blIdt0WWMWa69vqLPJlezL9dQEkyiBRtDSThjBJu1XWFfUV9ujcCgwpePE5j+lm80Z4NxCxaULngX1V7lYa+hzqj+qRfmBYMgW7kSFpX9W969HY4lqF+6+CCuHWknMLM0ISi5c0KzQjenna0Sim7yC9Epq4gCCL8Rs9/p9ZHxQ8ickPVMDSmAzpn38egnybClNTUJosnoJQ4QdB6IAKCvjxqG+kdwHTBly/WrtYf3xtXxfCMyA+MPSIos5aPxI8Iko9aWn/vs0l0QnVCaMWz/KAHPLnxD2yMdZ0fNO8pFr9gGH7pBiDnuDLHwMRmhjubw7UbyaCMNOZkGVhRI/2JfCl+YfBLQBVAhdP2aZwErN4CL3k2JVask5iSvdZ3JMCiWPAp0x9VLn3BpGKKABNxpjPTQRONoHQzkWmr45exXcfvQk41cPCr4fJzOFfG14qun21JXiDHM81i4V8ASBIVH6fdgOpdscdnN2sgNhL7DS3xgz9syhSAbvjOmwIQIKQ1Njtz+6ipVNFTSe0YvFoR79J9gjFHuwGgyz67AzSXSxJYRD2khJwhMXMS74hG6TdyhIqKMaoky1p/uVdTOWPGyZW4NDnY6+YKxDX3ys0MMRv1IEZ9ZQ3W7urNKMErv7hMZsUDg5LxHEtc4vR 5Y/hN/I8 xwtg92Yj/XDOJi7iV7dladu5l2jiz0jjqB26wyU4SZJaQ/1ocT5uEyp4N7fkxupdRQmHx+kkvt5ZQm8gSb9tzHxWm5VNSsElUL6eU9I3MMOkbnAvEhTtWeC6FQylIK+PXvEvy2I9i4nNzs8Uuy2wz38QNlhnDzIQiwhN5R4iH4BqDdzWLKB8CNL/dpfMAoDpNCqA6TSrqBbgFe0zvBjLTZgi6hdeET7ZgUpQanMQkuK36WSXHXtJ1xut3Ove5S3bsB7o1F6yZdjx1BEBl0fYnqGEk9B7KYaRq/dRVOTYEx34xiwbd89hXTMEmRryRYAEFMC1G/xwsfW/cDr4i/j5d97KazQkUWbCMGdVY2l/RVkVTWF/aIdOjcdiNcMIQBld0qqjjG/JH320WuxEDta5KueKmOoRQy0UiBkM2Xygwn5aqZUMYX7LOHbC/pucgTQDb9ZtmHFe06Mh5lRhQ6IdKieU/kv5fcLJbWxWUGY8ZT5KvoDMMiB1/6ams5W7O9A5rUvBGCRY9jPd2wzITS+cFMVBj0nVTQD8JK6dRod+0HwbOVhLsYU5F/jXKK9lpjX7ImbOCNqWI+tNrhDqWVEoGxCHfaEVntSKJIZsEJ6odRTii/iGGHE6yGI2DnzYw6Drg0t/W0KTVwd9ZZ8WvIarqpssbWpwaL47UwnZlcHPGnOAVqPW+S0XSp+kCmSDhdW3O+bzsSA1uhREHRQNoPAQVUpmXgAjoV8xvptRcbxtxpi3W2uC++PLsoGRqPTubzNPNAstI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jun 06, 2024 at 04:03:55PM -0700, Yosry Ahmed wrote: > On Thu, Jun 6, 2024 at 3:36 PM Minchan Kim wrote: > > > > On Tue, Jun 04, 2024 at 05:53:40PM +0000, Yosry Ahmed wrote: > > > Zswap creates multiple zpools to improve concurrency. Each zsmalloc > > > zpool creates its own 'zs_handle' and 'zspage' slab caches. Currently we > > > end up with 32 slab caches of each type. > > > > > > Since each slab cache holds some free objects, we end up with a lot of > > > free objects distributed among the separate zpool caches. Slab caches > > > are designed to handle concurrent allocations by using percpu > > > structures, so having a single instance of each cache should be enough, > > > and avoids wasting more memory than needed due to fragmentation. > > > > > > Additionally, having more slab caches than needed unnecessarily slows > > > down code paths that iterate slab_caches. > > > > > > In the results reported by Eric in [1], the amount of unused slab memory > > > in these caches goes down from 242808 bytes to 29216 bytes (-88%). This > > > is calculated by (num_objs - active_objs) * objsize for each 'zs_handle' > > > and 'zspage' cache. Although this patch did not help with the allocation > > > failure reported by Eric with zswap + zsmalloc, I think it is still > > > worth merging on its own. > > > > > > [1]https://lore.kernel.org/lkml/20240604134458.3ae4396a@yea/ > > > > I doubt this is the right direction. > > > > Zsmalloc is used for various purposes, each with different object > > lifecycles. For example, swap operations relatively involve short-lived > > objects, while filesystem use cases might have longer-lived objects. > > This mix of lifecycles could lead to fragmentation with this approach. > > Even in a swapfile, some objects can be short-lived and some objects > can be long-lived, and the line between swap and file systems both > becomes blurry with shmem/tmpfs. I don't think having separate caches Many allocators differentiate object lifecycles to minimize fragmentation. While this isn't a new concept, you argue it's irrelevant without a clearcut use case. > here is vital, but I am not generally familiar with the file system > use cases and I don't have data to prove/disprove it. The use case I had in mind was build output directories (e.g., Android). These consume object files in zram until the next build. Other potential scenarios involve separate zrams: one for foreground apps (short-term) and another for cached apps (long-term). Even zswap and zram could have different object lifecycles, as zswap might write back more aggressively. While you see no clear use cases, I disagree with dismissing this concept without strong justification. > > > > > I believe the original problem arose when zsmalloc reduced its lock > > granularity from the class level to a global level. And then, Zswap went > > to mitigate the issue with multiple zpools, but it's essentially another > > bandaid on top of the existing problem, IMO. > > IIRC we reduced the granularity when we added writeback support to > zsmalloc, which was relatively recent. I think we have seen lock > contention with zsmalloc long before that. We have had a similar patch > internally to use multiple zpools in zswap for many years now. > > +Yu Zhao > > Yu has more historical context about this, I am hoping he will shed > more light about this. > > > > > The correct approach would be to further reduce the zsmalloc lock > > granularity. > > I definitely agree that the correct approach should be to fix the lock > contention at the source and drop zswap's usage of multiple zpools. > Nonetheless, I think this patch provides value in the meantime. The > fragmentation within the slab caches is real with zswap's use case. > OTOH, sharing a cache between swap and file system use cases leading > to fragmentation within the same slab cache is a less severe problem > in my opinion. > > That being said, I don't feel strongly. If you really don't like this > patch I am fine with dropping it. How about introducing a flag like "bool slab_merge" in zs_create_pool? This would allow zswap to unify slabs while others don't.