From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74430C27C65 for ; Tue, 11 Jun 2024 22:13:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D3FB36B00F7; Tue, 11 Jun 2024 18:13:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CEF3A6B00F8; Tue, 11 Jun 2024 18:13:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B69336B00F9; Tue, 11 Jun 2024 18:13:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 950666B00F7 for ; Tue, 11 Jun 2024 18:13:21 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 137B2121052 for ; Tue, 11 Jun 2024 22:13:21 +0000 (UTC) X-FDA: 82220009802.06.1C5F9A2 Received: from mail-qk1-f175.google.com (mail-qk1-f175.google.com [209.85.222.175]) by imf03.hostedemail.com (Postfix) with ESMTP id 2FAA620010 for ; Tue, 11 Jun 2024 22:13:19 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=fC9PKUNX; spf=pass (imf03.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.175 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718143999; a=rsa-sha256; cv=none; b=fTUsjn7Fq8EJIf/9WwB42k2/1dZQqrX712n6yExNTDiBgpMHU8dYtpXO7c8PB96vjQHXeR 95gkX8eNzPkoHm7vlNVFVrN+rnIEJ7VNw+j2nusZfAnRcVtHTePJpwbUghR3IWS+cQNfZr 8iQyU9+6WQJ5cR7ukCPGZmg8QZubuoo= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=fC9PKUNX; spf=pass (imf03.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.175 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718143999; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RPKZqSlAbaQuMLhwthwfzm4L0DNHV7L8sJdV9gURtUM=; b=PCXVjeEulGLNkLIyzGXduWbUsdzflDO+M6QvUgywR9ShLwuBzPtU23m9vWRWChNSiKx0OV NYDhkPtD0sIH2swwycYedxHmVTSrmw3Wc7nd3X4JeCiGfDMYbT2CqRUGMthLyx5AvwLd80 knUhRhqsLC4A0QHwb6hKfCIRicI6Fx0= Received: by mail-qk1-f175.google.com with SMTP id af79cd13be357-7961fb2d1cfso184122285a.0 for ; Tue, 11 Jun 2024 15:13:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718143998; x=1718748798; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=RPKZqSlAbaQuMLhwthwfzm4L0DNHV7L8sJdV9gURtUM=; b=fC9PKUNXuG+XJmfntcwlOojN30AuMXqK25c2Zh86xefNAyL47vpUqe2wwdoHcbctxG nYa6BrxaInIjrQL1/bXd57RwByGvCyKoRy9H4kotiFxm391zYDE6WEODmdqsnBn5SpZH yPemNHZZ1wbyW61qv4fG+cyZnflXrMI2uvPRy4dEYNaAHSLfj83Fu/rWS7vXd3TsxEz8 WJjTLxc3LpMTn+Hv+uQSihsDt1nfVW8Ta6EPvJRGSICTmAaux0g/jKvuCor7GE9x02hY oX3cXtoLUg6+VHTNslbyQItKlxUQTwcKizwolsairltI0k1xeLMZVhR2kdUK9ZIgPTUm EkdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718143998; x=1718748798; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RPKZqSlAbaQuMLhwthwfzm4L0DNHV7L8sJdV9gURtUM=; b=XV448YyQlaPZVJI100/6teW3LguMOjqqV8cKi+O0ne6ArpSQBrOWJ9nZ17TjzpKS/n Ai0MRStu8Gm4TpLiurONzXHAKroLgwu6yoiNA5GKeYZGwDE13NhdrzQbYVU21BMxVJYg CBZiIBzY9HLzp6pqKzUU+VDbMDndodofLxY5w2kbkEGZ4zfUdU0KrysjTKUpyIeoX6hc ThX4xytMi21ColescoZuRRceR/ADkiYTIkbWOM04/KXfzPHt99CMnOkP8H6CuRvHv84q bDX4yO607ck7Fa8UJ76tV9e91/y/vydjjk4SNY/riAmOLI0VKMXekICDOs09RJ/85l17 vJlQ== X-Forwarded-Encrypted: i=1; AJvYcCVaBOHTHbne1eapCFOOMC5+s94St69YldOdfuKMi3MXjzUn0Mqd6CzbGe5nHHCHsLzNbFyijX4gUuyhJAVdAw3BBG8= X-Gm-Message-State: AOJu0Yx0JBbuBzxwd4PqblZB558tdQrMGm96Nbf444yfDkriZRBM1SGF gOnxDoTQsuDSnOxnB4jkbHCjrHfi7k40NdvDWNxbH+7uuSpArPV6bWB/z2GmObdrWGCAgEj+zf5 4oA+UgEOECm31GxTv0LDTNhDx124= X-Google-Smtp-Source: AGHT+IHRKW+4TAFuiyd6TaF9HhkOMg9tjED6ruWvkwLADjsp/TEZ1jg4S5wZWE6+/Q3li0FjiSMZUY57hDIlWn0R1tY= X-Received: by 2002:a05:6214:2e44:b0:6b0:6525:4115 with SMTP id 6a1803df08f44-6b191d472e3mr18116d6.19.1718143998205; Tue, 11 Jun 2024 15:13:18 -0700 (PDT) MIME-Version: 1.0 References: <20240304081348.197341-1-21cnbao@gmail.com> <20240304081348.197341-6-21cnbao@gmail.com> In-Reply-To: From: Barry Song <21cnbao@gmail.com> Date: Wed, 12 Jun 2024 10:13:06 +1200 Message-ID: Subject: Re: [RFC PATCH v3 5/5] mm: support large folios swapin as a whole To: Shakeel Butt Cc: Chuanhua Han , Ryan Roberts , akpm@linux-foundation.org, linux-mm@kvack.org, chengming.zhou@linux.dev, chrisl@kernel.org, david@redhat.com, hannes@cmpxchg.org, kasong@tencent.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, mhocko@suse.com, nphamcs@gmail.com, shy828301@gmail.com, steven.price@arm.com, surenb@google.com, wangkefeng.wang@huawei.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com, yosryahmed@google.com, yuzhao@google.com, Chuanhua Han , Barry Song Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 2FAA620010 X-Rspam-User: X-Rspamd-Server: rspam12 X-Stat-Signature: ajpgnggywh5zn81gq99nfch9prju4obz X-HE-Tag: 1718143999-283547 X-HE-Meta: U2FsdGVkX1+1kPVHX7LmeNVrsSVrA161HX8DPEdCXKIgDUKCq3N5R/me0MaJiJ9PZ4xjtPoyOLjYA2q0OqTg+1i2apKIp8uuxkqp8B41yAIDOJthkvw06uH98kDKrUsNn5q/qou+tsTA64e/7UraDoRYNhvyy1QgOiSP1ZU+ad270WVbFvrrWjoN5aZb9U2VacuHOlaFXhmG3x63wq6IHS7/1nzKAwCMZwK1eeVpZTY7ht1O+NsW1kg7bIOnfmdXmxHVW0aIJMLo85BIu0/E1ZFgjMTnNnT+CqCeZvPB2kstKNg5+X3j6bM/hsg84Nt9//dODL3L/4XGpkP5RTX7J9qDHqohverskgxyUovI+NdaM4hBngXCzCqwnQ2ct5tWpCE0W5LgizZKzx8u5fxWjpmxGzIR3HYko/O4oFDXu3T+jncYvQt+mFgRHoVAYeMw/oFmULMM9YpDZ2wObw9HDb6bCwa9DRAefZj++w0ZwuUtxOw97SxMeOG8vSNOQt8Kt8hXPLbdC4WG5cfx1MyVc5BBToRROrMqYPRNDgQOav1xmoAnBo+Avt5mCJFQLfejBq6pYuUzYnPwGmlyslH4qqAQdvtbbTFwML3UdcS9BKXk7oG17A4/CZoERxXtFQ8lqf6zV7RitpSr4O5NGQhe0xUFXlwnAv9ilVMIGx/GSYDoK+6iMLEfa52jw8sG1v3qMdMzIrTzZR1x2s/Vr1qfe9yCx5nXDqVEpHkWCx4uBu05HV7a1MCbiZog3nZfW1x1DVuM1o3xQUQEauapU1M931OnnEjPRHd8wG4USuquGdHw1x69ttx73R8pFHzRibUYo0hqvQoW4M9cAQuWORu2gZdD3qD0sx7jFgg0KGnXpBjj9F7WDoWGgUe8fVV0WFZ0pthsoAapiDXB34I8Hza9Urb8swCWuIer3wRCxA/WcKUpzs7erMtpHohUhlv787HmSbzwVGN/BLiqRGuHFcQ mqcWphAK erLR4s1cv3S/YrRmht8WQP7aB29f93cJdzIWp9dGhSA51q30aRQWHyyoD4HkuOs0z12PdN1rEN6uhGGQ6u4peV+w4YLxRwQxk17Na5uzChXU1i2g8LYPgdYdbghx5FNQ5FVX7gICRHrpXLr4qyUy663e49TT30bV+gYHCeUc6FmemdV9TFV9ScntqNQ30rOBEXg+cXb2hNVu8oBl0HTnPCv9ICpoG2GFroeDxHHvrRS3/yEm1qZaaQvUdXIx5jgQ/1ZM+lExw7d7ZUsmKFtk3JNPyXLVvgGVbWhoc6TlJ26NwhyNOXAB/CwJyfnHRZbwysxznhlqbhIp69q+7aDAWLA2EKTDW6SxhO5wTCC4sVs1ccVB/PPCGVTv+f9XnIoH7113OmWxIpu4fLBu1fAnr6gZivq6dxMakA0twJHSDArm2l8uqCZkGTyh0ZA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jun 12, 2024 at 5:24=E2=80=AFAM Shakeel Butt wrote: > > On Tue, Jun 11, 2024 at 12:23:41PM GMT, Barry Song wrote: > > On Tue, Jun 11, 2024 at 8:43=E2=80=AFAM Shakeel Butt wrote: > > > > > > On Thu, Mar 14, 2024 at 08:56:17PM GMT, Chuanhua Han wrote: > > > [...] > > > > > > > > > > So in the common case, swap-in will pull in the same size of foli= o as was > > > > > swapped-out. Is that definitely the right policy for all folio si= zes? Certainly > > > > > it makes sense for "small" large folios (e.g. up to 64K IMHO). Bu= t I'm not sure > > > > > it makes sense for 2M THP; As the size increases the chances of a= ctually needing > > > > > all of the folio reduces so chances are we are wasting IO. There = are similar > > > > > arguments for CoW, where we currently copy 1 page per fault - it = probably makes > > > > > sense to copy the whole folio up to a certain size. > > > > For 2M THP, IO overhead may not necessarily be large? :) > > > > 1.If 2M THP are continuously stored in the swap device, the IO > > > > overhead may not be very large (such as submitting bio with one > > > > bio_vec at a time). > > > > 2.If the process really needs this 2M data, one page-fault may perf= orm > > > > much better than multiple. > > > > 3.For swap devices like zram,using 2M THP might also improve > > > > decompression efficiency. > > > > > > > > > > Sorry for late response, do we have any performance data backing the > > > above claims particularly for zswap/swap-on-zram cases? > > > > no need to say sorry. You are always welcome to give comments. > > > > this, combining with zram modification, not only improves compression > > ratio but also reduces CPU time significantly. you may find some data > > here[1]. > > > > granularity orig_data_size compr_data_size time(us) > > 4KiB-zstd 1048576000 246876055 50259962 > > 64KiB-zstd 1048576000 199763892 18330605 > > > > On mobile devices, We tested the performance of swapin by running > > 100 iterations of swapping in 100MB of data ,and the results were > > as follows.the swapin speed increased by about 45%. > > > > time consumption of swapin(ms) > > lz4 4k 45274 > > lz4 64k 22942 > > > > zstdn 4k 85035 > > zstdn 64k 46558 > > Thanks for the response. Above numbers are actually very fascinating and > counter intuitive (at least to me). Do you also have numbers for 2MiB > THP? I am assuming 64k is the right balance between too small or too > large. Did you experiment on server machines as well? I don=E2=80=99t possess data on 2MiB, and regrettably, I lack a server mach= ine for testing. However, I believe that this type of higher compression ratio and lower CPU consumption generally holds true for generic anonymous memory. 64KB is a right balance. But nothing can stop THP from using 64KB to swapin, compression and decompression. as you can see from the zram/zsmalloc series, we actually have a configuration CONFIG_ZSMALLOC_MULTI_PAGES_ORDER The default value is 4. That means a 2MB THP can be compressed/decompressed as 32 * 64KB. If we use 64KB as the swapin granularity, we still have the balance and all the benefits if 2MB is a too large swap-in granularity which might caus= e memory waste. > > > > > [1] https://lore.kernel.org/linux-mm/20240327214816.31191-1-21cnbao@gma= il.com/ > > Thanks Barry