From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60D31C47072 for ; Wed, 15 Nov 2023 06:54:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CFE746B032B; Wed, 15 Nov 2023 01:54:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CAF606B032D; Wed, 15 Nov 2023 01:54:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B75666B032E; Wed, 15 Nov 2023 01:54:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A86C16B032B for ; Wed, 15 Nov 2023 01:54:38 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8ABFC160113 for ; Wed, 15 Nov 2023 06:54:38 +0000 (UTC) X-FDA: 81459275436.01.1B0B321 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) by imf11.hostedemail.com (Postfix) with ESMTP id 8DE4540008 for ; Wed, 15 Nov 2023 06:54:34 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Zqn1gMAS; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf11.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700031276; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DrPdU2SUZzn05TeKMGZZvxUsOIXfDJyEiiBY7/4c19Q=; b=kq+4gxk68i5AdZ+7U83MJbP6s+nUPSh2KLdZyGmGwIY3QPohtO728h3UOeHF84Nn2x3lT0 Y1/OAj4n+U2Xg4DZydwolAHNy3MtRpWNUhth62zwOwBd6FW6ROplLtAnZYisVbcOC455bU +vck5pvng/MQrSiC1fdGb4ysEVj4k5M= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Zqn1gMAS; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf11.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700031276; a=rsa-sha256; cv=none; b=sTIuIWuo+pf8tFzq0+mF5S6ct0/b1gMwJqcqi4BOJRFi8FD/oRju/V/11DzDailbiS6jkv nzT/3V+1L168eehYhhiNdLw47d42hykjQv6wqPeixMyj04G760wJMnWzco/6DSNgcWZoBR zxyAbxNgxd9D9BNkxGPJpn6Jk1E5ycU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1700031276; x=1731567276; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version:content-transfer-encoding; bh=KQs4vJm/jx2KWNtm77V1LxYhHIJm4bwsu8GghKrRV7U=; b=Zqn1gMASkIzJqUzYt3Uex3d4ykO0bRaXJMD2rL6Km+Wv2i7mAZVZ4tPo 9kvHcZjrxHE6XUT6zjmiol656lDzlhhmzA/Nbrd+UAD6gqPJ4ek1mx3Ed Z5ixMOIFmLYGbUyPj3x17qF6TFMZUEb7TqPfWybSh+EnfirTgG+2J4+mY 5+7zMUU+4MwGTvTNgErOtNAU9/0EsXmq5cP34u8TBdzJnJuYGrAx0GU3q d/r5+/J+qL5wKfJR6EWXOlgnJQl+Xe5KQCla2Z8ZI/dBSmuPbASb+cx3f h+dvZkN68gExrjieyswgX03J7rJ618Xh+SnzgNL7zGTd6/LqCHEGn530o g==; X-IronPort-AV: E=McAfee;i="6600,9927,10894"; a="3937283" X-IronPort-AV: E=Sophos;i="6.03,304,1694761200"; d="scan'208";a="3937283" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2023 22:54:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.03,304,1694761200"; d="scan'208";a="6048782" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2023 22:54:26 -0800 From: "Huang, Ying" To: Huan Yang Cc: Michal Hocko , Tejun Heo , Zefan Li , Johannes Weiner , "Jonathan Corbet" , Roman Gushchin , "Shakeel Butt" , Muchun Song , "Andrew Morton" , David Hildenbrand , Matthew Wilcox , Kefeng Wang , Peter Xu , "Vishal Moola (Oracle)" , Yosry Ahmed , "Liu Shixin" , Hugh Dickins , , , , , Subject: Re: [RFC 0/4] Introduce unbalance proactive reclaim In-Reply-To: <97a3dbb3-9e73-4dcc-877d-f491ff47363b@vivo.com> (Huan Yang's message of "Mon, 13 Nov 2023 16:26:00 +0800") References: <87msvniplj.fsf@yhuang6-desk2.ccr.corp.intel.com> <1e699ff2-0841-490b-a8e7-bb87170d5604@vivo.com> <6b539e16-c835-49ff-9fae-a65960567657@vivo.com> <87edgufakm.fsf@yhuang6-desk2.ccr.corp.intel.com> <87a5rif58s.fsf@yhuang6-desk2.ccr.corp.intel.com> <97a3dbb3-9e73-4dcc-877d-f491ff47363b@vivo.com> Date: Wed, 15 Nov 2023 14:52:24 +0800 Message-ID: <87jzqjecev.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 8DE4540008 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 97zuec9xiu8fjit5gt1ynet89iujrxj1 X-HE-Tag: 1700031274-438836 X-HE-Meta: U2FsdGVkX1/aB+5l7Wvu8gNWRcY2zBnkUj4NAs7GSMTGEy56n/M04o0fdZaaA/YkVx9sJTA0bQ19EfZ7+pQN1djSzITb+tapWLtwpTlXIjWsuTxIGdALhYvIFBivhG5uL//KZ4Evhm+AykZJ4gVnVJjfMcZckHSmew7TJXvK+4BBjU3efZG0gNgq94GvqxsE4AAYilwm6+1CGtgDuG2+9oxJ8cN7zHZUJd4FTY57w9JDyLW2vhKtB/daI9kejr1RURYRCTAe6oF8mz7hxXWT7Z13G89TTeDV8SktHcVDUZbZ8L6VlN1ybClMCmJc6aq28hz2cySKGI28KvvYLYi9wetXxxIE4G30ZwxzGgwESGUHBcRVyhzTf5SPYqMXlm76Gh1QQ/zgOttvfar7LJJ6JQI+uA944LIPEgvLeaXWokd5TWCl79WdAPZK1/4n4yfozwzTEnML5BGM+FcjSqI/LKH+H9IzsvalNcpSfRNfI+UmjG/Gh3nhBta2kE47WL5EUvnmky75EwrLwEuKgKCFLPVSLjjIhTOslqqLsv26KUjbrQPYPIglGiW4Nc+2STbPjqvOcqzL21B2IbZlhWfiWHb2iweA5M3SzIcEQXs8hkOu0s8NmlTWtieozcFjcKlFDJZJ+7uuo2fX43fjNNBaGtvqthgP9tVMSWL1nySL1v7HBoCK5kAQrlLM67o/eRbr9c+SqGQvc4gVWQ6I1fmpcSl4wXhTEEDFS/he4A43L2W0IwvByLD28QoY8dWp4WXDzCEMIrk2Uj53JBfPTs+jgQzisEMpo7PS0BmKm92gOrlNGwemb+UQi27UnZ5l8PR/cZh6kec11EDrajXpSTIf6EdU4mMX9TM8IDoNCsOLBwnxy3IgvR8u44oBXElBX7j813x4Ox3J4dQOcddk8JdPexEM2aCHIHKamul9bxXAwZ7hijiSf3uW1DHwLDLaRI1+ljWI8KzSc08qDVqwGnR P6wjKWhT RNZ71EkkyINwuGcizBn1GTdMWO6fbHAX/3r/4mgK3pSD/yPvKkfhHqdH0AD7n6WBnDNpmc9XBQfzvm4b98YmlSxzzmY1nLwByeOSPSRFI/I45s/m9SBYZOQRQpxe5GgLpY/yjFX2TmQ2T9cM1BDbCrtKyghgp4yBYlKcCsx/wt24HqY1AZVTFZIdiT/5YQUBK5oOOc8cUg15Osbv54EBtQLZOVbV8iWeko2b2zksjIpR1JNxTYLrDraG1HVBeNSGOftrb1t9rMu022R/1SDr8acDfRQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Huan Yang writes: > =E5=9C=A8 2023/11/13 16:05, Huang, Ying =E5=86=99=E9=81=93: >> Huan Yang writes: >> >>> =E5=9C=A8 2023/11/13 14:10, Huang, Ying =E5=86=99=E9=81=93: >>>> Huan Yang writes: >>>> >>>>> =E5=9C=A8 2023/11/10 20:24, Michal Hocko =E5=86=99=E9=81=93: >>>>>> On Fri 10-11-23 11:48:49, Huan Yang wrote: >>>>>> [...] >>>>>>> Also, When the application enters the foreground, the startup speed >>>>>>> may be slower. Also trace show that here are a lot of block I/O. >>>>>>> (usually 1000+ IO count and 200+ms IO Time) We usually observe very >>>>>>> little block I/O caused by zram refault.(read: 1698.39MB/s, write: >>>>>>> 995.109MB/s), usually, it is faster than random disk reads.(read: >>>>>>> 48.1907MB/s write: 49.1654MB/s). This test by zram-perf and I chang= e a >>>>>>> little to test UFS. >>>>>>> >>>>>>> Therefore, if the proactive reclamation encounters many file pages, >>>>>>> the application may become slow when it is opened. >>>>>> OK, this is an interesting information. From the above it seems that >>>>>> storage based IO refaults are order of magnitude more expensive than >>>>>> swap (zram in this case). That means that the memory reclaim should >>>>>> _in general_ prefer anonymous memory reclaim over refaulted page cac= he, >>>>>> right? Or is there any reason why "frozen" applications are any >>>>>> different in this case? >>>>> Frozen applications mean that the application process is no longer ac= tive, >>>>> so once its private anonymous page data is swapped out, the anonymous >>>>> pages will not be refaulted until the application becomes active agai= n. >>>>> >>>>> On the contrary, page caches are usually shared. Even if the >>>>> application that >>>>> first read the file is no longer active, other processes may still >>>>> read the file. >>>>> Therefore, it is not reasonable to use the proactive reclamation >>>>> interface to >>>>> reclaim=C2=A0page caches without considering memory pressure. >>>> No. Not all page caches are shared. For example, the page caches used >>>> for use-once streaming IO. And, they should be reclaimed firstly. >>> Yes, but this part is done very well in MGLRU and does not require our >>> intervention. >>> Moreover, the reclaim speed of clean files is very fast, but compared t= o it, >>> the reclaim speed of anonymous pages is a bit slower. >>>> So, your solution may work good for your specific use cases, but it's >>> Yes, this approach is not universal. >>>> not a general solution. Per my understanding, you want to reclaim only >>>> private pages to avoid impact the performance of other applications. >>>> Privately mapped anonymous pages is easy to be identified (And I sugge= st >>>> that you can find a way to avoid reclaim shared mapped anonymous pages= ). >>> Yes, it is not good to reclaim shared anonymous pages, and it needs to = be >>> identified. In the future, we will consider how to filter them. >>> Thanks. >>>> There's some heuristics to identify use-once page caches in reclaiming >>>> code. Why doesn't it work for your situation? >>> As mentioned above, the default reclaim algorithm is suitable for recyc= ling >>> file pages, but we do not need to intervene in it. >>> Direct reclaim or kswapd of these use-once file pages is very fast and = will >>> not cause lag or other effects. >>> Our overall goal is to actively and reasonably compress unused anonymous >>> pages based on certain strategies, in order to increase available memor= y to >>> a certain extent, avoid lag, and prevent applications from being killed. >>> Therefore, using the proactive reclaim interface, combined with LRU >>> algorithm >>> and reclaim tendencies, is a good way to achieve our goal. >> If so, why can't you just use the proactive reclaim with some large >> enough swappiness? That will reclaim use-once page caches and compress > This works very well for proactive memory reclaim that is only > executed once. > However, considering that we need to perform proactive reclaim in batches, > suppose that only 5% of the use-once page cache in this memcg can be > reclaimed, > but we need to call proactive memory reclaim step by step, such as 5%, > 10%, 15% ... 100%. > Then, the page cache may be reclaimed due to the balancing adjustment > of reclamation, > even if the 5% of use-once pages are reclaimed. We may still touch on > shared file pages. > (If I misunderstood anything, please correct me.) If the proactive reclaim amount is less than the size of anonymous pages, I think that you are safe. For example, if the size of anonymous pages is 100MB, the size of use-once file pages is 10MB, the size of shared file pages is 20MB. Then if you reclaim 100MB proactively with swappiness=3D200, you will reclaim 10MB use-once file pages and 90MB anonymous pages. In the next time, if you reclaim 10MB proactively, you will still not reclaim shared file pages. > We previously used the two values of modifying swappiness to 200 and 0 > to adjust reclaim > tendencies. However, the debug interface showed that some file pages > were reclaimed, > and after being actively reclaimed, some applications and the reopened > applications that were > reclaimed had some block IO and startup lag. If so, please research why use-once file page heuristics not work and try to fix it or raise the issue. > This way of having incomplete control over the process maybe is not > suitable for proactive memory > reclaim. Instead, with an proactive reclaim interface with tendencies, > we can issue a > 5% page cache trim once and then gradually reclaim anonymous pages. >> anonymous pages. So, more applications can be kept in memory before >> passive reclaiming or killing background applications? -- Best Regards, Huang, Ying