From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2164C4332F for ; Mon, 13 Nov 2023 08:07:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C26B16B01A0; Mon, 13 Nov 2023 03:07:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BD55D6B01A1; Mon, 13 Nov 2023 03:07:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A75CC6B01A2; Mon, 13 Nov 2023 03:07:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 977326B01A0 for ; Mon, 13 Nov 2023 03:07:28 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5C1BD802F7 for ; Mon, 13 Nov 2023 08:07:28 +0000 (UTC) X-FDA: 81452201376.11.77C1B47 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by imf23.hostedemail.com (Postfix) with ESMTP id 7DF8C14000E for ; Mon, 13 Nov 2023 08:07:25 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ENLLO7xc; spf=pass (imf23.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699862846; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AFB+KH/fQsEOsmX/CZNCsEjkHZnucW6GZPcLD5VhHq4=; b=cNhz1filMqf71mSMsDbg/IMcVpS3cbAJP0/KpD1tMVs/W67qoeU6zilGBit3YF0ANaiy7L jkusDUOSr5QsnNPKkBaVQ5NDKdPUjjoOuOxt379lLNwvXCxBpWOzp1M50Adacgf1TqSVhg qFsbeGF8+v/HCHBQUeuAbPrm3guw/mI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699862846; a=rsa-sha256; cv=none; b=lWXvKDwYFdyugNiocNvtHcsbmAt/zdKC7Z1kOVH/02Pa5Z+dPbQTCSI5i6dYOWGsK4zDjh WBcVffqSIbvzLS4vULwEIUlNamwqROEZCEVIT+F5Uaj01tUP/S7Jnf/74CFeiP8jspoips oeD5W1MX5dOLAWjNcfycWt/6W3HNYJ4= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ENLLO7xc; spf=pass (imf23.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1699862846; x=1731398846; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version:content-transfer-encoding; bh=Gz1jc3j/vUIQC1LcsowIbxsUZ+JEI4/ZUHS7gaVZnYI=; b=ENLLO7xcDgesBWWZYQXsMwMJ1DDXtc9OsLFX+M4DCptab7zkPMW5xb3j iitHTeP8T4KTHU3xvdvx/A2V1N0NiiM6aIauuAUJe90zWq6BnRg43nPjz e0iaAOI+ZBvQFMbDBuxMSuvvXJuiZQ27V23WYCOs2MxfxkVn2a+wHrSa9 yR9T+Ay5suFhKPsa1Y7le2PXdM7xhVdkzuRj8HfNNmZpQt3C4j4Tvxf9w Ik/D6Ii+Cm6CMfCKkn60btlzKTDVpRtnSy/5Ms5qTuhRBSwAmhXlOmwRp s0fPFYjh1YI544Vko++VSW1RibUUtOHHolXjRUQTlKeFFIrWU2p5D3eUt w==; X-IronPort-AV: E=McAfee;i="6600,9927,10892"; a="9039189" X-IronPort-AV: E=Sophos;i="6.03,298,1694761200"; d="scan'208";a="9039189" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Nov 2023 00:07:15 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10892"; a="740698058" X-IronPort-AV: E=Sophos;i="6.03,298,1694761200"; d="scan'208";a="740698058" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Nov 2023 00:07:09 -0800 From: "Huang, Ying" To: Huan Yang Cc: Michal Hocko , Tejun Heo , Zefan Li , Johannes Weiner , "Jonathan Corbet" , Roman Gushchin , "Shakeel Butt" , Muchun Song , "Andrew Morton" , David Hildenbrand , Matthew Wilcox , Kefeng Wang , Peter Xu , "Vishal Moola (Oracle)" , Yosry Ahmed , "Liu Shixin" , Hugh Dickins , , , , , Subject: Re: [RFC 0/4] Introduce unbalance proactive reclaim In-Reply-To: (Huan Yang's message of "Mon, 13 Nov 2023 14:28:20 +0800") References: <87msvniplj.fsf@yhuang6-desk2.ccr.corp.intel.com> <1e699ff2-0841-490b-a8e7-bb87170d5604@vivo.com> <6b539e16-c835-49ff-9fae-a65960567657@vivo.com> <87edgufakm.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Mon, 13 Nov 2023 16:05:07 +0800 Message-ID: <87a5rif58s.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 7DF8C14000E X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 3b4btm4owfxhgnwkmnjjkiqrk3m3kmdq X-HE-Tag: 1699862845-942916 X-HE-Meta: U2FsdGVkX19LF9bmKpN053NUTmIEXCglKqunsGhvsB9Mgysq3KGWXVmGP46phIET1yIoB0D5s8xAYWhqvuOfrHjNfBo2/qdHNXXKXBCUwju0SwDnnBXIRhYdh8WODQzLcjF8C55752T0Vp16amzfzXPEPGfegs3590cY5A/vrVvhM89DXno9CU0B/qXHjkmlCtwdcNOSJR9x9mbqEvHEin5KWPuv3UmmAbHafEoTutcwvvH/VRE4HXqjYFWj5Iplx4YcVpa3EXg1+3h92MlHkk71jkyRTvsclzVP6DWdgwpj7uizc0cHxXlmeuisJ/Grdql4dswZqm0K35umfC8tHRiKc0WYT1H2W4KdP3ii1yKvM/N7Qi/vJZ0I5TJhbXBXrvfe5h4EXYbfxQVPRTVGmnANnikU8jg3avuMjX1KZ6cVkt6PYLeXAE3OKPqFjBafY/E44KzYPFv606AtHd4BJcfnneF62q60SEAesZRZXIOnFmeBvYdyVTh77L4OFj0SbY2TaVZZ0qA1f0CtIdpamdyvrg/Cg7oAw3AcEvjM5c/h/202WSGxHtq5tCem8LZR1rbcnCuhSlxkNuTyV+9nZMdgLLotD8cS//c3PhprRba7zs7l9bPHlB9gJotmImLht18ebTOZXAdVgdpsCfeS/+cyn9SpKQvOsDumLgoybbk9JfE1OEUgK9/7OiVaw3zhBERi4ln49MTkPWnhT4LGL9nfklk77TzNdszjsS89FQt6LsfGHK1KSxdajSnL+qdUIADkOcYpBKs2wXJyXJ3sdOTbfrxtdqK+/suykpIrJC5gMild7y5DXypz9ypVzCkU+xg67Mxt2oPy4xqPwLRtuP39HGsoK92neRd9nnEpLvBV9VGlPDk6zc2vfxIJcd69vSe7Sao1r0Q7alNdZK3Bd15qz/Hl4aMvBLDxVEnjXLo8i8+7wtUV32by1RYxNUzmVXQ0WaaF1mS/Qo56QYT N4LnbXRE A82fuFEpZ0xqwWZS2eQ9vj9LYzcf9FDJO+PJyM4rciY4SKjxRdnjws8d11OKgz5xn/OURO/DK/GEGOukfjxKBrFwnug6vqmu6m9/QVN88q8iAeBlr3RskkAWWmI7E2HdEAcAkuLMJd4ycwBuH076nEY4t1Lr2agW/kgKuImY5Lgw1Su3xLHCMKIKyivUuAV5ywPKLPrUtd5TZlxfrmTjtP+gwzFB70cCmm3gG5EZjyTb9ORMYivUe6lVEtbGSaH7CNNFn+nD9UYKHGdBVEG4PPew0Rw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Huan Yang writes: > =E5=9C=A8 2023/11/13 14:10, Huang, Ying =E5=86=99=E9=81=93: >> Huan Yang writes: >> >>> =E5=9C=A8 2023/11/10 20:24, Michal Hocko =E5=86=99=E9=81=93: >>>> On Fri 10-11-23 11:48:49, Huan Yang wrote: >>>> [...] >>>>> Also, When the application enters the foreground, the startup speed >>>>> may be slower. Also trace show that here are a lot of block I/O. >>>>> (usually 1000+ IO count and 200+ms IO Time) We usually observe very >>>>> little block I/O caused by zram refault.(read: 1698.39MB/s, write: >>>>> 995.109MB/s), usually, it is faster than random disk reads.(read: >>>>> 48.1907MB/s write: 49.1654MB/s). This test by zram-perf and I change a >>>>> little to test UFS. >>>>> >>>>> Therefore, if the proactive reclamation encounters many file pages, >>>>> the application may become slow when it is opened. >>>> OK, this is an interesting information. From the above it seems that >>>> storage based IO refaults are order of magnitude more expensive than >>>> swap (zram in this case). That means that the memory reclaim should >>>> _in general_ prefer anonymous memory reclaim over refaulted page cache, >>>> right? Or is there any reason why "frozen" applications are any >>>> different in this case? >>> Frozen applications mean that the application process is no longer acti= ve, >>> so once its private anonymous page data is swapped out, the anonymous >>> pages will not be refaulted until the application becomes active again. >>> >>> On the contrary, page caches are usually shared. Even if the >>> application that >>> first read the file is no longer active, other processes may still >>> read the file. >>> Therefore, it is not reasonable to use the proactive reclamation >>> interface to >>> reclaim=C2=A0page caches without considering memory pressure. >> No. Not all page caches are shared. For example, the page caches used >> for use-once streaming IO. And, they should be reclaimed firstly. > Yes, but this part is done very well in MGLRU and does not require our > intervention. > Moreover, the reclaim speed of clean files is very fast, but compared to = it, > the reclaim speed of anonymous pages is a bit slower. >> >> So, your solution may work good for your specific use cases, but it's > Yes, this approach is not universal. >> not a general solution. Per my understanding, you want to reclaim only >> private pages to avoid impact the performance of other applications. >> Privately mapped anonymous pages is easy to be identified (And I suggest >> that you can find a way to avoid reclaim shared mapped anonymous pages). > Yes, it is not good to reclaim shared anonymous pages, and it needs to be > identified. In the future, we will consider how to filter them. > Thanks. >> There's some heuristics to identify use-once page caches in reclaiming >> code. Why doesn't it work for your situation? > As mentioned above, the default reclaim algorithm is suitable for recycli= ng > file pages, but we do not need to intervene in it. > Direct reclaim or kswapd of these use-once file pages is very fast and wi= ll > not cause lag or other effects. > Our overall goal is to actively and reasonably compress unused anonymous > pages based on certain strategies, in order to increase available memory = to > a certain extent, avoid lag, and prevent applications from being killed. > Therefore, using the proactive reclaim interface, combined with LRU > algorithm > and reclaim tendencies, is a good way to achieve our goal. If so, why can't you just use the proactive reclaim with some large enough swappiness? That will reclaim use-once page caches and compress anonymous pages. So, more applications can be kept in memory before passive reclaiming or killing background applications? -- Best Regards, Huang, Ying