From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 231D6C4167B for ; Tue, 14 Nov 2023 10:04:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A3608D003B; Tue, 14 Nov 2023 05:04:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 952EE8D002E; Tue, 14 Nov 2023 05:04:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F2438D003B; Tue, 14 Nov 2023 05:04:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6C6FA8D002E for ; Tue, 14 Nov 2023 05:04:39 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 46953A0945 for ; Tue, 14 Nov 2023 10:04:39 +0000 (UTC) X-FDA: 81456125478.29.C379A80 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf23.hostedemail.com (Postfix) with ESMTP id 056A0140014 for ; Tue, 14 Nov 2023 10:04:36 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=Euj6Kvtz; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf23.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699956277; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HikEMLbWxEVzvyuBuhlgY1v61UUO53h7x3i8QbDBxPA=; b=0n1osMnzmEte4ifhNSRtbJPeYsx6IEWFjwqsve2VBHs14PIdCp0GapcB+jHvKRJj5j6UB3 BXN8qmmllatG0u+KCUKhE2QWx2sW3HRumDCwkvA5UBmc0KyBTJuGuBvuqBg8OjKh8g508L OplE31N0rt5AJRALQoulX1sp0X7Fla8= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=Euj6Kvtz; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf23.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699956277; a=rsa-sha256; cv=none; b=gibh/MzNlOu/XkaQawqE5AyMpdqtllyCjYJgyfXz0oLUMqNEOCSLzzZyxhQu+AGAxzIAY4 Uja4IyZLvQRiu30tf23yu3bXIiNyuxd2HKKgvAotqKV5QTzR+DkU5lty8TCgaOEkc7LmvO 0a9P67c3Jq46A3wIQppza9M/GVS8InA= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 0F4852189A; Tue, 14 Nov 2023 10:04:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1699956275; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HikEMLbWxEVzvyuBuhlgY1v61UUO53h7x3i8QbDBxPA=; b=Euj6KvtzEFXW41rjtYwsll8g5sMbKlmsGymplUNyLRBZpuR27W1ZceHjpTXOQ1UdSepabS T4dvs6dnt0gQOvgDjtusoAqjz6hnK9Lj5Y8C8y0QWuwt9s8Zsm4WLzyJIQdFSdqSNyjAEL 7lFNShTf6XgGpx3WCRg3UMLmgqrBS+Q= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D7CBE13416; Tue, 14 Nov 2023 10:04:34 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id YqE3MjJGU2WnIwAAMHmgww (envelope-from ); Tue, 14 Nov 2023 10:04:34 +0000 Date: Tue, 14 Nov 2023 11:04:34 +0100 From: Michal Hocko To: Huan Yang Cc: "Huang, Ying" , Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , David Hildenbrand , Matthew Wilcox , Kefeng Wang , Peter Xu , "Vishal Moola (Oracle)" , Yosry Ahmed , Liu Shixin , Hugh Dickins , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, opensource.kernel@vivo.com Subject: Re: [RFC 0/4] Introduce unbalance proactive reclaim Message-ID: References: <6b539e16-c835-49ff-9fae-a65960567657@vivo.com> <87a5rmiewp.fsf@yhuang6-desk2.ccr.corp.intel.com> <878r76gsvz.fsf@yhuang6-desk2.ccr.corp.intel.com> <78128117-ce70-47ef-b7fd-10c772b1c933@vivo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: 056A0140014 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 7ppwo8bsc46y978ocrrqwn7miz7si8e6 X-HE-Tag: 1699956276-799409 X-HE-Meta: U2FsdGVkX18O6cbJ2h28AvyUtWINvH3gedXntl73GwihE1ljc9XKhvkjxyOKB9wH7av/pLWKN3YabYi0QLOOJn0KJM+MMSz2gr+ggujRonhT12y0a+YiaS4VqM/nMdwP16lSVucRlT0U08lSRPfOoEG+C/DUCrnW8FblknmjD5eWM9BDBpULg+7alDNeUm7Kn97emUWgokioVayOBWgkbIWvl38FVEC4+3Qx+ImQ5Yp9sCMc+2/6RUaf1YeVtH4T5r2ehY8IfCney9wt106tIETONcjLwsftuFFwkBgrshmQf1jNLtGQwMC36yxKFExQ26Mzos+4g9IQ1K44yKoynSRue+t/nfiL1j0W724adi2pWPdN3dYApXHN1TFs0jW/XLmWcCUF3xEY1PEMok42/Q9b5fEr8PFBNVPDwnLDW/8lDRdU+ZCvdp7LImwbD3mCEUTIOgoE5M8aqSyKKPe5MgxMP1lBLR2/twz+1b6p1HKqVMM8WafJ7JIdv8ggadWY5XOmSRrjFOkjFFDekXzWDbGIKZUvufBh9Jnc6JY/YMZ0Nlh7NnQe04lLXxLcbIpbl8/nzjB4Hmpx+paWCJ7mswDiWMgXYxJHfTeyJeoy3l0UeXgn1X1wWnSxkJQeBv0oUW+qmLl6lHiOZBijpWvPHV4wlIU10zczix+gp8wN+HqMvAx0VdND+rO+hqxwu7pGmMJxoEOG0Oma4+tUBzJXIRAdDvrYhU10f/wjmDgQ/fzJIe/VTVlBsLjBNNXclNcjyKiGFzmM38PkuIg1cGF5N42jxJlvxtu5k7q66uk+IR2dFxGkkLg2hU0GjCmatOHtL1QbHRaWpa/PuTZ8PZgkKqHJuO3GrmStHb8p7CneVBUu+BNxoxW3OSf5Uvs8ANCdTd2f23Wvll2w1Km9BrBtkvWJSC5u01s8byaeRnEUPROpJV2RtgIUuBZ0ciSOLBkxBiHNmIEBCtZqvsOG0NR 3j9mkDtM GE8iot7bjapxgwCJKE9F9LeLbuXp4mkj8cHtW4VnFluyoRJiBxDgrsxXKwtN5H9n3AE3F33oWGMxv1iZrwdV4WEqyu6lMyf+Fh8t6lodNwDEHulBvAHlVfEDAFdPqqpqu2I+EwpxE6yDgctOaz84913K1etXpwcUdDsfmpcDZPaWkAXn96SiR0K7dPWGInqDyrAPb8ulF7/B+JQ1HMoyRKJ9b3NA0B1nqDlxVerm3sbsM1Jn+s7SPRyQE0IRaWtj4uQSA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon 13-11-23 09:54:55, Huan Yang wrote: > > 在 2023/11/10 20:32, Michal Hocko 写道: > > On Fri 10-11-23 14:21:17, Huan Yang wrote: > > [...] > > > > BTW: how do you know the number of pages to be reclaimed proactively in > > > > memcg proactive reclaiming based solution? > > > One point here is that we are not sure how long the frozen application > > > will be opened, it could be 10 minutes, an hour, or even days. So we > > > need to predict and try, gradually reclaim anonymous pages in > > > proportion, preferably based on the LRU algorithm. For example, if > > > the application has been frozen for 10 minutes, reclaim 5% of > > > anonymous pages; 30min:25%anon, 1hour:75%, 1day:100%. It is even more > > > complicated as it requires adding a mechanism for predicting failure > > > penalties. > > Why would make your reclaiming decisions based on time rather than the > > actual memory demand? I can see how a pro-active reclaim could make a > > head room for an unexpected memory pressure but applying more pressure > > just because of inactivity sound rather dubious to me TBH. Why cannot > > you simply wait for the external memory pressure (e.g. from kswapd) to > > deal with that based on the demand? > Because the current kswapd and direct memory reclamation are a passive > memory reclamation based on the watermark, and in the event of triggering > these reclamation scenarios, the smoothness of the phone application cannot > be guaranteed. OK, so you are worried about latencies on spike memory usage. > (We often observe that when the above reclamation is triggered, there > is a delay in the application startup, usually accompanied by block > I/O, and some concurrency issues caused by lock design.) Does that mean you do not have enough head room for kswapd to keep with the memory demand? It is really hard to discuss this without some actual numbers or more specifics. > To ensure the smoothness of application startup, we have a module in > Android called LMKD (formerly known as lowmemorykiller). Based on a > certain algorithm, LMKD detects if application startup may be delayed > and proactively kills inactive applications. (For example, based on > factors such as refault IO and swap usage.) > > However, this behavior may cause the applications we want to protect > to be killed, which will result in users having to wait for them to > restart when they are reopened, which may affect the user > experience.(For example, if the user wants to reopen the application > interface they are working on, or re-enter the order interface they > were viewing.) This suggests that your LMKD doesn't pick up the right victim to kill. And I suspect this is a fundamental problem of those pro-active oom killer solutions. > Therefore, the above proactive reclamation interface is designed to > compress memory types with minimal cost for upper-layer applications > based on reasonable strategies, in order to avoid triggering LMKD or > memory reclamation as much as possible, even if it is not balanced. This would suggest that MADV_PAGEOUT is really what you are looking for. If you really aim at compressing a specific type of memory then tweking reclaim to achieve that sounds like a shortcut because madvise based solution is more involved. But that is not a solid justification for adding a new interface. -- Michal Hocko SUSE Labs