From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F050FC4167D for ; Thu, 9 Nov 2023 12:41:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 449C38D00D6; Thu, 9 Nov 2023 07:41:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3F9B68D0073; Thu, 9 Nov 2023 07:41:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C3538D00D6; Thu, 9 Nov 2023 07:41:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1DB078D0073 for ; Thu, 9 Nov 2023 07:41:00 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 94C431CB48A for ; Thu, 9 Nov 2023 12:40:59 +0000 (UTC) X-FDA: 81438375438.06.F44448E Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf13.hostedemail.com (Postfix) with ESMTP id 9768720015 for ; Thu, 9 Nov 2023 12:40:57 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b="UERc19i/"; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf13.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699533658; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kBDurIQ5uqPNYzPg069Va7iaYE5q2cMCH1L1AF71VnM=; b=rj0wvtW6WFjXqjjwdKfW5jxLcbNbyno/xtsqGeK87qLE9b1CGNl+O0zZnIQuekVh6joihi +40Uwl7DcGraCnnobHJmlMfSUD4zRX/OJkYC11NCZ0+KNruRt5ZQipzu5zk3qYPNspzClL th+LnW2O880OFf+SL3AxbPPb2vI+EGQ= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b="UERc19i/"; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf13.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699533658; a=rsa-sha256; cv=none; b=4COT1g7WY5WSFEc6QGPzpBYneM3z2WjgZzx76NV/hEqhgmS1Osi+z1Gwn2/GWhJsLqhe46 GQUiQuZ6nI2OvVNV34vIxTb+z9IleZ7MD1uAJ2pyc8poPtiBzjPgXMmb7SzVDO/st+GG2d YsfqagzzqR8Dhb3yVYNj7JZfPUWrkAU= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6DC4721977; Thu, 9 Nov 2023 12:40:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1699533655; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kBDurIQ5uqPNYzPg069Va7iaYE5q2cMCH1L1AF71VnM=; b=UERc19i/58CHPOKnAxtzhjGF01dQY0FXYRsKov+eSSNOfzWQe1jdNK5iYYpdsSYM/fcpwE ELjxm3MvWOmhxh0/QsGhga74/a/gvcVEA4gfm2pVMLR2UEmZXM+loZLcxmXyirLLXFFRZ9 zeOAw0f3HvdE5jghAXBM9HsWJaP1iQE= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 5E92513524; Thu, 9 Nov 2023 12:40:55 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id iioKF1fTTGVhOQAAMHmgww (envelope-from ); Thu, 09 Nov 2023 12:40:55 +0000 Date: Thu, 9 Nov 2023 13:40:54 +0100 From: Michal Hocko To: Huan Yang Cc: "Huang, Ying" , Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , David Hildenbrand , Matthew Wilcox , Kefeng Wang , Peter Xu , "Vishal Moola (Oracle)" , Yosry Ahmed , Liu Shixin , Hugh Dickins , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, opensource.kernel@vivo.com Subject: Re: [RFC 0/4] Introduce unbalance proactive reclaim Message-ID: References: <20231108065818.19932-1-link@vivo.com> <87msvniplj.fsf@yhuang6-desk2.ccr.corp.intel.com> <1e699ff2-0841-490b-a8e7-bb87170d5604@vivo.com> <6b539e16-c835-49ff-9fae-a65960567657@vivo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspam-User: X-Stat-Signature: e93cyseybsrnu9aac75tzu7upe9eo5zn X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9768720015 X-HE-Tag: 1699533657-918432 X-HE-Meta: U2FsdGVkX198Cn2B/hEdch1bIhH/2ucKQtm6KiS+b080HKOyITgADVT3z/6Mq316CHr8gsqnTegJ6y5J31GTOlURblShfsxO78+djrCiN1DxP6Br2/QiN5uWT0aVFjDGhDkosjPPPhLgVH8L/eyxdNVgVerJ/eGJbcSEHHI8cga4vkk+74TDwNFBomz9xwXIAdhzrl33T2gUEqn8tEP2NiKCPsMYqK0HF9/aYjdUV7bMM/N+Bm2eR56rHhbecpJ/kibIvjXRxMfb7vitE53c1+DVL7zwtt7n7K66OUEOtmu3tyykDZBi+VBOIsU8AqcxTjFolLF3xe/DvTC5n0UvdnkiIckjuejAPPbgc1EksOst+alujaVS2PE8MMCt4fRmN29RcZSd2QhxD3wPYU2OJL7dVT9pLNpkl7YJKoJdUsv1WZ7x/Rl37zeJO9MmrzsNlI0EsytbjzXIQCbRp4/Jg2KEhQTP6MXYnygtoNEzxZAYZrZhOROhKBO3kV3cai8/+mjubL3ZU55fQnMVqZtjCTcrsGDtmrkkdp4ICoPJrphS1wFiNcdSYaLrTiEOz0LxPI5FKVjbjqcrBehRBqYIE+mZIYIHaEW2djp/E4zyfbATJy6tytMi+qY7JVTORxOPl8NjtC4ho7BJFXjdCKZnbgL4xjqFv8jxvpf/YBynfCgJWLKfsMl2lXPZ80gIR/zcamm52lVKNBNM15+n8ukOOOhkuRzfGVPBltMr/t5BYFO8HGngd1U6ZZn9IPuIRkn2mkICtIuz6jV/tPrsk7/eJeWV9X+C88awxFIHKLIzaWFNSsJQSthSIAuMrQAN6UI4/PJDKQuotyLw8qUuXhIYoENaGQXV3TmZeNhb0DD7HIoOXSzST5ErHkBYDAbn82Hvqk6aO/DKYItfwk4myCt6TZm6o0loGul4jxd7U0gPybZ4F5ahCykeP4TYGqeCb6JDLl5r8l9HgfsgLaklLRj 34GLzsvW DsVwcJ+0IqhvnlYHkK8JVjvet1+Ex7lE0Du5unV6p/+aphSbaB61nKrChlFyIG6ZXAjStkUZcW4ID1vhbt4AJbdLaw1GvW9OeaF3JlIAJj9vBf/m58wwWeNxpXIK8EtMhiA2t1BTwrGJBBB7ugiUjqAMdKcJKXniq3F5Sa3pbfrIC6zaJ0Ej6mOfqBZVf6HAVQMOw65+rrs2Yff/c9qyodRn/5Trh4thykNf2vKaqvPXpphYdbm9Qk8+0zQR90UrHNH/mQJOwe+Pahfzy/wWBwueSUnp1EvHt8orchBMcukBR8m3jELnj9QswyfhBYme/21JH9sjMOgD4UyPiVRargYD84dZoQypqNPunyfGN1WroDqf/nCWiBIF9XpRj85XoI4ojwyk5LYZX5V3xezfazGQoAA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu 09-11-23 18:50:36, Huan Yang wrote: > > 在 2023/11/9 18:39, Michal Hocko 写道: > > [Some people who received this message don't often get email from mhocko@suse.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] > > > > On Thu 09-11-23 18:29:03, Huan Yang wrote: > > > HI Michal Hocko, > > > > > > Thanks for your suggestion. > > > > > > 在 2023/11/9 17:57, Michal Hocko 写道: > > > > [Some people who received this message don't often get email from mhocko@suse.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] > > > > > > > > On Thu 09-11-23 11:38:56, Huan Yang wrote: > > > > [...] > > > > > > If so, is it better only to reclaim private anonymous pages explicitly? > > > > > Yes, in practice, we only proactively compress anonymous pages and do not > > > > > want to touch file pages. > > > > If that is the case and this is mostly application centric (which you > > > > seem to be suggesting) then why don't you use madvise(MADV_PAGEOUT) > > > > instead. > > > Madvise may not be applicable in this scenario.(IMO) > > > > > > This feature is aimed at a core goal, which is to compress the anonymous > > > pages > > > of frozen applications. > > > > > > How to detect that an application is frozen and determine which pages can be > > > safely reclaimed is the responsibility of the policy part. > > > > > > Setting madvise for an application is an active behavior, while the above > > > policy > > > is a passive approach.(If I misunderstood, please let me know if there is a > > > better > > > way to set madvise.) > > You are proposing an extension to the pro-active reclaim interface so > > this is an active behavior pretty much by definition. So I am really not > > following you here. Your agent can simply scan the address space of the > > application it is going to "freeze" and call pidfd_madvise(MADV_PAGEOUT) > > on the private memory is that is really what you want/need. > > There is a key point here. We want to use the grouping policy of memcg > to perform proactive reclamation with certain tendencies. Your > suggestion is to reclaim memory by scanning the task process space. > However, in the mobile field, memory is usually viewed at the > granularity of an APP. OK, sthis is likely a terminology gap on my end. By application you do not really mean a process but rather a whole cgroup. That would have been really useful to be explicit about. > Therefore, after an APP is frozen, we hope to reclaim memory uniformly > according to the pre-grouped APP processes. > > Of course, as you suggested, madvise can also achieve this, but > implementing it in the agent may be more complex.(In terms of > achieving the same goal, using memcg to group all the processes of an > APP and perform proactive reclamation is simpler than using madvise > and scanning multiple processes of an application using an agent?) It might be more involved but the primary question is whether it is usable for the specific use case. Madvise interface is not LRU aware but you are not really talking about that to be a requirement? So it would really help if you go deeper into details on how is the interface actually supposed to be used in your case. Also make sure to exaplain why you cannot use other existing interfaces. For example, why you simply don't decrease the limit of the frozen cgroup and rely on the normal reclaim process to evict the most cold memory? What are you basing your anon vs. file proportion decision on? In other words more details, ideally with some numbers and make sure to describe why existing APIs cannot be used. -- Michal Hocko SUSE Labs