From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47C7DC4167B for ; Thu, 9 Nov 2023 13:46:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8E5E480030; Thu, 9 Nov 2023 08:46:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 895D48D0073; Thu, 9 Nov 2023 08:46:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 75E3E80030; Thu, 9 Nov 2023 08:46:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 65E868D0073 for ; Thu, 9 Nov 2023 08:46:56 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 31AC1B6110 for ; Thu, 9 Nov 2023 13:46:56 +0000 (UTC) X-FDA: 81438541632.27.9A5BD4C Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf09.hostedemail.com (Postfix) with ESMTP id 36F94140006 for ; Thu, 9 Nov 2023 13:46:53 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=l2biBl5p; spf=pass (imf09.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699537614; a=rsa-sha256; cv=none; b=xD9EgDbMWTPyh6aW0KH8wwGp6/jVguVUkAfCtTf1WeLW+tTX08HRxjJu4eVNirqTLeOt4g aRVDXNAFCCrAa5gXJPm0cABSme86Vbqt1tB2EXGLzH+2t7WT1aWwFhgbMBkxIniln3eXWV 1g8EyYXqn9+gSr4k7y0xlcCcZF8/91g= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=l2biBl5p; spf=pass (imf09.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699537614; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YKHzFm/oapSlwZcHTR2PFLFJfozxMVapfiyCQwE/TOI=; b=J7xNoRxorH00p8xkNQUQnGhe2gayYg6547RjtEhgsg3LkyZ6pWOKQ+MdMGkQfPL8ANGnTx 30STx5PQWp7hi2KmHJr15qixT2vLmfEIB2aDoV80W+7fq7GyG5JNtwz50nVCxbB6WLMQUT Dts+ZXHhN+BBx7hnba6qhzltUlLwKWo= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id CDB6F21986; Thu, 9 Nov 2023 13:46:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1699537611; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YKHzFm/oapSlwZcHTR2PFLFJfozxMVapfiyCQwE/TOI=; b=l2biBl5ppzUx3t2WvUqfOlGuhAfLsWkWFrakSkJ7AEDjtiNP/PL9a27vmTyG8Nrx/jD+bw oJ58v0OKNARNz+eoBZAndvkafUxkWLACRq+aDkmdIKQt3L6z2X9ol2MiSYNo4nkBV9F9EO ehe9y+r+mKOMEcUbFUMWzfXuAF9IY24= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id BCEFE138E5; Thu, 9 Nov 2023 13:46:51 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id JSQVLsviTGWSWQAAMHmgww (envelope-from ); Thu, 09 Nov 2023 13:46:51 +0000 Date: Thu, 9 Nov 2023 14:46:51 +0100 From: Michal Hocko To: Huan Yang Cc: "Huang, Ying" , Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , David Hildenbrand , Matthew Wilcox , Kefeng Wang , Peter Xu , "Vishal Moola (Oracle)" , Yosry Ahmed , Liu Shixin , Hugh Dickins , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, opensource.kernel@vivo.com Subject: Re: [RFC 0/4] Introduce unbalance proactive reclaim Message-ID: References: <87msvniplj.fsf@yhuang6-desk2.ccr.corp.intel.com> <1e699ff2-0841-490b-a8e7-bb87170d5604@vivo.com> <6b539e16-c835-49ff-9fae-a65960567657@vivo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 36F94140006 X-Stat-Signature: rknu3p1fznsz5qheq37dgo5xstt7ppsg X-Rspam-User: X-HE-Tag: 1699537613-195771 X-HE-Meta: U2FsdGVkX19taMuka3P5M5KA3Kx+G7pYuD+VpyscT+jeaLewu+J8HRuoQmDqRcF046QYBn3B1ca8dwGh6IRq2qic1HS7+BDI8LibZHD5899zasQtcqZXXuPPXS3LER3TIA8LuipwcfEZGsABcKAcS2nrKU+DsOSlCwBX7Lz0UOiab96Op9HnsEInR/bH8APSx17ny5ts9XuzReO1jrE/SDdnNiOWmfMqxS+DaTDPkrG4a6l1FZxvNNHqJ9SoVDRJnAP2gWhhEgOZYqxu5X6iLFyKkidnF34RIilkFc0mrOPKcWxSCeVUY0k24QNJgt8JloEXa3W6kZ193v3V2Cr3QJRvP6dwyM0tnoe06kaqKmaG5G1eVQzaYNXtFgGAfH0rhl+0ANhMNa8GFdM/Z7IpMZ/4ihkf0lY5mm2NFnjv7kumZM5GJenRUqYd4jGc+Lp68zpWom1aUP2ruHaMnb0ZbeKRfGuYr3sfhobW9zw09qDhzGojy301huJ0WNIz/JkKSSzS1P42pzcYNxTHG8NhDo+x3tIOmH2CmsCV1xf80KmMQiJBqbZ+JJR0B/kbWR1TyQJlau6a95oLNkWO1pjlH57UkxA3EqMzvqny+owzyL0UrcuXBmkAzRA5G7/5n9SJVc/37HvLN/2kVwpxDSOF8VNXHHqB4DA/S1OtBMnyTg/0R0HA6W7kO6sY+wn6frDI/X5N2F9BQjCaihFtQ5RF1RfJdu0mzT9quGjXm//H9zoEeswIwNe4bTgfhUqiNKGmyC5DHWJ1coIML4PvoCXX8glHQTmmh3ugd4KItmkqkIeByWnP5hQMN1Io+1GBidStKOLsMUH9nsCs2e95oNztu/Q4i2cD1knDPP+4l4Xy0n+ClfYhaOESojMJpb2eho9aIbhZTArTziO6/FaMtzdxL9SYcZf1bWD7lLoVdmYr+9j5O1PaZf4uipg/NmVXDhSXvsjq65kQgNrFLO2WaMW yZdrnxBi LyzJ+mJiXVvR5H9MZOqrlAuqgLjacBn3f97yT7QT1NOxZSgoH2bSVioBYbaS9lYU++dC8A31UCw0iE05ZjpZECYOwh6UeDLUu9hO1lV92qnMX/uotweLEqybArlJOvENjBmATBrMwBtFZ2kKVADxeqKdR+xyUpBFihnrQzRZeuIeOThFITp6iMGGRUJ1n2U45yEWLJy/t6mmtYFfHqM4kaiwrvXmq698XvZZv4ImbRjelTdFRD+HIp+7iBEHctkviz6nmYDJiO/LyjiM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu 09-11-23 21:07:29, Huan Yang wrote: [...] > > > Of course, as you suggested, madvise can also achieve this, but > > > implementing it in the agent may be more complex.(In terms of > > > achieving the same goal, using memcg to group all the processes of an > > > APP and perform proactive reclamation is simpler than using madvise > > > and scanning multiple processes of an application using an agent?) > > It might be more involved but the primary question is whether it is > > usable for the specific use case. Madvise interface is not LRU aware but > > you are not really talking about that to be a requirement? So it would > > really help if you go deeper into details on how is the interface > > actually supposed to be used in your case. > In mobile field, we usually configure zram to compress anonymous page. > We can approximate to expand memory usage with limited hardware memory > by using zram. > > With proper strategies, an 8GB RAM phone can approximate the usage of a 12GB > phone > (or more). > > In our strategy, we group memcg by application. When the agent detects that > an > application has entered the background, then frozen, and has not been used > for a long time, > the agent will slowly issue commands to reclaim the anonymous page of that > application. > > With this interface, `echo memory anon > memory.reclaim` This doesn't really answer my questions above. > > Also make sure to exaplain why you cannot use other existing interfaces. > > For example, why you simply don't decrease the limit of the frozen > > cgroup and rely on the normal reclaim process to evict the most cold > This is a question of reclamation tendency, and simply decreasing the limit > of the frozen cgroup cannot achieve this. Why? > > memory? What are you basing your anon vs. file proportion decision on? > When zram is configured and anonymous pages are reclaimed proactively, the > refault > probability of anonymous pages is low when an application is frozen and not > reopened. > Also, the cost of refaulting from zram is relatively low. > > However, file pages usually have shared properties, so even if an > application is frozen, > other processes may still access the file pages. If a limit is set and the > reclamation encounters > file pages, it will cause a certain amount of refault I/O, which is costly > for mobile devices. Two points here (and the reason why I am repeatedly asking for some data) 1) are you really seeing shared and actively used page cache pages being reclaimed? 2) Is the refault IO really a problem. What kind of storage those phone have that this is more significant than potentially GB of compressed anonymous memory which would need CPU to refaulted back. I mean do you have any actual numbers to show that the default reclaim strategy would lead to a less utilized or less performant system? -- Michal Hocko SUSE Labs