From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47062C38142 for ; Fri, 27 Jan 2023 09:29:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF9286B0071; Fri, 27 Jan 2023 04:29:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B81E06B0072; Fri, 27 Jan 2023 04:29:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A223E6B0074; Fri, 27 Jan 2023 04:29:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8EB286B0071 for ; Fri, 27 Jan 2023 04:29:35 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 4CFCF1C608D for ; Fri, 27 Jan 2023 09:29:35 +0000 (UTC) X-FDA: 80400056310.27.D4E9380 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf11.hostedemail.com (Postfix) with ESMTP id 6181F40011 for ; Fri, 27 Jan 2023 09:29:33 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=d3UxIkYc; spf=pass (imf11.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674811773; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bEY5UCtEKCeTLdzAmSIUQZvUYZL+4hOTPkRByerNhaI=; b=3ca8xQ61lID/Ue0Jkl5l2skJlWwhC7ipg3JFRgnpxn6q0FwdfyKKYvqTU0YgvGFnsFu6ah 3V3lvo29dnlY6X0tPJPAfzwwKzAS55J9HAGvxd+5hLyCpMCYtZ4v16B1B0LuN6BOj8jGBf WQsIs4/LGzhLiSWNOYiR0kSVQGLO2GI= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=d3UxIkYc; spf=pass (imf11.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674811773; a=rsa-sha256; cv=none; b=zFQuGyzk0F9MAcne/0swLkoP3crY476SDH8gU8ueHGp/8psUSda7x0R2Jl2tkvVHQs6J55 lCtRwkDBwKDmrT6GGY6FoOOeUl+2lmgUXudrCZyJ4jcQ6LzM69g0VVyK0Y0pSK0nbn9PuB vgtxlUtDjmqpw00srwIcJ0LwlPTIyEI= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id EF59121D67; Fri, 27 Jan 2023 09:29:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1674811771; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bEY5UCtEKCeTLdzAmSIUQZvUYZL+4hOTPkRByerNhaI=; b=d3UxIkYcjOecRmojVKRzC++3+Ldu/RdeTs9vhHpP3QlZwel4R8UkrVo/RGAv81LU0tBUB2 abmN2m9OZnLCKqCTBqauh3umAFlA2jpYLBTXUIqyef0R7XUhLn8B3mkHnh8WlIPW9b/jSx MMfeL0YJkgibZC+S3aOyPha7Srcc/kI= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id CEB46138E3; Fri, 27 Jan 2023 09:29:31 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id +EwtMHuZ02OxHAAAMHmgww (envelope-from ); Fri, 27 Jan 2023 09:29:31 +0000 Date: Fri, 27 Jan 2023 10:29:31 +0100 From: Michal Hocko To: Leonardo =?iso-8859-1?Q?Br=E1s?= Cc: Roman Gushchin , Marcelo Tosatti , Johannes Weiner , Shakeel Butt , Muchun Song , Andrew Morton , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 0/5] Introduce memcg_stock_pcp remote draining Message-ID: References: <20230125073502.743446-1-leobras@redhat.com> <9e61ab53e1419a144f774b95230b789244895424.camel@redhat.com> <55ac6e3cbb97c7d13c49c3125c1455d8a2c785c3.camel@redhat.com> <15c605f27f87d732e80e294f13fd9513697b65e3.camel@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <15c605f27f87d732e80e294f13fd9513697b65e3.camel@redhat.com> X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 6181F40011 X-Rspam-User: X-Stat-Signature: wj4mfqzw6xbjbrmu9j5wg1iytw4eytb4 X-HE-Tag: 1674811773-551326 X-HE-Meta: U2FsdGVkX1/d3OKqZ0kWk9N7Q6eSnQjGN9DjZ3rPr38/7PafMDb9Hi5MRWRDPAwmy5/vAD0/cI6jQMF4iDrXm0K4qx5rNvcHCfL7hzmtvco3iP8dpQTKPyD9L46ozb+WzO4PyVV+ZQCdZHZRXiq7vyOxF7C2q66SM+PWqXAV6sWZQbKuurXZjCArfrqbpbPEKsZTsQH1RP/1VgZeJOXez5uYOwrCrH7PjTznnBuP3U0ldl1iESXOIdGyVaxKI6afCsmkW9w2aRXHA5JR1DruObTqYbdR7B1gHVPctfjCrv8mzozEfUTBcR3PXak6Tn60Fy2uuJV84a1ZrazwmlBQa5OvjbChO8LWdg/eiSeBZ6AtSqlKCCZiDp3BVPhLYlOdZJ2S5Iz7PfRv711EHBf8HafYYdj4TIM6cOy3qeeRC20m6bNaAL3FBU8g6URYvCQY5ipAt57iIXDJ3nANIHqRQEJHxhZk12j1w3EJT7nLRE54uJIUjWu7U/x1md77H70R6+ZVDSu5VB/pgwclIqpyReP7le6Wx9f+EXrUiv3XTwWQvJTMI3A6GqopQ+jo0/cRN+gPfboLIT8NUWZxsrxUYExEpDJniCW6dwJ6P94r1q4eUm2eXDWFXP8vNTJffOCEZf3+MnjXt4D8EOw1X5dIXUjfbzBvM/Cb4W9ELd2/Xj6djJXlRFuDOO2md5Y4FrWYfnlSHAOexL3hcBzuwZJrZE7M15JEzSmDVHlzhofbqApr286wp9EkXu4Z0oaeBIdXt7QkwHdbqUCUHaLX7KIhrGQ7nnpZYpnDgS+67Vc20WdHAt1eqIV+H6aAUF3Eof4BF+pkazpmooy6ORJGBW3DcvPnI6SzP4VaxiRUkzhogv3dyCSSg3k5avAJBSuluXJ1/6dlXYE6lpprhMs60Pzr+HamIMFXQ7oEPtd8BGEsB2Jj6oCdt9tSe68F77JP8k5BC/NzkZe2W9d77lUqCM2 Kwod8fH+ Ks3Fz/fynk0WXqvovPg/teSdQNhO4L804di2Vi0oKIuMOFWU4Dzj2UNqiPHsscBzXcO0zeDFXUM/s86bxGIsgnJeUP76U/j+IhS7ONc5TeWo/7012XeaGeqLwVG9afa6Jz8cSkoEhKedIFMq+BWPbGtoD0r+HtZ87mBX2cwdjz93rjqPVYPcFV7hIgKVLtPRBvQupYPMiXGxJPT8hWwSJfUG/ZQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri 27-01-23 04:35:22, Leonardo Brás wrote: > On Fri, 2023-01-27 at 08:20 +0100, Michal Hocko wrote: > > On Fri 27-01-23 04:14:19, Leonardo Brás wrote: > > > On Thu, 2023-01-26 at 15:12 -0800, Roman Gushchin wrote: > > [...] > > > > I'd rather opt out of stock draining for isolated cpus: it might slightly reduce > > > > the accuracy of memory limits and slightly increase the memory footprint (all > > > > those dying memcgs...), but the impact will be limited. Actually it is limited > > > > by the number of cpus. > > > > > > I was discussing this same idea with Marcelo yesterday morning. > > > > > > The questions had in the topic were: > > > a - About how many pages the pcp cache will hold before draining them itself?  > > > > MEMCG_CHARGE_BATCH (64 currently). And one more clarification. The cache > > doesn't really hold any pages. It is a mere counter of how many charges > > have been accounted for the memcg page counter. So it is not really > > consuming proportional amount of resources. It just pins the > > corresponding memcg. Have a look at consume_stock and refill_stock > > I see. Thanks for pointing that out! > > So in worst case scenario the memcg would have reserved 64 pages * (numcpus - 1) s@numcpus@num_isolated_cpus@ > that are not getting used, and may cause an 'earlier' OOM if this amount is > needed but can't be freed. s@OOM@memcg OOM@ > In the wave of worst case, supposing a big powerpc machine, 256 CPUs, each > holding 64k * 64 pages => 1GB memory - 4MB (one cpu using resources). > It's starting to get too big, but still ok for a machine this size. It is more about the memcg limit rather than the size of the machine. Again, let's focus on actual usacase. What is the usual memcg setup with those isolcpus > The thing is that it can present an odd behavior: > You have a cgroup created before, now empty, and try to run given application, > and hits OOM. The application would either consume those cached charges or flush them if it is running in a different memcg. Or what do you have in mind? > You then restart the cgroup, run the same application without an issue. > > Even though it looks a good possibility, this can be perceived by user as > instability. > > > > > > b - Would it cache any kind of bigger page, or huge page in this same aspect? > > > > The above should answer this as well as those following up I hope. If > > not let me know. > > IIUC we are talking normal pages, is that it? We are talking about memcg charges and those have page granularity. -- Michal Hocko SUSE Labs