From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F97FC433E0 for ; Mon, 18 Jan 2021 16:07:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BECF322C9C for ; Mon, 18 Jan 2021 16:07:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BECF322C9C Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1A37D6B020C; Mon, 18 Jan 2021 11:07:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1538D6B024A; Mon, 18 Jan 2021 11:07:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F10D86B024C; Mon, 18 Jan 2021 11:07:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0181.hostedemail.com [216.40.44.181]) by kanga.kvack.org (Postfix) with ESMTP id D795F6B020C for ; Mon, 18 Jan 2021 11:07:28 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A37AC181AC9C6 for ; Mon, 18 Jan 2021 16:07:28 +0000 (UTC) X-FDA: 77719375776.19.body28_010280d2754a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 77E751AD1B1 for ; Mon, 18 Jan 2021 16:07:28 +0000 (UTC) X-HE-Tag: body28_010280d2754a X-Filterd-Recvd-Size: 4679 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Mon, 18 Jan 2021 16:07:27 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1610986046; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=JNQs8E/IK8Nde3e0PCSuz8+Pj2HndUD2dN/6u3bCedk=; b=GWaRX4IuWfbZ7SJ3piEOKp5lCKRKnVuAMZA0GxbUsuyMiIw769sw7d63xngi+B0XzJ7z1q iZd7uzetTEUuA8Pj04XAG8y0FmFg0394YpeSxQ3H3mONYynGJSdsgFrekw7GewRvVsmGkY RYDV1qqdOX2EBuFcEeUp3XWI/ljG/4Q= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 584BEAD19; Mon, 18 Jan 2021 16:07:26 +0000 (UTC) Date: Mon, 18 Jan 2021 17:07:22 +0100 From: Michal Hocko To: Christoph Lameter Cc: Vlastimil Babka , Jann Horn , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Linux-MM , kernel list , Thomas Gleixner , Sebastian Andrzej Siewior , Roman Gushchin , Johannes Weiner , Shakeel Butt , Suren Baghdasaryan , Minchan Kim Subject: Re: SLUB: percpu partial object count is highly inaccurate, causing some memory wastage and maybe also worse tail latencies? Message-ID: <20210118160722.GG14336@dhcp22.suse.cz> References: <20210118110319.GC14336@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon 18-01-21 15:46:43, Cristopher Lameter wrote: > On Mon, 18 Jan 2021, Michal Hocko wrote: > > > > Hm this would be similar to recommending a periodical echo > drop_caches > > > operation. We actually discourage from that (and yeah, some tools do that, and > > > we now report those in dmesg). I believe the kernel should respond to memory > > > pressure and not OOM prematurely by itself, including SLUB. > > > > Absolutely agreed! Partial caches are a very deep internal > > implementation detail of the allocator and admin has no bussiness into > > fiddling with that. This would only lead to more harm than good. > > Comparision to drop_caches is really exact! > > Really? The maximum allocation here has a upper boundary that depends on > the number of possible partial per cpu slabs. And number of cpus and caches... > There is a worst case > scenario that is not nice and wastes some memory but it is not an OOM > situation and the system easily recovers from it. There is no pro-active shrinking of those when we are close to the OOM so we still can go and kill a task while there is quite some memory sitting in a freeable slub caches unless I am missing something. We have learned about this in a memcg environment on our distribution kernels where the problem is amplified by the use in memcgs with a small limit. This is an older kernel and I would expect the current upstream will behave better with Roman's accounting rework. But still it would be great if the allocator could manage its caches depending on the memory demand. > The slab shrinking is not needed but if you are concerned about reclaiming > more memory right now then I guess you may want to run the slab shrink > operation. Yes, you can do that. In a same way you can shrink the page cache. Moreover it is really hard to do that somehow inteligently because you would need to watch the system very closely in order to shrink when it is really needed. That requires a deep understanding of the allocator. > Dropping the page cache is bad? Well sometimes you want more free memory > due to a certain operation that needs to be started and where you do not > want the overhead of page cache processing. It is not bad if used properly. My experience is that people have developed instinct to drop caches whenever something is not quite right because Internet has recommended that. -- Michal Hocko SUSE Labs