From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE78CC34056 for ; Wed, 19 Feb 2020 19:17:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8FB1D24670 for ; Wed, 19 Feb 2020 19:17:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="HSImXs47" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8FB1D24670 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chrisdown.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 400276B0006; Wed, 19 Feb 2020 14:17:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 388956B0007; Wed, 19 Feb 2020 14:17:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 251926B0008; Wed, 19 Feb 2020 14:17:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0181.hostedemail.com [216.40.44.181]) by kanga.kvack.org (Postfix) with ESMTP id 0A31F6B0006 for ; Wed, 19 Feb 2020 14:17:08 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id C42D38248D52 for ; Wed, 19 Feb 2020 19:17:07 +0000 (UTC) X-FDA: 76507834494.04.wash61_7865be2eb432c X-HE-Tag: wash61_7865be2eb432c X-Filterd-Recvd-Size: 4430 Received: from mail-qt1-f194.google.com (mail-qt1-f194.google.com [209.85.160.194]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Wed, 19 Feb 2020 19:17:07 +0000 (UTC) Received: by mail-qt1-f194.google.com with SMTP id i14so1026091qtv.13 for ; Wed, 19 Feb 2020 11:17:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=rANX/Xd0q5oumfVjHme1j9P+031qDgStPNCqoS/CWNk=; b=HSImXs472qEzrnVI+WL7Nop5RBlDH02Ti0aEwsTdZ4cSXbYVeBHGUJ469Zly4yC2gR jBMOl0dEmEC36yW5IHEYw0JYVWKAnqu8OxEHSMuV/K+joeaLqY3cLHhwcgZsCGoXuMjX t8PEhn5xfL7/e2amuMCadASeZa+BmzA8yLgCo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=rANX/Xd0q5oumfVjHme1j9P+031qDgStPNCqoS/CWNk=; b=GFIJ1DC4bUX99rnHoET+0DQB9fPKS6QQ3cNh3oBaFtkmh9cpcs4k5gkaklBN38pkR9 DDByEV8/yLHNTxZ/fQyUvVpDOcxQenZx6QXuqOEqx4ukbRBYo3aqBdUog3Es7HzGJI/N 5x4DfQob1trryzwcNX/6pCZihap8U3CAE5O9C+kBZHFYa7Ctm1Qb0ajqGjvWNCl3AvAV 3rZ+5FsEYTzwihEUs6bsz2tZ4lwjMlfPS/tDrw1qizqImv6d/u7ncuEzcszCaY41SHaL 7XntqYZstxYiCIZX7teG408OblDqCkDk//7F3PaZQ84IiOR/x4aAukwfIRIY8+EyyKG3 9TYw== X-Gm-Message-State: APjAAAXaPje+kPrdszkH6oelBVCGNTyygU2p26fJV00uJK1vI6/GZ7yl pbVsjhA4yjNFWAc2ZggTYGwH3A== X-Google-Smtp-Source: APXvYqzuBiEv0vvgKtn5PNq1EZZbkyC5xuZeib29M2sEV7JjYUoe8S+nrCLA/Pu970K3tyH+0yzpyQ== X-Received: by 2002:aed:2584:: with SMTP id x4mr23824321qtc.343.1582139826677; Wed, 19 Feb 2020 11:17:06 -0800 (PST) Received: from localhost ([2620:10d:c091:500::37cd]) by smtp.gmail.com with ESMTPSA id w202sm324923qkb.89.2020.02.19.11.17.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2020 11:17:06 -0800 (PST) Date: Wed, 19 Feb 2020 14:17:05 -0500 From: Chris Down To: Michal Hocko Cc: Johannes Weiner , Andrew Morton , Tejun Heo , Roman Gushchin , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: Re: [PATCH] mm: memcontrol: asynchronous reclaim for memory.high Message-ID: <20200219191705.GA1357312@chrisdown.name> References: <20200219181219.54356-1-hannes@cmpxchg.org> <20200219183731.GC11847@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <20200219183731.GC11847@dhcp22.suse.cz> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Michal Hocko writes: >On Wed 19-02-20 13:12:19, Johannes Weiner wrote: >> We have received regression reports from users whose workloads moved >> into containers and subsequently encountered new latencies. For some >> users these were a nuisance, but for some it meant missing their SLA >> response times. We tracked those delays down to cgroup limits, which >> inject direct reclaim stalls into the workload where previously all >> reclaim was handled my kswapd. > >I am curious why is this unexpected when the high limit is explicitly >documented as a throttling mechanism. Throttling is what one expects if one's workload does not respect the high threshold (ie. if it's not possible to reclaim the requisite number of pages), but you don't expect throttling just because you're brushing up against the threshold, because we only reclaim at certain points (eg. return to usermode). That is, the workload may be well-behaved and it's just we didn't get around to completing reclaim yet. In that case, throttling the workload when there's no evidence it's misbehaving seems unduly harsh, hence the ~4% grace, with exponential penalties as there's more evidence the workload is pathological. So sure, memory.high is a throttling mechanism if you *exceed* the stated bounds of your allocation, but this is about even those applications which are well-behaved, and just brushing against the bounds of it, as is expected on a system where the bottleneck is at the cgroup rather than being global.