From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 673C2C10F04 for ; Fri, 1 Dec 2023 17:10:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D0676B04EA; Fri, 1 Dec 2023 12:10:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9587F6B04EB; Fri, 1 Dec 2023 12:10:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D1FD6B04EC; Fri, 1 Dec 2023 12:10:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 66FDF6B04EA for ; Fri, 1 Dec 2023 12:10:00 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 31E80A02C6 for ; Fri, 1 Dec 2023 17:10:00 +0000 (UTC) X-FDA: 81518886960.24.40DD0E6 Received: from mail-qv1-f49.google.com (mail-qv1-f49.google.com [209.85.219.49]) by imf09.hostedemail.com (Postfix) with ESMTP id 017A814000A for ; Fri, 1 Dec 2023 17:09:57 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=NevaA0P0; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf09.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.49 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701450598; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UHSBlrWKXm1wAl594fX7qblvpqL30ZdbgTyCgGt+Df8=; b=u/rIWymRtu+Cja3B7YoBrsDE1L/OiSOffMlnkV/UoW8nW2glUIajxE1gLIYPmhlg/paOOp /8yVBuvIs98LMUPuVqvw7wD2vlb4yKJv3rMccZ2XajRVmpP5FVyjCiHt7Spv7YF+eQR5Yk /KUwAgktpMA8VPBHJvwMJLri1QoSkiU= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=NevaA0P0; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf09.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.49 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701450598; a=rsa-sha256; cv=none; b=nwQGSMb4zjCBasRskhQzZhkgwNhho8RjJuerwuli+1H4Afq1Ktw0YkzPGcBwAAzNeyaoc2 MIb5IiSzbN6WtHw//kp1us2v0qKOhpzKVAO1MT55yKRve7DEnGXgvr80ikCUWbJgzVFPFB RbNRzz/E/thzTtQG0tbKK9/gFdZHBio= Received: by mail-qv1-f49.google.com with SMTP id 6a1803df08f44-67a12079162so12065116d6.1 for ; Fri, 01 Dec 2023 09:09:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1701450597; x=1702055397; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=UHSBlrWKXm1wAl594fX7qblvpqL30ZdbgTyCgGt+Df8=; b=NevaA0P0FEyTHxjwOZeu6msYoioppDRDAtGLYGI5dTCfUSI/LVif1FTOuUliwuWEtH 9PDXbe2FUuVm8UjQ7g06QbW1JfRz/z8LYbvBfO5lgSEA3UyevVYBWfwZqgHtqvrqsNZM g5I8JXRBhJf7GoR5cXshzK9jvDIRHA7A2D9k3vT9V8ejRLh3EuD4VReL7y8dfctDLm3I L4BFfTuhBRIKYk109EgOBfW6IVsBE6e0PpFUoRsAw3EejVMs55I9NU8652A6zyzP4ywT 1lbP5aff5G12Y3qo5qjTXaRmHact8p4FkyM0daG+Kvnv7yOGhzbB+VzOkDJHhyh8iJjY PRKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701450597; x=1702055397; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=UHSBlrWKXm1wAl594fX7qblvpqL30ZdbgTyCgGt+Df8=; b=qhfarE9aU2PZ/6ujle9MUcuu5RF7kwe2DADm1lKooQDlKSC+ybhwQShDbkAoyfeUju iSYN63C35FMLNoUg6cxHdA++Oa4hCFWauTc3IV2aiwW4OIftpICxZ9oYQyKQXcbA6FR4 X+Nloa7JJknnO85+PwyKy/2rYILRntlFTa1A6s9ktl4nvwc+25frNRdej7fIZwcCDhjR nsoHzbjTXUbz7IUQCly9r3PsaI8PhOakcBMHRWMP8YxoIoXJKCRc6S4LHOCgCCbYyS63 eaW4e5HqkKrLq/YS1EIKQ4sYV68MFYhesGrVdVDKclyEjPaRW7Wd5HgWYMO+TViD3wnm kq/g== X-Gm-Message-State: AOJu0YwhOhsOExaWCOSk3/T26MONyYh54UBUwlLMy9UZ7GfxVuuZ856B G64SFm/X+7uW0y7SzE+M/fA9tw== X-Google-Smtp-Source: AGHT+IHqQfbSInVj9aP1pUXmAYQeii4YG2nj7kgSlMdhi5m0/8lKOirnLYXwBxYrgyzVTwed0GPvbA== X-Received: by 2002:a0c:e90e:0:b0:67a:fd5:24a9 with SMTP id a14-20020a0ce90e000000b0067a0fd524a9mr30266352qvo.19.1701450596905; Fri, 01 Dec 2023 09:09:56 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id du5-20020a05621409a500b0067aa28ac616sm255221qvb.113.2023.12.01.09.09.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Dec 2023 09:09:56 -0800 (PST) Date: Fri, 1 Dec 2023 12:09:55 -0500 From: Johannes Weiner To: Michal Hocko Cc: Dan Schatzberg , Roman Gushchin , Yosry Ahmed , Huan Yang , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Shakeel Butt , Muchun Song , Andrew Morton , David Hildenbrand , Matthew Wilcox , Huang Ying , Kefeng Wang , Peter Xu , "Vishal Moola (Oracle)" , Yue Zhao , Hugh Dickins Subject: Re: [PATCH 0/1] Add swappiness argument to memory.reclaim Message-ID: <20231201170955.GA694615@cmpxchg.org> References: <20231130153658.527556-1-schatzberg.dan@gmail.com> <20231130165642.GA386439@cmpxchg.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 017A814000A X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: bocxktgrzzwmymhp9x3uzusthj8qxhhm X-HE-Tag: 1701450597-21935 X-HE-Meta: U2FsdGVkX1954QLFLdXbsvb66pYtILG/nIv/euCUrP5ZT2ESS02jW1xCaiGn5GrFFjpxSyZkDXx/HMPPXqB6Pajh3I9c70qFJ4Nz34wI/nlQrQ2u+Sz26QoJMIfyZwBvIB/p09gPz1GjZ8IrJzPbOccGF2Laj8ui6wGzO781Vjy6dalVkNEWGB+G4Mc15Rct+X+vZX11Pz6TdNQ2xX/wfLU9S9AAJKKIBrvmqoEosBxRVPu5Leo4nvlKadKgXuSJpW0N/AUWhURQ16+xw9XfQbiB+DP7IGOBLKcbqrIOLXp9l5q4eZDADxENtbegWAEN74uxpC5jg611e6ExeNhSyvLY0cgM3/xqavYs9RUzUSOV/Lt1OsjonHgT4OMmMhP3t++murU/+PXlIl5QzoIw/h7OofOk/BmBYDQcXuuapaE4GVrwxIrGawq1UuIDqXwnpC84gEucgU0Xqm3/S0Z7JoxMEdjdFtCE02EZ49XEheCOj/l2RKfMQifpTzjnaGrHWT7IvM91zxBiWb76YtLbw7PEJaIhIE3rQNyXfmrXEtckkeA265SRQsx8CT2AnOrNRkR1TCbE1UZ77wrTMVK8bC2DraD4MQBWU0k+c2mItO+Y9JMY+gU0LS+wf1+mnixmUGLolgsmUgZX106KVh4ML/Vcdq+xQ5PbgdYagG6jcDobsTXAokeJe+RZ09citJmLL+Pl0HZFc0mldzl28zy3dRRhwXo4NqDqGOgPFGcEq7CiEYpuf7uPzvkOUTbdhGjgfnOzjPQNTlQQRoiyZPvldClDJ7ntwggx2aHm4zyCD/Hmsq032ZYcuEsiMucKz3wLBzOr7cX9k/6mLzvl7bq/hKomJrIJKic0dqFXKQ/g7Vuds5EhkBoU0cX58dke1h4HI3Vpq6yaD79gK8ogS7R2ySAy89utgjtgpPuRB3gD0IZhR8XYjdUT786dloQS8FCDaURA320Ov1BK8vEaMd2 bFy7r6Pw TjdWm25iMLIkuuISYoMAFyfUnomdIKvYy0ymGa/4PFTaELmJwupG1JNMggyX4btLUlqKV+inRRX4z9FIHHmKrAZuZc6h+gtwnpYLnNwCMp/gDHa1NUMu6efS5H+m8+Tw6njVP7kmOBUmOfRUeQE6w5K5XhJVCd1a2vihVs+uIJwG29yZmi35A3Z+8kv32txT7Uvu/elzFpeCBp+BpXRvC2KY3ql5/GE2q7Wa/8vzBvtRH/CBjiUYRsYIgr8GqTzBLbE3QiGdCAMWP1Ov0eo53zseuMLuAldq9OP6NxPqudRzWk3prxZEwVI6KzMPMJB1bG0Ca0a9qBw81ZkD9MkGjRKVzPdavQovNwJcYjOMBzKeC1am3y2knRJQXeHyGkmM5QBkQpaY3PKel4n5pKFaPRvgM+WC2PHaWtdExQnBQLB4BRfQPZScvJkTLbgfVQkpbIUDfNOxL0H9jKg3FAeBd811pqg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Dec 01, 2023 at 10:33:01AM +0100, Michal Hocko wrote: > On Thu 30-11-23 11:56:42, Johannes Weiner wrote: > [...] > > So I wouldn't say it's merely a reclaim hint. It controls a very > > concrete and influential factor in VM decision making. And since the > > global swappiness is long-established ABI, I don't expect its meaning > > to change significantly any time soon. > > As I've said I am more worried about potential future changes which > would modify existing, reduce or add more corner cases which would be > seen as a change of behavior from the user space POV. That means that we > would have to be really explicit about the fact that the reclaim is free > to override the swappiness provided by user. So essentially a best > effort interface without any actual guarantees. That surely makes it > harder to use. Is it still useable? But it's not free to override the setting as it pleases. I wrote a detailed list of the current exceptions, and why the user wouldn't have strong expectations of swappiness being respected in those cases. Having reasonable limitations is not the same as everything being up for grabs. Again, the swappiness setting is ABI, and people would definitely complain if we ignored their request in an unexpected situation and regressed their workloads. I'm not against documenting the exceptions and limitations. Not just for proactive reclaim, but for swappiness in general. But I don't think it's fair to say that there are NO rules and NO userspace contract around this parameter (and I'm the one who wrote most of the balancing code that implements the swappiness control). So considering what swappiness DOES provide, and the definition and behavior to which we're tied by ABI rules, yes I do think it's useful to control this from the proactive reclaim context. In fact, we know it's useful, because we've been doing it for a while in production now - just in a hacky way, and this patch is merely making it less hacky. > Btw. IIRC these concerns were part of the reason why memcg v2 doesn't > have swappiness interface. If we decide to export swappiness via > memory.reclaim interface does it mean we will do so on per-memcg level > as well? Well I'm the person who wrote the initial cgroup2 memory interface, and I left it out because there was no clear usecase for why you'd want to tweak it on a per-container basis. But Dan did bring up a new and very concrete usecase: controlling for write endurance. And it's not just a theoretical one, but a proven real world application. As far as adding a static memory.swappiness goes, I wouldn't add it just because, but wait for a concrete usecase for that specifically. I don't think Dan's rationale extends to it. But if a usecase comes up and is convincing, I wouldn't be opposed to it.