From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7A32C4167B for ; Thu, 30 Nov 2023 16:56:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 69CA56B048B; Thu, 30 Nov 2023 11:56:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 64C346B048C; Thu, 30 Nov 2023 11:56:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4ED6C6B048D; Thu, 30 Nov 2023 11:56:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 3CCB26B048B for ; Thu, 30 Nov 2023 11:56:52 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D5A4C4020B for ; Thu, 30 Nov 2023 16:56:51 +0000 (UTC) X-FDA: 81515225022.10.5D65098 Received: from mail-qv1-f49.google.com (mail-qv1-f49.google.com [209.85.219.49]) by imf09.hostedemail.com (Postfix) with ESMTP id C881F140004 for ; Thu, 30 Nov 2023 16:56:49 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="Y/czfOWW"; spf=pass (imf09.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.49 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701363410; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/OnsmpUH3uPWb9adT3MVl22lO/c/KEyWJNg1p4mH6C4=; b=Vdj6wwm14h35xqOL+4c9zwvNscjM+P0PwVxjvS9zKQVyCNYR+dWyiramS9CHc8O4xik9tf +AKp9fatPLeh04UDsDttJl/WceUJfMK5Xbi6JEsA6rUFBE5gXX3lF1mSCJcLAXflqR/BmE 2SlOGaeWKt6RNFK8FoM5jti8GGAYQIo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701363410; a=rsa-sha256; cv=none; b=cTYyDE3vIVVgQmPEtenoEovUWjXyKg2+hc4SgFF8VkT+mVkLGgV2W692kwvd0/1m1izYH0 eIKvwctM5l5ri8PRD9qM0Hz6tYtxMVa8GXBfLXfFz7ZopuO0Y2fA8RvTc2FN94vjwnnL7q /x0gDBCzWTuCVMyj/YpJbg5eHNWWmEw= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="Y/czfOWW"; spf=pass (imf09.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.49 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org Received: by mail-qv1-f49.google.com with SMTP id 6a1803df08f44-67a73619fc4so6498396d6.2 for ; Thu, 30 Nov 2023 08:56:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1701363409; x=1701968209; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=/OnsmpUH3uPWb9adT3MVl22lO/c/KEyWJNg1p4mH6C4=; b=Y/czfOWWsK1gs9dGrb3niaCwYink9xjwk0dq00IyB1iZjVLn2xsAZIY6fdxbXDml70 Nm4tdbYgsnipCvtE9bcJ7gi1S1DjBKZRaQ0fr9AYasBgGxWVEUcQ0R4RjSPn9X791EZx qoV4cKsfsma8qKwbqxOpoio67/58mvwgMmrGw6wUsDYQwG4fQM6YsO+5P9h57cSQVIBh iBVsgXln3nls8H6EhnLMATSgMYyW83uKHT9fX55/HTFt1M2bQEDNqC/Gh6WU031GAu7t IRA4k8oxveRn0vIfGqyUMs3PyoKdtJnVXt0EjzFBeSJ39yDQwFTt4wx7BuzYXbtUnG1H 0HFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701363409; x=1701968209; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=/OnsmpUH3uPWb9adT3MVl22lO/c/KEyWJNg1p4mH6C4=; b=RL0NiDQ0i5a6twIy/+4IfE8qAkdoOdaf5/6u2jOtniwRsuXzt2OGE9lqT3o4vfykXo qn2Su4hQqMDEKRpDAHr0dcEZEm/t5Da6lqD28k/OMEJwrZ46EIC0aS8w5IPKfEzrG4HX +/gyA/7Y0N4vPJ+P36NroLUeQ6T+9JbMVO7Ostxd5KGZBYETGGCj45q6S1vLqz9r9LZ4 BYqCv1mIShywXE+ZG47hhGzEdqXdE9+EJv6DY5ErYttpETPc/Z3YWmRRKrUN26fW0mCK Humaree0ITvZHB7HRPWBIAFszxxWgIlKNcTehzDkBeLG2uDnsRAgyu4YZRAN2OxAXPSN Eq+A== X-Gm-Message-State: AOJu0Yy1QMPjTBEgzjLg25e8MRRgXybo0B8V82CdYtjkCMBBcdrUXPIa uIdnjUvF3yjsDPm5A2qPP+IRqw== X-Google-Smtp-Source: AGHT+IHcJpQUxaVA/+lad1maiVUeE9yNLpI5CPPwm06wwTHhVR25W6sk4PQ/92Cm1s0DXwkSDvsBGA== X-Received: by 2002:a05:6214:1387:b0:67a:4ab3:991a with SMTP id pp7-20020a056214138700b0067a4ab3991amr14101831qvb.60.1701363408836; Thu, 30 Nov 2023 08:56:48 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id n3-20020a0ce483000000b0067a33133420sm645030qvl.110.2023.11.30.08.56.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Nov 2023 08:56:48 -0800 (PST) Date: Thu, 30 Nov 2023 11:56:42 -0500 From: Johannes Weiner To: Michal Hocko Cc: Dan Schatzberg , Roman Gushchin , Yosry Ahmed , Huan Yang , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Shakeel Butt , Muchun Song , Andrew Morton , David Hildenbrand , Matthew Wilcox , Huang Ying , Kefeng Wang , Peter Xu , "Vishal Moola (Oracle)" , Yue Zhao , Hugh Dickins Subject: Re: [PATCH 0/1] Add swappiness argument to memory.reclaim Message-ID: <20231130165642.GA386439@cmpxchg.org> References: <20231130153658.527556-1-schatzberg.dan@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: he1b8fo6f4qy9ofb8xtkmdjw7m8oufwr X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: C881F140004 X-Rspam-User: X-HE-Tag: 1701363409-818198 X-HE-Meta: U2FsdGVkX1+OjP5ViIg1767NFlEwzrx1Hz5MvWhnKFwZEvZiYbiS9GSV6wYExAdWsGT2HhRPc6Go0H0xgMjinGlkAOaJ8Xue/zf7C1P5g6kQ8JQrX+0W+FzoUuBKeAIhSg4yP6HqHti46XREUWD8wSfCL78XVjn5lgzBZ1mfXgTfwDYSMwNJphfQFeKgiLplHe4hmedtNh5dmHiaVB4nn0nDYjYgmOVfxjzYZzKTI0EOJTgoKrjBYz5aC/bRh++MOA5HLATuwVkX1r43js/l7hWDyj4NbEV/kCQGNhrgMuZHEX2hRA8Q1Q/O6SyxBTIGhQoxdeKQZFJavxEQ+nv4X0ZC/dmONFauU7ebS4+e9W76kpThuyGXKKM+ZiPCjYd86QAPOJtwkpm5I9CmaI0ilRH0lHXql2OkEdVZriPY5ULGjIbvAradKjMfO8iWaKSrgbq9bqBeHekFUY0DBJMZ3zD2bnDeIGO2Htx3fZxoasumIWNcoSfs2V4D01lz6tWDjocq3TMjd1GLs54dty2G+zmvSoZrIGYRgANwuv3T3yy+N/rr8/GPTpEyKpLFux1vo0DZky04iHd+K3GFig+uQTssIf+PTjh9piRYRCocswMtOasRoQEs/kZdQY8/W1C2o9Oa8d1TALIPMaYYqAgs9xvjqrwzuAcdNxtP6BEwhMigsePCj2wzR88+Db6/n6f85QlcIusjYSBatMcraEPxzM9VFlvdtdtKPry0TZc72P+T84GEOmsgY8GxeSI5K57NU+yurC1REGPfe8dLcp4ujD8zsC8dKOp8o2zzJMlC8KOG5myjWpd6TlVTlat2APHx+iKKvdreurBqZjPt0qRw4P37eKTWdy0gfZ8Nih09k9m4w+uqa2WP7hucoe0+541k8cNE5lVum3qanLGUIpNMi/XuU9Y/U9oC/+5AVJ4AJLu/hu5HFOdY5zw42KmtImSQ84ZTrXJ/bUMyRqAbcqh Red8JPNA GCYBvE2JIHz3MTGttdJz5DNyksF80fPIQHtjau8tPlMBtPCnuMh1ZkuswlmW51V6F9YJPuTcnBDS02xgLcN539VrgDWTaMCQBPRofadMkJRWOm1BzeyjXVWZSnxmdEwhvulZ3TJSFQFLQDs9qHDH5+YJsKPvQRCHeJAZjv08MgKloPytiF8Sx6E85ud8/hVk/qIhIi4yXac3gD4VtWqGQQAAgJHHn2O/dfAWeHBK/v+uaXqBuDyj/ycLlsXPxc6T61UeRc4ZGIt1sWaDh1acB9F3VZLAeRHAtEkF//I3OKWxZOfaugbr1KAglmnTfocetaFzpglRJySGHBrrnyX41Un5iuFjKCk/uF81sCz0Hkaa3U1L4OmbV1mQ0Gd7Ac1IH7Yi5XO99IOm6sAgYSeiM1en80fJNqaTy+1ZM3ka1mn/z8OKlOZphqm+w0evgOjldyx3Wt1mLke2XQ5lA/PW5m2PXHQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Nov 30, 2023 at 04:57:41PM +0100, Michal Hocko wrote: > On Thu 30-11-23 07:36:53, Dan Schatzberg wrote: > [...] > > In contrast, I argue in favor of a swappiness setting not as a way to implement > > custom reclaim algorithms but rather to bias the balance of anon vs file due to > > differences of proactive vs reactive reclaim. In this context, swappiness is the > > existing interface for controlling this balance and this patch simply allows for > > it to be configured differently for proactive vs reactive reclaim. > > I do agree that swappiness is a better interface than explicit anon/file > but the problem with swappiness is that it is more of a hint for the reclaim > rather than a real control. Just look at get_scan_count and its history. > Not only its range has been extended also the extent when it is actually > used has been changing all the time and I think it is not a stretch to > assume that trend to continue. Right, we did tweak the edge behavior of e.g. swappiness=0. And we extended the range to express "anon is cheaper than file", which wasn't possible before, to support the compressed memory case. However, its meaning and impact has been remarkably stable over the years: it allows userspace to specify the relative cost of paging IO between file and anon pages. This comment is from 2.6.28: /* * With swappiness at 100, anonymous and file have the same priority. * This scanning priority is essentially the inverse of IO cost. */ anon_prio = sc->swappiness; file_prio = 200 - sc->swappiness; And this is it today: /* * Calculate the pressure balance between anon and file pages. * * The amount of pressure we put on each LRU is inversely * proportional to the cost of reclaiming each list, as * determined by the share of pages that are refaulting, times * the relative IO cost of bringing back a swapped out * anonymous page vs reloading a filesystem page (swappiness). * * Although we limit that influence to ensure no list gets * left behind completely: at least a third of the pressure is * applied, before swappiness. * * With swappiness at 100, anon and file have equal IO cost. */ total_cost = sc->anon_cost + sc->file_cost; anon_cost = total_cost + sc->anon_cost; file_cost = total_cost + sc->file_cost; total_cost = anon_cost + file_cost; ap = swappiness * (total_cost + 1); ap /= anon_cost + 1; fp = (200 - swappiness) * (total_cost + 1); fp /= file_cost + 1; So swappiness still means the same it did 15 years ago. We haven't changed the default swappiness setting, and we haven't broken any existing swappiness configurations through VM changes in that time. There are a few scenarios where swappiness doesn't apply: - No swap. Oh well, that seems reasonable. - Priority=0. This applies to near-OOM situations where the MM system tries to save itself. This isn't a range in which proactive reclaimers (should) operate. - sc->file_is_tiny. This doesn't apply to cgroup reclaim and thus proactive reclaim. - sc->cache_trim_mode. This implements clean cache dropbehind, and applies in the presence of large, non-refaulting inactive cache. The assumption there is that this data is reclaimable without involving IO to evict, and without the expectation of refault IO in the future. Without IO involvement, the relative IO cost isn't a factor. This will back off when refaults are observed, and the IO cost setting is then taken into account again as expected. If you consider swappiness to mean "reclaim what I ask you to", then this would override that, yes. But in the definition of relative IO cost, this decision making is permissible. Note that this applies to the global swappiness setting as well, and nobody has complained about it. So I wouldn't say it's merely a reclaim hint. It controls a very concrete and influential factor in VM decision making. And since the global swappiness is long-established ABI, I don't expect its meaning to change significantly any time soon.