From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9368DC77B72 for ; Mon, 17 Apr 2023 11:14:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0DFD7900005; Mon, 17 Apr 2023 07:14:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 08F958E0001; Mon, 17 Apr 2023 07:14:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E734D900005; Mon, 17 Apr 2023 07:13:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D53338E0001 for ; Mon, 17 Apr 2023 07:13:59 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 9F9C6ABEC2 for ; Mon, 17 Apr 2023 11:13:59 +0000 (UTC) X-FDA: 80690623398.25.ECDB7B9 Received: from outbound-smtp23.blacknight.com (outbound-smtp23.blacknight.com [81.17.249.191]) by imf23.hostedemail.com (Postfix) with ESMTP id 179D214000B for ; Mon, 17 Apr 2023 11:13:55 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf23.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.191 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681730038; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M1Q82Roz42OPHZlHxq9vgIdt8tbFUEU3QEd3VndCNz0=; b=ERZUhXHZPpPhxGqeDLNOTVpzR7Qy58caLWPquhSOn4qHZJrfBkl3MP22qktwJtw7+rck+J K1Vy7jtAOAQrp7OAQrouZmqTgknxGUIYYuCiEufLONfpcmdSNvPmZSs4vb2CWaGBXeZBYn FZxzdsBymc8U4Cqt+XGIo5uM9/pOUhQ= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf23.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.191 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681730038; a=rsa-sha256; cv=none; b=V+zIaJUIYeZpKIJTjrYnGixcgsowvD98wcPBs/zoR7Xl+sZoY8doHR/6wz+a1PPIvoG6y5 OZpdAZjaOnYNn87ytOLpa7gGy0ApA882lwiD9Om1l/D6j5ucan90L1by9d9qoN2YN40hu0 PhYExyFv6EADJAKwkGJPYkxgN3jEI9o= Received: from mail.blacknight.com (pemlinmail04.blacknight.ie [81.17.254.17]) by outbound-smtp23.blacknight.com (Postfix) with ESMTPS id 28DF4BED0B for ; Mon, 17 Apr 2023 12:13:54 +0100 (IST) Received: (qmail 19061 invoked from network); 17 Apr 2023 11:13:54 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[84.203.21.103]) by 81.17.254.9 with ESMTPSA (AES256-SHA encrypted, authenticated); 17 Apr 2023 11:13:54 -0000 Date: Mon, 17 Apr 2023 12:13:52 +0100 From: Mel Gorman To: Wen Yang Cc: Andrew Morton , Oscar Salvador , William Lam , Fu Wei , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] mm: compaction: optimize compact_memory to comply with the admin-guide Message-ID: <20230417111352.v26slrcmz4qo3tnn@techsingularity.net> References: <20230411134801.a4aadef5aba0f51e0d44bb7a@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: 179D214000B X-Stat-Signature: 7u9raf6at59xxmxug5mzj4wgfracaeii X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1681730035-689654 X-HE-Meta: U2FsdGVkX18fSRl9oCSSQ/4ghcPBa6pEulNJvGc4NcWes+XjCJuZg7Irv2o3KDvNqC0rtqHx13IWYtZoy2w7laNfv+S/0pCvy+3D1L8oGNlmDvmab3WYmARuta03UfZjFu8nNgYJMq/K06ukUdMzvQxGDF3cnsc/SxCPRR4I++X4/5DLS6uXJSa8vJdKMF7rx+O4JGWCHC0u906cwMA2n2yqBso7Pf31OMRd/KSBSFvP+DxIKv99scV5UAx8xCzthlIzQEBhvb67RIINgUFq2x8BqUlKUua00PSWKfuAjhbCCapZCPUmfDV7/aLCxgf0r+wud73WCHb5TuL+t5OeAUwkOa4S8/8e5L2VlDUkXqqhfjzd4ShtE/KTbhQ+8QDCOWwL114fFizNY5ROvBJRyuOH9cIOVLR7iCJdGyA2lSMZoPpzwxyGmUB/gxx9O7xyRKi2nLU9TjJde5/Jpaoau4ylum4/kDiPtk+EGRjIZP0PS0BjyNg2WBkzBojLn07DMv6F9AbOpwa9qnqRNo2+2MirEET1J8gWDhQFdXwY4VYSEghPOqoIWTsyqQwdYZ30UfECmmELWNOlxlxWFRGnSeX+Iq6/GneIqhc3mFmdeMCxiWCpANGy3YFNNE1loYgTJY1eUIVsfV1lZP+zZXXY8EYLRcZK+gvfV5qMZdmqhU+6QfVWzn4HMlZODb8SDf5shjo0DY/WUcTheav64aaH5URxEFFsmgg1cVZ07hA7kTODv5IQQ+5v0lHMKef0IRXGPRHj3ORVfu+xMvKiUU4EShe0Eur5yPnsfDaOmqh0DWZa66S340GXKX80eaTXd3jEZB1FHX0IbHTnlGMJpRHuWTWTKNDLKxjP+kJrZ/gfALvI0+y9iPlPkhYNFnIM9UL1OflkVrpHphqlW49TvZ5X2citljvWA15iqYoBSVDpRL3zBaIPHzM7R9DwgAsw3o84GLueS6idMukBw54snD7 kUAojw+X aOMuBCcUGVQZFajNpgoSiREF/3wiozfuiuA+Bjaw0NbM/zW4Y+q4HBdnmn+5L105cQp6Aahm58f7Pfgqot7ui7KeQsmbUnTE516EKHrfFyyf6X+kwPqPkFHrnp5DtYHX78UnT32ajH4QX/SdV6PmFgDKmYTbSLn+y5+ZI94igVsMhvY7xR9jXJhaUAkAo+G2Nw72n0zJQwxH9Nl77hLopZkmKUSGY61dZwmpp8GwAQrxKnhTJdXwuEZJIOyokHSfyV1zX4GFm0oDGc5qdCO/SLWU4+nyECXv6dRNJX0LlFtop4bl48aoc5JA6bDYabOaN7jJgxCILt5gewROPF9wkMNLDpZFpwXPAsC6N X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, Apr 16, 2023 at 01:42:44AM +0800, Wen Yang wrote: > > ??? 2023/4/13 00:54, Wen Yang ??????: > > > > ??? 2023/4/12 04:48, Andrew Morton ??????: > > > On Wed, 12 Apr 2023 02:24:26 +0800 wenyang.linux@foxmail.com wrote: > > > > > > > For the /proc/sys/vm/compact_memory file, the admin-guide states: > > > > When 1 is written to the file, all zones are compacted such that free > > > > memory is available in contiguous blocks where possible. This can be > > > > important for example in the allocation of huge pages although > > > > processes > > > > will also directly compact memory as required > > > > > > > > But it was not strictly followed, writing any value would cause all > > > > zones to be compacted. In some critical scenarios, some applications > > > > operating it, such as echo 0, have caused serious problems. > > > Really?  You mean someone actually did this and didn't observe the > > > effect during their testing? > > > > Thanks for your reply. > > > > Since /proc/sys/vm/compact_memory has been well documented for over a > > decade: > > > > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/Documentation/admin-guide/sysctl/vm.rst#n109 > > > > > > it is believed that only writing 1 will trigger trigger all zones to be > > compacted. > > > > Especially for those who write applications, they may only focus on > > documentation and generally do not read kernel code.  Moreover, such > > problems are not easily detected through testing on low pressure > > machines. > > > > Writing any meaningful or meaningless values will trigger it and affect > > the entire server: > > > > # echo 1 > /proc/sys/vm/compact_memory > > # echo 0 > /proc/sys/vm/compact_memory > > # echo dead > /proc/sys/vm/compact_memory > > # echo "hello world" > /proc/sys/vm/compact_memory > > > > The implementation of this high-risk operation may require following the > > admin-guides. > > > > -- > > > > Best wishes, > > > > Wen > > > > > Hello, do you think it's better to optimize the sysctl_compaction_handler > code or update the admin-guide document? > Enforce the 1 on the unlikely chance that the sysctl handler is ever extended to do something different and expects a bitmask. The original intent intent of the sysctl was debugging -- demonstrating a contiguous allocation failure when aggressive compaction should have succeeded. Later some machines dedicated to batch jobs used the compaction sysctl to compact memory before a new job started to reduce startup latencies. Drop the justification "In some critical scenarios, some applications operating it, such as echo 0, have caused serious problems." from the changelog. I cannot imagine a sane "critical scenario" where an application running as root is writing expected garbage to proc or sysfs files and then surprised when something unexpected happens. -- Mel Gorman SUSE Labs