From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8394EC433EF for ; Mon, 10 Jan 2022 15:39:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ED1B46B0071; Mon, 10 Jan 2022 10:39:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E33936B0072; Mon, 10 Jan 2022 10:39:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD4706B0074; Mon, 10 Jan 2022 10:39:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0195.hostedemail.com [216.40.44.195]) by kanga.kvack.org (Postfix) with ESMTP id B6DA76B0071 for ; Mon, 10 Jan 2022 10:39:54 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 74B1C901B7 for ; Mon, 10 Jan 2022 15:39:54 +0000 (UTC) X-FDA: 79014787908.10.0BF5850 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf03.hostedemail.com (Postfix) with ESMTP id ECBC420017 for ; Mon, 10 Jan 2022 15:39:53 +0000 (UTC) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id B460421124; Mon, 10 Jan 2022 15:39:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1641829192; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=FpgD/z1tH1n4z25ssRiC8BwrRHCQmq0t20cgjahtksA=; b=a5nL7FD8WAScsrFn7oClSv1bqemNsF2dJoGD/wehxxpMVPytSTcqwnKSQ5wCUv8KKG6s8k yn3Yq4JQCE1YyY4GlQKkiO31jokIJkxJ8FCsP95m7oEwzL+LHmPqye4nUZny4PxI+IhEIx Lmrz78siT39fe0YgQ6PngSztc4docRw= Received: from suse.cz (unknown [10.100.201.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 46A64A3B83; Mon, 10 Jan 2022 15:39:52 +0000 (UTC) Date: Mon, 10 Jan 2022 16:39:51 +0100 From: Michal Hocko To: Yu Zhao Cc: Andrew Morton , Linus Torvalds , Andi Kleen , Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Jesse Barnes , Johannes Weiner , Jonathan Corbet , Matthew Wilcox , Mel Gorman , Michael Larabel , Rik van Riel , Vlastimil Babka , Will Deacon , Ying Huang , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, page-reclaim@google.com, x86@kernel.org Subject: Re: [PATCH v6 0/9] Multigenerational LRU Framework Message-ID: References: <20220104202227.2903605-1-yuzhao@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: yuahbt3x9ix443omu9pf9chdrac5s5w1 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: ECBC420017 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=a5nL7FD8; spf=pass (imf03.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com X-HE-Tag: 1641829193-778102 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri 07-01-22 11:45:40, Yu Zhao wrote: [...] > Next, I argue that the benefits of this patchset outweigh its risks, > because, drawing from my past experience, > 1. There have been many larger and/or riskier patchsets taken; I'll > assemble a list if you disagree. No question about that. Changes in the reclaim path are paved with failures and reverts and fine tuning on top of existing fine tuning. The difference from your patchset is that they tend to be much much smaller and go incremental and therefore easier to review. > And this patchset is fully guarded > by #ifdef; Linus has also assessed on this point. I appreciate you made the new behavior an opt-in and therefore existing workloads are less likely to regress. I do not think ifdefs help all that much, though, because a) realistically the config will likely be enabled for most distribution kernels and b) the parallel reclaim implementation adds a maintenance overhead regardless of those ifdef. The later point is especially worrying because the memory reclaim is a complex and hard to review beast already. Any future changes would need to consider both reclaim algorithms of course. Hence I argue we really need a wider consensus this is the right direction we want to pursue. > 2. There have been none that came with the testing/benchmarking > coverage as this one did. Please point me to some if I'm mistaken, > and I'll gladly match them. I do appreciate your numbers but you should realize that this is an area that is really hard to get any conclusive testing for. We keep learning about fallouts on workloads we haven't really anticipated or where the runtime effects happen to disagree with our intuition. So while those numbers are nice there are other important aspects to consider like the maintenance cost for example. > The numbers might not materialize in the real world; the code is not > perfect; and many other risks... But all the top eight open source > memory hogs were covered, which is unprecedented; memcached and fio > showed significant improvements and it only takes a few commands to > see for yourselves. > > Regarding the acks and the reviewed-bys, I certainly can ask people > who have reaped the benefits of this patchset to do them, if it's > required. But I see less fun in that. I prefer to provide empirical > evidence and convince people who are on the other side of the aisle. I like to hear from users who benefit from your work and that certainly gives more credit to it. But it will be the MM community to maintain the code and address future issues. We do not have a dedicated maintainer for the memory reclaim but certainly there are people who have helped shaping the existing code and have learned a lot from the past issues - like Johannes, Rik, Mel just to name few. If I were you I would be really looking into finding an agreement with them. I myself can help you with memcg and oom side of the things (we already have discussions about those). Thanks! -- Michal Hocko SUSE Labs