From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA073C4332F for ; Fri, 2 Dec 2022 21:53:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5422B6B0071; Fri, 2 Dec 2022 16:53:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F29C6B0072; Fri, 2 Dec 2022 16:53:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BB096B0073; Fri, 2 Dec 2022 16:53:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 27A5B6B0071 for ; Fri, 2 Dec 2022 16:53:08 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id DD5791C6F0D for ; Fri, 2 Dec 2022 21:53:07 +0000 (UTC) X-FDA: 80198717214.10.47265EA Received: from mail-vk1-f175.google.com (mail-vk1-f175.google.com [209.85.221.175]) by imf07.hostedemail.com (Postfix) with ESMTP id 8B6F44000C for ; Fri, 2 Dec 2022 21:53:07 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Ydy+x0SU; spf=pass (imf07.hostedemail.com: domain of almasrymina@google.com designates 209.85.221.175 as permitted sender) smtp.mailfrom=almasrymina@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670017987; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ct0cPrSi092x9+JQYVwzrM1Jm4HkMAgjSbtFI3tq/NY=; b=bhmFVREGLZYDbedPXl8I37xoFgZ8f5tNNDsY0qR4lO0ygVhxwOEXvlAQwK7hD0lyt6fC5M 3yPMrGHHRzLkq69UdWo7ooFecOB4/fG95SIVowyTvu5wrkuX6UzwWoWf4LeO/gC8R/aJYR 0M/B0K0GPs6Pzo5HaTf5AL6+g33GvjE= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Ydy+x0SU; spf=pass (imf07.hostedemail.com: domain of almasrymina@google.com designates 209.85.221.175 as permitted sender) smtp.mailfrom=almasrymina@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670017987; a=rsa-sha256; cv=none; b=QTkJC/GnpWMt5dhtj8eTLBRq0g8hOdLZcMmq75OIaNTPExSFKvtCU8BFScBiHtVaMYqYsE S3aBKXPF9bVs6Vm8hfRv8wq1PZ/fPh5kTxgh3SH3+pyyNKsle9732/6m53viV/85xda5if u2pHQYukTs3Kr2+KYReT+QfE78YwrdU= Received: by mail-vk1-f175.google.com with SMTP id o5so591095vkn.7 for ; Fri, 02 Dec 2022 13:53:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Ct0cPrSi092x9+JQYVwzrM1Jm4HkMAgjSbtFI3tq/NY=; b=Ydy+x0SUjPBd8rlBTfLQvfQhclcGeQye4ZJmZb6dCvZmpCoe1U4EPuT6V4IWgDo77G N69hl/0x/n+puIAwK3nUedXq/dZflpq6qY0nHg45KIFbrO74J91Csdi2EiO7QkOi4gO4 Y9zAZMNEcOL5DXQw+DycdP2OtblBh/zNMx8RFiA9MAFSl2Xopv5koYj2nlbU5p/jKQTc wYV/PAhb8PEIqGl32evepYaBJayxX/B6Zsd8supMTxEIfzswC++WMSjXttHkVv/PBReS XwJEoU6Mgl+nWIRqszyVLX3tBpYnhjyfJV6zvgMGPTVfvbDzAjarPL31HOxD1u7j1vE5 SWZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Ct0cPrSi092x9+JQYVwzrM1Jm4HkMAgjSbtFI3tq/NY=; b=yOZ7CaAeLfoVJpjzE8QWLXOkxiMleVLUrebiYJY0ZUyo3anjLsZ+zNhpvFXlWAIw2L M+XiDj6eaOoWk3n+1U/mHSeUOQrqw4p7RiPcCUAK9ecP3/qek+85nDSXS+f53+Y/Figb b6i8aCAiSlEgvRoBczLzhyrLmyCyrMiq3MAmNYP0Wmts0T1BKbAASmMDJjeyerfMUItg G3aDUpERRyc2OlG8Ve08RfyN5sBSn9C4wwEnU/i6hDxjnjD58p/uSDImpFvsJlJc3Q6f GNJ+uk3AaJPfF6eNGRf+fQnXz3XNcGDpE2fsZhnKRx/nIw0nAD4aA+6xYnHXUGeZgYkK zqfA== X-Gm-Message-State: ANoB5plmj1IlQOu0EeR4kMea+v+TaDk1H6iK1dhVx1caqOrCw6uq75cy krw/1Z4eoYsWoZ0ScwWdSPcDwlSBENr5WXYcbyEPyA== X-Google-Smtp-Source: AA0mqf4Zn/JmvGfUC7Y8Dukg3p9YPL5jtHAQXNAsLflE5CoMvabxjvRudsoThZhBWzF1392f10iPs0qJZGYVrFCli1Q= X-Received: by 2002:a05:6122:482:b0:3bc:d827:7113 with SMTP id o2-20020a056122048200b003bcd8277113mr16603367vkn.0.1670017986741; Fri, 02 Dec 2022 13:53:06 -0800 (PST) MIME-Version: 1.0 References: <20221201233317.1394958-1-almasrymina@google.com> <20221202133840.5cdd4270cf73eaaa1d9d0345@linux-foundation.org> In-Reply-To: <20221202133840.5cdd4270cf73eaaa1d9d0345@linux-foundation.org> From: Mina Almasry Date: Fri, 2 Dec 2022 13:52:55 -0800 Message-ID: Subject: Re: [PATCH v1] mm: disable top-tier fallback to reclaim on proactive reclaim To: Andrew Morton Cc: Huang Ying , Yang Shi , Yosry Ahmed , Tim Chen , weixugc@google.com, shakeelb@google.com, gthelen@google.com, fvdl@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [1.72 / 9.00]; SORBS_IRL_BL(3.00)[209.85.221.175:from]; BAYES_HAM(-1.38)[83.20%]; BAD_REP_POLICIES(0.10)[]; RCVD_NO_TLS_LAST(0.10)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DMARC_POLICY_ALLOW(0.00)[google.com,reject]; RCPT_COUNT_SEVEN(0.00)[11]; DKIM_TRACE(0.00)[google.com:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[linux-mm@kvack.org]; R_DKIM_ALLOW(0.00)[google.com:s=20210112]; ARC_SIGNED(0.00)[hostedemail.com:s=arc-20220608:i=1]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:209.85.128.0/17]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[] X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 8B6F44000C X-Stat-Signature: kh8izjgtzsi3uz7kdsucqgmay3oamyqn X-HE-Tag: 1670017987-974516 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Dec 2, 2022 at 1:38 PM Andrew Morton wrote: > > On Thu, 1 Dec 2022 15:33:17 -0800 Mina Almasry wrote: > > > Reclaiming directly from top tier nodes breaks the aging pipeline of > > memory tiers. If we have a RAM -> CXL -> storage hierarchy, we > > should demote from RAM to CXL and from CXL to storage. If we reclaim > > a page from RAM, it means we 'demote' it directly from RAM to storage, > > bypassing potentially a huge amount of pages colder than it in CXL. > > > > However disabling reclaim from top tier nodes entirely would cause ooms > > in edge scenarios where lower tier memory is unreclaimable for whatever > > reason, e.g. memory being mlocked() or too hot to reclaim. In these > > cases we would rather the job run with a performance regression rather > > than it oom altogether. > > > > However, we can disable reclaim from top tier nodes for proactive reclaim. > > That reclaim is not real memory pressure, and we don't have any cause to > > be breaking the aging pipeline. > > > > Is this purely from code inspection, or are there quantitative > observations to be shared? > This is from code inspection, but also it is by definition. Proactive reclaim is when the userspace does: echo "1m" > /path/to/cgroup/memory.reclaim At that point the kernel tries to proactively reclaim 1 MB from that cgroup at the userspace's behest, regardless of the actual memory pressure in the cgroup, so proactive reclaim is not real memory pressure as I state in the commit message. Proactive reclaim is triggered in the code by memory_reclaim(): https://elixir.bootlin.com/linux/v6.1-rc7/source/mm/memcontrol.c#L6572 Which sets MEMCG_RECLAIM_PROACTIVE: https://elixir.bootlin.com/linux/v6.1-rc7/source/mm/memcontrol.c#L6586 Which in turn sets sc->proactive: https://elixir.bootlin.com/linux/v6.1-rc7/source/mm/vmscan.c#L6743 In my patch I only allow falling back to reclaim from top tier nodes if !sc->proactive. I was in the process of sending a v2 with the comment fix btw, but I'll hold back on that since it seems you already merged the patch to unstable. Thanks! If I end up sending another version of the patch it should come with the comment fix.