From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A75AC4167B for ; Wed, 30 Nov 2022 02:15:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D185E6B0072; Tue, 29 Nov 2022 21:15:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CC8406B0073; Tue, 29 Nov 2022 21:15:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B91586B0074; Tue, 29 Nov 2022 21:15:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id AA2636B0072 for ; Tue, 29 Nov 2022 21:15:10 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2DA8140E94 for ; Wed, 30 Nov 2022 02:15:10 +0000 (UTC) X-FDA: 80188491180.30.AEF810E Received: from mail-vs1-f52.google.com (mail-vs1-f52.google.com [209.85.217.52]) by imf13.hostedemail.com (Postfix) with ESMTP id DCD6C2000D for ; Wed, 30 Nov 2022 02:15:01 +0000 (UTC) Received: by mail-vs1-f52.google.com with SMTP id d185so15952239vsd.0 for ; Tue, 29 Nov 2022 18:15:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=0JX7OVGXqhjLT5U0sxZCrCFZrJK+/Ptd8iza3dr3fMU=; b=fK/t1R6jEYGIaIccsQFMtY0W0OhO5tYVELmfG/KKVlOA90f7AmEAgqSwrhjlT087Ov Zjr15NzihKSnGTMQUHUxIhxuT3ou17X5W1y0nDS2O36heNDIwEE+Pb52dW4ajbGO+LkP ngDaqhU7EMfSMN9uIjJO7R0Ip1IQctKb8QDGKuZHd+m476Mm8rGCxaaFIYNrZdpXaBJf GEWTEfzEMM+r+wFLkgmMVmLT7rvvYEaYMTS1ZtZZJzieimTZJRwh5gxyuIutcw30B49Y CkYhspbjwY1xiX3TSmLqHlbBg7HvFlesXloEhio5Vw20Ve3MDTLyWhmqgf5/z0lQgg+n qnEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0JX7OVGXqhjLT5U0sxZCrCFZrJK+/Ptd8iza3dr3fMU=; b=Dri4nNnh4ufq5OceMLKr/B/tmuNH24Qe/yiRj+yIbiev+Zxq3ipJh65YG01DakEA9s inTijTKByt+aU5+c10jkKt/xYa/6sWq0T+1fjwTYYAaFB4YMmjI/w/2/QY7xqlmsDsUa owbsL1eMVJrcta9wC7rV6ShDQvYWFUDknOJiZEkbtr1r2po/94xyxmSUw0VPo7tm5oXr DYr0Qw0O5eEDi33ix+yoLIiPs1QRMaXmtv8gRBNznqLOGk9RCVaYQoepFywI7qPux1t2 MY2jL24eQFDDfioUWg29ilnPbgAJktMvixoXdCpHB9kDz0ZfmScOHgkbcOpE6xv3P1CS 3zSQ== X-Gm-Message-State: ANoB5plac7AjYHYUAw1xIuUrYoZLshwMn+ZT37ETRb7JxT3FWbo817nz l1+LBkKF9MkXa1Mrgzc38NCZPpGpX8WJP/7BGR4STg== X-Google-Smtp-Source: AA0mqf5qCx7brp/8Dmkd70vgLRjX378XaQj83zJ+INAgJDVwo6ip+MvAfzH9/EPsv41BaKjD1If4S16ThnMaeXL/KMA= X-Received: by 2002:a05:6102:54a5:b0:3b0:7462:a88c with SMTP id bk37-20020a05610254a500b003b07462a88cmr18688692vsb.49.1669774500963; Tue, 29 Nov 2022 18:15:00 -0800 (PST) MIME-Version: 1.0 References: <20221122203850.2765015-1-almasrymina@google.com> <874juonbmv.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <874juonbmv.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Mina Almasry Date: Tue, 29 Nov 2022 18:14:49 -0800 Message-ID: Subject: Re: [RFC PATCH V1] mm: Disable demotion from proactive reclaim To: "Huang, Ying" Cc: Johannes Weiner , Yang Shi , Yosry Ahmed , Tim Chen , weixugc@google.com, shakeelb@google.com, gthelen@google.com, fvdl@google.com, Michal Hocko , Roman Gushchin , Muchun Song , Andrew Morton , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669774501; a=rsa-sha256; cv=none; b=QmXIMvP3ylJfNtm+Jr4cuvhLDhm39tkgqjA2tHdiiwEgBoOLBedU0RAHEAk4B+OZg6exrb TbI9IN3aLCDJvtEpOVS5TjEY2PgzUdUIPbwvce58T/+/l02JJXn+T31FsKSjXIFxxBXCHD 6vAqgFwmPzRQkgZ93QBcWKW0QpnqRP8= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="fK/t1R6j"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of almasrymina@google.com designates 209.85.217.52 as permitted sender) smtp.mailfrom=almasrymina@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669774501; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0JX7OVGXqhjLT5U0sxZCrCFZrJK+/Ptd8iza3dr3fMU=; b=rEn5OXRNVYGUhr1ib0+cviXfoViHba6QKetTOp6L5elL27ru950I0/3K4SX6QtHRD2MdQZ 3fM2uXeT1ufh8/hAlAnP3GIXUNssByKJLcgI97RVyUhqQ42PLm66nTwURo2ursrBJ0WOpS 5Ak1ZgldWaaoRG+nh2YWdzALrtrDy7c= X-Rspamd-Queue-Id: DCD6C2000D Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="fK/t1R6j"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of almasrymina@google.com designates 209.85.217.52 as permitted sender) smtp.mailfrom=almasrymina@google.com X-Rspamd-Server: rspam12 X-Rspam-User: X-Stat-Signature: 18oygmj34gx4cgt9s8tgp5t7ofapmmcn X-HE-Tag: 1669774501-249924 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Nov 23, 2022 at 9:52 PM Huang, Ying wrote: > > Hi, Johannes, > > Johannes Weiner writes: > [...] > > > > The fallback to reclaim actually strikes me as wrong. > > > > Think of reclaim as 'demoting' the pages to the storage tier. If we > > have a RAM -> CXL -> storage hierarchy, we should demote from RAM to > > CXL and from CXL to storage. If we reclaim a page from RAM, it means > > we 'demote' it directly from RAM to storage, bypassing potentially a > > huge amount of pages colder than it in CXL. That doesn't seem right. > > > > If demotion fails, IMO it shouldn't satisfy the reclaim request by > > breaking the layering. Rather it should deflect that pressure to the > > lower layers to make room. This makes sure we maintain an aging > > pipeline that honors the memory tier hierarchy. > > Yes. I think that we should avoid to fall back to reclaim as much as > possible too. Now, when we allocate memory for demotion > (alloc_demote_page()), __GFP_KSWAPD_RECLAIM is used. So, we will trigger I may be missing something but as far I can tell reclaim is disabled for allocations from lower tier memory: https://elixir.bootlin.com/linux/v6.1-rc7/source/mm/vmscan.c#L1583 I think this is maybe a good thing when doing proactive demotion. In this case we probably don't want to try to reclaim from lower tier nodes and instead fail the proactive demotion. However I can see this being desirable when the top tier nodes are under real memory pressure to deflect that pressure to the lower tier nodes. > kswapd reclaim on lower tier node to free some memory to avoid fall back > to reclaim on current (higher tier) node. This may be not good enough, > for example, the following patch from Hasan may help via waking up > kswapd earlier. > > https://lore.kernel.org/linux-mm/b45b9bf7cd3e21bca61d82dcd1eb692cd32c122c.1637778851.git.hasanalmaruf@fb.com/ > > Do you know what is the next step plan for this patch? > > Should we do even more? > > From another point of view, I still think that we can use falling back > to reclaim as the last resort to avoid OOM in some special situations, > for example, most pages in the lowest tier node are mlock() or too hot > to be reclaimed. > > > So I'm hesitant to design cgroup controls around the current behavior. I sent RFC v2 patch: https://lore.kernel.org/linux-mm/20221130020328.1009347-1-almasrymina@google.com/T/#u Please take a look when convenient. Thanks! > > > > Best Regards, > Huang, Ying >