From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 630BFC4332F for ; Tue, 29 Nov 2022 18:08:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 094576B0074; Tue, 29 Nov 2022 13:08:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 044F56B0075; Tue, 29 Nov 2022 13:08:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4E996B007B; Tue, 29 Nov 2022 13:08:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id D5CFC6B0074 for ; Tue, 29 Nov 2022 13:08:05 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5AE291A03AE for ; Tue, 29 Nov 2022 18:08:05 +0000 (UTC) X-FDA: 80187263730.10.09DC1B0 Received: from mail-qv1-f49.google.com (mail-qv1-f49.google.com [209.85.219.49]) by imf16.hostedemail.com (Postfix) with ESMTP id DBD81180016 for ; Tue, 29 Nov 2022 18:08:03 +0000 (UTC) Received: by mail-qv1-f49.google.com with SMTP id d2so5334402qvp.12 for ; Tue, 29 Nov 2022 10:08:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20210112.gappssmtp.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=uLGeaz8NT3BELQqUMecnx7omQ6ldn4HqO+/oulNlF3g=; b=iJplMCRi2LhhQadE374lqFmED+tpdcxnO/27Tz7GhsxsGYaov3AHSLgVredAl4EKwf y8y410DKLDpSpT3Vg/nc8WJi8/kmTtcmzjrB7+f6ia7nIR8Fx3ZTXcjIUFvPzTGKcA7N sWqZP2O1KSGzmGfWdZulbDtgFgQt+fgC6FqTMrTCWJYjAcIvClChZJLfa3V7OLx7EIiD 3/UMSLouRXc6/u20V8fBpyyQyw4VpjI0d+y3dKINM0DN3bj8C8bVcVVlf7k657Y7eC0l C1PNjG/V/Im5mFkoETZImcdpJ9/+aiX7oXo7pgjfoqk1pwugErWo42C1OW9+7z/62bmZ EwPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=uLGeaz8NT3BELQqUMecnx7omQ6ldn4HqO+/oulNlF3g=; b=xYM1bSjjSrDoP6IoQ8SRTSuq6rggcXCUBBovoj8xQQi9VG4pjR3JS5KGF56/JCMReM fyCDFtd51z0dcSqLKVtosZ1jKI1/x6MOOXYmF5Q8DqOsQNvfbGNiEczQB5JXZpPPjaEV cgXUGhm4ePQZZ9KWZ4roVAR3Z4ppXx8btH+bgMFTIwODcDUWTkaELpOA5RaUDIBjt/yp ED38dCfDOzgNxBDJmD+Mu7ZtFPHv0r1BiOEGEtuH4I76j7gWNZHHjTbJwYwJRtCFC9W/ w5ABvMfrEkZHGa/ftQpOT5hadRgbyxdxkMa+1WqflIv/laj/rkuekiQgkFD1saJmhVMu Dv7g== X-Gm-Message-State: ANoB5plKGDqawoE8GBOj3+YsCrKXwVFOAZpUYJzIpVGTX8HIHD9TU2uP sYqq0KDbtlWxt28ahsypMyKfzw== X-Google-Smtp-Source: AA0mqf4IJkpBeqMkP10/WQYwbB4b2vUFv97YcL7SBezywirDtOwEYT22VSC2x2m31tDlZYOy8QPELw== X-Received: by 2002:a0c:90f1:0:b0:4c6:8f2e:9a2 with SMTP id p104-20020a0c90f1000000b004c68f2e09a2mr52264929qvp.100.1669745282935; Tue, 29 Nov 2022 10:08:02 -0800 (PST) Received: from localhost ([2620:10d:c091:480::1:ea9a]) by smtp.gmail.com with ESMTPSA id f3-20020a05622a114300b003434d3b5938sm8911215qty.2.2022.11.29.10.08.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Nov 2022 10:08:02 -0800 (PST) Date: Tue, 29 Nov 2022 13:08:01 -0500 From: Johannes Weiner To: "Huang, Ying" Cc: Mina Almasry , Yang Shi , Yosry Ahmed , Tim Chen , weixugc@google.com, shakeelb@google.com, gthelen@google.com, fvdl@google.com, Michal Hocko , Roman Gushchin , Muchun Song , Andrew Morton , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: Re: [RFC PATCH V1] mm: Disable demotion from proactive reclaim Message-ID: References: <20221122203850.2765015-1-almasrymina@google.com> <874juonbmv.fsf@yhuang6-desk2.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <874juonbmv.fsf@yhuang6-desk2.ccr.corp.intel.com> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669745284; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uLGeaz8NT3BELQqUMecnx7omQ6ldn4HqO+/oulNlF3g=; b=Tbi7FTsYshGboEU74SNFo6d1mjBwpHDqOYsX8zFeK1iOuQLc+X6iz6v7l25WvvhssJx6lL JxHGL05Z35XAUvjVe+71KSM3DPUJ3qCllwmRXc4WVMQNCvpyHDyrQaDU06eXJ956n4c/Sk k6V+ceIeYTxSqsj8iOFaO7Nm9iRLB8s= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=iJplMCRi; spf=pass (imf16.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.49 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669745284; a=rsa-sha256; cv=none; b=0Q9ZtaKZcd0XJ/5RwIvGbSAEnXaJ43/WP13550QiipYH6UiAplct8YlwGKtbaUsRyBrDkj DYc4xzak9Wk0m7Aa4kUiN4ts8x5PpOFl+dchAxpVLuAadadDb9HXZ6ZvW6bv7LkhiHvZFb sVotGIV5XRKIV2pTlfApmqmq8yG0RS0= X-Stat-Signature: yiph1db6oiwbkcnu7ombrime735wmqdj X-Rspamd-Queue-Id: DBD81180016 Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=iJplMCRi; spf=pass (imf16.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.49 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org X-Rspamd-Server: rspam06 X-Rspam-User: X-HE-Tag: 1669745283-267216 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hello Ying, On Thu, Nov 24, 2022 at 01:51:20PM +0800, Huang, Ying wrote: > Johannes Weiner writes: > > The fallback to reclaim actually strikes me as wrong. > > > > Think of reclaim as 'demoting' the pages to the storage tier. If we > > have a RAM -> CXL -> storage hierarchy, we should demote from RAM to > > CXL and from CXL to storage. If we reclaim a page from RAM, it means > > we 'demote' it directly from RAM to storage, bypassing potentially a > > huge amount of pages colder than it in CXL. That doesn't seem right. > > > > If demotion fails, IMO it shouldn't satisfy the reclaim request by > > breaking the layering. Rather it should deflect that pressure to the > > lower layers to make room. This makes sure we maintain an aging > > pipeline that honors the memory tier hierarchy. > > Yes. I think that we should avoid to fall back to reclaim as much as > possible too. Now, when we allocate memory for demotion > (alloc_demote_page()), __GFP_KSWAPD_RECLAIM is used. So, we will trigger > kswapd reclaim on lower tier node to free some memory to avoid fall back > to reclaim on current (higher tier) node. This may be not good enough, > for example, the following patch from Hasan may help via waking up > kswapd earlier. > > https://lore.kernel.org/linux-mm/b45b9bf7cd3e21bca61d82dcd1eb692cd32c122c.1637778851.git.hasanalmaruf@fb.com/ > > Do you know what is the next step plan for this patch? > > Should we do even more? > > From another point of view, I still think that we can use falling back > to reclaim as the last resort to avoid OOM in some special situations, > for example, most pages in the lowest tier node are mlock() or too hot > to be reclaimed. If they're hotter than reclaim candidates on the toptier, shouldn't they get promoted instead and make room that way? We may have to tweak the watermark logic a bit to facilitate that (allow promotions where regular allocations already fail?). But this sort of resorting would be preferable to age inversions. The mlock scenario sounds possible. In that case, it wouldn't be an aging inversion, since there is nothing colder on the CXL node. Maybe a bypass check should explicitly consult the demotion target watermarks against its evictable pages (similar to the file_is_tiny check in prepare_scan_count)? Because in any other scenario, if there is a bug in the promo/demo coordination, I think we'd rather have the OOM than deal with age inversions causing intermittent performance issues that are incredibly hard to track down.