From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A9FFC4332F for ; Wed, 30 Nov 2022 03:56:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8536E6B0072; Tue, 29 Nov 2022 22:56:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 803EF6B0073; Tue, 29 Nov 2022 22:56:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F24D6B0074; Tue, 29 Nov 2022 22:56:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 609E66B0072 for ; Tue, 29 Nov 2022 22:56:11 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1D71B40497 for ; Wed, 30 Nov 2022 03:56:11 +0000 (UTC) X-FDA: 80188745742.04.956627D Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf18.hostedemail.com (Postfix) with ESMTP id C02451C000B for ; Wed, 30 Nov 2022 03:56:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669780569; x=1701316569; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=wstP/Vj3b/bx7v6BDCKgttREqYEPOTLSqNXoVk7Grsg=; b=ffSrjkcSx/VvMFg6+5kniC4dKtTtaZOASOWW9jFvQ1KFO5lgvigIARvT KPbNUCbIG44ypIB8RlOsrsb5hloPxgy3S0ySxMet1f8eaH2gRaToYxaFD p/ai4kPS4Uf3+XXozyySdtFjfcKg2uR3ss5faevQmhy/mHE0hb5EHsPgS WGJyl5RGzBQqwG59uNqQ9xLR6jYdjUkS4jARkpA4HiCs7zgm97ggH0bES NNRvU3OdSXtkIMf+sqQ2ddG99ZFVx2v7Ou5gscyTyWxLmIyGSAeQbhifO tknwI07uu9GDNK9d+NR9sZhX4FvN6M7mN4hhLvhYS7CZRsTGLLm9BmWMW w==; X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="312911102" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="312911102" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 19:56:06 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="644055798" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="644055798" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 19:56:02 -0800 From: "Huang, Ying" To: Johannes Weiner Cc: Mina Almasry , Yang Shi , Yosry Ahmed , Tim Chen , weixugc@google.com, shakeelb@google.com, gthelen@google.com, fvdl@google.com, Michal Hocko , Roman Gushchin , Muchun Song , Andrew Morton , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: Re: [RFC PATCH V1] mm: Disable demotion from proactive reclaim References: <20221122203850.2765015-1-almasrymina@google.com> <874juonbmv.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Wed, 30 Nov 2022 11:55:12 +0800 In-Reply-To: (Johannes Weiner's message of "Tue, 29 Nov 2022 13:08:01 -0500") Message-ID: <87wn7dayfz.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669780570; a=rsa-sha256; cv=none; b=T7go2R6XXRSFTaBpCrHSC6ncrE6g8UopKK5AHwSwdHQw0wobYEVO4W9Vs8pKGcmQjzKPjm +ZoObN05NpHFdeNg2POzC1a6QHX1fwSqrNvXzctDbR1rWRGqrIgAxJXbvVKaUIFkBPx1Gy JsDXufxqZghJXaKJiHybGMXZk2EUHNg= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=ffSrjkcS; spf=pass (imf18.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669780570; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=P7R4r89eCSQA+3OXNm+GQHNuYGeQiD/0LwVfwJ8s9mk=; b=IJ1iL4T5rgKuRoeoeeBt4/4zMOOGNhcs2m9ka4dwq1l2PcdYHXJylJHE+Vn6ZmR/PrUZ62 n7AttBu9638hMxegvHUmjYLQ7CdpLoCVpHowPXtPh4tRmNuhDKlFQW37hPjCzfpJFVke5E x7AhNlsQSaoWfW3EM8fTFLTrdb56I8g= X-Stat-Signature: yx5ebwehdfagudrfx6uuyhqc855fqcut Authentication-Results: imf18.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=ffSrjkcS; spf=pass (imf18.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: C02451C000B X-Rspam-User: X-HE-Tag: 1669780568-289622 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Johannes Weiner writes: > Hello Ying, > > On Thu, Nov 24, 2022 at 01:51:20PM +0800, Huang, Ying wrote: >> Johannes Weiner writes: >> > The fallback to reclaim actually strikes me as wrong. >> > >> > Think of reclaim as 'demoting' the pages to the storage tier. If we >> > have a RAM -> CXL -> storage hierarchy, we should demote from RAM to >> > CXL and from CXL to storage. If we reclaim a page from RAM, it means >> > we 'demote' it directly from RAM to storage, bypassing potentially a >> > huge amount of pages colder than it in CXL. That doesn't seem right. >> > >> > If demotion fails, IMO it shouldn't satisfy the reclaim request by >> > breaking the layering. Rather it should deflect that pressure to the >> > lower layers to make room. This makes sure we maintain an aging >> > pipeline that honors the memory tier hierarchy. >> >> Yes. I think that we should avoid to fall back to reclaim as much as >> possible too. Now, when we allocate memory for demotion >> (alloc_demote_page()), __GFP_KSWAPD_RECLAIM is used. So, we will trigger >> kswapd reclaim on lower tier node to free some memory to avoid fall back >> to reclaim on current (higher tier) node. This may be not good enough, >> for example, the following patch from Hasan may help via waking up >> kswapd earlier. >> >> https://lore.kernel.org/linux-mm/b45b9bf7cd3e21bca61d82dcd1eb692cd32c122c.1637778851.git.hasanalmaruf@fb.com/ >> >> Do you know what is the next step plan for this patch? >> >> Should we do even more? >> >> From another point of view, I still think that we can use falling back >> to reclaim as the last resort to avoid OOM in some special situations, >> for example, most pages in the lowest tier node are mlock() or too hot >> to be reclaimed. > > If they're hotter than reclaim candidates on the toptier, shouldn't > they get promoted instead and make room that way? We may have to tweak > the watermark logic a bit to facilitate that (allow promotions where > regular allocations already fail?). But this sort of resorting would > be preferable to age inversions. Now it's legal to enable demotion and disable promotion. Yes, this is wrong configuration in general. But should we trigger OOM for these users? And now promotion only works for default NUMA policy (and MPOL_BIND to both promotion source and target nodes with MPOL_F_NUMA_BALANCING). If we use some other NUMA policy, the pages cannot be promoted too. > The mlock scenario sounds possible. In that case, it wouldn't be an > aging inversion, since there is nothing colder on the CXL node. > > Maybe a bypass check should explicitly consult the demotion target > watermarks against its evictable pages (similar to the file_is_tiny > check in prepare_scan_count)? Yes. This sounds doable. > Because in any other scenario, if there is a bug in the promo/demo > coordination, I think we'd rather have the OOM than deal with age > inversions causing intermittent performance issues that are incredibly > hard to track down. Previously, I thought that people will always prefer performance regression than OOM. Apparently, I am wrong. Anyway, I think that we need to reduce the possibility of OOM or falling back to reclaim as much as possible firstly. Do you agree? One possibility, can we fall back to reclaim only if the sc->priority is small enough (even 0)? Best Regards, Huang, Ying