From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA6B6C61D97 for ; Thu, 23 Nov 2023 17:19:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EDD456B0673; Thu, 23 Nov 2023 12:19:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E8CF66B0674; Thu, 23 Nov 2023 12:19:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D56B56B0675; Thu, 23 Nov 2023 12:19:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C81BB6B0673 for ; Thu, 23 Nov 2023 12:19:21 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7A1811418A4 for ; Thu, 23 Nov 2023 17:19:21 +0000 (UTC) X-FDA: 81489880122.15.D4AB157 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf13.hostedemail.com (Postfix) with ESMTP id 6A69D20024 for ; Thu, 23 Nov 2023 17:19:18 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=ZuPn6na8; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf13.hostedemail.com: domain of chrisl@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=chrisl@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700759958; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fPpJ2QrJUEGgW/k94SDUjYvfNCM1OcypubpbYnCTs88=; b=gSzH73DJQLaFQzy/dnAsPTBWBb9NM1qdx3U6wvbt7vC1Ex5v09yztkubX8flywYNuKCDL0 Yt8ZAK3JgLt873SlxEqP+sOO0I5lAFzAN4ZdOqghKYJcewk1hEEPgRBV5ObgNH/uvJblE9 RhbtOdy5d/fp/WwD7H6DCUscWkRO+Zs= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=ZuPn6na8; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf13.hostedemail.com: domain of chrisl@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=chrisl@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700759958; a=rsa-sha256; cv=none; b=8dREx9rv2t/MS1hAQyjtxAfPMwYZdu1pkMVoZuIDMZ2y8osKAhdmbMw+D4kVQ4isOWMljG kXFRXHDW28YxYhr0BsbUBMZXrmL6UDXPS9wyIS+A5S1gOW9eug3p4UArs3/bLyKOmTVNKR hqrXAcVkF4hIY+PDhgH6PZurHWn/ZK8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id 5919BB82EA7 for ; Thu, 23 Nov 2023 17:19:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 873D7C41679 for ; Thu, 23 Nov 2023 17:19:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700759955; bh=IdgHI2lDjcmt4qEFhrLPvoD8tNuqkVTcDgzNfvZ85sc=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=ZuPn6na8swcN5XZ87KuRw8VlUEvGyFw4xJWJoFCixX95Ila96E2rGljhlyRXKIpU6 XPHCeBAiqw8KqqxLoD2ZURqP4BB2cq9o2XPeI9dwYYx6MOy0AI0PwRbBcuTjxlzHFf IeY1u/FostorSZJYlmbbIFp1QI2c+3JRYJhU8nywhJ+Th9L1qDHc32QNaiTezSdYEC 4Y/TsxpwtntcLo7g+1wxrHO9bSyMgm4n5YWULvHTpkVcjsVSlnDk73qotD5LH5iDzw 745G/cvwvkWhLbM5fVF3+2qlkKeTY+GIjhcjeUs+lE+rozOfbJCowoBTwOMxlmqUm6 s2e7GiMNzqDPA== Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-2851cc359c1so828616a91.3 for ; Thu, 23 Nov 2023 09:19:15 -0800 (PST) X-Gm-Message-State: AOJu0YwxM5C34EHB8ei54nuooEIoUONASEKe8DkVpZCCxYwAQF18t0i9 FQF6jT3t/bqctoVsRZ1Zn62/Egu1uUBG5aO6VgvVcw== X-Google-Smtp-Source: AGHT+IE3pl7gcLotSh+vx8sl8UGkyahEEMwps0Q2TRd4gl8uT97Aak8+By2TDnyAl7ynK5oyAvf5WB7kcMP9vxTQL70= X-Received: by 2002:a17:90b:4b4f:b0:27d:8ad:69f9 with SMTP id mi15-20020a17090b4b4f00b0027d08ad69f9mr144759pjb.2.1700759954942; Thu, 23 Nov 2023 09:19:14 -0800 (PST) MIME-Version: 1.0 References: <20231121090624.1814733-1-liushixin2@huawei.com> In-Reply-To: <20231121090624.1814733-1-liushixin2@huawei.com> From: Chris Li Date: Thu, 23 Nov 2023 09:19:00 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v10] mm: vmscan: try to reclaim swapcache pages if no swap space To: Liu Shixin Cc: Yu Zhao , Andrew Morton , Yosry Ahmed , Huang Ying , Sachin Sant , Michal Hocko , Johannes Weiner , Kefeng Wang , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 6A69D20024 X-Stat-Signature: r93d3pnt7sc1pwmrsyhouf3sstjywhuo X-Rspam-User: X-HE-Tag: 1700759958-929943 X-HE-Meta: U2FsdGVkX19LTL5OgijfXABqc/CIh70dUDB53qcjVjqGX6pUUluBDi4ubR3r9ipSRe9X2EHBDoYcq2AOcTvorZvWESlBEaZ4bkb+6nqWi8Grwyv9bRCW4zrephtl8f94hnsocw+oYkKUXNmLkwnq3DXx3HiV5TIVXCKDXolX7JByxyrk2SShV6rl11U+9JQO2c01rLVPkBiYxYiPS3y3bnPbDz618PclOVJJhjQ83iMgAoyetZYQQSov4vnSD8mS0WJpVOlPweU6FBb5Ri716QFstbo5GcCMhK6C8BmQZn2qE15HG4K70nqOMDcpAMVA3o6hRYA5rsT6fw3RuRZ9ToIDRG/jCreEmJ+JY1tBqUlq6nIzZKY90RR1uHDhcYpIN2mY4werOrkNCNvz1vSHe2OavmnmHxKx6arI7gzRAby7q2KrI7EF/cujA1hdHNV/N04cCfRFwzrBjHH55mmPwDLzvsTlWVKdDQicvDu6CJde5bL+e/qG+y+6ia0uUIw+jVEIzqP6n0gt1yfHrzji1uRZQGBeql1GPT1eP0+URV8Se21KnISh1KOjevZzcvVvfussrLl7+mmMyGV2Ac9RdNndKgAh73QK13oiH6MueJn/n3GcAs3aMBuXLA0sNsvB7jVSDJ4Sre00RauSbUA7/k0OBwusFa5QSuBOhT0kNLK+Ou/1N09PZ4HFfqnXJXExtLWR4bQtqhlpsP+L8L9mDLM1TUQiOdr1kfkOqFsPm98k5Si2nNZHWKqhcIWhCO8xghkHONO7eRfVz1I8thRoHjKl6X4vFmmIq4MZn593qX1rIxFz0y7IvRViuSpiWPW+HpOicgPk+4PFxE9partLxyLAy/Kj6hSlI/uyCI44FnEDXMH5pp/EgdfgsyBN8s1XezcfwmXY5P3fhKp0Vibwg//y2h3a4fxIJ23F+JMmH2MZcW17qOZg61WU5HipavCq6tQAcu9cXwuzPp9xqkc v2RqRHsk VkQWPdhRCAAL5PS+Ef9ixc8WAIJtCfOe+Uqv3AGpTgTQruxNYmtVCKuf7/KRmZqKfrXRU21mJ++018UTRoEKQwP6Uu2am8cTVJoG9DCju+BV/DAM44ESv4nyqbS1rJKCaXM6SG68uwQDMQSIPe1trrZHCUgaXbTB7PIjACNeFG58ve+bafCcUQ17TV+AQDvT8Z6dhck8Tww89uA66eFyOMPJvLBl0JN6Hot5GxldVZ0WPhiAwtfQhrV2BNycqdh4dq+DtxosLJOfAMN3OCIPQp2JBWsl6v958Cg2YvNZqTB7Cnd9YD/s/Lsv+Mu3Vzo9lbUzFBXSHjJdE3MZl5YqaKQ4dKsdfB4MtOWSK08kdmwHwQa2MQCaE7jhPIg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Shixin, On Tue, Nov 21, 2023 at 12:08=E2=80=AFAM Liu Shixin = wrote: > > When spaces of swap devices are exhausted, only file pages can be > reclaimed. But there are still some swapcache pages in anon lru list. > This can lead to a premature out-of-memory. > > The problem is found with such step: > > Firstly, set a 9MB disk swap space, then create a cgroup with 10MB > memory limit, then runs an program to allocates about 15MB memory. > > The problem occurs occasionally, which may need about 100 times [1]. Just out of my curiosity, in your usage case, how much additional memory in terms of pages or MB can be freed by this patch, using current code as base line? Does the swap cache page reclaimed in swapcache_only mode, all swap count drop to zero, and the only reason to stay in swap cache is to void page IO write if we need to swap that page out again? > Fix it by checking number of swapcache pages in can_reclaim_anon_pages(). > If the number is not zero, return true and set swapcache_only to 1. > When scan anon lru list in swapcache_only mode, non-swapcache pages will > be skipped to isolate in order to accelerate reclaim efficiency. Here you said non-swapcache will be skipped if swapcache_only =3D=3D 1 > > However, in swapcache_only mode, the scan count still increased when scan > non-swapcache pages because there are large number of non-swapcache pages > and rare swapcache pages in swapcache_only mode, and if the non-swapcache Here you suggest non-swapcache pages will also be scanned even when swapcache_only =3D=3D 1. It seems to contradict what you said above. I feel that I am missing something here. > is skipped and do not count, the scan of pages in isolate_lru_folios() ca= n Can you clarify which "scan of pages", are those pages swapcache pages or non-swapcache pages? > eventually lead to hung task, just as Sachin reported [2]. > > By the way, since there are enough times of memory reclaim before OOM, it > is not need to isolate too much swapcache pages in one times. > > [1]. https://lore.kernel.org/lkml/CAJD7tkZAfgncV+KbKr36=3DeDzMnT=3D9dZOT0= dpMWcurHLr6Do+GA@mail.gmail.com/ > [2]. https://lore.kernel.org/linux-mm/CAJD7tkafz_2XAuqE8tGLPEcpLngewhUo= =3D5US14PAtSM9tLBUQg@mail.gmail.com/ > > Signed-off-by: Liu Shixin > Tested-by: Yosry Ahmed > Reviewed-by: "Huang, Ying" > Reviewed-by: Yosry Ahmed > --- > v9->v10: Use per-node swapcache suggested by Yu Zhao. > v8->v9: Move the swapcache check after can_demote() and refector > can_reclaim_anon_pages() a bit. > v7->v8: Reset swapcache_only at the beginning of can_reclaim_anon_pages()= . > v6->v7: Reset swapcache_only to zero after there are swap spaces. > v5->v6: Fix NULL pointing derefence and hung task problem reported by Sac= hin. > > mm/vmscan.c | 50 +++++++++++++++++++++++++++++++++++++++++++++++++- > 1 file changed, 49 insertions(+), 1 deletion(-) > > diff --git a/mm/vmscan.c b/mm/vmscan.c > index 506f8220c5fe..1fcc94717370 100644 > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -136,6 +136,9 @@ struct scan_control { > /* Always discard instead of demoting to lower tier memory */ > unsigned int no_demotion:1; > > + /* Swap space is exhausted, only reclaim swapcache for anon LRU *= / > + unsigned int swapcache_only:1; > + > /* Allocation order */ > s8 order; > > @@ -308,10 +311,36 @@ static bool can_demote(int nid, struct scan_control= *sc) > return true; > } > > +#ifdef CONFIG_SWAP > +static bool can_reclaim_swapcache(struct mem_cgroup *memcg, int nid) > +{ > + struct pglist_data *pgdat =3D NODE_DATA(nid); > + unsigned long nr_swapcache; > + > + if (!memcg) { > + nr_swapcache =3D node_page_state(pgdat, NR_SWAPCACHE); > + } else { > + struct lruvec *lruvec =3D mem_cgroup_lruvec(memcg, pgdat)= ; > + > + nr_swapcache =3D lruvec_page_state_local(lruvec, NR_SWAPC= ACHE); > + } > + > + return nr_swapcache > 0; > +} > +#else > +static bool can_reclaim_swapcache(struct mem_cgroup *memcg, int nid) > +{ > + return false; > +} > +#endif > + > static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg, > int nid, > struct scan_control *sc) > { > + if (sc) > + sc->swapcache_only =3D 0; > + Minor nitpick. The sc->swapcache_only is first set to 0 then later set to 1. Better use a local variable then write to sc->swapcache_only in one go. If the scan_control has more than one thread accessing it, the threads can see the flicker of 0->1 change. I don't think that is the case in our current code, sc is created on stack. There are other minor benefits as The "if (sc) test" only needs to be done once, one store instruction. Chris > if (memcg =3D=3D NULL) { > /* > * For non-memcg reclaim, is there > @@ -330,7 +359,17 @@ static inline bool can_reclaim_anon_pages(struct mem= _cgroup *memcg, > * > * Can it be reclaimed from this node via demotion? > */ > - return can_demote(nid, sc); > + if (can_demote(nid, sc)) > + return true; > + > + /* Is there any swapcache pages to reclaim in this node? */ > + if (can_reclaim_swapcache(memcg, nid)) { > + if (sc) > + sc->swapcache_only =3D 1; > + return true; > + } > + > + return false; > } > > /* > @@ -1642,6 +1681,15 @@ static unsigned long isolate_lru_folios(unsigned l= ong nr_to_scan, > */ > scan +=3D nr_pages; > > + /* > + * Count non-swapcache too because the swapcache pages ma= y > + * be rare and it takes too much times here if not count > + * the non-swapcache pages. > + */ > + if (unlikely(sc->swapcache_only && !is_file_lru(lru) && > + !folio_test_swapcache(folio))) > + goto move; > + > if (!folio_test_lru(folio)) > goto move; > if (!sc->may_unmap && folio_mapped(folio)) > -- > 2.25.1 > >