From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28485C433E2 for ; Mon, 14 Sep 2020 13:26:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5FA0B206DB for ; Mon, 14 Sep 2020 13:26:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="XdMDfjGg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5FA0B206DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A301A900003; Mon, 14 Sep 2020 09:26:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9BA08900002; Mon, 14 Sep 2020 09:26:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85A32900003; Mon, 14 Sep 2020 09:26:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id 687F8900002 for ; Mon, 14 Sep 2020 09:26:23 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 0E3002CBBB for ; Mon, 14 Sep 2020 13:26:23 +0000 (UTC) X-FDA: 77261741046.01.lift70_5907a3c27108 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 3C80C101A7FFC for ; Mon, 14 Sep 2020 13:26:13 +0000 (UTC) X-HE-Tag: lift70_5907a3c27108 X-Filterd-Recvd-Size: 17093 Received: from mail-lj1-f193.google.com (mail-lj1-f193.google.com [209.85.208.193]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Mon, 14 Sep 2020 13:26:12 +0000 (UTC) Received: by mail-lj1-f193.google.com with SMTP id w3so18631652ljo.5 for ; Mon, 14 Sep 2020 06:26:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=6brFbSdj4B5UNNBTgTiEkKGOU9q1PiIGHAL3/sZivGQ=; b=XdMDfjGgDWovM6FBEquiU9Z7gNsNAMRxsuXF9AQtkTqU1K1BA8rFLWMiawfHp8lzaT oq/O2f1uavYIXelFNR+yIe0WULBi10hJdk8WtzTARCo/hThgqkfgeAI0TCi9VJCFn0Ka ClElEJC8UfxpOJj0X7C2DyV30EtqfI/+CX/89NngWgiYd7k2DIk9Udvt+ZqTSl8hJ7zZ q/KFDiAsbNgNCFE3+MXqT0wDVcP4ov1LtbWuWL3iqDZvtRnlOjBFta2XKBw/QX052sLm Z2oXgmAOEIPwqru3xPTh9uPBEyhjkPjLNfVIO0cODV5oh5VAzqzHD/AsCtgq1NEYmG7U EpTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=6brFbSdj4B5UNNBTgTiEkKGOU9q1PiIGHAL3/sZivGQ=; b=NGMKi7tHEm0WqwC8ekUNNgZB64l/KJIJmSUExhQxtv+2ikHfk9NvbzT99VoK6EHSzv /IBcdf9W48HKGPTKWXHUqUGafP/YVYSBUkmOsKoFBE4hECSrzoLK9yZ5LHYi99FaoSK1 egVpgkT2H7EqmHiDxvAL6I/K00kmZk4d0o135WCmwWjHhavwk0/Esem0htd03MEran3X 540ioQKvgOBWJhJvlw3vDdgs8e3ohjAGvHmZw62DqGuNWIVgPQk+RqXa0dd/prfxQ8Q8 TvhU/N37tf6IScEJ0NnuSbQzBO7U84PgW7As0heIoIB5qhiRSmpAYnv3QkykDvbhg0Ti h4DQ== X-Gm-Message-State: AOAM533Rpq2Jda1ySYrHRSPiR/NcstSvHbahNR4iW4uHzUiq5aSX9hb7 aGzwIJ7q7IaxcNOmo0PfBx4kPh8l3bEHn78GdI/orc/t5n/iasvx X-Google-Smtp-Source: ABdhPJxYpvJljuOA4ZyYGofTT/j3s4nLJ/4PIM/C0FzcX7amoTnNdDp1SoAgnVXQpBg3UFAkTfrL0pLXFLIFlq/cBUI= X-Received: by 2002:a2e:889a:: with SMTP id k26mr4918598lji.214.1600089970568; Mon, 14 Sep 2020 06:26:10 -0700 (PDT) MIME-Version: 1.0 References: <20200909152047.27905-1-zangchunxin@bytedance.com> <20200914093032.GG16999@dhcp22.suse.cz> In-Reply-To: <20200914093032.GG16999@dhcp22.suse.cz> From: Chunxin Zang Date: Mon, 14 Sep 2020 21:25:59 +0800 Message-ID: Subject: Re: [External] Re: [PATCH v2] mm/vmscan: fix infinite loop in drop_slab_node To: Michal Hocko Cc: Andrew Morton , Linux Memory Management List , LKML , Muchun Song Content-Type: multipart/alternative; boundary="0000000000004e0d9c05af45fac3" X-Rspamd-Queue-Id: 3C80C101A7FFC X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.002424, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --0000000000004e0d9c05af45fac3 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Sep 14, 2020 at 5:30 PM Michal Hocko wrote: > The subject is misleading because this patch doesn't fix an infinite > loop, right? It just allows the userspace to interrupt the operation. > > Yes, so we are making a separate patch follow Vlastimil's recommendations. Use double of threshold to end the loop. On Thu, Sep 10, 2020 at 1:59 AM Vlastimil Babka wrote: > > From: Chunxin Zang > > > ... > - IMHO it's still worth to bail out in your scenario even without a > signal, e.g. > by the doubling of threshold. But it can be a separate patch. > Thanks! > ... On Wed 09-09-20 23:20:47, zangchunxin@bytedance.com wrote: > > From: Chunxin Zang > > > > On our server, there are about 10k memcg in one machine. They use memor= y > > very frequently. When I tigger drop caches=EF=BC=8Cthe process will inf= inite loop > > in drop_slab_node. > > Is this really an infinite loop, or it just takes a lot of time to > process all the metadata in that setup? If this is really an infinite > loop then we should look at it. My current understanding is that the > operation would finish at some time it just takes painfully long to get > there. > Yes, it's really an infinite loop. Every loop spends a lot of time. In this time, memcg will alloc/free memory, so the next loop, the total of 'freed' always bigger than 10. > > There are two reasons: > > 1.We have too many memcgs, even though one object freed in one memcg, t= he > > sum of object is bigger than 10. > > > > 2.We spend a lot of time in traverse memcg once. So, the memcg who > > traversed at the first have been freed many objects. Traverse memcg > next > > time, the freed count bigger than 10 again. > > > > We can get the following info through 'ps': > > > > root:~# ps -aux | grep drop > > root 357956 ... R Aug25 21119854:55 echo 3 > > /proc/sys/vm/drop_caches > > root 1771385 ... R Aug16 21146421:17 echo 3 > > /proc/sys/vm/drop_caches > > root 1986319 ... R 18:56 117:27 echo 3 > /proc/sys/vm/drop_caches > > root 2002148 ... R Aug24 5720:39 echo 3 > /proc/sys/vm/drop_caches > > root 2564666 ... R 18:59 113:58 echo 3 > /proc/sys/vm/drop_caches > > root 2639347 ... R Sep03 2383:39 echo 3 > /proc/sys/vm/drop_caches > > root 3904747 ... R 03:35 993:31 echo 3 > /proc/sys/vm/drop_caches > > root 4016780 ... R Aug21 7882:18 echo 3 > /proc/sys/vm/drop_caches > > > > Use bpftrace follow 'freed' value in drop_slab_node: > > > > root:~# bpftrace -e 'kprobe:drop_slab_node+70 {@ret=3Dhist(reg("bp"))= ; }' > > Attaching 1 probe... > > ^B^C > > > > @ret: > > [64, 128) 1 | > | > > [128, 256) 28 | > | > > [256, 512) 107 |@ > | > > [512, 1K) 298 |@@@ > | > > [1K, 2K) 613 |@@@@@@@ > | > > [2K, 4K) 4435 > |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| > > [4K, 8K) 442 |@@@@@ > | > > [8K, 16K) 299 |@@@ > | > > [16K, 32K) 100 |@ > | > > [32K, 64K) 139 |@ > | > > [64K, 128K) 56 | > | > > [128K, 256K) 26 | > | > > [256K, 512K) 2 | > | > > > > In the while loop, we can check whether the TASK_KILLABLE signal is set= , > > if so, we should break the loop. > > I would make it explicit that this is not fixing the above scenario. It > just helps to cancel to operation which is a good thing in general. > > > Signed-off-by: Chunxin Zang > > Signed-off-by: Muchun Song > > With updated changelog > Acked-by: Michal Hocko > > > --- > > changelogs in v2: > > 1) Via check TASK_KILLABLE signal break loop. > > > > mm/vmscan.c | 3 +++ > > 1 file changed, 3 insertions(+) > > > > diff --git a/mm/vmscan.c b/mm/vmscan.c > > index b6d84326bdf2..c3ed8b45d264 100644 > > --- a/mm/vmscan.c > > +++ b/mm/vmscan.c > > @@ -704,6 +704,9 @@ void drop_slab_node(int nid) > > do { > > struct mem_cgroup *memcg =3D NULL; > > > > + if (fatal_signal_pending(current)) > > + return; > > + > > freed =3D 0; > > memcg =3D mem_cgroup_iter(NULL, NULL, NULL); > > do { > > -- > > 2.11.0 > > > > -- > Best wishes > Chunxin > --0000000000004e0d9c05af45fac3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

On Mon, Sep 14, 2020 at 5:30 PM Michal Hocko <mhocko@suse.com> wrote:
The subject is misleading bec= ause this patch doesn't fix an infinite
loop, right? It just allows the userspace to interrupt the operation.

=C2=A0
Yes,=C2=A0 so we are making a=C2=A0s= eparate patch follow=C2=A0Vlastimil's=C2=A0recommendations. Use=C2=A0
double of threshold to end the loop.

On Thu, Sep 10, 2020 at 1:59 = AM Vlastimil Babka <vbabka@suse.cz= > wrote:
> From: Chunxin Zang <zangchunxin@bytedance.com>
>
...=C2=A0
- IMHO it's still worth to bail out in your scenario even without a sig= nal, e.g.
by the doubling of threshold. But it can be a separate patch.
Thanks!
...


On Wed 09-09-20 23:20:47, zangchunxin@bytedance.com wrote:
> From: Chunxin Zang <zangchunxin@bytedance.com>
>
> On our server, there are about 10k memcg in one machine. They use memo= ry
> very frequently. When I tigger drop caches=EF=BC=8Cthe process will in= finite loop
> in drop_slab_node.

Is this really an infinite loop, or it just takes a lot of time to
process all the metadata in that setup? If this is really an infinite
loop then we should look at it. My current understanding is that the
operation would finish at some time it just takes painfully long to get
there.

Yes,=C2=A0 it's really an in= finite loop.=C2=A0 Every loop=C2=A0spends a lot of time.=C2=A0In this=C2=A0= time,=C2=A0
memcg will alloc/free memory,=C2=A0 so the next loop,= the total of=C2=A0 'freed' always=C2=A0bigger than 10.
= =C2=A0
> There are two reasons:
> 1.We have too many memcgs, even though one object freed in one memcg, = the
>=C2=A0 =C2=A0sum of object is bigger than 10.
>
> 2.We spend a lot of time in traverse memcg once. So, the memcg who
>=C2=A0 =C2=A0traversed at the first have been freed many objects. Trave= rse memcg next
>=C2=A0 =C2=A0time, the freed count bigger than 10 again.
>
> We can get the following info through 'ps':
>
>=C2=A0 =C2=A0root:~# ps -aux | grep drop
>=C2=A0 =C2=A0root=C2=A0 357956 ... R=C2=A0 =C2=A0 Aug25 21119854:55 ech= o 3 > /proc/sys/vm/drop_caches
>=C2=A0 =C2=A0root 1771385 ... R=C2=A0 =C2=A0 Aug16 21146421:17 echo 3 &= gt; /proc/sys/vm/drop_caches
>=C2=A0 =C2=A0root 1986319 ... R=C2=A0 =C2=A0 18:56 117:27 echo 3 > /= proc/sys/vm/drop_caches
>=C2=A0 =C2=A0root 2002148 ... R=C2=A0 =C2=A0 Aug24 5720:39 echo 3 > = /proc/sys/vm/drop_caches
>=C2=A0 =C2=A0root 2564666 ... R=C2=A0 =C2=A0 18:59 113:58 echo 3 > /= proc/sys/vm/drop_caches
>=C2=A0 =C2=A0root 2639347 ... R=C2=A0 =C2=A0 Sep03 2383:39 echo 3 > = /proc/sys/vm/drop_caches
>=C2=A0 =C2=A0root 3904747 ... R=C2=A0 =C2=A0 03:35 993:31 echo 3 > /= proc/sys/vm/drop_caches
>=C2=A0 =C2=A0root 4016780 ... R=C2=A0 =C2=A0 Aug21 7882:18 echo 3 > = /proc/sys/vm/drop_caches
>
> Use bpftrace follow 'freed' value in drop_slab_node:
>
>=C2=A0 =C2=A0root:~# bpftrace -e 'kprobe:drop_slab_node+70 {@ret=3D= hist(reg("bp")); }'
>=C2=A0 =C2=A0Attaching 1 probe...
>=C2=A0 =C2=A0^B^C
>
>=C2=A0 =C2=A0@ret:
>=C2=A0 =C2=A0[64, 128)=C2=A0 =C2=A0 =C2=A0 =C2=A0 1 |=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 |
>=C2=A0 =C2=A0[128, 256)=C2=A0 =C2=A0 =C2=A0 28 |=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 |
>=C2=A0 =C2=A0[256, 512)=C2=A0 =C2=A0 =C2=A0107 |@=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0|
>=C2=A0 =C2=A0[512, 1K)=C2=A0 =C2=A0 =C2=A0 298 |@@@=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0|
>=C2=A0 =C2=A0[1K, 2K)=C2=A0 =C2=A0 =C2=A0 =C2=A0613 |@@@@@@@=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0|
>=C2=A0 =C2=A0[2K, 4K)=C2=A0 =C2=A0 =C2=A0 4435 |@@@@@@@@@@@@@@@@@@@@@@@= @@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
>=C2=A0 =C2=A0[4K, 8K)=C2=A0 =C2=A0 =C2=A0 =C2=A0442 |@@@@@=C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0|
>=C2=A0 =C2=A0[8K, 16K)=C2=A0 =C2=A0 =C2=A0 299 |@@@=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0|
>=C2=A0 =C2=A0[16K, 32K)=C2=A0 =C2=A0 =C2=A0100 |@=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0|
>=C2=A0 =C2=A0[32K, 64K)=C2=A0 =C2=A0 =C2=A0139 |@=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0|
>=C2=A0 =C2=A0[64K, 128K)=C2=A0 =C2=A0 =C2=A056 |=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 |
>=C2=A0 =C2=A0[128K, 256K)=C2=A0 =C2=A0 26 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 |
>=C2=A0 =C2=A0[256K, 512K)=C2=A0 =C2=A0 =C2=A02 |=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 |
>
> In the while loop, we can check whether the TASK_KILLABLE signal is se= t,
> if so, we should break the loop.

I would make it explicit that this is not fixing the above scenario. It
just helps to cancel to operation which is a good thing in general.

> Signed-off-by: Chunxin Zang <zangchunxin@bytedance.com>
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>

With updated changelog
Acked-by: Michal Hocko <mhocko@suse.com>

> ---
>=C2=A0 =C2=A0 =C2=A0 =C2=A0changelogs in v2:
>=C2=A0 =C2=A0 =C2=A0 =C2=A01) Via check TASK_KILLABLE signal break loop= .
>
>=C2=A0 mm/vmscan.c | 3 +++
>=C2=A0 1 file changed, 3 insertions(+)
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index b6d84326bdf2..c3ed8b45d264 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -704,6 +704,9 @@ void drop_slab_node(int nid)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0do {
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0struct mem_cgrou= p *memcg =3D NULL;
>=C2=A0
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (fatal_signal_pend= ing(current))
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0return;
> +
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0freed =3D 0;
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0memcg =3D mem_cg= roup_iter(NULL, NULL, NULL);
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0do {
> --
> 2.11.0
>

--
Best wishes
Chunxin
--0000000000004e0d9c05af45fac3--