From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 926626B7938 for ; Thu, 6 Sep 2018 10:39:07 -0400 (EDT) Received: by mail-ed1-f72.google.com with SMTP id w44-v6so3766237edb.16 for ; Thu, 06 Sep 2018 07:39:07 -0700 (PDT) Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15]) by mx.google.com with ESMTPS id k5-v6si1756234edd.380.2018.09.06.07.39.06 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Sep 2018 07:39:06 -0700 (PDT) Date: Thu, 6 Sep 2018 16:39:05 +0200 From: Michal Hocko Subject: Re: [PATCH] mm, oom: Introduce time limit for dump_tasks duration. Message-ID: <20180906143905.GC14951@dhcp22.suse.cz> References: <0252ad5d-46e6-0d7f-ef91-4e316657a83d@i-love.sakura.ne.jp> <201809060553.w865rmpj036017@www262.sakura.ne.jp> <58aa0543-86d0-b2ad-7fb9-9bed7c6a1f6c@i-love.sakura.ne.jp> <20180906112306.GO14951@dhcp22.suse.cz> <1611e45d-235e-67e9-26e3-d0228255fa2f@i-love.sakura.ne.jp> <20180906115320.GS14951@dhcp22.suse.cz> <7f50772a-f2ef-d16e-4d09-7f34f4bf9227@i-love.sakura.ne.jp> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7f50772a-f2ef-d16e-4d09-7f34f4bf9227@i-love.sakura.ne.jp> Sender: owner-linux-mm@kvack.org List-ID: To: Tetsuo Handa Cc: Dmitry Vyukov , Andrew Morton , David Rientjes , syzbot , 'Dmitry Vyukov' via syzkaller-upstream-moderation , linux-mm On Thu 06-09-18 22:45:26, Tetsuo Handa wrote: > On 2018/09/06 20:53, Michal Hocko wrote: > > On Thu 06-09-18 20:40:34, Tetsuo Handa wrote: > >> On 2018/09/06 20:23, Michal Hocko wrote: > >>> On Thu 06-09-18 19:58:25, Tetsuo Handa wrote: > >>> [...] > >>>> >From 18876f287dd69a7c33f65c91cfcda3564233f55e Mon Sep 17 00:00:00 2001 > >>>> From: Tetsuo Handa > >>>> Date: Thu, 6 Sep 2018 19:53:18 +0900 > >>>> Subject: [PATCH] mm, oom: Introduce time limit for dump_tasks duration. > >>>> > >>>> Since printk() is slow, printing one line takes nearly 0.01 second. > >>>> As a result, syzbot is stalling for 52 seconds trying to dump 5600 > >>>> tasks at for_each_process() under RCU. Since such situation is almost > >>>> inflight fork bomb attack (the OOM killer will print similar tasks for > >>>> so many times), it makes little sense to print all candidate tasks. > >>>> Thus, this patch introduces 3 seconds limit for printing. > >>>> > >>>> Signed-off-by: Tetsuo Handa > >>>> Cc: Dmitry Vyukov > >>> > >>> You really love timeout based solutions with randomly chosen timeouts, > >>> don't you. This is just ugly as hell. We already have means to disable > >>> tasks dumping (see /proc/sys/vm/oom_dump_tasks). > >> > >> I know /proc/sys/vm/oom_dump_tasks . Showing some entries while not always > >> printing all entries might be helpful. > > > > Not really. It could be more confusing than helpful. The main purpose of > > the listing is to double check the list to understand the oom victim > > selection. If you have a partial list you simply cannot do that. > > It serves as a safeguard for avoiding RCU stall warnings. > > > > > If the iteration takes too long and I can imagine it does with zillions > > of tasks then the proper way around it is either release the lock > > periodically after N tasks is processed or outright skip the whole thing > > if there are too many tasks. The first option is obviously tricky to > > prevent from duplicate entries or other artifacts. > > > > Can we add rcu_lock_break() like check_hung_uninterruptible_tasks() does? This would be a better variant of your timeout based approach. But it can still produce an incomplete task list so it still consumes a lot of resources to print a long list of tasks potentially while that list is not useful for any evaluation. Maybe that is good enough. I don't know. I would generally recommend to disable the whole thing with workloads with many tasks though. -- Michal Hocko SUSE Labs