From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C839C433EF for ; Tue, 7 Sep 2021 08:44:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E9FB461101 for ; Tue, 7 Sep 2021 08:44:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E9FB461101 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 44B876B0071; Tue, 7 Sep 2021 04:44:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3FBA2900002; Tue, 7 Sep 2021 04:44:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E9A66B0073; Tue, 7 Sep 2021 04:44:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0048.hostedemail.com [216.40.44.48]) by kanga.kvack.org (Postfix) with ESMTP id 1EF4A6B0071 for ; Tue, 7 Sep 2021 04:44:38 -0400 (EDT) Received: from smtpin33.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C6FE0180ABF42 for ; Tue, 7 Sep 2021 08:44:37 +0000 (UTC) X-FDA: 78560141394.33.062DA4D Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf21.hostedemail.com (Postfix) with ESMTP id 3D63DD0290A7 for ; Tue, 7 Sep 2021 08:44:37 +0000 (UTC) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 2BACB21FE4; Tue, 7 Sep 2021 08:44:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1631004276; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=KLbWJgt7Pq7iTo2+u+wRsltkx42moH/GKLU6EPym12c=; b=fb8TAUdnRMI1pKMM76wf/il79XFi0Xe4TReUmU6/lhZUL3XOJc62vvSBdlMVUv9YqHGlKm veoFeOD4ahD2cCGJRlkIqS0cRLa/UWqJsPQNi4VfQP43kHbcnp0xPo7hTXoEha1qFpV3H1 bxfKDDTXbJyIg0mAxgPGQLyk7Oh4nGw= Received: from suse.cz (unknown [10.100.201.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id EFDD5A3B81; Tue, 7 Sep 2021 08:44:35 +0000 (UTC) Date: Tue, 7 Sep 2021 10:44:32 +0200 From: Michal Hocko To: Feng Tang Cc: Andrew Morton , David Rientjes , Mel Gorman , Vlastimil Babka , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] mm/page_alloc: detect allocation forbidden by cpuset and bail out early Message-ID: References: <1631003150-96935-1-git-send-email-feng.tang@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1631003150-96935-1-git-send-email-feng.tang@intel.com> Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=fb8TAUdn; spf=pass (imf21.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 3D63DD0290A7 X-Stat-Signature: g8p7u17rso38gfw9rqcakctipftodgyb X-HE-Tag: 1631004277-816175 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue 07-09-21 16:25:50, Feng Tang wrote: > There was report that starting an Ubuntu in docker while using cpuset > to bind it to movlabe nodes (a node only has movable zone, like a node s@movlabe@movable@ > for hotplug or a Persistent Memory node in normal usage) will fail > due to memory allocation failure, and then OOM is involved and many > other innocent processes got killed. It can be reproduced with command: > $docker run -it --rm --cpuset-mems 4 ubuntu:latest bash -c > "grep Mems_allowed /proc/self/status" (node 4 is a movable node) > > The reason is, in the case, the target cpuset nodes only have movable > zone, while the creation of an OS in docker sometimes needs to allocate > memory in non-movable zones (dma/dma32/normal) like GFP_HIGHUSER, and > the cpuset limit forbids the allocation, then out-of-memory killing is > involved even when normal nodes and movable nodes both have many free > memory. It would be great to add a oom report here as an example. > The failure is reasonable, but still there is one problem, that when > the usage fails as it's an mission impossible due to the cpuset limit, > the allocation should just not trigger reclaim/compaction, and more > importantly, not get any innocent process oom-killed. I would reformulate to something like: " The OOM killer cannot help to resolve the situation as there is no usable memory for the request in the cpuset scope. The only reasonable measure to take is to fail the allocation right away and have the caller to deal with it. " > So add detection for cases like this in the slowpath of allocation, > and bail out early returning NULL for the allocation. > > We've run some cases of malloc/mmap/page_fault/lru-shm/swap from > will-it-scale and vm-scalability, and didn't see obvious performance > change (all inside +/- 1%), test boxes are 2 socket Cascade Lake and > Icelake servers. > > [thanks to Micho Hocko and David Rientjes for suggesting not handle > it inside OOM code] While this is a good fix from the functionality POV I believe you can go a step further. Please add a detection to the cpuset code and complain to the kernel log if somebody tries to configure movable only cpuset. Once you have that in place you can easily create a static branch for cpuset_insane_setup() and have zero overhead for all reasonable configuration. There shouldn't be any reason to pay a single cpu cycle to check for something that almost nobody does. What do you think? -- Michal Hocko SUSE Labs