From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BDC4C433E7 for ; Wed, 2 Sep 2020 13:50:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B821220767 for ; Wed, 2 Sep 2020 13:50:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B821220767 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 19C5A6B005C; Wed, 2 Sep 2020 09:50:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 14C66900002; Wed, 2 Sep 2020 09:50:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 039D26B006E; Wed, 2 Sep 2020 09:50:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0250.hostedemail.com [216.40.44.250]) by kanga.kvack.org (Postfix) with ESMTP id DE8ED6B005C for ; Wed, 2 Sep 2020 09:50:22 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 9B19A181AEF15 for ; Wed, 2 Sep 2020 13:50:22 +0000 (UTC) X-FDA: 77218255884.22.shoe91_430649b270a1 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 67ADA18038E68 for ; Wed, 2 Sep 2020 13:50:22 +0000 (UTC) X-HE-Tag: shoe91_430649b270a1 X-Filterd-Recvd-Size: 3840 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Wed, 2 Sep 2020 13:50:21 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id BB1DFB609; Wed, 2 Sep 2020 13:50:20 +0000 (UTC) Date: Wed, 2 Sep 2020 15:50:18 +0200 From: Michal Hocko To: Pavel Tatashin Cc: David Hildenbrand , Vlastimil Babka , Roman Gushchin , Bharata B Rao , "linux-mm@kvack.org" , Andrew Morton , Johannes Weiner , Shakeel Butt , Vladimir Davydov , "linux-kernel@vger.kernel.org" , Kernel Team , Yafang Shao , stable , Linus Torvalds , Sasha Levin , Greg Kroah-Hartman , David Hildenbrand Subject: Re: [PATCH v2 00/28] The new cgroup slab memory controller Message-ID: <20200902135018.GF4617@dhcp22.suse.cz> References: <6469324e-afa2-18b4-81fb-9e96466c1bf3@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 67ADA18038E68 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 02-09-20 08:42:13, Pavel Tatashin wrote: > > > Am 02.09.2020 um 11:53 schrieb Vlastimil Babka : > > > > > > =EF=BB=BFOn 8/28/20 6:47 PM, Pavel Tatashin wrote: > > >> There appears to be another problem that is related to the > > >> cgroup_mutex -> mem_hotplug_lock deadlock described above. > > >> > > >> In the original deadlock that I described, the workaround is to > > >> replace crash dump from piping to Linux traditional save to files > > >> method. However, after trying this workaround, I still observed > > >> hardware watchdog resets during machine shutdown. > > >> > > >> The new problem occurs for the following reason: upon shutdown sys= temd > > >> calls a service that hot-removes memory, and if hot-removing fails= for > > > > > > Why is that hotremove even needed if we're shutting down? Are there= any > > > (virtualization?) platforms where it makes some difference over pla= in > > > shutdown/restart? > > > > If all it=E2=80=98s doing is offlining random memory that sounds unne= cessary and dangerous. Any pointers to this service so we can figure out = what it=E2=80=98s doing and why? (Arch? Hypervisor?) >=20 > Hi David, >=20 > This is how we are using it at Microsoft: there is a very large > number of small memory machines (8G each) with low downtime > requirements (reboot must be under a second). There is also a large > state ~2G of memory that we need to transfer during reboot, otherwise > it is very expensive to recreate the state. We have 2G of system > memory memory reserved as a pmem in the device tree, and use it to > pass information across reboots. Once the information is not needed we > hot-add that memory and use it during runtime, before shutdown we > hot-remove the 2G, save the program state on it, and do the reboot. I still do not get it. So what does guarantee that the memory is offlineable in the first place? Also what is the difference between offlining and simply shutting the system down so that the memory is not used in the first place. In other words what kind of difference hotremove makes? --=20 Michal Hocko SUSE Labs