From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7164C2BAEE for ; Wed, 11 Mar 2020 16:55:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E393E20737 for ; Wed, 11 Mar 2020 16:55:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FQT4Wc+J" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E393E20737 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ED9E16B0003; Wed, 11 Mar 2020 12:55:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E89AB6B0005; Wed, 11 Mar 2020 12:55:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D52206B0006; Wed, 11 Mar 2020 12:55:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id BA36B6B0003 for ; Wed, 11 Mar 2020 12:55:55 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 6BB80F026 for ; Wed, 11 Mar 2020 16:55:55 +0000 (UTC) X-FDA: 76583683470.11.wine11_6de2685373741 X-HE-Tag: wine11_6de2685373741 X-Filterd-Recvd-Size: 8845 Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Mar 2020 16:55:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1583945754; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=C9E1mdmP6Hr606pPo5qeYGkzmwHPOmC4Eba6XbgosPs=; b=FQT4Wc+JXDPVF9K6/0gMqxKs0WK10W67ttJ3xFRogyh6R1j0FtAhK0hZPmVpLuWVRjg/jc DbFZNXkfLrN4RyOQJ/OcL1fTCdyyaZAm5VfIehfdlKnsO72XgsfB3UQvs5hnl7Tcgg9kWs HuvxHQPa21tw0edmnXbMk6z4/5itLwc= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-380-ktddTzC2O2qCiwhaMoaXGg-1; Wed, 11 Mar 2020 12:55:52 -0400 X-MC-Unique: ktddTzC2O2qCiwhaMoaXGg-1 Received: by mail-wm1-f72.google.com with SMTP id t2so868676wmj.2 for ; Wed, 11 Mar 2020 09:55:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=HSgjM26zIX3PavBkrCBluq3yJAmMXPt5zdmFUceEPUk=; b=SKUzmr1rt2HKDFwxRdgQfwjIWBD8XDNSt3L5qC4FyAk4JAieGPSBRBp/o6MwFls6Bs rqkkpKuk/SMPFENZK1lnDEtlNWN8ih3Z9m/NSTiZ13uJ4T4Iptu8RJtRKxveoYF6QrPt lVKz4ApDZvjOMQh9Dj+ybZVWcxPJ7v4b5KdFkstZ2TDSMBKrGdv8shYOuPYXiihb0c+N 6yAh4vRYJbt2wK9RmRJB/g4l4enlubU5jJB1rpFn4q2YSEvrXxu86yo+yf3RAY3+/FAw F7s3/1CWZ3tVFnIEvvxNEM/qnMBx6yp+PD5DuxyUb7Nb2yGwl4ToAgbezdRAUZZe+xce 9YqA== X-Gm-Message-State: ANhLgQ3V73hcumFohgnMJFA2q20V8Nt4QqI9H7kiGbRhBQSEXOQDoAZT 9iyE1OnG2Ngmor4MYy7kjdyttMNz8nz8+ikkSRz5S/muBQ97GwnqbLObw+x5L4QBc976MIjgqmy fxPwCutffV5A= X-Received: by 2002:a5d:63c7:: with SMTP id c7mr5326987wrw.384.1583945751469; Wed, 11 Mar 2020 09:55:51 -0700 (PDT) X-Google-Smtp-Source: ADFU+vsW35ore7Qm3vjEYXtbIoqC8m+BW/xNy5Vy9kjOPP+WoiYbSKW84175pQgJX/okAmoW7A9FTg== X-Received: by 2002:a5d:63c7:: with SMTP id c7mr5326967wrw.384.1583945751193; Wed, 11 Mar 2020 09:55:51 -0700 (PDT) Received: from vitty.brq.redhat.com (g-server-2.ign.cz. [91.219.240.2]) by smtp.gmail.com with ESMTPSA id p10sm7252926wru.4.2020.03.11.09.55.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Mar 2020 09:55:49 -0700 (PDT) From: Vitaly Kuznetsov To: David Hildenbrand , linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, linux-hyperv@vger.kernel.org, David Hildenbrand , Greg Kroah-Hartman , Andrew Morton , Michal Hocko , Oscar Salvador , "Rafael J. Wysocki" , Baoquan He , Wei Yang Subject: Re: [PATCH v1 5/5] mm/memory_hotplug: allow to specify a default online_type In-Reply-To: <20200311123026.16071-6-david@redhat.com> References: <20200311123026.16071-1-david@redhat.com> <20200311123026.16071-6-david@redhat.com> Date: Wed, 11 Mar 2020 17:55:48 +0100 Message-ID: <877dzqsuej.fsf@vitty.brq.redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: David Hildenbrand writes: > For now, distributions implement advanced udev rules to essentially > - Don't online any hotplugged memory (s390x) > - Online all memory to ZONE_NORMAL (e.g., most virt environments like > hyperv) > - Online all memory to ZONE_MOVABLE in case the zone imbalance is taken > care of (e.g., bare metal, special virt environments) > > In summary: All memory is usually onlined the same way, however, the > kernel always has to ask userspace to come up with the same answer. > E.g., HyperV always waits for a memory block to get onlined before > continuing, otherwise it might end up adding memory faster than > hotplugging it, which can result in strange OOM situations. > > Let's allow to specify a default online_type, not just "online" and > "offline". This allows distributions to configure the default online_type > when booting up and be done with it. > > We can now specify "offline", "online", "online_movable" and > "online_kernel" via > - "memhp_default_state=3D" on the kernel cmdline > - /sys/devices/systemn/memory/auto_online_blocks > just like we are able to specify for a single memory block via > /sys/devices/systemn/memory/memoryX/state > Thank you for picking this up!=20 It's been awhile since I've added CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE but I vaguely recall one problem: memory hotplug may happen *very* early (just because some memory is presented to a VM as hotplug memory, it is not in e820). It happens way before we launch userspace (including udev). The question is -- which ZONE will this memory be assigned too? 'memhp_default_state=3D' resolves the issue but nobody likes additional kernel parameters for anything but debug. CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE was supposed to help, but it is binary and distro-wide (so *all* deployments will get the same default and as you validly stated we want it differently). We could've added something like your example onlining script to the kernel itself but this is likely going to be hard to sell: "policies belong to userspace!" will likely be the answer.=20 So if we don't want to start the endless discussions (again), your proposal is 'good enough'. > Cc: Greg Kroah-Hartman > Cc: Andrew Morton > Cc: Michal Hocko > Cc: Oscar Salvador > Cc: "Rafael J. Wysocki" > Cc: Baoquan He > Cc: Wei Yang > Signed-off-by: David Hildenbrand > --- > drivers/base/memory.c | 11 +++++------ > include/linux/memory_hotplug.h | 2 ++ > mm/memory_hotplug.c | 8 ++++---- > 3 files changed, 11 insertions(+), 10 deletions(-) > > diff --git a/drivers/base/memory.c b/drivers/base/memory.c > index 8d3e16dab69f..2b09b68b9f78 100644 > --- a/drivers/base/memory.c > +++ b/drivers/base/memory.c > @@ -35,7 +35,7 @@ static const char *const online_type_to_str[] =3D { > =09[MMOP_ONLINE_MOVABLE] =3D "online_movable", > }; > =20 > -static int memhp_online_type_from_str(const char *str) > +int memhp_online_type_from_str(const char *str) > { > =09int i; > =20 > @@ -394,13 +394,12 @@ static ssize_t auto_online_blocks_store(struct devi= ce *dev, > =09=09=09=09=09struct device_attribute *attr, > =09=09=09=09=09const char *buf, size_t count) > { > -=09if (sysfs_streq(buf, "online")) > -=09=09memhp_default_online_type =3D MMOP_ONLINE; > -=09else if (sysfs_streq(buf, "offline")) > -=09=09memhp_default_online_type =3D MMOP_OFFLINE; > -=09else > +=09const int online_type =3D memhp_online_type_from_str(buf); > + > +=09if (online_type < 0) > =09=09return -EINVAL; > =20 > +=09memhp_default_online_type =3D online_type; > =09return count; > } > =20 > diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplu= g.h > index c6e090b34c4b..ef55115320fb 100644 > --- a/include/linux/memory_hotplug.h > +++ b/include/linux/memory_hotplug.h > @@ -117,6 +117,8 @@ extern int arch_add_memory(int nid, u64 start, u64 si= ze, > =09=09=09struct mhp_restrictions *restrictions); > extern u64 max_mem_size; > =20 > +extern int memhp_online_type_from_str(const char *str); > + > /* Default online_type (MMOP_*) when new memory blocks are added. */ > extern int memhp_default_online_type; > /* If movable_node boot option specified */ > diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c > index 01443c70aa27..4a96273eafa7 100644 > --- a/mm/memory_hotplug.c > +++ b/mm/memory_hotplug.c > @@ -75,10 +75,10 @@ EXPORT_SYMBOL_GPL(memhp_default_online_type); > =20 > static int __init setup_memhp_default_state(char *str) > { > -=09if (!strcmp(str, "online")) > -=09=09memhp_default_online_type =3D MMOP_ONLINE; > -=09else if (!strcmp(str, "offline")) > -=09=09memhp_default_online_type =3D MMOP_OFFLINE; > +=09const int online_type =3D memhp_online_type_from_str(str); > + > +=09if (online_type >=3D 0) > +=09=09memhp_default_online_type =3D online_type; > =20 > =09return 1; > } --=20 Vitaly