From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DCE7C433F5 for ; Wed, 6 Oct 2021 08:01:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BD4CF61019 for ; Wed, 6 Oct 2021 08:01:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BD4CF61019 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 54D1C900002; Wed, 6 Oct 2021 04:01:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4FC106B0071; Wed, 6 Oct 2021 04:01:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C3DC900002; Wed, 6 Oct 2021 04:01:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0171.hostedemail.com [216.40.44.171]) by kanga.kvack.org (Postfix) with ESMTP id 2CCBD6B006C for ; Wed, 6 Oct 2021 04:01:46 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D8DCD32094 for ; Wed, 6 Oct 2021 08:01:45 +0000 (UTC) X-FDA: 78665268570.19.38E68D1 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf03.hostedemail.com (Postfix) with ESMTP id 6192D3000C82 for ; Wed, 6 Oct 2021 08:01:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633507304; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c35aybOfo9pBjuNizm0LjauEBD0q9opIWAMl2tyAep0=; b=LdrdiA+2xxFC59rAs2/qm2JXf1/wUBotuHiI+5uVmNi1J5E9/QkoK3qz+zslMFgOKfR3yd QaGbqpS/YcgwAbtRXMk2UNRG+rm8cj6aVmsRB0FDyci8jna6VTdGak9yw/qpSW2Tdg4WMg JDNSzyol2C2d0sbJ4EAJHWuGwu/DlK0= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-343-dmIfcxcKNzKfX5HjzVB9vw-1; Wed, 06 Oct 2021 04:01:41 -0400 X-MC-Unique: dmIfcxcKNzKfX5HjzVB9vw-1 Received: by mail-wr1-f71.google.com with SMTP id r25-20020adfab59000000b001609ddd5579so1324422wrc.21 for ; Wed, 06 Oct 2021 01:01:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:to:cc:references:from:organization:subject :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=c35aybOfo9pBjuNizm0LjauEBD0q9opIWAMl2tyAep0=; b=yba2bVs4Yeafk0nkLyuCDWQg11Zy1l4h0T4t73bpJNjeL/E4SNvucdHmL1vaJN4xTZ rAtLZ3nj82hrgkjfJ5sBa+DEXzfCnXWLcE6eSrOq5i5/RjcR81Ao1lF5jN7cAvR03897 jNsVg68KqaBVaIhQUMgDZRVi8MkFi5/rDDbkjXnlNr/4gJaZ2FpSrcb5u+NTXtq595N0 5vVQWpLglfbkSVQU8e6lUxMfBWhG3boKt405m/mvGj9ekJOAHwuOg0s6Ut+cw9VGP2BP SJCWgofEHIO0PqUmW4WxagVFlXLeCNZEsuE/lKxIVVnG/+/efhpcxQTUVpVphaLy1a5F jkcw== X-Gm-Message-State: AOAM533NMfcI4Rd4iQt0zK25Iqzo+V2+6ZaYKItpqUiW/KC/Kwh7Fa82 jQGjZ7jf8g5i4STqOvrP+cFXinJmXjOE/s0TmLsmVQvvwgEF1U3CCcgV59rteu9jeuuP+a8xn/C LSkuyOt0J+GxdZTGk3429k8UXT5CCGrWkTrMd5biZu/vz4yHIgRXttWhH4v4= X-Received: by 2002:a05:600c:1992:: with SMTP id t18mr2699813wmq.48.1633507300661; Wed, 06 Oct 2021 01:01:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwyJvYkKNT0PnZsl8y/R4yENzonm/o0NU+J2mMwAgmnH5NfAQA3pCU7qmNSg1WRzAjBTWX+Xg== X-Received: by 2002:a05:600c:1992:: with SMTP id t18mr2699788wmq.48.1633507300300; Wed, 06 Oct 2021 01:01:40 -0700 (PDT) Received: from [192.168.3.132] (p5b0c6529.dip0.t-ipconnect.de. [91.12.101.41]) by smtp.gmail.com with ESMTPSA id a2sm6465397wru.82.2021.10.06.01.01.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 06 Oct 2021 01:01:39 -0700 (PDT) To: Mike Rapoport Cc: linux-kernel@vger.kernel.org, Andrew Morton , Jonathan Corbet , Michal Hocko , Oscar Salvador , linux-doc@vger.kernel.org, linux-mm@kvack.org References: <20210930144117.23641-1-david@redhat.com> <20210930144117.23641-4-david@redhat.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH v1 3/3] memory-hotplug.rst: document the "auto-movable" online policy Message-ID: <4bab9000-0b49-a852-d574-1c8b2fe10de1@redhat.com> Date: Wed, 6 Oct 2021 10:01:39 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 6192D3000C82 X-Stat-Signature: ezkzygpfijogy4nj5e1wn5ebouu6zohy Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=LdrdiA+2; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf03.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=david@redhat.com X-HE-Tag: 1633507305-317626 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 06.10.21 02:35, Mike Rapoport wrote: > On Thu, Sep 30, 2021 at 04:41:17PM +0200, David Hildenbrand wrote: >> In commit e83a437faa62 ("mm/memory_hotplug: introduce "auto-movable" o= nline >> policy") we introduced a new memory online policy to automatically >> select a zone for memory blocks to be onlined. We added a way to >> set the active online policy and tunables for the auto-movable online >> policy. In follow-up commits we tweaked the "auto-movable" policy to a= lso >> consider memory device details when selecting zones for memory blocks = to >> be onlined. >> >> Let's document the new toggles and how the two online policies we have >> work. >> >> Signed-off-by: David Hildenbrand >> --- >> .../admin-guide/mm/memory-hotplug.rst | 128 +++++++++++++++-= -- >> 1 file changed, 108 insertions(+), 20 deletions(-) >> >> diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documen= tation/admin-guide/mm/memory-hotplug.rst >> index ee00b70dedde..c20a2c0031cf 100644 >> --- a/Documentation/admin-guide/mm/memory-hotplug.rst >> +++ b/Documentation/admin-guide/mm/memory-hotplug.rst >> @@ -165,9 +165,8 @@ Or alternatively:: >> =20 >> % echo 1 > /sys/devices/system/memory/memoryXXX/online >> =20 >> -The kernel will select the target zone automatically, usually default= ing to >> -``ZONE_NORMAL`` unless ``movable_node`` has been specified on the ker= nel >> -command line or if the memory block would intersect the ZONE_MOVABLE = already. >> +The kernel will select the target zone automatically, depending on th= e >> +configured ``online_policy``. >> =20 >> One can explicitly request to associate an offline memory block with >> ZONE_MOVABLE by:: >> @@ -198,6 +197,9 @@ Auto-onlining can be enabled by writing ``online``= , ``online_kernel`` or >> =20 >> % echo online > /sys/devices/system/memory/auto_online_blocks >> =20 >> +Similarly to manual onlining, with ``online`` the kernel will select = the >> +target zone automatically, depending on the configured ``online_polic= y``. >> + >> Modifying the auto-online behavior will only affect all subsequently= added >> memory blocks only. >> =20 >> @@ -393,9 +395,11 @@ command line parameters are relevant: >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D >> ``memhp_default_state`` configure auto-onlining by essentially sett= ing >> ``/sys/devices/system/memory/auto_online_bl= ocks``. >> -``movable_node`` configure automatic zone selection in the kernel. W= hen >> - set, the kernel will default to ZONE_MOVABLE, unless >> - other zones can be kept contiguous. >> +``movable_node`` configure automatic zone selection in the kernel wh= en >> + using the ``contig-zones`` online policy. When >> + set, the kernel will default to ZONE_MOVABLE when >> + onlining a memory block, unless other zones can be kept >> + contiguous. >=20 > The movable_node main purpose is to allow unplugging an entire node. Zo= ne > selection is a consequence of this. You may want to cite the descriptio= n of > movable_node in kernel-paramenters.txt here. Right, I only document the effects of these parameters on memory=20 hot(un)plug. What about: diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst=20 b/Documentation/admin-guide/mm/memory-hotplug.rst index c20a2c0031cf..f8976ded0863 100644 --- a/Documentation/admin-guide/mm/memory-hotplug.rst +++ b/Documentation/admin-guide/mm/memory-hotplug.rst @@ -402,6 +402,9 @@ command line parameters are relevant: contiguous. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D +See Documentation/admin-guide/kernel-parameters.txt for a more generic +description of these command line parameters. + Module Parameters ------------------ >=20 > And, pardon my ignorance, how movable_node will play with auto-movable > policy? It's essentially ignored with the auto-movable policy for memory=20 hotplugged after boot (!MEMBLOCK_HOTPLUG). That's why only the=20 description of "contig-zones" below describes how it interacts with the=20 ``movable_node``, and we make it clear here that it's restricted to the=20 "contig-zones" policy as well.
Bare metal, where we care about reliably unplugging hotplugged memory=20 usually configures auto-onlining to "online_movable": for example,=20 that's the case on RHEL. auto-movable doesn't make too much sense for=20 bare metal: the nature of "movable_node" is to essentially online=20 anything that might get hotunplugged MOVABLE, especially after=20 hotplugging memory and rebooting: that is highly dangerous especially in=20 virtualized environments. "auto-movable" is valuable in virtualized environments, where we add=20 memory via: * add_memory_driver_managed() like virtio-mem, whereby such memory is never part of the firmware provided memory-map, so the policy is always in control even after a reboot. * Hotplugged virtual DIMMs, such as provided by x86-64/arm64, whereby we don't include these DIMMs in the firmware-provided memory map, but ACPI code adds them after early boot, making it behave similar to add_memory_driver_managed() -- the policy is always in control even after a reboot.
Thanks! --=20 Thanks, David / dhildenb