From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C4F4C433F5 for ; Thu, 27 Jan 2022 12:47:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DC57B6B0071; Thu, 27 Jan 2022 07:46:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D75306B0072; Thu, 27 Jan 2022 07:46:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3C6A6B0073; Thu, 27 Jan 2022 07:46:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0073.hostedemail.com [216.40.44.73]) by kanga.kvack.org (Postfix) with ESMTP id B5ED46B0071 for ; Thu, 27 Jan 2022 07:46:59 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 77781181E7860 for ; Thu, 27 Jan 2022 12:46:59 +0000 (UTC) X-FDA: 79076041758.26.013ABAE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf29.hostedemail.com (Postfix) with ESMTP id DE63C12001B for ; Thu, 27 Jan 2022 12:46:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643287618; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ALFiThAZRei2Ll9xqqRD4pC1dm+F070gAZM4sCDYH80=; b=Z01l5c3vzXqj4q2pXscLHL0+2vXvuGtvRCe5U6fsxnz/Q9Inuq4LV40AuSc7pYSglxIivZ KNwVnrMCZDFHQv/axYNc1ZQD/cI/oon3CSVwfkge4WSnxQAgWwDgAISBKuLaMaOiv+4b+J eRKJa9+AnJ2mZfOpeI8axiav0vw7VwE= Received: from mail-ej1-f71.google.com (mail-ej1-f71.google.com [209.85.218.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-172-L7tElClRMqmXZRP09BLxlQ-1; Thu, 27 Jan 2022 07:46:57 -0500 X-MC-Unique: L7tElClRMqmXZRP09BLxlQ-1 Received: by mail-ej1-f71.google.com with SMTP id v2-20020a170906292200b006a94a27f903so1291886ejd.8 for ; Thu, 27 Jan 2022 04:46:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:organization:in-reply-to :content-transfer-encoding; bh=ALFiThAZRei2Ll9xqqRD4pC1dm+F070gAZM4sCDYH80=; b=sHIYWX85fZJkPOQp5btaA3Uj7JwoOl5ELyqS56TTWbxbtb+cets4tvvsGkdhEtIb4Z PM1Wy9Z/R8/wcbfjccs8Qcumcmukqw9fTpVUeWEZhbmWoG9ohE+zONo8bODyZR/Rc1sd Ov8ijlCnAylCelyG4oIJk8qE75eEpOAMcnZ34PpVOxONbaV/7oLvhZ95nE3ehafpnJY+ LPdvNKsjXSBZDoAVOMYCTWCBIPTwuPw2/ljukzYwNwS8GsEuQYa9beZUjx9afIKH7Yfe KYoVGcsiDRAk80rFncAeaQMUj6b46iQklrCyLxYveEafUiHEX16DxlQTBguG6S1fpooQ 4YrQ== X-Gm-Message-State: AOAM532r2+vk1euTZnOsS7CPsDjkd7YaO2TaBQvUuYc+oS+IUc9SJU+v RJULwqVZBb2QnkU3ey2aDgjhM3FtzpnSSlwq7wZi7Z2LInQ525jx1o2kwpFXVaqeGnPMwkZY2gT Uo0pEmCLNut4= X-Received: by 2002:a17:907:a089:: with SMTP id hu9mr2860969ejc.680.1643287615815; Thu, 27 Jan 2022 04:46:55 -0800 (PST) X-Google-Smtp-Source: ABdhPJyKNRDuVEw6gMC+JHMbjWpjotLdKjF4OPfTstmx43Retq5GPIkHdVhIpby1+MaA873czLwcvg== X-Received: by 2002:a17:907:a089:: with SMTP id hu9mr2860944ejc.680.1643287615503; Thu, 27 Jan 2022 04:46:55 -0800 (PST) Received: from ?IPV6:2003:cb:c70d:8300:4812:9d4f:6cd8:7f47? (p200300cbc70d830048129d4f6cd87f47.dip0.t-ipconnect.de. [2003:cb:c70d:8300:4812:9d4f:6cd8:7f47]) by smtp.gmail.com with ESMTPSA id kq16sm8632834ejb.163.2022.01.27.04.46.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 27 Jan 2022 04:46:54 -0800 (PST) Message-ID: <6db33bb0-c72a-5539-5873-14039702e2a3@redhat.com> Date: Thu, 27 Jan 2022 13:46:53 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.4.0 Subject: Re: [PATCH 4/6] mm, memory_hotplug: reorganize new pgdat initialization To: Michal Hocko , Andrew Morton Cc: linux-mm@kvack.org, LKML , Alexey Makhalov , Dennis Zhou , Eric Dumazet , Oscar Salvador , Tejun Heo , Christoph Lameter , Nico Pache , Wei Yang , Rafael Aquini , Michal Hocko References: <20220127085305.20890-1-mhocko@kernel.org> <20220127085305.20890-5-mhocko@kernel.org> From: David Hildenbrand Organization: Red Hat In-Reply-To: <20220127085305.20890-5-mhocko@kernel.org> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspam-User: nil X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: DE63C12001B X-Stat-Signature: k5onscgb7j36kpgt1arg1gfbefpuaf7q Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Z01l5c3v; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf29.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=david@redhat.com X-HE-Tag: 1643287618-449081 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 27.01.22 09:53, Michal Hocko wrote: > From: Michal Hocko > > When a !node_online node is brought up it needs a hotplug specific > initialization because the node could be either uninitialized yet or it > could have been recycled after previous hotremove. hotadd_init_pgdat is > responsible for that. > > Internal pgdat state is initialized at two places currently > - hotadd_init_pgdat > - free_area_init_core_hotplug > There is no real clear cut what should go where but this patch's chosen to > move the whole internal state initialization into free_area_init_core_hotplug. > hotadd_init_pgdat is still responsible to pull all the parts together - > most notably to initialize zonelists because those depend on the overall topology. > > This patch doesn't introduce any functional change. > > Acked-by: Rafael Aquini > Signed-off-by: Michal Hocko > --- > include/linux/memory_hotplug.h | 2 +- > mm/memory_hotplug.c | 28 +++------------------------- > mm/page_alloc.c | 25 +++++++++++++++++++++++-- > 3 files changed, 27 insertions(+), 28 deletions(-) > > diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h > index 60f09d3ebb3d..76bf2de86def 100644 > --- a/include/linux/memory_hotplug.h > +++ b/include/linux/memory_hotplug.h > @@ -319,7 +319,7 @@ extern void set_zone_contiguous(struct zone *zone); > extern void clear_zone_contiguous(struct zone *zone); > > #ifdef CONFIG_MEMORY_HOTPLUG > -extern void __ref free_area_init_core_hotplug(int nid); > +extern void __ref free_area_init_core_hotplug(struct pglist_data *pgdat); > extern int __add_memory(int nid, u64 start, u64 size, mhp_t mhp_flags); > extern int add_memory(int nid, u64 start, u64 size, mhp_t mhp_flags); > extern int add_memory_resource(int nid, struct resource *resource, > diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c > index 875cdc7ffa58..ddc62f8b591f 100644 > --- a/mm/memory_hotplug.c > +++ b/mm/memory_hotplug.c > @@ -1166,39 +1166,16 @@ static pg_data_t __ref *hotadd_init_pgdat(int nid) > { > struct pglist_data *pgdat; > > - pgdat = NODE_DATA(nid); > - > /* > * NODE_DATA is preallocated (free_area_init) but its internal > * state is not allocated completely. Add missing pieces. > * Completely offline nodes stay around and they just need > * reintialization. > */ > - if (pgdat->per_cpu_nodestats == &boot_nodestats) { > - pgdat->per_cpu_nodestats = > - alloc_percpu(struct per_cpu_nodestat); > - } else { > - int cpu; > - /* > - * Reset the nr_zones, order and highest_zoneidx before reuse. > - * Note that kswapd will init kswapd_highest_zoneidx properly > - * when it starts in the near future. > - */ > - pgdat->nr_zones = 0; > - pgdat->kswapd_order = 0; > - pgdat->kswapd_highest_zoneidx = 0; > - for_each_online_cpu(cpu) { > - struct per_cpu_nodestat *p; > - > - p = per_cpu_ptr(pgdat->per_cpu_nodestats, cpu); > - memset(p, 0, sizeof(*p)); > - } > - } > - > - pgdat->node_start_pfn = 0; > + pgdat = NODE_DATA(nid); > > /* init node's zones as empty zones, we don't have any present pages.*/ > - free_area_init_core_hotplug(nid); > + free_area_init_core_hotplug(pgdat); > > /* > * The node we allocated has no zone fallback lists. For avoiding > @@ -1210,6 +1187,7 @@ static pg_data_t __ref *hotadd_init_pgdat(int nid) > * When memory is hot-added, all the memory is in offline state. So > * clear all zones' present_pages because they will be updated in > * online_pages() and offline_pages(). > + * TODO: should be in free_area_init_core_hotplug? > */ > reset_node_managed_pages(pgdat); > reset_node_present_pages(pgdat); > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > index 1a05669044d3..32d0189de4c5 100644 > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -7506,12 +7506,33 @@ static void __meminit zone_init_internals(struct zone *zone, enum zone_type idx, > * NOTE: this function is only called during memory hotplug > */ > #ifdef CONFIG_MEMORY_HOTPLUG > -void __ref free_area_init_core_hotplug(int nid) > +void __ref free_area_init_core_hotplug(struct pglist_data *pgdat) > { > + int nid = pgdat->node_id; > enum zone_type z; > - pg_data_t *pgdat = NODE_DATA(nid); > + int cpu; > > pgdat_init_internals(pgdat); > + > + if (pgdat->per_cpu_nodestats == &boot_nodestats) > + pgdat->per_cpu_nodestats = alloc_percpu(struct per_cpu_nodestat); > + > + /* > + * Reset the nr_zones, order and highest_zoneidx before reuse. > + * Note that kswapd will init kswapd_highest_zoneidx properly > + * when it starts in the near future. > + */ > + pgdat->nr_zones = 0; > + pgdat->kswapd_order = 0; > + pgdat->kswapd_highest_zoneidx = 0; > + pgdat->node_start_pfn = 0; > + for_each_online_cpu(cpu) { > + struct per_cpu_nodestat *p; > + > + p = per_cpu_ptr(pgdat->per_cpu_nodestats, cpu); > + memset(p, 0, sizeof(*p)); > + } > + > for (z = 0; z < MAX_NR_ZONES; z++) > zone_init_internals(&pgdat->node_zones[z], z, nid, 0); > } I feel like we should be initializing all of this only once, just after allocating the node. There should be no difference between a node we're reusing and a "fresh" node. IOW, memory offlining should be resetting all state accordingly when the last memory goes away. But I might be wrong and this patch looks like an improvement, as you say, without functional change Acked-by: David Hildenbrand -- Thanks, David / dhildenb