Re: [PATCH v2] mm/sparse: set section nid for hot-add memory

[Michal Hocko <mhocko@suse.com>]

On Wed 19-06-19 11:07:30, David Hildenbrand wrote:
> On 19.06.19 11:04, Michal Hocko wrote:
> > On Wed 19-06-19 10:51:47, David Hildenbrand wrote:
> >> On 19.06.19 09:53, Oscar Salvador wrote:
> >>> On Wed, Jun 19, 2019 at 08:23:30AM +0200, Michal Hocko wrote:
> >>>> On Tue 18-06-19 08:55:37, Wei Yang wrote:
> >>>>> In case of NODE_NOT_IN_PAGE_FLAGS is set, we store section's node id in
> >>>>> section_to_node_table[]. While for hot-add memory, this is missed.
> >>>>> Without this information, page_to_nid() may not give the right node id.
> >>>>
> >>>> Which would mean that NODE_NOT_IN_PAGE_FLAGS doesn't really work with
> >>>> the hotpluged memory, right? Any idea why nobody has noticed this
> >>>> so far? Is it because NODE_NOT_IN_PAGE_FLAGS is rare and essentially
> >>>> unused with the hotplug? page_to_nid providing an incorrect result
> >>>> sounds quite serious to me.
> >>>
> >>> The thing is that for NODE_NOT_IN_PAGE_FLAGS to be enabled we need to run out of
> >>> space in page->flags to store zone, nid and section. 
> >>> Currently, even with the largest values (with pagetable level 5), that is not
> >>> possible on x86_64.
> >>> It is possible though, that somewhere in the future, when the values get larger
> >>> (e.g: we add more zones, NODE_SHIFT grows, or we need more space to store
> >>> the section) we finally run out of room for the flags though.
> >>>
> >>> I am not sure about the other arches though, we probably should audit them
> >>> and see which ones can fall in there.
> >>>
> >>
> >> I'd love to see NODE_NOT_IN_PAGE_FLAGS go.
> > 
> > NODE_NOT_IN_PAGE_FLAGS is an implementation detail on where the
> > information is stored.
> 
> Yes and no. Storing it per section clearly doesn't allow storing node
> information on smaller granularity, like storing in page->flags does.
> 
> So no, it is not only an implementation detail.

Let me try to put it differently. NODE_NOT_IN_PAGE_FLAGS is not about
storing the mapping per section. You can do what ever other data
structure. NODE_NOT_IN_PAGE_FLAGS is in fact about telling that it is
not in page->flags.

> > I cannot say how much it is really needed now but
> > I can see there will be a demand for it in a longer term because
> > page->flags space is scarce and very interesting storage. So I do not
> > see it go away I am afraid.
> Depends on how performance-critical pfn_to_nid() is. I can't tell.

page_to_node is used in many important code paths. Not in the hotest
ones I believe but many of them are quite hot I would say.

-- 
Michal Hocko
SUSE Labs