Re: [External Mail] Re: [External Mail] [RFC PATCH] mm/mempolicy: Weighted interleave auto-tuning

[Hyeonggon Yoo <hyeonggon.yoo@sk.com>]

On 2024-12-30 3:48 AM, Huang, Ying wrote:
> Gregory Price <gourry@gourry.net> writes:
> 
>> On Fri, Dec 27, 2024 at 09:59:30AM +0800, Huang, Ying wrote:
>>> Gregory Price <gourry@gourry.net> writes:
>>>> This still allows 0 to be a manual "reset specific node to default"
>>>> mechanism for a specific node, and gives us a clean override.
>>>
>>> The difficulty is that users don't know the default value when they
>>> reset a node's weight.  We don't have an interface to show them.  So, I
>>> suggest to disable the functionality: "reset specific node to default".
>>> They can still use "echo 1 > use_defaults" to reset all nodes to
>>> default.
>>>
>>
>> Good point, and agree.  So lets just ditch 0.  Since that "feature"
>> wasn't even functional in the current code (it just reset it to 1 at
>> this point), it's probably safe to just ditch it.  Worst case scenario
>> if someone takes issues, we can just have it revert the weight to 1.
> 
> Before implementing the new version, it's better to summarize the user
> space interface design first.  So, we can check whether we have some
> flaws.

Hi, hope you all had a nice year-end holiday :)

Let me summarize the points we've discussed:

- A new knob 'weightiness' is unnecessary until it's proven useful.
   Just using an internal default weightiness value will be enough.

- It will be counter-intuitive to update the value previously set by
   user, even if the value will no longer be valid (e.g. due to CXL
   memory hot-plug). User should update the weights accordingly in that
   case, instead of the kernel updating automatically overwriting them.

- Ditch the way of using 0 as 'system default' value because the user
   won't know what will be the default when setting it anyway. 0 value
   now means the kernel won't weight-interleave the node.

- Setting a node weight to default value (e.g. via the previous
   semantic of '0') could be problematic because it's not atomic -
   the system may be updating default values while the user's
   trying to set a node weight to default value.

   To deal with that, Huang suggested 'use_defaults' to atomically update
   all the weights to system default.

Please let me know if there's any point we discussed that I am missing.

Additionally I would like to mention that within an internal discussion
my colleague Honggyu suggested introducing 'mode' parameter which can be
either 'manual' or 'auto' instead of 'use_defaults' to be provide more
intuitive interface.

With Honggyu's suggestion and the points we've discussed,
I think the interface could be:

# At booting, the mode is 'auto' where the kernel can automatically
# update any weights.

mode             auto         # User hasn't specified any weight yet.
effective        [2, 1, -, -] # Using system defaults for node 0-1,
                               # and node 2-3 not populated yet.

# When a new NUMA node is added (e.g. via hotplug) in the 'auto' mode,
# all weights are re-calculated based on ACPI HMAT table, including the
# weight of the new node.

mode             auto         # User hasn't specified weights yet.
effective        [2, 1, 1, -] # Using system defaults for node 0-2,
                               # and node 3 not populated yet.

# When user set at least one weight value, change the mode to 'manual'
# where the kernel does not update any weights automatically without
# user's consent.

mode             manual       # User changed the weight of node 0 to 4,
                               # changing the mode to manual config mode.
effective        [4, 1, 1, -]


# When a new NUMA node is added (e.g. via hotplug) in the manual mode,
# the new node's weight is zero because it's in manual mode and user
# did not specify the weight for the new node yet.

mode             manual
effective        [4, 1, 1, 0]

# When user changes the mode to 'auto', all weights are changed to
# system defaults based on the ACPI HMAT table.

mode             auto
effective        [2, 1, 1, 1]  # system defaults

In the example I did not distinguish 'default weights' and 'user
weights' because it's not important where the weight values came from --
but it's important to know 1) what's the effective weights now and 2) if
the kernel can update them.

Any thoughts?

---
Best,
Hyeonggon