[PATCH] maple_tree: Make maple state reusable after mas_empty_area()

[PATCH] maple_tree: Make maple state reusable after mas_empty_area()

[Liam R. Howlett]

Do not update the min and max of the maple state to the slot of the leaf
node.  Leaving the min and max to the node entry allows for the maple
state to be used in other operations.

Users would get unexpected results from other operations on the maple
state after calling the affected function.

Reported-by: "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>
Reported-by: Tad <support@spotco.us>
Reported-by: Michael Keyes <mgkeyes@vigovproductions.net>
Link: https://lore.kernel.org/linux-mm/32f156ba80010fd97dbaf0a0cdfc84366608624d.camel@intel.com/
Link: https://lore.kernel.org/linux-mm/e6108286ac025c268964a7ead3aab9899f9bc6e9.camel@spotco.us/
Fixes: Fixes: 54a611b60590 ("Maple Tree: add new data structure")
Cc: <Stable@vger.kernel.org>
Signed-off-by: Liam R. Howlett <Liam.Howlett@oracle.com>
---
 lib/maple_tree.c | 15 +--------------
 1 file changed, 1 insertion(+), 14 deletions(-)

diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index 110a36479dced..1c4bc7a988ed3 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -5285,10 +5285,6 @@ static inline int mas_sparse_area(struct ma_state *mas, unsigned long min,
 int mas_empty_area(struct ma_state *mas, unsigned long min,
 		unsigned long max, unsigned long size)
 {
-	unsigned char offset;
-	unsigned long *pivots;
-	enum maple_type mt;
-
 	if (min >= max)
 		return -EINVAL;
 
@@ -5311,18 +5307,9 @@ int mas_empty_area(struct ma_state *mas, unsigned long min,
 	if (unlikely(mas_is_err(mas)))
 		return xa_err(mas->node);
 
-	offset = mas->offset;
-	if (unlikely(offset == MAPLE_NODE_SLOTS))
+	if (unlikely(mas->offset == MAPLE_NODE_SLOTS))
 		return -EBUSY;
 
-	mt = mte_node_type(mas->node);
-	pivots = ma_pivots(mas_mn(mas), mt);
-	if (offset)
-		mas->min = pivots[offset - 1] + 1;
-
-	if (offset < mt_pivots[mt])
-		mas->max = pivots[offset];
-
 	if (mas->index < mas->min)
 		mas->index = mas->min;
 
-- 
2.39.2

Re: [PATCH] maple_tree: Make maple state reusable after mas_empty_area()

[Edgecombe, Rick P]

On Thu, 2023-05-04 at 13:55 -0400, Liam R. Howlett wrote:
> Do not update the min and max of the maple state to the slot of the
> leaf
> node.  Leaving the min and max to the node entry allows for the maple
> state to be used in other operations.
> 
> Users would get unexpected results from other operations on the maple
> state after calling the affected function.
> 
> Reported-by: "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>
> Reported-by: Tad <support@spotco.us>
> Reported-by: Michael Keyes <mgkeyes@vigovproductions.net>
> Link:
> https://lore.kernel.org/linux-mm/32f156ba80010fd97dbaf0a0cdfc84366608624d.camel@intel.com/
> Link:
> https://lore.kernel.org/linux-mm/e6108286ac025c268964a7ead3aab9899f9bc6e9.camel@spotco.us/
> Fixes: Fixes: 54a611b60590 ("Maple Tree: add new data structure")
> Cc: <Stable@vger.kernel.org>
> Signed-off-by: Liam R. Howlett <Liam.Howlett@oracle.com>
> ---

This fixes it for all the cases I encountered, thanks!

Re: [PATCH] maple_tree: Make maple state reusable after mas_empty_area()

[Michael Keyes]

On 04.05.23 20:10, Edgecombe, Rick P wrote:
> On Thu, 2023-05-04 at 13:55 -0400, Liam R. Howlett wrote:
>> Do not update the min and max of the maple state to the slot of the
>> leaf
>> node.  Leaving the min and max to the node entry allows for the maple
>> state to be used in other operations.
>>
>> Users would get unexpected results from other operations on the maple
>> state after calling the affected function.
>>
>> Reported-by: "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>
>> Reported-by: Tad <support@spotco.us>
>> Reported-by: Michael Keyes <mgkeyes@vigovproductions.net>
>> Link:
>> https://lore.kernel.org/linux-mm/32f156ba80010fd97dbaf0a0cdfc84366608624d.camel@intel.com/
>> Link:
>> https://lore.kernel.org/linux-mm/e6108286ac025c268964a7ead3aab9899f9bc6e9.camel@spotco.us/
>> Fixes: Fixes: 54a611b60590 ("Maple Tree: add new data structure")
>> Cc: <Stable@vger.kernel.org>
>> Signed-off-by: Liam R. Howlett <Liam.Howlett@oracle.com>
>> ---
> This fixes it for all the cases I encountered, thanks!
Me too. I had issues running an old version of LuaJIT, and it seems to
be running better than it had in a long time now! Thank you!

-- 
Michael

Re: [PATCH] maple_tree: Make maple state reusable after mas_empty_area()

[Peng Zhang]

在 2023/5/5 01:55, Liam R. Howlett 写道:
> Do not update the min and max of the maple state to the slot of the leaf
> node.  Leaving the min and max to the node entry allows for the maple
> state to be used in other operations.
> 
> Users would get unexpected results from other operations on the maple
> state after calling the affected function.
> 
> Reported-by: "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>
> Reported-by: Tad <support@spotco.us>
> Reported-by: Michael Keyes <mgkeyes@vigovproductions.net>
> Link: https://lore.kernel.org/linux-mm/32f156ba80010fd97dbaf0a0cdfc84366608624d.camel@intel.com/
> Link: https://lore.kernel.org/linux-mm/e6108286ac025c268964a7ead3aab9899f9bc6e9.camel@spotco.us/
> Fixes: Fixes: 54a611b60590 ("Maple Tree: add new data structure")
> Cc: <Stable@vger.kernel.org>
> Signed-off-by: Liam R. Howlett <Liam.Howlett@oracle.com>
> ---
>   lib/maple_tree.c | 15 +--------------
>   1 file changed, 1 insertion(+), 14 deletions(-)
> 
> diff --git a/lib/maple_tree.c b/lib/maple_tree.c
> index 110a36479dced..1c4bc7a988ed3 100644
> --- a/lib/maple_tree.c
> +++ b/lib/maple_tree.c
> @@ -5285,10 +5285,6 @@ static inline int mas_sparse_area(struct ma_state *mas, unsigned long min,
>   int mas_empty_area(struct ma_state *mas, unsigned long min,
>   		unsigned long max, unsigned long size)
>   {
> -	unsigned char offset;
> -	unsigned long *pivots;
> -	enum maple_type mt;
> -
>   	if (min >= max)
>   		return -EINVAL;
>   
> @@ -5311,18 +5307,9 @@ int mas_empty_area(struct ma_state *mas, unsigned long min,
>   	if (unlikely(mas_is_err(mas)))
>   		return xa_err(mas->node);
>   
> -	offset = mas->offset;
> -	if (unlikely(offset == MAPLE_NODE_SLOTS))
> +	if (unlikely(mas->offset == MAPLE_NODE_SLOTS))
>   		return -EBUSY;
>   
> -	mt = mte_node_type(mas->node);
> -	pivots = ma_pivots(mas_mn(mas), mt);
> -	if (offset)
> -		mas->min = pivots[offset - 1] + 1;
> -
> -	if (offset < mt_pivots[mt])
> -		mas->max = pivots[offset];
> -
>   	if (mas->index < mas->min)
>   		mas->index = mas->min;
This will bring new bugs, mas->index should take the maximum
value with mas->index and mas_safe_min(mas, pivots, offset),
otherwise there will be overwriting allocation.

Maybe you have forgotten, I have posted a patch[1] with the same
function last week. I didn't know of a place where mas was used
after mas_empty_area() before. That patch does not introduce new
bugs, but the code style has not been updated yet. If using this
patch will bring more conflicts with my patch set, so what should
I do? 😁

[1] 
https://lore.kernel.org/lkml/20230425110511.11680-3-zhangpeng.00@bytedance.com/
>   

Re: [PATCH] maple_tree: Make maple state reusable after mas_empty_area()

[Liam R. Howlett]

* Peng Zhang <perlyzhang@gmail.com> [230504 23:23]:
> 
> 
> 在 2023/5/5 01:55, Liam R. Howlett 写道:
> > Do not update the min and max of the maple state to the slot of the leaf
> > node.  Leaving the min and max to the node entry allows for the maple
> > state to be used in other operations.
> > 
> > Users would get unexpected results from other operations on the maple
> > state after calling the affected function.
> > 
> > Reported-by: "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>
> > Reported-by: Tad <support@spotco.us>
> > Reported-by: Michael Keyes <mgkeyes@vigovproductions.net>
> > Link: https://lore.kernel.org/linux-mm/32f156ba80010fd97dbaf0a0cdfc84366608624d.camel@intel.com/
> > Link: https://lore.kernel.org/linux-mm/e6108286ac025c268964a7ead3aab9899f9bc6e9.camel@spotco.us/
> > Fixes: Fixes: 54a611b60590 ("Maple Tree: add new data structure")
> > Cc: <Stable@vger.kernel.org>
> > Signed-off-by: Liam R. Howlett <Liam.Howlett@oracle.com>
> > ---
> >   lib/maple_tree.c | 15 +--------------
> >   1 file changed, 1 insertion(+), 14 deletions(-)
> > 
> > diff --git a/lib/maple_tree.c b/lib/maple_tree.c
> > index 110a36479dced..1c4bc7a988ed3 100644
> > --- a/lib/maple_tree.c
> > +++ b/lib/maple_tree.c
> > @@ -5285,10 +5285,6 @@ static inline int mas_sparse_area(struct ma_state *mas, unsigned long min,
> >   int mas_empty_area(struct ma_state *mas, unsigned long min,
> >   		unsigned long max, unsigned long size)
> >   {
> > -	unsigned char offset;
> > -	unsigned long *pivots;
> > -	enum maple_type mt;
> > -
> >   	if (min >= max)
> >   		return -EINVAL;
> > @@ -5311,18 +5307,9 @@ int mas_empty_area(struct ma_state *mas, unsigned long min,
> >   	if (unlikely(mas_is_err(mas)))
> >   		return xa_err(mas->node);
> > -	offset = mas->offset;
> > -	if (unlikely(offset == MAPLE_NODE_SLOTS))
> > +	if (unlikely(mas->offset == MAPLE_NODE_SLOTS))
> >   		return -EBUSY;
> > -	mt = mte_node_type(mas->node);
> > -	pivots = ma_pivots(mas_mn(mas), mt);
> > -	if (offset)
> > -		mas->min = pivots[offset - 1] + 1;
> > -
> > -	if (offset < mt_pivots[mt])
> > -		mas->max = pivots[offset];
> > -
> >   	if (mas->index < mas->min)
> >   		mas->index = mas->min;
> This will bring new bugs, mas->index should take the maximum
> value with mas->index and mas_safe_min(mas, pivots, offset),
> otherwise there will be overwriting allocation.

Yes, you are right.  Both mas->index and mas->last should be set when
the gap is found, but we aren't currently doing this.

> 
> Maybe you have forgotten, I have posted a patch[1] with the same
> function last week. I didn't know of a place where mas was used
> after mas_empty_area() before. That patch does not introduce new
> bugs, but the code style has not been updated yet. If using this
> patch will bring more conflicts with my patch set, so what should
> I do? 😁
> 
> [1] https://lore.kernel.org/lkml/20230425110511.11680-3-zhangpeng.00@bytedance.com/

I did forget about that, my apologies.  I really wish I had remembered
instead of tracking down the same bug.

This has become an issue because the maple state is reused for the stack
guard checks.

We should use your patch as you sent it first and both need revisions.
We need this soon and it will need to be backported.

Can you please take it out of your series and make the necessary
adjustments and send v2?  It seems isolated enough that it won't be
difficult.  Be sure to Cc everyone I have on my patch and include the
fixes tag so it can be backported as necessary.

In the future, I'd like mas->index/mas->last to point to the gap located
to better align with mas_*_range() interface that will soon be added.
It makes more sense to record the entire gap found when returning from
the search.  I think this entire area needs modernisation/attention and
optimisation, but for now, we should fix the bug.

Thanks,
Liam

Re: [PATCH] maple_tree: Make maple state reusable after mas_empty_area()

[Peng Zhang]

在 2023/5/5 21:16, Liam R. Howlett 写道:
> * Peng Zhang <perlyzhang@gmail.com> [230504 23:23]:
>>
>>
>> 在 2023/5/5 01:55, Liam R. Howlett 写道:
>>> Do not update the min and max of the maple state to the slot of the leaf
>>> node.  Leaving the min and max to the node entry allows for the maple
>>> state to be used in other operations.
>>>
>>> Users would get unexpected results from other operations on the maple
>>> state after calling the affected function.
>>>
>>> Reported-by: "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>
>>> Reported-by: Tad <support@spotco.us>
>>> Reported-by: Michael Keyes <mgkeyes@vigovproductions.net>
>>> Link: https://lore.kernel.org/linux-mm/32f156ba80010fd97dbaf0a0cdfc84366608624d.camel@intel.com/
>>> Link: https://lore.kernel.org/linux-mm/e6108286ac025c268964a7ead3aab9899f9bc6e9.camel@spotco.us/
>>> Fixes: Fixes: 54a611b60590 ("Maple Tree: add new data structure")
>>> Cc: <Stable@vger.kernel.org>
>>> Signed-off-by: Liam R. Howlett <Liam.Howlett@oracle.com>
>>> ---
>>>    lib/maple_tree.c | 15 +--------------
>>>    1 file changed, 1 insertion(+), 14 deletions(-)
>>>
>>> diff --git a/lib/maple_tree.c b/lib/maple_tree.c
>>> index 110a36479dced..1c4bc7a988ed3 100644
>>> --- a/lib/maple_tree.c
>>> +++ b/lib/maple_tree.c
>>> @@ -5285,10 +5285,6 @@ static inline int mas_sparse_area(struct ma_state *mas, unsigned long min,
>>>    int mas_empty_area(struct ma_state *mas, unsigned long min,
>>>    		unsigned long max, unsigned long size)
>>>    {
>>> -	unsigned char offset;
>>> -	unsigned long *pivots;
>>> -	enum maple_type mt;
>>> -
>>>    	if (min >= max)
>>>    		return -EINVAL;
>>> @@ -5311,18 +5307,9 @@ int mas_empty_area(struct ma_state *mas, unsigned long min,
>>>    	if (unlikely(mas_is_err(mas)))
>>>    		return xa_err(mas->node);
>>> -	offset = mas->offset;
>>> -	if (unlikely(offset == MAPLE_NODE_SLOTS))
>>> +	if (unlikely(mas->offset == MAPLE_NODE_SLOTS))
>>>    		return -EBUSY;
>>> -	mt = mte_node_type(mas->node);
>>> -	pivots = ma_pivots(mas_mn(mas), mt);
>>> -	if (offset)
>>> -		mas->min = pivots[offset - 1] + 1;
>>> -
>>> -	if (offset < mt_pivots[mt])
>>> -		mas->max = pivots[offset];
>>> -
>>>    	if (mas->index < mas->min)
>>>    		mas->index = mas->min;
>> This will bring new bugs, mas->index should take the maximum
>> value with mas->index and mas_safe_min(mas, pivots, offset),
>> otherwise there will be overwriting allocation.
> 
> Yes, you are right.  Both mas->index and mas->last should be set when
> the gap is found, but we aren't currently doing this.
> 
>>
>> Maybe you have forgotten, I have posted a patch[1] with the same
>> function last week. I didn't know of a place where mas was used
>> after mas_empty_area() before. That patch does not introduce new
>> bugs, but the code style has not been updated yet. If using this
>> patch will bring more conflicts with my patch set, so what should
>> I do? 😁
>>
>> [1] https://lore.kernel.org/lkml/20230425110511.11680-3-zhangpeng.00@bytedance.com/
> 
> I did forget about that, my apologies.  I really wish I had remembered
> instead of tracking down the same bug.
> 
> This has become an issue because the maple state is reused for the stack
> guard checks.
> 
> We should use your patch as you sent it first and both need revisions.
> We need this soon and it will need to be backported.
> 
> Can you please take it out of your series and make the necessary
> adjustments and send v2?  It seems isolated enough that it won't be
> difficult.  Be sure to Cc everyone I have on my patch and include the
> fixes tag so it can be backported as necessary.
> 
> In the future, I'd like mas->index/mas->last to point to the gap located
> to better align with mas_*_range() interface that will soon be added.
> It makes more sense to record the entire gap found when returning from
> the search.  I think this entire area needs modernisation/attention and
> optimisation, but for now, we should fix the bug.
Yes, alignment can be done in mas_*_range() interface. When you
post the code, I can review it. I am very interested in various
data structures especially maple tree and have read most of maple
tree code and verify its correctness. If there is work around
maple tree in the future, I can help a little. I have some small
maple tree optimization ideas, but I need some time to verify that
it works. Also, I found that many codes written in maple tree are
very complicated, and it will be difficult to maintain, so I will
make some simplifications appropriately.

Thanks,
Peng
> 
> Thanks,
> Liam
> 
>