[PATCH] hugetlb: detect race if fail to COW

[PATCH] hugetlb: detect race if fail to COW

[Hillf Danton]

In the error path that we fail to allocate new huge page, before try again, we
have to check race since page_table_lock is re-acquired.

If racing, our job is done.

Signed-off-by: Hillf Danton <dhillf@gmail.com>
---

--- a/mm/hugetlb.c	Fri Nov 18 21:38:30 2011
+++ b/mm/hugetlb.c	Fri Nov 18 21:48:15 2011
@@ -2407,7 +2407,14 @@ retry_avoidcopy:
 				BUG_ON(page_count(old_page) != 1);
 				BUG_ON(huge_pte_none(pte));
 				spin_lock(&mm->page_table_lock);
-				goto retry_avoidcopy;
+				ptep = huge_pte_offset(mm, address & huge_page_mask(h));
+				if (likely(pte_same(huge_ptep_get(ptep), pte)))
+					goto retry_avoidcopy;
+				/*
+				 * race occurs while re-acquiring page_table_lock, and
+				 * our job is done.
+				 */
+				return 0;
 			}
 			WARN_ON_ONCE(1);
 		}

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] hugetlb: detect race if fail to COW

[John Kacur]

On Fri, Nov 18, 2011 at 3:04 PM, Hillf Danton <dhillf@gmail.com> wrote:
> In the error path that we fail to allocate new huge page, before try again, we
> have to check race since page_table_lock is re-acquired.
>
> If racing, our job is done.
>
> Signed-off-by: Hillf Danton <dhillf@gmail.com>
> ---
>
> --- a/mm/hugetlb.c      Fri Nov 18 21:38:30 2011
> +++ b/mm/hugetlb.c      Fri Nov 18 21:48:15 2011
> @@ -2407,7 +2407,14 @@ retry_avoidcopy:
>                                BUG_ON(page_count(old_page) != 1);
>                                BUG_ON(huge_pte_none(pte));
>                                spin_lock(&mm->page_table_lock);
> -                               goto retry_avoidcopy;
> +                               ptep = huge_pte_offset(mm, address & huge_page_mask(h));
> +                               if (likely(pte_same(huge_ptep_get(ptep), pte)))
> +                                       goto retry_avoidcopy;
> +                               /*
> +                                * race occurs while re-acquiring page_table_lock, and
> +                                * our job is done.
> +                                */
> +                               return 0;
>                        }
>                        WARN_ON_ONCE(1);
>                }


I'm not sure about the veracity of the race condition, but you better
do spin_unlock before you return.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] hugetlb: detect race if fail to COW

[John Kacur]

On Fri, Nov 18, 2011 at 3:16 PM, John Kacur <jkacur@redhat.com> wrote:
> On Fri, Nov 18, 2011 at 3:04 PM, Hillf Danton <dhillf@gmail.com> wrote:
>> In the error path that we fail to allocate new huge page, before try again, we
>> have to check race since page_table_lock is re-acquired.
>>
>> If racing, our job is done.
>>
>> Signed-off-by: Hillf Danton <dhillf@gmail.com>
>> ---
>>
>> --- a/mm/hugetlb.c      Fri Nov 18 21:38:30 2011
>> +++ b/mm/hugetlb.c      Fri Nov 18 21:48:15 2011
>> @@ -2407,7 +2407,14 @@ retry_avoidcopy:
>>                                BUG_ON(page_count(old_page) != 1);
>>                                BUG_ON(huge_pte_none(pte));
>>                                spin_lock(&mm->page_table_lock);
>> -                               goto retry_avoidcopy;
>> +                               ptep = huge_pte_offset(mm, address & huge_page_mask(h));
>> +                               if (likely(pte_same(huge_ptep_get(ptep), pte)))
>> +                                       goto retry_avoidcopy;
>> +                               /*
>> +                                * race occurs while re-acquiring page_table_lock, and
>> +                                * our job is done.
>> +                                */
>> +                               return 0;
>>                        }
>>                        WARN_ON_ONCE(1);
>>                }
>
>
> I'm not sure about the veracity of the race condition, but you better
> do spin_unlock before you return.
>

Ugh, sorry for the noise, I see that's not how it works here.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] hugetlb: detect race if fail to COW

[Hillf Danton]

On Fri, Nov 18, 2011 at 10:21 PM, John Kacur <jkacur@redhat.com> wrote:
> On Fri, Nov 18, 2011 at 3:16 PM, John Kacur <jkacur@redhat.com> wrote:
>> On Fri, Nov 18, 2011 at 3:04 PM, Hillf Danton <dhillf@gmail.com> wrote:
>>> In the error path that we fail to allocate new huge page, before try again, we
>>> have to check race since page_table_lock is re-acquired.
>>>
>>> If racing, our job is done.
>>>
>>> Signed-off-by: Hillf Danton <dhillf@gmail.com>
>>> ---
>>>
>>> --- a/mm/hugetlb.c      Fri Nov 18 21:38:30 2011
>>> +++ b/mm/hugetlb.c      Fri Nov 18 21:48:15 2011
>>> @@ -2407,7 +2407,14 @@ retry_avoidcopy:
>>>                                BUG_ON(page_count(old_page) != 1);
>>>                                BUG_ON(huge_pte_none(pte));
>>>                                spin_lock(&mm->page_table_lock);
>>> -                               goto retry_avoidcopy;
>>> +                               ptep = huge_pte_offset(mm, address & huge_page_mask(h));
>>> +                               if (likely(pte_same(huge_ptep_get(ptep), pte)))
>>> +                                       goto retry_avoidcopy;
>>> +                               /*
>>> +                                * race occurs while re-acquiring page_table_lock, and
>>> +                                * our job is done.
>>> +                                */
>>> +                               return 0;
>>>                        }
>>>                        WARN_ON_ONCE(1);
>>>                }
>>
>>
>> I'm not sure about the veracity of the race condition, but you better
>> do spin_unlock before you return.
>>
>
> Ugh, sorry for the noise, I see that's not how it works here.

Welcome:)

Re: [PATCH] hugetlb: detect race if fail to COW

[Michal Hocko]

On Fri 18-11-11 22:04:37, Hillf Danton wrote:
> In the error path that we fail to allocate new huge page, before try again, we
> have to check race since page_table_lock is re-acquired.

I do not think we can race here because we are serialized by
hugetlb_instantiation_mutex AFAIU. Without this lock, however, we could
fall into avoidcopy and shortcut despite the fact that other thread has
already did the job.

The mutex usage is not obvious in hugetlb_cow so maybe we want to be
explicit about it (either a comment or do the recheck).

> 
> If racing, our job is done.
> 
> Signed-off-by: Hillf Danton <dhillf@gmail.com>
> ---
> 
> --- a/mm/hugetlb.c	Fri Nov 18 21:38:30 2011
> +++ b/mm/hugetlb.c	Fri Nov 18 21:48:15 2011
> @@ -2407,7 +2407,14 @@ retry_avoidcopy:
>  				BUG_ON(page_count(old_page) != 1);
>  				BUG_ON(huge_pte_none(pte));
>  				spin_lock(&mm->page_table_lock);
> -				goto retry_avoidcopy;
> +				ptep = huge_pte_offset(mm, address & huge_page_mask(h));
> +				if (likely(pte_same(huge_ptep_get(ptep), pte)))
> +					goto retry_avoidcopy;
> +				/*
> +				 * race occurs while re-acquiring page_table_lock, and
> +				 * our job is done.
> +				 */
> +				return 0;
>  			}
>  			WARN_ON_ONCE(1);
>  		}
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Michal Hocko
SUSE Labs
SUSE LINUX s.r.o.
Lihovarska 1060/12
190 00 Praha 9    
Czech Republic

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] hugetlb: detect race if fail to COW

[Hillf Danton]

On Fri, Nov 18, 2011 at 11:07 PM, Michal Hocko <mhocko@suse.cz> wrote:
> On Fri 18-11-11 22:04:37, Hillf Danton wrote:
>> In the error path that we fail to allocate new huge page, before try again, we
>> have to check race since page_table_lock is re-acquired.
>
> I do not think we can race here because we are serialized by
> hugetlb_instantiation_mutex AFAIU. Without this lock, however, we could
> fall into avoidcopy and shortcut despite the fact that other thread has
> already did the job.
>
> The mutex usage is not obvious in hugetlb_cow so maybe we want to be
> explicit about it (either a comment or do the recheck).
>

Then the following check is unnecessary, no?

Thanks,
Hillf

	/*
	 * Retake the page_table_lock to check for racing updates
	 * before the page tables are altered
	 */
	spin_lock(&mm->page_table_lock);
	ptep = huge_pte_offset(mm, address & huge_page_mask(h));
	if (likely(pte_same(huge_ptep_get(ptep), pte))) {
		/* Break COW */

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] hugetlb: detect race if fail to COW

[Michal Hocko]

On Fri 18-11-11 23:23:12, Hillf Danton wrote:
> On Fri, Nov 18, 2011 at 11:07 PM, Michal Hocko <mhocko@suse.cz> wrote:
> > On Fri 18-11-11 22:04:37, Hillf Danton wrote:
> >> In the error path that we fail to allocate new huge page, before try again, we
> >> have to check race since page_table_lock is re-acquired.
> >
> > I do not think we can race here because we are serialized by
> > hugetlb_instantiation_mutex AFAIU. Without this lock, however, we could
> > fall into avoidcopy and shortcut despite the fact that other thread has
> > already did the job.
> >
> > The mutex usage is not obvious in hugetlb_cow so maybe we want to be
> > explicit about it (either a comment or do the recheck).
> >
> 
> Then the following check is unnecessary, no?

Hmm, thinking about it some more, I guess we have to recheck because we
can still race with page migration. So we need you patch.

Reviewed-by: Michal Hocko <mhocko@suse.cz>

> 
> Thanks,
> Hillf
> 
> 	/*
> 	 * Retake the page_table_lock to check for racing updates
> 	 * before the page tables are altered
> 	 */
> 	spin_lock(&mm->page_table_lock);
> 	ptep = huge_pte_offset(mm, address & huge_page_mask(h));
> 	if (likely(pte_same(huge_ptep_get(ptep), pte))) {
> 		/* Break COW */
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Michal Hocko
SUSE Labs
SUSE LINUX s.r.o.
Lihovarska 1060/12
190 00 Praha 9    
Czech Republic

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] hugetlb: detect race if fail to COW

[Andrew Morton]

On Fri, 18 Nov 2011 17:11:28 +0100
Michal Hocko <mhocko@suse.cz> wrote:

> On Fri 18-11-11 23:23:12, Hillf Danton wrote:
> > On Fri, Nov 18, 2011 at 11:07 PM, Michal Hocko <mhocko@suse.cz> wrote:
> > > On Fri 18-11-11 22:04:37, Hillf Danton wrote:
> > >> In the error path that we fail to allocate new huge page, before try again, we
> > >> have to check race since page_table_lock is re-acquired.
> > >
> > > I do not think we can race here because we are serialized by
> > > hugetlb_instantiation_mutex AFAIU. Without this lock, however, we could
> > > fall into avoidcopy and shortcut despite the fact that other thread has
> > > already did the job.
> > >
> > > The mutex usage is not obvious in hugetlb_cow so maybe we want to be
> > > explicit about it (either a comment or do the recheck).
> > >
> > 
> > Then the following check is unnecessary, no?
> 
> Hmm, thinking about it some more, I guess we have to recheck because we
> can still race with page migration. So we need you patch.
> 
> Reviewed-by: Michal Hocko <mhocko@suse.cz>

So we need a new changelog.  How does this look?


From: Hillf Danton <dhillf@gmail.com>
Subject: hugetlb: detect race upon page allocation failure during COW

In the error path where we failed to allocate a new huge page, we should
check whether a racing thread has added this page for us while this thread
waited for the page_table_lock.

We are serialized by hugetlb_instantiation_mutex on the pagefault patch
but this race can occur when another thread is performing page migration.

Signed-off-by: Hillf Danton <dhillf@gmail.com>
Reviewed-by: Michal Hocko <mhocko@suse.cz>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Johannes Weiner <jweiner@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/hugetlb.c |    9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff -puN mm/hugetlb.c~hugetlb-detect-race-upon-page-allocation-failure-during-cow mm/hugetlb.c
--- a/mm/hugetlb.c~hugetlb-detect-race-upon-page-allocation-failure-during-cow
+++ a/mm/hugetlb.c
@@ -2407,7 +2407,14 @@ retry_avoidcopy:
 				BUG_ON(page_count(old_page) != 1);
 				BUG_ON(huge_pte_none(pte));
 				spin_lock(&mm->page_table_lock);
-				goto retry_avoidcopy;
+				ptep = huge_pte_offset(mm, address & huge_page_mask(h));
+				if (likely(pte_same(huge_ptep_get(ptep), pte)))
+					goto retry_avoidcopy;
+				/*
+				 * race occurs while re-acquiring page_table_lock, and
+				 * our job is done.
+				 */
+				return 0;
 			}
 			WARN_ON_ONCE(1);
 		}
_

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] hugetlb: detect race if fail to COW

[Hillf Danton]

On Sat, Nov 19, 2011 at 3:39 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Fri, 18 Nov 2011 17:11:28 +0100
> Michal Hocko <mhocko@suse.cz> wrote:
>
>> On Fri 18-11-11 23:23:12, Hillf Danton wrote:
>> > On Fri, Nov 18, 2011 at 11:07 PM, Michal Hocko <mhocko@suse.cz> wrote:
>> > > On Fri 18-11-11 22:04:37, Hillf Danton wrote:
>> > >> In the error path that we fail to allocate new huge page, before try again, we
>> > >> have to check race since page_table_lock is re-acquired.
>> > >
>> > > I do not think we can race here because we are serialized by
>> > > hugetlb_instantiation_mutex AFAIU. Without this lock, however, we could
>> > > fall into avoidcopy and shortcut despite the fact that other thread has
>> > > already did the job.
>> > >
>> > > The mutex usage is not obvious in hugetlb_cow so maybe we want to be
>> > > explicit about it (either a comment or do the recheck).
>> > >
>> >
>> > Then the following check is unnecessary, no?
>>
>> Hmm, thinking about it some more, I guess we have to recheck because we
>> can still race with page migration. So we need you patch.
>>
>> Reviewed-by: Michal Hocko <mhocko@suse.cz>
>
> So we need a new changelog.  How does this look?
>
Thanks Andrew and Michal:)

Best regards
Hillf

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] hugetlb: detect race if fail to COW

[Michal Hocko]

On Fri 18-11-11 17:11:28, Michal Hocko wrote:
> On Fri 18-11-11 23:23:12, Hillf Danton wrote:
> > On Fri, Nov 18, 2011 at 11:07 PM, Michal Hocko <mhocko@suse.cz> wrote:
> > > On Fri 18-11-11 22:04:37, Hillf Danton wrote:
> > >> In the error path that we fail to allocate new huge page, before try again, we
> > >> have to check race since page_table_lock is re-acquired.
> > >
> > > I do not think we can race here because we are serialized by
> > > hugetlb_instantiation_mutex AFAIU. Without this lock, however, we could
> > > fall into avoidcopy and shortcut despite the fact that other thread has
> > > already did the job.
> > >
> > > The mutex usage is not obvious in hugetlb_cow so maybe we want to be
> > > explicit about it (either a comment or do the recheck).
> > >
> > 
> > Then the following check is unnecessary, no?
> 
> Hmm, thinking about it some more, I guess we have to recheck because we
> can still race with page migration. So we need you patch.

OK, so looked at it again and we cannot race with page migration because
the page is locked (by unmap_and_move_*page) migration and we have the
old page locked here as well (hugetlb_fault).

Or am I missing something?

-- 
Michal Hocko
SUSE Labs
SUSE LINUX s.r.o.
Lihovarska 1060/12
190 00 Praha 9    
Czech Republic

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] hugetlb: detect race if fail to COW

[Michal Hocko]

On Mon 21-11-11 13:23:03, Michal Hocko wrote:
> On Fri 18-11-11 17:11:28, Michal Hocko wrote:
> > On Fri 18-11-11 23:23:12, Hillf Danton wrote:
> > > On Fri, Nov 18, 2011 at 11:07 PM, Michal Hocko <mhocko@suse.cz> wrote:
> > > > On Fri 18-11-11 22:04:37, Hillf Danton wrote:
> > > >> In the error path that we fail to allocate new huge page, before try again, we
> > > >> have to check race since page_table_lock is re-acquired.
> > > >
> > > > I do not think we can race here because we are serialized by
> > > > hugetlb_instantiation_mutex AFAIU. Without this lock, however, we could
> > > > fall into avoidcopy and shortcut despite the fact that other thread has
> > > > already did the job.
> > > >
> > > > The mutex usage is not obvious in hugetlb_cow so maybe we want to be
> > > > explicit about it (either a comment or do the recheck).
> > > >
> > > 
> > > Then the following check is unnecessary, no?
> > 
> > Hmm, thinking about it some more, I guess we have to recheck because we
> > can still race with page migration. So we need you patch.
> 
> OK, so looked at it again and we cannot race with page migration because
> the page is locked (by unmap_and_move_*page) migration and we have the
> old page locked here as well (hugetlb_fault).
> 
> Or am I missing something?

And the updated patch:
--- 

[PATCH] hugetlb: clarify hugetlb_instantiation_mutex usage

[Michal Hocko]

And the follow up documentation patch:
---