[PATCH 1/1] vmalloc: purge_fragmented_blocks: Acquire spinlock before reading vmap_block

[PATCH 1/1] vmalloc: purge_fragmented_blocks: Acquire spinlock before reading vmap_block

[Kautuk Consul]

The purge_fragmented_blocks will loop over all vmap_blocks in the
vmap_block_queue to create the purge list.
Currently, the code in the loop does not acquire the &vb->lock before
reading the vb->free and vb->dirty.

Due to this, there might be a possibility of vb->free and vb->dirty being
changed in parallel which could lead to the current vmap_block not being
selected for purging.

Changing the code to acquire this spinlock before the check for vb->free
and vb->dirty.

Signed-off-by: Kautuk Consul <consul.kautuk@gmail.com>
---
 mm/vmalloc.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 3231bf3..2228971 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -855,11 +855,14 @@ static void purge_fragmented_blocks(int cpu)
 
 	rcu_read_lock();
 	list_for_each_entry_rcu(vb, &vbq->free, free_list) {
+		spin_lock(&vb->lock);
 
-		if (!(vb->free + vb->dirty == VMAP_BBMAP_BITS && vb->dirty != VMAP_BBMAP_BITS))
+		if (!(vb->free + vb->dirty == VMAP_BBMAP_BITS &&
+			  vb->dirty != VMAP_BBMAP_BITS)) {
+			spin_unlock(&vb->lock);
 			continue;
+		}
 
-		spin_lock(&vb->lock);
 		if (vb->free + vb->dirty == VMAP_BBMAP_BITS && vb->dirty != VMAP_BBMAP_BITS) {
 			vb->free = 0; /* prevent further allocs after releasing lock */
 			vb->dirty = VMAP_BBMAP_BITS; /* prevent purging it again */
-- 
1.7.6

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/1] vmalloc: purge_fragmented_blocks: Acquire spinlock before reading vmap_block

[David Rientjes]

On Thu, 8 Dec 2011, Kautuk Consul wrote:

> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 3231bf3..2228971 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -855,11 +855,14 @@ static void purge_fragmented_blocks(int cpu)
>  
>  	rcu_read_lock();
>  	list_for_each_entry_rcu(vb, &vbq->free, free_list) {
> +		spin_lock(&vb->lock);
>  
> -		if (!(vb->free + vb->dirty == VMAP_BBMAP_BITS && vb->dirty != VMAP_BBMAP_BITS))
> +		if (!(vb->free + vb->dirty == VMAP_BBMAP_BITS &&
> +			  vb->dirty != VMAP_BBMAP_BITS)) {
> +			spin_unlock(&vb->lock);
>  			continue;
> +		}
>  
> -		spin_lock(&vb->lock);
>  		if (vb->free + vb->dirty == VMAP_BBMAP_BITS && vb->dirty != VMAP_BBMAP_BITS) {
>  			vb->free = 0; /* prevent further allocs after releasing lock */
>  			vb->dirty = VMAP_BBMAP_BITS; /* prevent purging it again */

Nack, this is wrong because the if-clause you're modifying isn't the 
criteria that is used to determine whether the purge occurs or not.  It's 
merely an optimization to prevent doing exactly what your patch is doing: 
taking vb->lock unnecessarily.

In the original code, if the if-clause fails, the lock is only then taken 
and the exact same test occurs again while protected.  If the test now 
fails, the lock is immediately dropped.  A branch here is faster than a 
contented spinlock.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/1] vmalloc: purge_fragmented_blocks: Acquire spinlock before reading vmap_block

[Kautuk Consul]

On Thu, Dec 8, 2011 at 12:37 PM, David Rientjes <rientjes@google.com> wrote:
> On Thu, 8 Dec 2011, Kautuk Consul wrote:
>
>> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
>> index 3231bf3..2228971 100644
>> --- a/mm/vmalloc.c
>> +++ b/mm/vmalloc.c
>> @@ -855,11 +855,14 @@ static void purge_fragmented_blocks(int cpu)
>>
>>       rcu_read_lock();
>>       list_for_each_entry_rcu(vb, &vbq->free, free_list) {
>> +             spin_lock(&vb->lock);
>>
>> -             if (!(vb->free + vb->dirty == VMAP_BBMAP_BITS && vb->dirty != VMAP_BBMAP_BITS))
>> +             if (!(vb->free + vb->dirty == VMAP_BBMAP_BITS &&
>> +                       vb->dirty != VMAP_BBMAP_BITS)) {
>> +                     spin_unlock(&vb->lock);
>>                       continue;
>> +             }
>>
>> -             spin_lock(&vb->lock);
>>               if (vb->free + vb->dirty == VMAP_BBMAP_BITS && vb->dirty != VMAP_BBMAP_BITS) {
>>                       vb->free = 0; /* prevent further allocs after releasing lock */
>>                       vb->dirty = VMAP_BBMAP_BITS; /* prevent purging it again */
>
> Nack, this is wrong because the if-clause you're modifying isn't the
> criteria that is used to determine whether the purge occurs or not.  It's
> merely an optimization to prevent doing exactly what your patch is doing:
> taking vb->lock unnecessarily.

I agree.

>
> In the original code, if the if-clause fails, the lock is only then taken
> and the exact same test occurs again while protected.  If the test now
> fails, the lock is immediately dropped.  A branch here is faster than a
> contented spinlock.

But, if there is some concurrent change happening to vb->free and
vb->dirty, dont you think
that it will continue and then go to the next vmap_block ?

If yes, then it will not be put into the purge list.


So, can we make a change where we simply remove the first check ?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/1] vmalloc: purge_fragmented_blocks: Acquire spinlock before reading vmap_block

[David Rientjes]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 868 bytes --]

On Thu, 8 Dec 2011, Kautuk Consul wrote:

> > In the original code, if the if-clause fails, the lock is only then taken
> > and the exact same test occurs again while protected. A If the test now
> > fails, the lock is immediately dropped. A A branch here is faster than a
> > contented spinlock.
> 
> But, if there is some concurrent change happening to vb->free and
> vb->dirty, dont you think
> that it will continue and then go to the next vmap_block ?
> 
> If yes, then it will not be put into the purge list.
> 

That's intentional as an optimization, we don't care if 
vb->free + vb->dirty == VMAP_BBMAP_BITS && vb->dirty != VMAP_BBMAP_BITS 
would speculatively be true after we grab vb->lock, we'll have to purge it 
next time instead.  We certainly don't want to grab vb->lock for blocks 
that aren't candidates, so this optimization is a singificant speedup.

Re: [PATCH 1/1] vmalloc: purge_fragmented_blocks: Acquire spinlock before reading vmap_block

[Kautuk Consul]

>
> That's intentional as an optimization, we don't care if
> vb->free + vb->dirty == VMAP_BBMAP_BITS && vb->dirty != VMAP_BBMAP_BITS
> would speculatively be true after we grab vb->lock, we'll have to purge it
> next time instead.  We certainly don't want to grab vb->lock for blocks
> that aren't candidates, so this optimization is a singificant speedup.

Ah, I agree.
Anyway, the probability of there being too many vmap_blocks being
missed due to concurrent changes
is not quite high, so I guess its okay that a few vmap_blocks get
purged next time.

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>