Re: [PATCH/RFC 0/11] Shared Policy Overview

[Lee Schermerhorn <Lee.Schermerhorn@hp.com>]

On Tue, 2007-06-26 at 20:25 -0700, Christoph Lameter wrote:
> On Wed, 27 Jun 2007, Andi Kleen wrote:
> 
> > > You are sure that this works? Just by looking at the description: It
> > > cannot work. Any allocator use of a memory policy must use rcu locks
> > > otherwise the memory policy can vanish from under us while allocating a
> > > page. This means you need to add this to alloc_pages_current
> > > and alloc_pages_node.  Possible all of __alloc_pages must be handled
> > > under RCU. This is a significant increase of RCU use.
> > 
> > I've been actually looking at using RCUs for the shared policies 
> > too to plug the recent reference count issue.  I don't think it's a problem 
> > because the RCU use can be limited to when policies are actually
> > used. Besides rcu_read_lock() is a nop on non preemptible kernels
> > anyways and users of preemptible kernels will probably not notice
> > it among all the other overhead they have anyways.

Hi, Andi:

I see that Christoph has already responded, so I'll respond in the
context of his message.
> 
> If a system policy is set then it will be used all of the time.
> Could be a signficant increase in RCU use.

Generally, I don't think you need to use RCU for the system policy, as
it is statically allocated.  Now, if "default_policy" were changed to a
pointer to the actual policy, AND you could replace the pointer at
run-time, there might be a use for RCU.

>  
> > > If we can make this work then RCU should be used for all policies so that
> > > we can get rid of the requirement that policies can only be modified from
> > > the task context that created it.
> > 
> > Huh? RCU doesn't give you locking against multiple writers. Just existence
> > guarantees. And you can have those already by just holding the reference 
> > count.

Right.  It only works for shared policies, because shared policies have
a spin lock that protects the rb-tree from concurrent updates.  [And the
policies stored in the tree seem to be reference counted properly.]
However, I think RCU could be used for changing, including deleting
[more below], the task/process policy and a given VMA policy in a
similar fashion to the way I'm deleting shared file policy on removal of
last shared mapping.

RE: deleting:  it occurs to me that installing a "DEFAULT" policy could
actually delete the corresponding policy without changing semantics.  I
plan on looking at this after OLS.

> 
> If you want to replace one policy by another then RCU ensures 
> that the old policy can still be used for the remainder of the rcu period.

As Christoph indicates, I'm using RCU to replace the shared policy on a
regular file with NULL [== default!] on unmap of last shared mapping.  I
need to protect against any references that come from other than a
shared mapping.  This includes accesses via regular file system IO and
faults from private mappings of the file.   Unlike shared mappings,
which as you say are protected from disappearing by reference counts,
page cache allocations to satisfy normal file descriptor based IO or
faults in private mappings can't guarantee that the policy won't go away
when some other task removes the last shared mapping. 

> 
> If RCU is not used then the updating of a policy is not possible since
> there is currently no locking and there may be concurrent uses of the 
> policy or the zonelist generated by a policy. One thread may acquire
> the pointer to a policy while another changes the policy.
> 
> If the old policy is immediately freed then the first thread may access 
> invalid data.

Yep.  So, deleting a policy or replacing it with another can be done
safely under RCU, assuming everyone who gains access to the policy takes
a proper reference [AND releases it when finished...].

Lee

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>