Re: Store Buffers (was Re: Is it OK to pass non-acquired objects to kfree?)

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Wed, Sep 09, 2015 at 06:23:11PM -0500, Christoph Lameter wrote:
> On Wed, 9 Sep 2015, Paul E. McKenney wrote:
> 
> > > > > A processor that can randomly defer writes to cachelines in the face of
> > > > > other processors owning cachelines exclusively does not seem sane to me.
> > > > > In fact its no longer exclusive.
> > > >
> > > > Welcome to the wonderful world of store buffers, which are present even
> > > > on strongly ordered systems such as x86 and the mainframe.
> > >
> > > Store buffers hold complete cachelines that have been written to by a
> > > processor.
> >
> > In many cases, partial cachelines.  If the cacheline is not available
> > locally, the processor cannot know the contents of the rest of the cache
> > line, only the contents of the portion that it recently stored into.
> 
> For a partial cacheline it would have to read the rest of the cacheline
> before updating. And I would expect the processor to have exclusive access
> to the cacheline that is held in a store buffer. If not then there is
> trouble afoot.

Yep.  The store buffer would hold part of the cacheline, gain exclusive
access to that cacheline, then update it.

> > >            Hmmm... Guess I need to think more about this. Dont know the
> > > detail here on how they interact with cacheline exclusivity and stuff.
> >
> > A large number of stores to the same variable can happen concurrently,
> > and the system can stitch together the order of these stores after
> > the fact.
> 
> Well thats what I know. The exact way the store buffers interact with
> cache coherency is what I do not know.

That would vary among systems and be highly optimized.

> > > > > Sounds like the atomic_read needs more barriers.
> > > >
> > > > We all know that this won't happen.
> > >
> > > Well then welcome to the wonderful world of a broken kernel. Still
> > > wondering what this has to do with slab allocators.
> >
> > The concern I have is that the compiler might be able to reorder the
> > running CPU's last accesses to an object that is to be kfree()ed with
> > kfree()'s accesses.  The issue being that the compiler is within its
> > rights to assume pointers to different types don't alias unless one of the
> > types is char * (or some such, Dmitry can correct me if I am confused).
> 
> Hmmm... Yeah if one assumes that the object is going to be handled by a
> different processor then that is a valid concern but if its on the same
> processor then the guarantee is that the changes become visible to the
> exeucting thread in program order. That is enough.

The CPU is indeed constrained in this way, but the compiler is not.
In particular, the CPU must do exact alias analysis, while the compiler
is permitted to do approximate alias analysis in some cases.  However,
in gcc builds of the Linux kernel, I believe that the -fno-strict-aliasing
gcc command-line argument forces exact alias analysis.

Dmitry, anything that I am missing?

> The transfer to another processor is guarded by locks and I think that
> those are enough to ensure that the cachelines become visible in a
> controlled fashion.

For the kfree()-to-kmalloc() path, I do believe that you are correct.
Dmitry's question was leading up to the kfree().

							Thanx, Paul

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>