[PATCH 0/6] mm/slab: optimize allocation fastpath

[Joonsoo Kim <iamjoonsoo.kim@lge.com>]

SLAB always disable irq before executing any object alloc/free operation.
This is really painful in terms of performance. Benchmark result that does
alloc/free repeatedly shows that each alloc/free is rougly 2 times slower
than SLUB's one (27 ns : 14 ns). To improve performance, this patchset
try to implement allocation fastpath without disabling irq.

This is a similar way to implement allocation fastpath in SLUB.
Transaction id is introduced and updated on every operation. In allocation
fastpath, object in array cache is read speculartively. And then, pointer
pointing object position in array cache and transaction id are updated
simultaneously through this_cpu_cmpxchg_double(). If tid is unchanged
until this updating, it ensures that there is no concurrent clients
allocating/freeing object to this slab. So allocation could succeed
without disabling irq.

Above mentioned benchmark shows that alloc/free fastpath performance
is improved roughly 22%. (27 ns -> 21 ns).

Unfortunately, I cannot optimize free fastpath, because speculartively
writing freeing object pointer into array cache cannot be possible.
If anyone have a good idea to optimize free fastpath, please let me know.

Thanks.

Joonsoo Kim (6):
  mm/slab: fix gfp flags of percpu allocation at boot phase
  mm/slab: remove kmemleak_erase() call
  mm/slab: clean-up __ac_get_obj() to prepare future changes
  mm/slab: rearrange irq management
  mm/slab: cleanup ____cache_alloc()
  mm/slab: allocation fastpath without disabling irq

 include/linux/kmemleak.h |    8 --
 mm/slab.c                |  257 +++++++++++++++++++++++++++++++---------------
 2 files changed, 176 insertions(+), 89 deletions(-)

-- 
1.7.9.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>