From: Christoph Lameter <cl@linux.com>
To: Matthew Wilcox <willy@infradead.org>
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Pekka Enberg <penberg@cs.helsinki.fi>,
akpm@linux-foundation.org, Mel Gorman <mel@skynet.ie>,
andi@firstfloor.org, Rik van Riel <riel@redhat.com>,
Dave Chinner <dchinner@redhat.com>,
Christoph Hellwig <hch@lst.de>, Michal Hocko <mhocko@suse.com>,
Mike Kravetz <mike.kravetz@oracle.com>
Subject: [RFC 0/7] Slab object migration for xarray V2
Date: Thu, 20 Dec 2018 19:21:55 +0000 [thread overview]
Message-ID: <01000167cd1130c8-c9bebcb9-1f95-4f7c-b24a-90600d56c62f-000000@email.amazonses.com> (raw)
V1->V2:
- Works now on top of 4.20-rc7
- A couple of bug fixes
This is a patchset on top of Matthew Wilcox Xarray code and implements
slab object migration for xarray nodes. The migration is integrated into
the defragmetation and shrinking logic of the slab allocator.
Defragmentation will ensure that all xarray slab pages have
less objects available than specified by the slab defrag ratio.
Slab shrinking will create a slab cache with optimal object
density. Only one slab page will have available objects per node.
To test apply this patchset and run a workload that uses lots of radix tree objects
Then go to
/sys/kernel/slab/radix_tree_node
Inspect the number of total objects that the slab can handle
cat total_objects
qmdr:/sys/kernel/slab/radix_tree_node# cat objects
868 N0=448 N1=168 N2=56 N3=196
And the number of slab pages used for those
cat slabs
qmdr:/sys/kernel/slab/radix_tree_node# cat slabs
31 N0=16 N1=6 N2=2 N3=7
Perform a cache shrink operation
echo 1 >shrink
Now see how the slab has changed:
qmdr:/sys/kernel/slab/radix_tree_node# cat slabs
30 N0=15 N1=6 N2=2 N3=7
qmdr:/sys/kernel/slab/radix_tree_node# cat objects
713 N0=349 N1=141 N2=52 N3=171
This is just a barebones approach using a special mode
of the slab migration patchset that does not require refcounts.
If this is acceptable then additional functionality can be added:
1. Migration of objects to a specific node. Not sure how to implement
that. Using another sysfs field?
2. Dispersion of objects across all nodes (MPOL_INTERLEAVE)
3. Subsystems can request to move an object to a specific node.
How to design such functionality best?
4. Tying into the page migration and page defragmentation logic so
that so far unmovable pages that are in the way of creating a
contiguous block of memory will become movable.
This would mean checking for slab pages in the migration logic
and calling slab to see if it can move the page by migrating
all objects.
This is only possible for xarray for now but it would be worthwhile
to extend this to dentries and inodes.
WARNING: multiple messages have this Message-ID
From: Christoph Lameter <cl@linux.com>
To: Matthew Wilcox <willy@infradead.org>
Cc: linux-mm@kvack.org
Cc: linux-kernel@vger.kernel.org
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: akpm@linux-foundation.org
Cc: Mel Gorman <mel@skynet.ie>
Cc: andi@firstfloor.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Dave Chinner <dchinner@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Subject: [RFC 0/7] Slab object migration for xarray V2
Date: Thu, 20 Dec 2018 19:21:55 +0000 [thread overview]
Message-ID: <01000167cd1130c8-c9bebcb9-1f95-4f7c-b24a-90600d56c62f-000000@email.amazonses.com> (raw)
Message-ID: <20181220192155.k5n-hY2kUhcNtKH2JyaS3YlvdNAOh5H_lW8Vh-MXWH0@z> (raw)
V1->V2:
- Works now on top of 4.20-rc7
- A couple of bug fixes
This is a patchset on top of Matthew Wilcox Xarray code and implements
slab object migration for xarray nodes. The migration is integrated into
the defragmetation and shrinking logic of the slab allocator.
Defragmentation will ensure that all xarray slab pages have
less objects available than specified by the slab defrag ratio.
Slab shrinking will create a slab cache with optimal object
density. Only one slab page will have available objects per node.
To test apply this patchset and run a workload that uses lots of radix tree objects
Then go to
/sys/kernel/slab/radix_tree_node
Inspect the number of total objects that the slab can handle
cat total_objects
qmdr:/sys/kernel/slab/radix_tree_node# cat objects
868 N0=448 N1=168 N2=56 N3=196
And the number of slab pages used for those
cat slabs
qmdr:/sys/kernel/slab/radix_tree_node# cat slabs
31 N0=16 N1=6 N2=2 N3=7
Perform a cache shrink operation
echo 1 >shrink
Now see how the slab has changed:
qmdr:/sys/kernel/slab/radix_tree_node# cat slabs
30 N0=15 N1=6 N2=2 N3=7
qmdr:/sys/kernel/slab/radix_tree_node# cat objects
713 N0=349 N1=141 N2=52 N3=171
This is just a barebones approach using a special mode
of the slab migration patchset that does not require refcounts.
If this is acceptable then additional functionality can be added:
1. Migration of objects to a specific node. Not sure how to implement
that. Using another sysfs field?
2. Dispersion of objects across all nodes (MPOL_INTERLEAVE)
3. Subsystems can request to move an object to a specific node.
How to design such functionality best?
4. Tying into the page migration and page defragmentation logic so
that so far unmovable pages that are in the way of creating a
contiguous block of memory will become movable.
This would mean checking for slab pages in the migration logic
and calling slab to see if it can move the page by migrating
all objects.
This is only possible for xarray for now but it would be worthwhile
to extend this to dentries and inodes.
next reply other threads:[~2018-12-20 19:21 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-20 19:21 Christoph Lameter [this message]
2018-12-20 19:21 ` Christoph Lameter
2018-12-20 22:32 ` Randy Dunlap
2018-12-21 1:43 ` [RFC 5/7] Slab defrag core Christopher Lameter
2018-12-21 1:43 ` Christopher Lameter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=01000167cd1130c8-c9bebcb9-1f95-4f7c-b24a-90600d56c62f-000000@email.amazonses.com \
--to=cl@linux.com \
--cc=akpm@linux-foundation.org \
--cc=andi@firstfloor.org \
--cc=dchinner@redhat.com \
--cc=hch@lst.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mel@skynet.ie \
--cc=mhocko@suse.com \
--cc=mike.kravetz@oracle.com \
--cc=penberg@cs.helsinki.fi \
--cc=riel@redhat.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox