linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Dennis Zhou <dennisz@fb.com>
To: Tejun Heo <tj@kernel.org>, Christoph Lameter <cl@linux.com>,
	Josef Bacik <josef@toxicpanda.com>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	kernel-team@fb.com, Dennis Zhou <dennisszhou@gmail.com>
Subject: [PATCH v2 23/23] percpu: update header to contain bitmap allocator explanation.
Date: Mon, 24 Jul 2017 19:02:20 -0400	[thread overview]
Message-ID: <20170724230220.21774-24-dennisz@fb.com> (raw)
In-Reply-To: <20170724230220.21774-1-dennisz@fb.com>

From: "Dennis Zhou (Facebook)" <dennisszhou@gmail.com>

The other patches contain a lot of information, so adding this
information in a separate patch. It adds my copyright and a brief
explanation of how the bitmap allocator works. There is a minor typo as
well in the prior explanation so that is fixed.

Signed-off-by: Dennis Zhou <dennisszhou@gmail.com>
---
 mm/percpu.c | 32 ++++++++++++++++++--------------
 1 file changed, 18 insertions(+), 14 deletions(-)

diff --git a/mm/percpu.c b/mm/percpu.c
index ffa9da7..a4dd0c8 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -4,6 +4,9 @@
  * Copyright (C) 2009		SUSE Linux Products GmbH
  * Copyright (C) 2009		Tejun Heo <tj@kernel.org>
  *
+ * Copyright (C) 2017		Facebook Inc.
+ * Copyright (C) 2017		Dennis Zhou <dennisszhou@gmail.com>
+ *
  * This file is released under the GPLv2 license.
  *
  * The percpu allocator handles both static and dynamic areas.  Percpu
@@ -25,7 +28,7 @@
  *
  * There is special consideration for the first chunk which must handle
  * the static percpu variables in the kernel image as allocation services
- * are not online yet.  In short, the first chunk is structure like so:
+ * are not online yet.  In short, the first chunk is structured like so:
  *
  *                  <Static | [Reserved] | Dynamic>
  *
@@ -34,19 +37,20 @@
  * percpu variables from kernel modules.  Finally, the dynamic section
  * takes care of normal allocations.
  *
- * Allocation state in each chunk is kept using an array of integers
- * on chunk->map.  A positive value in the map represents a free
- * region and negative allocated.  Allocation inside a chunk is done
- * by scanning this map sequentially and serving the first matching
- * entry.  This is mostly copied from the percpu_modalloc() allocator.
- * Chunks can be determined from the address using the index field
- * in the page struct. The index field contains a pointer to the chunk.
- *
- * These chunks are organized into lists according to free_size and
- * tries to allocate from the fullest chunk first. Each chunk maintains
- * a maximum contiguous area size hint which is guaranteed to be equal
- * to or larger than the maximum contiguous area in the chunk. This
- * helps prevent the allocator from iterating over chunks unnecessarily.
+ * The allocator organizes chunks into lists according to free size and
+ * tries to allocate from the fullest chunk first.  Each chunk is managed
+ * by a bitmap with metadata blocks.  The allocation map is updated on
+ * every allocation and free to reflect the current state while the boundary
+ * map is only updated on allocation.  Each metadata block contains
+ * information to help mitigate the need to iterate over large portions
+ * of the bitmap.  The reverse mapping from page to chunk is stored in
+ * the page's index.  Lastly, units are lazily backed and grow in unison.
+ *
+ * There is a unique conversion that goes on here between bytes and bits.
+ * Each bit represents a fragment of size PCPU_MIN_ALLOC_SIZE.  The chunk
+ * tracks the number of pages it is responsible for in nr_pages.  Helper
+ * functions are used to convert from between the bytes, bits, and blocks.
+ * All hints are managed in bits unless explicitly stated.
  *
  * To use this allocator, arch code should do the following:
  *
-- 
2.9.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2017-07-24 23:03 UTC|newest]

Thread overview: 51+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-07-24 23:01 [PATCH v2 00/23] percpu: replace percpu area map allocator with bitmap allocator Dennis Zhou
2017-07-24 23:01 ` [PATCH v2 01/23] percpu: setup_first_chunk enforce dynamic region must exist Dennis Zhou
2017-07-25 18:03   ` Josef Bacik
2017-07-24 23:01 ` [PATCH v2 02/23] percpu: introduce start_offset to pcpu_chunk Dennis Zhou
2017-07-25 18:04   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 03/23] percpu: remove has_reserved from pcpu_chunk Dennis Zhou
2017-07-25 18:05   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 04/23] percpu: setup_first_chunk remove dyn_size and consolidate logic Dennis Zhou
2017-07-25 18:07   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 05/23] percpu: unify allocation of schunk and dchunk Dennis Zhou
2017-07-25 18:10   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 06/23] percpu: end chunk area maps page aligned for the populated bitmap Dennis Zhou
2017-07-25 18:14   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 07/23] percpu: setup_first_chunk rename schunk/dchunk to chunk Dennis Zhou
2017-07-25 18:15   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 08/23] percpu: modify base_addr to be region specific Dennis Zhou
2017-07-25 18:24   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 09/23] percpu: combine percpu address checks Dennis Zhou
2017-07-25 18:25   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 10/23] percpu: change the number of pages marked in the first_chunk pop bitmap Dennis Zhou
2017-07-25 18:27   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 11/23] percpu: introduce nr_empty_pop_pages to help empty page accounting Dennis Zhou
2017-07-25 18:29   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 12/23] percpu: increase minimum percpu allocation size and align first regions Dennis Zhou
2017-07-25 18:33   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 13/23] percpu: generalize bitmap (un)populated iterators Dennis Zhou
2017-07-25 18:35   ` Josef Bacik
2017-07-26 14:08   ` Tejun Heo
2017-07-24 23:02 ` [PATCH v2 14/23] percpu: replace area map allocator with bitmap allocator Dennis Zhou
2017-07-25 17:36   ` [percpu] ec1f2e572e: WARNING:at_lib/list_debug.c:#__list_add_valid kernel test robot
2017-07-25 19:15   ` [PATCH v2 14/23] percpu: replace area map allocator with bitmap allocator Josef Bacik
2017-07-25 20:24   ` [PATCH updated 14/23] percpu: replace area map allocator with bitmap Dennis Zhou
2017-07-24 23:02 ` [PATCH v2 15/23] percpu: introduce bitmap metadata blocks Dennis Zhou
2017-07-25 19:20   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 16/23] percpu: add first_bit to keep track of the first free in the bitmap Dennis Zhou
2017-07-25 19:21   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 17/23] percpu: skip chunks if the alloc does not fit in the contig hint Dennis Zhou
2017-07-25 19:25   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 18/23] percpu: keep track of the best offset for contig hints Dennis Zhou
2017-07-25 19:29   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 19/23] percpu: update alloc path to only scan if contig hints are broken Dennis Zhou
2017-07-25 19:32   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 20/23] percpu: update free path to take advantage of contig hints Dennis Zhou
2017-07-25 19:38   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 21/23] percpu: use metadata blocks to update the chunk contig hint Dennis Zhou
2017-07-25 19:40   ` Josef Bacik
2017-07-24 23:02 ` [PATCH v2 22/23] percpu: update pcpu_find_block_fit to use an iterator Dennis Zhou
2017-07-25 19:47   ` Josef Bacik
2017-07-24 23:02 ` Dennis Zhou [this message]
2017-07-25 19:49   ` [PATCH v2 23/23] percpu: update header to contain bitmap allocator explanation Josef Bacik
2017-07-26 21:42   ` Tejun Heo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170724230220.21774-24-dennisz@fb.com \
    --to=dennisz@fb.com \
    --cc=cl@linux.com \
    --cc=dennisszhou@gmail.com \
    --cc=josef@toxicpanda.com \
    --cc=kernel-team@fb.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox