From: "Uladzislau Rezki (Sony)" <urezki@gmail.com>
To: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>,
Vlastimil Babka <vbabka@suse.cz>
Cc: RCU <rcu@vger.kernel.org>, LKML <linux-kernel@vger.kernel.org>,
Uladzislau Rezki <urezki@gmail.com>,
Oleksiy Avramchenko <oleksiy.avramchenko@sony.com>
Subject: [RFC v1 1/5] rcu/kvfree: Temporary reclaim over call_rcu()
Date: Tue, 10 Dec 2024 17:40:31 +0100 [thread overview]
Message-ID: <20241210164035.3391747-2-urezki@gmail.com> (raw)
In-Reply-To: <20241210164035.3391747-1-urezki@gmail.com>
This is to start a smooth process of moving a main functionality
to the SLAB. Therefore this patch:
- adds a support(temporary) to reclaim freed objects over call_rcu();
- disconnects a main functionality of kvfree_rcu() API by using call_rcu();
- directly reclaims an object for a single-argument variant;
- adds an rcu_barrier() call to the kvfree_rcu_barrier().
Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
---
kernel/rcu/tree.c | 27 +++++++++++++++++++++++----
1 file changed, 23 insertions(+), 4 deletions(-)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index b1f883fcd918..ab24229dfa73 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2559,13 +2559,19 @@ static void rcu_do_batch(struct rcu_data *rdp)
debug_rcu_head_unqueue(rhp);
rcu_lock_acquire(&rcu_callback_map);
- trace_rcu_invoke_callback(rcu_state.name, rhp);
f = rhp->func;
- debug_rcu_head_callback(rhp);
- WRITE_ONCE(rhp->func, (rcu_callback_t)0L);
- f(rhp);
+ /* This is temporary, it will be removed when migration is over. */
+ if (__is_kvfree_rcu_offset((unsigned long) f)) {
+ trace_rcu_invoke_kvfree_callback("", rhp, (unsigned long) f);
+ kvfree((void *) rhp - (unsigned long) f);
+ } else {
+ trace_rcu_invoke_callback(rcu_state.name, rhp);
+ debug_rcu_head_callback(rhp);
+ WRITE_ONCE(rhp->func, (rcu_callback_t)0L);
+ f(rhp);
+ }
rcu_lock_release(&rcu_callback_map);
/*
@@ -3787,6 +3793,16 @@ void kvfree_call_rcu(struct rcu_head *head, void *ptr)
struct kfree_rcu_cpu *krcp;
bool success;
+ if (head) {
+ call_rcu(head, (rcu_callback_t) ((void *) head - ptr));
+ } else {
+ synchronize_rcu();
+ kvfree(ptr);
+ }
+
+ /* Disconnect the rest. */
+ return;
+
/*
* Please note there is a limitation for the head-less
* variant, that is why there is a clear rule for such
@@ -3871,6 +3887,9 @@ void kvfree_rcu_barrier(void)
bool queued;
int i, cpu;
+ /* Temporary. */
+ rcu_barrier();
+
/*
* Firstly we detach objects and queue them over an RCU-batch
* for all CPUs. Finally queued works are flushed for each CPU.
--
2.39.5
next prev parent reply other threads:[~2024-12-10 16:40 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-12-10 16:40 [RFC v1 0/5] Move kvfree_rcu() into SLAB Uladzislau Rezki (Sony)
2024-12-10 16:40 ` Uladzislau Rezki (Sony) [this message]
2024-12-10 16:40 ` [RFC v1 2/5] mm/slab: Copy main data structures of kvfree_rcu() Uladzislau Rezki (Sony)
2024-12-10 16:40 ` [RFC v1 3/5] mm/slab: Copy internal functions " Uladzislau Rezki (Sony)
2024-12-10 16:40 ` [RFC v1 4/5] mm/slab: Copy a function of kvfree_rcu() initialization Uladzislau Rezki (Sony)
2024-12-10 16:40 ` [RFC v1 5/5] mm/slab: Move kvfree_rcu() into SLAB Uladzislau Rezki (Sony)
2024-12-11 16:12 ` [RFC v1 0/5] " Paul E. McKenney
2024-12-12 10:30 ` Vlastimil Babka
2024-12-12 18:04 ` Uladzislau Rezki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20241210164035.3391747-2-urezki@gmail.com \
--to=urezki@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=oleksiy.avramchenko@sony.com \
--cc=rcu@vger.kernel.org \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox