From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C32F6CD13CF for ; Tue, 3 Sep 2024 11:48:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 591EB8D0162; Tue, 3 Sep 2024 07:48:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5407A8D015F; Tue, 3 Sep 2024 07:48:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E11D8D0162; Tue, 3 Sep 2024 07:48:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1A0F18D015F for ; Tue, 3 Sep 2024 07:48:19 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id AE56EA8ECB for ; Tue, 3 Sep 2024 11:48:18 +0000 (UTC) X-FDA: 82523253876.29.4804BF8 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf07.hostedemail.com (Postfix) with ESMTP id 1578F40016 for ; Tue, 3 Sep 2024 11:48:16 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=giQfQLrz; spf=pass (imf07.hostedemail.com: domain of dakr@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=dakr@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725364002; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nAX6fwCWvLMYcE9qIC3UGyxUMiR2JtO1h3V1yr0swag=; b=JnlqX0G7fpZMkcR/8hcLGWroN42v2HMVdnrM1n7KAJXGnooTjv78lv0mACty2dllO3tfBV kM0oZ5i1eBkSGo3tUWBBHm1cUIxni+pOAM4UjK1YZ3jIB8YL4myqan56IZZ/CIZrHZVwAy PY0H9aA86xQFyrT8RgYgva/93NgeaPQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725364002; a=rsa-sha256; cv=none; b=JEVgKQ/JKDHbKSHtgGvC400A3P6x96GTUFRchA9XWtVAyuy6YmfFARZS3QouRazv8a8C+K DJtdj+BqKy9T173OX4DZXLS70NB4X40kn5EDqX/lyWwO6M+jvhF4/xFef3MTU1m7dTTIcD aUtj1KYl9wa41sylhORkVBUoU3/Zdiw= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=giQfQLrz; spf=pass (imf07.hostedemail.com: domain of dakr@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=dakr@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 23605A43307; Tue, 3 Sep 2024 11:48:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 44806C4CEC4; Tue, 3 Sep 2024 11:48:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725364096; bh=hO+C6A+eNj4TLJv4y9Ur2zuJIyl6632D7KN1QOitnwM=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=giQfQLrzY36BKYRDcJhzPoonO1XaZH7XMyEOZQCSMUhI7M5tJhJcaZikNwl4grBHf VX9XzP0K16WgKmhqZau2oAj//nc0Qf5KA73BbtZtoAP7ASbO3U2lHScDOaEdAzIs/S 7CWYIObPpKVOvv4SfugJXrirPWtQ21D3t/cc+ttnzOJEROnFRSpEuydTXAJcPiI7kb Ls3FYy4tVKWeoggy3QYThENIM2PwVS8MC/mW6mDS94DDqbzA1m4cNQzCL7b+jAAbcE q3HD+CT5xlChoD36S7MGox4Rvc9X+N0jdSj22wDkxxhiYcQgNXBuzxyKi8sFz71STk J7l9sdehKqyxQ== Date: Tue, 3 Sep 2024 13:48:08 +0200 From: Danilo Krummrich To: Benno Lossin Cc: ojeda@kernel.org, alex.gaynor@gmail.com, wedsonaf@gmail.com, boqun.feng@gmail.com, gary@garyguo.net, bjorn3_gh@protonmail.com, a.hindborg@samsung.com, aliceryhl@google.com, akpm@linux-foundation.org, daniel.almeida@collabora.com, faith.ekstrand@collabora.com, boris.brezillon@collabora.com, lina@asahilina.net, mcanal@igalia.com, zhiw@nvidia.com, cjia@nvidia.com, jhubbard@nvidia.com, airlied@redhat.com, ajanulgu@redhat.com, lyude@redhat.com, linux-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v6 04/26] rust: alloc: implement `Allocator` for `Kmalloc` Message-ID: References: <20240816001216.26575-1-dakr@kernel.org> <20240816001216.26575-5-dakr@kernel.org> <2dd02834-b2b6-4ff6-9e29-43c9d77b69e2@proton.me> <962b7014-4f8b-4abe-8774-636b612a051c@proton.me> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <962b7014-4f8b-4abe-8774-636b612a051c@proton.me> X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1578F40016 X-Stat-Signature: z3u89mt57nqanf3gadpseoh967mudcp6 X-HE-Tag: 1725364096-391322 X-HE-Meta: U2FsdGVkX18hf5v1MTcu0jHH4eHW9gLt1s1pEmJoy/qOgocFBLo1gBRqHZdTuIuSZIgAZp3jNepL06h1EpuAGio2mQWnmbZ5HvLe8SAJEWm3XvfqutiM/rIJmMkHqb5Co9SN/kIK4h/qUwdNq9PUuB3zOk+/Jb4Vg5vfYs6DPn4UahufdqPx2JJJ7OgKw1qvKP/LBUJbWyeRCiichiYe0+0GX8nvSI7hSumkzKvdMPFiDIlp/IBl0tKEykxNrlWuKJq64RWiNQq38uWjmYpLZDnno8dR7Yr5xJ/lFybr/r1kkZ6pXTvMlwUzXowjpVkHUpSqRcgp4gMg+FvSHWdhBAyudGW+KcZLn3of5Pd0dRIEQNpMCJYtMqnUDywNEcVs8yE+GSe3zBBt88fosRKSKG061Sp3HtfTuPvEWMrEwB67zzaD5T/1jzaUmSLpu9dfZ1qqt0sO8OmkGgRr40MSOoiPLvgVTKOnE8rDU8IrSA8zKVkXxH9Y6tYNSzD56FpQ5nvqg7A7YWJ9mje08udWvhxIGSrhgL4rp/fiuQA6Fub8jFe3jvVaRu/hBl6hJCxBSGuyH/ObSqhkdJDagdsk8kxrZDlrVh2KlhtmEDCIR1oyXU5Y97pHdYg23PbUu/QqwXwbgCZKv9Ta+6tFZgGmMkMGox8OQia9XuEEMa+m812Rmv00G3hTPYJh7ioVRXoxhTeRWXTGD8RntweRIne5XUCNugbTJVi3tKSa2gIIc4jQ39W+T48SAy95u3DXEZ3+2d26UUy02fo/VlbP3JCw1Dj5eDIRC25FwBiwhDrm1+NXA7ARksfwimN43+83GpRs8GChzvN+XlWmIm0qqaMDrq9rrJGodpnRBnIM3XzDa0NYCowJENVt27Frpa34KpjlbF1NUyCoi7AA5vKV787X1qX1/yIte284ISm581CvOEX9Qb6VkXD8XL4EieFPbI7Hs3yKrIQHy0rMNUSEwDu 39qQENtW tyxIThU4/OmPBZ7iJiQIIhyqNE5mcL7zELdOQlxLDbGyMoB7IBVn+qmDrGNMKTqAw34tUIdsAo3KDSY0LOwnquYq6zrLRfxtgKalkaGd0O1dwDJ+U6FacLpz/saz/K5u3y1aU4tQDl5jZJvoosbI0JQ9TkjYwExG7KuBppl4e9VXWYOlDvqiKDQSO1QXl2RsBsm13N5qEFXJPyGNjY8CZ6umVDlwc70zZ/0tc2WiyZKS7JruJABL0JKNv/qyJ9iJJGNQPftFu5/ZDB/HYwdOMuDTEl+hPn98+rDAxuOsb+hcQn6xN9f8ZLwwBFLXOEBoLxpCO5rxXr0qh7fo+Cgk4zQQOrtUoxqwllEXn0KpDZODfdvdHjjWYHkyUtg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Aug 30, 2024 at 02:45:35PM +0000, Benno Lossin wrote: > On 30.08.24 00:04, Danilo Krummrich wrote: > > On Thu, Aug 29, 2024 at 06:32:42PM +0000, Benno Lossin wrote: > >> On 16.08.24 02:10, Danilo Krummrich wrote: > >>> Implement `Allocator` for `Kmalloc`, the kernel's default allocator, > >>> typically used for objects smaller than page size. > >>> > >>> All memory allocations made with `Kmalloc` end up in `krealloc()`. > >>> > >>> It serves as allocator for the subsequently introduced types `KBox` and > >>> `KVec`. > >>> > >>> Signed-off-by: Danilo Krummrich > >>> --- > >>> rust/kernel/alloc.rs | 2 +- > >>> rust/kernel/alloc/allocator.rs | 72 +++++++++++++++++++++++++++++++++- > >>> 2 files changed, 72 insertions(+), 2 deletions(-) > >>> > >>> diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs > >>> index 9932f21b0539..477dbe3c5a2f 100644 > >>> --- a/rust/kernel/alloc.rs > >>> +++ b/rust/kernel/alloc.rs > >>> @@ -4,7 +4,7 @@ > >>> > >>> #[cfg(not(test))] > >>> #[cfg(not(testlib))] > >>> -mod allocator; > >>> +pub mod allocator; > >>> pub mod box_ext; > >>> pub mod vec_ext; > >>> > >>> diff --git a/rust/kernel/alloc/allocator.rs b/rust/kernel/alloc/allocator.rs > >>> index e32182f91167..78e7d5488843 100644 > >>> --- a/rust/kernel/alloc/allocator.rs > >>> +++ b/rust/kernel/alloc/allocator.rs > >>> @@ -1,12 +1,28 @@ > >>> // SPDX-License-Identifier: GPL-2.0 > >>> > >>> //! Allocator support. > >>> +//! > >>> +//! Documentation for the kernel's memory allocators can found in the "Memory Allocation Guide" > >>> +//! linked below. For instance, this includes the concept of "get free page" (GFP) flags and the > >>> +//! typical application of the different kernel allocators. > >>> +//! > >>> +//! Reference: > >> > >> Thanks, this nice. > >> > >>> > >>> use super::{flags::*, Flags}; > >>> use core::alloc::{GlobalAlloc, Layout}; > >>> use core::ptr; > >>> +use core::ptr::NonNull; > >>> > >>> -struct Kmalloc; > >>> +use crate::alloc::{AllocError, Allocator}; > >>> +use crate::bindings; > >>> + > >>> +/// The contiguous kernel allocator. > >>> +/// > >>> +/// `Kmalloc` is typically used for physically contiguous allocations up to page size, but also > >>> +/// supports larger allocations up to `bindings::KMALLOC_MAX_SIZE`, which is hardware specific. > >> > >> Does putting a link here work? (I guess we don't yet export the bindings > >> documentation, so it will probably fail... When we decide to enable it, > >> we should create an issue to add missing links) > >> > >>> +/// > >>> +/// For more details see [self]. > >>> +pub struct Kmalloc; > >>> > >>> /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. > >>> fn aligned_size(new_layout: Layout) -> usize { > >>> @@ -36,6 +52,60 @@ pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F > >>> unsafe { bindings::krealloc(ptr as *const core::ffi::c_void, size, flags.0) as *mut u8 } > >>> } > >>> > >>> +/// # Invariants > >>> +/// > >>> +/// One of the following `krealloc`, `vrealloc`, `kvrealloc`. > >>> +struct ReallocFunc( > >>> + unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void, > >>> +); > >>> + > >>> +impl ReallocFunc { > >>> + // INVARIANT: `krealloc` satisfies the type invariants. > >>> + const KREALLOC: Self = Self(bindings::krealloc); > >>> + > >>> + /// # Safety > >>> + /// > >>> + /// This method has the same safety requirements as [`Allocator::realloc`]. > >>> + unsafe fn call( > >>> + &self, > >>> + ptr: Option>, > >>> + layout: Layout, > >>> + flags: Flags, > >>> + ) -> Result, AllocError> { > >>> + let size = aligned_size(layout); > >>> + let ptr = match ptr { > >>> + Some(ptr) => ptr.as_ptr(), > >>> + None => ptr::null(), > >>> + }; > >>> + > >>> + // SAFETY: `ptr` is either NULL or valid by the safety requirements of this function. > >> > >> You need some justification as to why calling the three allowed > >> functions here. > > > > What kind of justification do I need? Can you please share some more details on > > what you think is missing here? > > So, you are calling a function pointer to an `unsafe` function. This > means that through some invariant you have to know what the safety > requirements are (otherwise how can you guarantee that this is OK?). You > have the invariant that the pointer points at one of the three functions > mentioned above. What are the safety requirements of those functions? I > would assume that the only one is that `ptr` is valid. So you can use: > > // SAFETY: > // - `self.0` is one of `krealloc`, `vrealloc`, `kvrealloc` and thus only requires that `ptr` is > // NULL or valid. I'm fine adding it, but I'd like to understand why you think it's required in the safety comment here? Isn't this implicit by being the type invariant? > // - `ptr` is either NULL or valid by the safety requirements of this function. This is the part I already have. > > >>> + let raw_ptr = unsafe { > >>> + // If `size == 0` and `ptr != NULL` the memory behind the pointer is freed. > >>> + self.0(ptr.cast(), size, flags.0).cast() > >>> + }; > >>> + > >>> + let ptr = if size == 0 { > >>> + NonNull::dangling() > >>> + } else { > >>> + NonNull::new(raw_ptr).ok_or(AllocError)? > >>> + }; > >>> + > >>> + Ok(NonNull::slice_from_raw_parts(ptr, size)) > >>> + } > >>> +} > >>> + > >>> +unsafe impl Allocator for Kmalloc { > >> > >> Missing SAFETY comment. > > > > Yeah, I think we came across this in an earlier version of the series. I asked > > you about the content and usefulness of a comment here, since I'd just end up > > re-iterating what the `Allocator` trait documentation says. > > > > IIRC, you replied that you want to think of something that'd make sense to add > > here. > > Oh yeah, sorry I forgot about that. > > > What do you think should be written here? > > I think the best way to do it, would be to push this question down into > `ReallocFunc::call`. So we would put this on the trait: > > // SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that > // - memory remains valid until it is explicitly freed, > // - passing a pointer to a vaild memory allocation is OK, > // - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same. So, we'd also need the same for: - `unsafe impl Allocator for Vmalloc` - `unsafe impl Allocator for KVmalloc` > > We then need to put this on `ReallocFunc::call`: > > /// # Guarantees > /// > /// This method has the same guarantees as `Allocator::realloc`. Additionally > /// - it accepts any pointer to a valid memory allocation allocated by this function. You propose this, since for `Allocator::realloc` memory allocated with `Allocator::alloc` would be fine too I guess. But if e.g. `Kmalloc` wouldn't use the default `Allocator::alloc`, this would be valid too. We could instead write something like: "it accepts any pointer to a valid memory allocation allocated with the same kernel allocator." > /// - memory allocated by this function remains valid until it is passed to this function. Same here, `Kmalloc` could implement its own `Allocator::free`. Maybe just "...until it is explicitly freed.". Anyway, I'm fine with both, since non of the kernel allocators uses anything else than `ReallocFunc::call` to allocate and free memory. > > Finally, we need a `GUARANTEE` comment (just above the return [^1] > value) that establishes these guarantees: > > // GUARANTEE: Since we called `self.0` with `size` above and by the type invariants of `Self`, > // `self.0` is one of `krealloc`, `vrealloc`, `kvrealloc`. Those functions provide the guarantees of > // this function. > > I am not really happy with the last sentence, but I also don't think > that there is value in listing out all the guarantees, only to then say > "all of this is guaranteed by us calling one of these three functions. > > > [^1]: I am not sure that there is the right place. If you have any > suggestions, feel free to share them. Either way, I'm fine with this proposal. > > > >>> + #[inline] > >>> + unsafe fn realloc( > >>> + ptr: Option>, > >>> + layout: Layout, > >>> + flags: Flags, > >>> + ) -> Result, AllocError> { > >>> + // SAFETY: `ReallocFunc::call` has the same safety requirements as `Allocator::realloc`. > >>> + unsafe { ReallocFunc::KREALLOC.call(ptr, layout, flags) } > >>> + } > >>> +} > > Oh one more thing, I know that you already have a lot of patches in this > series, but could you split this one into two? So the first one should > introduce `ReallocFunc` and the second one add the impl for `Kmalloc`? > I managed to confuse me twice because of that :) Generally, I'm fine with that, but I'm not sure if I can avoid an intermediate compiler warning about unused code doing that. > > --- > Cheers, > Benno > > >>> + > >>> unsafe impl GlobalAlloc for Kmalloc { > >>> unsafe fn alloc(&self, layout: Layout) -> *mut u8 { > >>> // SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety > >>> -- > >>> 2.46.0 > >>> > >> >