From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC3AFCA0EC7 for ; Thu, 29 Aug 2024 22:25:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 637DC6B0099; Thu, 29 Aug 2024 18:25:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E7FD6B009A; Thu, 29 Aug 2024 18:25:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4D6D56B009B; Thu, 29 Aug 2024 18:25:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3078F6B0099 for ; Thu, 29 Aug 2024 18:25:30 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B160DA0A0C for ; Thu, 29 Aug 2024 22:25:29 +0000 (UTC) X-FDA: 82506715578.05.FA46CDF Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf18.hostedemail.com (Postfix) with ESMTP id 0AA241C0019 for ; Thu, 29 Aug 2024 22:25:27 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=GtuPO+67; spf=pass (imf18.hostedemail.com: domain of dakr@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=dakr@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724970239; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0mwIF5IlN75i2+UdowXojjP+aivWkMiub0xGx5KXxfc=; b=ow1vsijfhe37HUDNk02ChhWtfBtx2AjXmyKMlUjACsozzSPLnpU0eqCL47tBvJXUGBHHWS VDLyHwfotAO1ZuuHkmZX1Taxy5HqGmctxl2DoCo1COcmO+Sk6mEGFrD9BNZMNLv/GtaZZS QbFZnUfa9MPO9qwhOpo5Oo4KnSDp3Fg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724970239; a=rsa-sha256; cv=none; b=tOgDe29blXj1AtvTkKUd4D6plDE90TLPdJ5ndRdvKSw8J5+CdiU0mGCxhtbLZO5w2Khd8q dVj4wfFV2pfy4vIsfqR2ItIQX0HXIsHhBfnjaQmDlU2fJNNMQ1CKr32O/Hsw1TA7Mxxluc eG4ZOO+KAno87DbuCg6/elILP1wL8+c= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=GtuPO+67; spf=pass (imf18.hostedemail.com: domain of dakr@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=dakr@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 4092AA43669; Thu, 29 Aug 2024 22:25:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3B050C4CEC1; Thu, 29 Aug 2024 22:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724970326; bh=9QL5tYH2tIhzGwNe5gZfGSxX1emUw9NxqJ0jPSVxO7E=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=GtuPO+67i2pbCILo9/yvl0O1IcwnF5psrFDSu6i7vpGbGfiFJWK/hCzVSF5NQgusX 1hkosOoEtrakTA6EwwDmKLwXlChRqzlI8G46GPB/pUdGVBJrKomGRlWI4Qthoe9qVg Pj/eegAHDRvoBTyDm3Qzao0TIlxKMYqw1zqB4sPMAFvYUrj9xi3EQbLtOXmAaOJ3vy QpEs2HSTWkYjulZl+OtwtbH3aFAjcqcHnaytrxrbXYKslkDHvLajcVWYBzb9Ib3Ri4 cFE2yDxI9tpvXtaw4jXhwJNaSeBtBvcDNxCKPCd9z9wcWdjDbU7rFfhzBWxZa7AvWc iu3x42vWOlOdw== Date: Fri, 30 Aug 2024 00:25:18 +0200 From: Danilo Krummrich To: Benno Lossin Cc: ojeda@kernel.org, alex.gaynor@gmail.com, wedsonaf@gmail.com, boqun.feng@gmail.com, gary@garyguo.net, bjorn3_gh@protonmail.com, a.hindborg@samsung.com, aliceryhl@google.com, akpm@linux-foundation.org, daniel.almeida@collabora.com, faith.ekstrand@collabora.com, boris.brezillon@collabora.com, lina@asahilina.net, mcanal@igalia.com, zhiw@nvidia.com, cjia@nvidia.com, jhubbard@nvidia.com, airlied@redhat.com, ajanulgu@redhat.com, lyude@redhat.com, linux-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v6 22/26] rust: alloc: implement `Cmalloc` in module allocator_test Message-ID: References: <20240816001216.26575-1-dakr@kernel.org> <20240816001216.26575-23-dakr@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 0AA241C0019 X-Stat-Signature: owkiwwc86hfijjoa8hbhaauqkf8g5k87 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1724970327-465875 X-HE-Meta: U2FsdGVkX1/8MXz7Bp9/x2YSQy8MoTRI/KwtWhBbzjGfIf81lz9BVHxqCpkuSleUdCakI9133IFTOMow/69pnIIsmrlwC6doNq8/GQy11KAVG6SCQU3QxYAcp0JLxB6c20Hkq1/mrlYXFDjCwMJnfKoDNTk939Doodz6+pALqcoNYJdWcH57IIc3i30XCDrpe+CBB8AbXNYd5C8YNfbuHNU0xaUvRTxbggUKlZjPGqcA57OPgzbb8baqD3XL9iatdl4CiPmtYVhvH6EN6WUTcHSV8J6TOlF5hy+YT+hiCcmLa3gaT1KWdXL4nRoF6UAgt6f4FIO3FtNcxPBXEDspQuXutMBqM4q8SVFC7/4eEOaqb5KqlbW+WvZJYRrgGSTHzPPUDNYuw/ZL3ct/H6HRVLr/EPlrMUE5EUP6ohdD2Sgjav6FYSbFx5F2rM4VurEdJNM5tbT9wpx5v5P13AtsLhlTQFbxpZr2c3IwPAqC5Prve5OPkbYtKGykEI6bwvWHlf65e+pFV4fDsWm8VtCnUgwSqMcZuMaNrohBhGcSiSUX4evtmrwxqpumiExEVUBQ+XsVERTMG9ul03eATdDHhLuA0mPwbKspOYxLzCNmSGu2rdegVxb/4nXB5613OgjkMIllpbs9OiDj5gHaDa2bV4e9EGBDI2QYtFfLTcS6vgrnH+wWOzsCewuA2K4Jrp2TFXpb6QdAQ14Qmej5dSwf9jMPcYnClng0XRxB4zr+gQER3kWvUWyEN1ctwu4g5WRUSESnjrlkEP0MwXC83xh6/aMrNzVhmGsGvdgVrE9/VzLGftnA0NM+rk2o4YceeP3h9X/23V4F5DaeAzpwPe+2LUe7sRsP2wPN82vwHtz8GFGPEFi/LG2OhmygPzrbdYOXFJTQZY5Zs9+kd+Qt6Ai12lZwwBzCVhrCxzCr+OXMZBUN00Sg+xkypFRupz061xzT872qDxlxwmwvmzYcIbj jV9k8ZdB jcdw9CASUN5gftBw/plrHvIhSMSIYL3qTx1w9FqMabXN/EcaHhfQCKaU4tfpuQ7fpabX2BKKArogXsauOQ03vJ5C/PgMcWd+oIHHTPjkfkr43pDViET3jzzM79Vff6wZI74bit3L5yvz/R9WphWx5NPkOtE6x61ou9MeDTMgt7L47JqTsqzh83/6S37Q0cdRQ5CTi3IJMPtAO2QvzXbC6RiWpBYdhvBOss0pX05jCvpWQTqnjpPsf8ux3Cr9WsRngV2dlCBdHizlm1A2gINb81DPFAJzoy9PWlhmbCBDPhplWGnMK97hUWRhS8sdCYMU47iubtpCsrqPgE57raOsozcS+mcVjBme+TKUdbBvnWYuNauI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Aug 29, 2024 at 07:14:18PM +0000, Benno Lossin wrote: > On 16.08.24 02:11, Danilo Krummrich wrote: > > So far the kernel's `Box` and `Vec` types can't be used by userspace > > test cases, since all users of those types (e.g. `CString`) use kernel > > allocators for instantiation. > > > > In order to allow userspace test cases to make use of such types as > > well, implement the `Cmalloc` allocator within the allocator_test module > > and type alias all kernel allocators to `Cmalloc`. The `Cmalloc` > > allocator uses libc's realloc() function as allocator backend. > > > > Signed-off-by: Danilo Krummrich > > --- > > rust/kernel/alloc/allocator_test.rs | 178 ++++++++++++++++++++++++++-- > > 1 file changed, 171 insertions(+), 7 deletions(-) > > > > diff --git a/rust/kernel/alloc/allocator_test.rs b/rust/kernel/alloc/allocator_test.rs > > index 1b2642c547ec..7fff308d02dc 100644 > > --- a/rust/kernel/alloc/allocator_test.rs > > +++ b/rust/kernel/alloc/allocator_test.rs > > @@ -2,20 +2,184 @@ > > > > Could add a short paragraph as the module description why this module > exists? Would probably be enough to paste the commit message. Yes, sounds good. > > > #![allow(missing_docs)] > > > > -use super::{AllocError, Allocator, Flags}; > > +use super::{flags::*, AllocError, Allocator, Flags}; > > use core::alloc::Layout; > > +use core::cmp; > > +use core::mem; > > +use core::ptr; > > use core::ptr::NonNull; > > > > -pub struct Kmalloc; > > +pub struct Cmalloc; > > +pub type Kmalloc = Cmalloc; > > pub type Vmalloc = Kmalloc; > > pub type KVmalloc = Kmalloc; > > > > -unsafe impl Allocator for Kmalloc { > > +extern "C" { > > + #[link_name = "aligned_alloc"] > > + fn libc_aligned_alloc(align: usize, size: usize) -> *mut core::ffi::c_void; > > + > > + #[link_name = "free"] > > + fn libc_free(ptr: *mut core::ffi::c_void); > > +} > > + > > +struct CmallocData { > > + // The actual size as requested through `Cmalloc::alloc` or `Cmalloc::realloc`. > > + size: usize, > > + // The offset from the pointer returned to the caller of `Cmalloc::alloc` or `Cmalloc::realloc` > > + // to the actual base address of the allocation. > > + offset: usize, > > +} > > + > > +impl Cmalloc { > > + /// Adjust the size and alignment such that we can additionally store `CmallocData` right > > + /// before the actual data described by `layout`. > > + /// > > + /// Example: > > + /// > > + /// For `CmallocData` assume an alignment of 8 and a size of 16. > > + /// For `layout` assume and alignment of 16 and a size of 64. > > This looks like you want it rendered as bulletpoints (but it won't). Actually, that wasn't my intention, but I'm fine changing that. > > > + /// > > + /// 0 16 32 96 > > + /// |----------------|----------------|------------------------------------------------| > > + /// empty CmallocData data > > Can you put this inside of '```'? Then it will render nicely in markdown > (don't forget to specify the type 'text') Sure. > > > + /// > > + /// For this example the returned `Layout` has an alignment of 32 and a size of 96. > > + fn layout_adjust(layout: Layout) -> Result { > > + let layout = layout.pad_to_align(); > > + > > + // Ensure that `CmallocData` fits into half the alignment. Additionally, this guarantees > > + // that advancing a pointer aligned to `align` by `align / 2` we still satisfy or exceed > > + // the alignment requested through `layout`. > > + let align = cmp::max( > > + layout.align(), > > + mem::size_of::().next_power_of_two(), > > + ) * 2; > > + > > + // Add the additional space required for `CmallocData`. > > + let size = layout.size() + mem::size_of::(); > > + > > + Ok(Layout::from_size_align(size, align) > > + .map_err(|_| AllocError)? > > + .pad_to_align()) > > + } > > + > > + fn alloc_store_data(layout: Layout) -> Result, AllocError> { > > + let requested_size = layout.size(); > > + > > + let layout = Self::layout_adjust(layout)?; > > + let min_align = layout.align() / 2; > > + > > + // SAFETY: Returns either NULL or a pointer to a memory allocation that satisfies or > > + // exceeds the given size and alignment requirements. > > + let raw_ptr = unsafe { libc_aligned_alloc(layout.align(), layout.size()) } as *mut u8; > > + > > + let priv_ptr = NonNull::new(raw_ptr).ok_or(AllocError)?; > > + > > + // SAFETY: Advance the pointer by `min_align`. The adjustments from `Self::layout_adjust` > > + // ensure that after this operation the original size and alignment requirements are still > > + // satisfied or exceeded. > > This SAFETY comment should address why it's OK to call `add`. You > justify something different, namely why the allocation still satisfies > the requirements of `layout`. That is something that this function > should probably guarantee. So, I guess you're arguing that instead I should say that, we're still within the bounds of the same allocated object and don't exceed `isize`? > > > + let ptr = unsafe { priv_ptr.as_ptr().add(min_align) }; > > + > > + // SAFETY: `min_align` is greater than or equal to the size of `CmallocData`, hence we > > + // don't exceed the allocation boundaries. > > + let data_ptr: *mut CmallocData = unsafe { ptr.sub(mem::size_of::()) }.cast(); > > + > > + let data = CmallocData { > > + size: requested_size, > > + offset: min_align, > > + }; > > + > > + // SAFETY: `data_ptr` is properly aligned and within the allocation boundaries reserved for > > + // `CmallocData`. > > + unsafe { data_ptr.write(data) }; > > + > > + NonNull::new(ptr).ok_or(AllocError) > > + } > > + > > + /// # Safety > > + /// > > + /// `ptr` must have been previously allocated with `Self::alloc_store_data`. > > You additionally need that you have shared access to the pointee. > > > + unsafe fn data<'a>(ptr: NonNull) -> &'a CmallocData { > > + // SAFETY: `Self::alloc_store_data` stores the `CmallocData` right before the address > > + // returned to callers of `Self::alloc_store_data`. > > + let data_ptr: *mut CmallocData = > > + unsafe { ptr.as_ptr().sub(mem::size_of::()) }.cast(); > > + > > + // SAFETY: The `CmallocData` has been previously stored at this offset with > > + // `Self::alloc_store_data`. > > + unsafe { &*data_ptr } > > + } > > + > > + /// # Safety > > + /// > > + /// This function must not be called more than once for the same allocation. > > + /// > > + /// `ptr` must have been previously allocated with `Self::alloc_store_data`. > > You additionally need that you have exclusive access to the pointee. > > > + unsafe fn free_read_data(ptr: NonNull) { > > + // SAFETY: `ptr` has been created by `Self::alloc_store_data`. > > + let data = unsafe { Self::data(ptr) }; > > + > > + // SAFETY: `ptr` has been created by `Self::alloc_store_data`. > > + let priv_ptr = unsafe { ptr.as_ptr().sub(data.offset) }; > > + > > + // SAFETY: `priv_ptr` has previously been allocatored with this `Allocator`. > > + unsafe { libc_free(priv_ptr.cast()) }; > > + } > > +} > > + > > +unsafe impl Allocator for Cmalloc { > > + fn alloc(layout: Layout, flags: Flags) -> Result, AllocError> { > > + if layout.size() == 0 { > > + return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0)); > > + } > > + > > + let ptr = Self::alloc_store_data(layout)?; > > + > > + if flags.contains(__GFP_ZERO) { > > + // SAFETY: `Self::alloc_store_data` guarantees that `ptr` points to memory of at least > > + // `layout.size()` bytes. > > + unsafe { ptr.as_ptr().write_bytes(0, layout.size()) }; > > + } > > This makes me wonder, what other flags should we handle for this > allocator? I don't think there are any other flags that we can handle. The only other one that'd make sense is __GFP_NOFAIL, but we can't guarantee that. If any specific gfp flags are needed, I think it's simply not a candidate for a userspace test. If we really want to do something here, we could whitelist the flags we ignore, since they do not matter (such as __GFP_NOWARN) and panic() for everything else. But I don't think that's really needed. > > > + > > + Ok(NonNull::slice_from_raw_parts(ptr, layout.size())) > > + } > > + > > unsafe fn realloc( > > - _ptr: Option>, > > - _layout: Layout, > > - _flags: Flags, > > + ptr: Option>, > > + layout: Layout, > > + flags: Flags, > > ) -> Result, AllocError> { > > - panic!(); > > + let src: NonNull = if let Some(src) = ptr { > > + src.cast() > > Why the cast? Probably a copy-paste mistake. > > > + } else { > > + return Self::alloc(layout, flags); > > + }; > > You should be able to write this instead: > > let Some(src) = ptr else { > return Self::alloc(layout, flags); > }; Yes, indeed. > > > + > > + if layout.size() == 0 { > > + // SAFETY: `src` has been created by `Self::alloc_store_data`. > > This is not true, consider: > > let ptr = alloc(size = 0); > free(ptr) > > Alloc will return a dangling pointer due to the first if statement and > then this function will pass it to `free_read_data`, even though it > wasn't created by `alloc_store_data`. > This isn't forbidden by the `Allocator` trait function's safety > requirements. > > > + unsafe { Self::free_read_data(src) }; > > + > > + return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0)); > > + } > > + > > + let dst = Self::alloc(layout, flags)?; > > + > > + // SAFETY: `src` has been created by `Self::alloc_store_data`. > > + let data = unsafe { Self::data(src) }; > > Same issue here, if the allocation passed in is zero size. I think you > have no other choice than to allocate even for zero size requests... > Otherwise how would you know that they are zero-sized. Good catch - gonna fix it. > > --- > Cheers, > Benno > > > + > > + // SAFETY: `src` has previously been allocated with this `Allocator`; `dst` has just been > > + // newly allocated. Copy up to the smaller of both sizes. > > + unsafe { > > + ptr::copy_nonoverlapping( > > + src.as_ptr(), > > + dst.as_ptr().cast(), > > + cmp::min(layout.size(), data.size), > > + ) > > + }; > > + > > + // SAFETY: `src` has been created by `Self::alloc_store_data`. > > + unsafe { Self::free_read_data(src) }; > > + > > + Ok(dst) > > } > > } > > -- > > 2.46.0 > > >