From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 565E9C3DA4A for ; Thu, 15 Aug 2024 00:16:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AFBA96B007B; Wed, 14 Aug 2024 20:16:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A85086B0082; Wed, 14 Aug 2024 20:16:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9259D6B0083; Wed, 14 Aug 2024 20:16:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 70E376B007B for ; Wed, 14 Aug 2024 20:16:25 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 18BB1121275 for ; Thu, 15 Aug 2024 00:16:25 +0000 (UTC) X-FDA: 82452563130.23.EAA6D62 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id 4E44F4000D for ; Thu, 15 Aug 2024 00:16:23 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=uHyTNH2Q; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of dakr@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=dakr@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723680902; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=H9+XRAn5i+C39ZSRj48zXPIZeKV7Q/fG87OxPG/y/Uk=; b=EKORHFkGUY+bi9C5J8bDHnfQh4TjzJTT9o8h6w+kWUz7GiivkekyiMMK8kxXD9VoTLqM9X yUUvk83+ZdB5Yy5JDbdNn08rJKUBI555GLOknoUfzWl/iHwFmEvGZ3Wkfel+casAcmWPyj UY/+Q1ZwjQAS/SYsCfldfVXQ1X8BAbA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723680902; a=rsa-sha256; cv=none; b=Ri5g9oTIS/WVfM8kZSsoVMnkdbNSobeZsEseSlsTqSBq0cjqfBQjWIr0kGcEYnrZoqiEhB ShOXKT80+kJkCO41gSNsgZD2gNxTjjHGYpdnMlE0StezP4Nwl5M+8dv8+eDcpUcxzMXbmG alEYYtZwsBPBheqsSh+9ymCnaV7FGbM= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=uHyTNH2Q; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of dakr@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=dakr@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 4D72660EA2; Thu, 15 Aug 2024 00:16:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4C3F8C116B1; Thu, 15 Aug 2024 00:16:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723680982; bh=8n714ePaEc1LnJejZ6lCUckFCOLvDCkdBz92Ksi4mCo=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=uHyTNH2QfiCQCfU8sclHYwXUoDgaB3GJCWBMNhzTG1hDk3Zt+z0BYOqDZoTBc72Wp a6mJZ0QaHN/kaKkbqtqwodHhs0p5LKL+Z85mD3Z9YEopckDKlMW1XexyTV+y7rdCFk JKlNa8bOVwGu82TXzNZG8lccY3aYprCupiPwBgxB6tX+TjYqjCqetK2u2DkAJ8GBKX OvmQbx/WDyP0XUIMC5AoKNpCxWSjTRtt9dEalFnIr20qn96wZXQ88zNRLC3s3atFKW KZ3/tyNCfExKeENCFoYeRgC6wHU5+7ao5ZiH9/3+Rz9ALTTGR4FaGsS7si4an5bURQ aBSj3fCJDheFA== Date: Thu, 15 Aug 2024 02:16:14 +0200 From: Danilo Krummrich To: Benno Lossin Cc: ojeda@kernel.org, alex.gaynor@gmail.com, wedsonaf@gmail.com, boqun.feng@gmail.com, gary@garyguo.net, bjorn3_gh@protonmail.com, a.hindborg@samsung.com, aliceryhl@google.com, akpm@linux-foundation.org, daniel.almeida@collabora.com, faith.ekstrand@collabora.com, boris.brezillon@collabora.com, lina@asahilina.net, mcanal@igalia.com, zhiw@nvidia.com, cjia@nvidia.com, jhubbard@nvidia.com, airlied@redhat.com, ajanulgu@redhat.com, lyude@redhat.com, linux-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v5 01/26] rust: alloc: add `Allocator` trait Message-ID: References: <20240812182355.11641-1-dakr@kernel.org> <20240812182355.11641-2-dakr@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 4E44F4000D X-Stat-Signature: 8knycty7r47o5qo7x81hafjfety3yxou X-Rspam-User: X-HE-Tag: 1723680983-147473 X-HE-Meta: U2FsdGVkX19l2obhzyFndoxo+Y02R/0jABbqTQOAGLp2zqO98Ke2/bGx67e0o1PQcdlsX9pOfRLmxgMgzUE7Emg00YTK5lhXV9isi6d8kM+toU5hYJXLuOaTRIUDMVPKVyvvECbpnkLbVqstQGbFyOFAvllyoB/ycVmH/eT4/sYS56a63zvLl7BkcCOmxxOhS1NVPvR+06RiQSf5VoC43QN4mCs8Z/05x02bOEOMrLp3z84VkS1mRU3eAHxBypkx0fFdXeKgAVok+jF1bfxdJ5lz/omlya3StPiuCYCVnZbwjpUkNUlSNvHBsXbeUEzaEL89tXVDDqm/dhR56WqSCXOBMP/0pYRB44uPS7kIvnJFFf1lro8+rUJvXlsxLqVYN5T6OmclOMenffJvGEQLfzODUPIXviX2aur6SF9N0h5EswmLvjHW8bS9Cg8a+KGbBbG1geXxuQE6e5OkQPObR+elU5BvyU1CNPWhIAQlTKPorPe7rSOADWlUK7uYD3Lj6c2kPB22/YwcJOT1Kt6GM2Hmg5hf3tgD6tGkp7vhmEzXHS3d7+b2mPexDtJR0BnC6DmaZW0hqQS8qvC+dU/1/+bX8RRWJ2OwpVDOCLPt4YBLAj7Fu0zy+gp/Sw7zWkCpuI7xUrbl5Qagu5L2VGpGHh/FPOexmBv+j5eYcx2kcamKZEs/7abiPahU8MlRNrAQtlkrHnhzvlOUBqDI+mBFWcDNwShTJXOVd6G56xwsr0dgqFIHjC4naqCTF84zmCS7zfRHPe0nvnNodS951eMxuNENYHSr+4kt5ybzmQO2RFKlO+rlLvHniWqZKTUqCYyQLZsra8Sho6vSv8bfLOHVE2WYJ9Xx9+IQfcg3EtgzO8cqU7ncQ0DJy+IoIFKQaLXfaBNdvUqtj1rt5EnDJBU8Uvu5lJDxc/sUcjLAgSGVlD6TOjzysanmPGuhkFU3z9eCfzAiFen3M/rit5bzqPS XfLrCBJK 7hTy2s6laTe/JkRj7oQcV0rQIPm83NXJ3ICbkqe98dqoCQ/WI5MMFxytzhZzh4Ni0/k1JpOq20KUCx4I6GGgYWlSdkEwTqDQaDTGGOza6w8sC/DhrLNhsoejlHvnKk/UPFSLPrcYVcjuooGkGQmxV58YIcts0sdrmQyv+N0z7lwiMi/hdCaCVxtu/CgV6ZFbC0sKMZ7tLLLrYIf8wuwkSlpWUPkHkMZBM8M9oNAMQb4SnHBNGNqlJ1I4xr5rMNM3WQSD963ibZ6zWLte+uDReDgX44TE/Cnj/NxtzTVqKeKXvcgj034dRU0I50UYa0W9LCS6vOxghliVj7VF+4VRAhQImFxIi/P4kh3GZcgsBax6owUN5jMeTjlSLzJ+UmZbS5l4zZJwvXy5D0hwlS06RWTprr6GMRWvDwb2pjSZoJnMvtMK4msOJztTIRw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Aug 14, 2024 at 04:13:06PM +0000, Benno Lossin wrote: > On 12.08.24 20:22, Danilo Krummrich wrote: > > Add a kernel specific `Allocator` trait, that in contrast to the one in > > Rust's core library doesn't require unstable features and supports GFP > > flags. > > > > Subsequent patches add the following trait implementors: `Kmalloc`, > > `Vmalloc` and `KVmalloc`. > > > > Reviewed-by: Alice Ryhl > > Signed-off-by: Danilo Krummrich > > --- > > rust/kernel/alloc.rs | 81 ++++++++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 81 insertions(+) > > > > diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs > > index 1966bd407017..194745498a75 100644 > > --- a/rust/kernel/alloc.rs > > +++ b/rust/kernel/alloc.rs > > @@ -11,6 +11,7 @@ > > /// Indicates an allocation error. > > #[derive(Copy, Clone, PartialEq, Eq, Debug)] > > pub struct AllocError; > > +use core::{alloc::Layout, ptr::NonNull}; > > > > /// Flags to be used when allocating memory. > > /// > > @@ -86,3 +87,83 @@ pub mod flags { > > /// small allocations. > > pub const GFP_NOWAIT: Flags = Flags(bindings::GFP_NOWAIT); > > } > > + > > +/// The kernel's [`Allocator`] trait. > > +/// > > +/// An implementation of [`Allocator`] can allocate, re-allocate and free memory buffer described > > +/// via [`Layout`]. > > +/// > > +/// [`Allocator`] is designed to be implemented as a ZST; [`Allocator`] functions do not operate on > > +/// an object instance. > > +/// > > +/// In order to be able to support `#[derive(SmartPointer)]` later on, we need to avoid a design > > +/// that requires an `Allocator` to be instantiated, hence its functions must not contain any kind > > +/// of `self` parameter. > > +/// > > +/// # Safety > > +/// > > +/// Memory returned from an allocator must point to a valid memory buffer and remain valid until > > +/// it is explicitly freed. > > I wouldn't say that an allocator "returns memory", and in general I > don't think the structure of the safety comment here is nice, how about > the following: we put "Implementers must ensure that all trait functions > abide by the guarantees documented in the `# Guarantees` sections."... Sounds reasonable to me. Additionally, I'd still keep the part below, that says that any pointer to a memory allocation must bbe valid to be passed to any other [`Allocator`] function of the same type. > > > +/// > > +/// Any pointer to a memory buffer which is currently allocated must be valid to be passed to any > > +/// other [`Allocator`] function of the same type. > > +/// > > +/// If `realloc` is called with: > > +/// - a size of zero, the given memory allocation, if any, must be freed > > +/// - `None`, a new memory allocation must be created > > +pub unsafe trait Allocator { > > + /// Allocate memory based on `layout` and `flags`. > > + /// > > + /// On success, returns a buffer represented as `NonNull<[u8]>` that satisfies the layout > > + /// constraints (i.e. minimum size and alignment as specified by `layout`). > > + /// > > + /// This function is equivalent to `realloc` when called with `None`. > > ... Then we can add this here: > > /// # Guarantees > /// > /// When the return value is `Ok(ptr)`, then `ptr` is > /// - valid for writes (and reads after the memory has been initialized) for `layout.size()` bytes, > /// until it is passed to [`Allocator::free`] or [`Allocator::realloc`], > /// - aligned to `layout.align()`, > /// - is valid for reads, if `flags.contains(flags::__GFP_ZERO)`, > > Do we need to handle other flags? The whole flags thing is a bit difficult to represent here properly. Theoretically, we'd need to add that it guarantees that the memory is zeroed for __GFP_ZERO, non-blocking for GFP_NOWAIT, etc. But, I think we shouldn't re-iterate all different behavior for the different flags. Another inconvenience is that not all page flags are honored or make sense for all allocators. This is especially inconvenient for `KVmalloc` where we can't even say if we end up in vrealloc() or krealloc(). kvmalloc() even contains a couple of flag fixups for this reason [2]. I think we should just point to [1], which should document everything already. [1] https://elixir.bootlin.com/linux/v6.10.4/source/include/linux/gfp_types.h [2] https://elixir.bootlin.com/linux/v6.10.4/source/mm/util.c#L612 > Also IIRC the memory given to us by C is considered initialized by Rust > (though it has a non-deterministic value), so we might have an > unconditional "valid for reads". Am I correct? Yes, but as you say, unless allocated with __GFP_ZERO, it contains non-deterministic data. It may even contain old data from previous allocations. > > > > + fn alloc(layout: Layout, flags: Flags) -> Result, AllocError> { > > + // SAFETY: Passing `None` to `realloc` is valid by it's safety requirements and asks for a > > + // new memory allocation. > > + unsafe { Self::realloc(None, layout, flags) } > > + } > > + > > + /// Re-allocate an existing memory allocation to satisfy the requested `layout`. If the > > + /// requested size is zero, `realloc` behaves equivalent to `free`. > > I don't think we want to include the second sentence in the short > description of this function, please add an empty line in between. > > > + /// > > + /// If the requested size is larger than the size of the existing allocation, a successful call > > + /// to `realloc` guarantees that the new or grown buffer has at least `Layout::size` bytes, but > > + /// may also be larger. > > + /// > > + /// If the requested size is smaller than the size of the existing allocation, `realloc` may or > > + /// may not shrink the buffer; this is implementation specific to the allocator. > > + /// > > + /// On allocation failure, the existing buffer, if any, remains valid. > > + /// > > + /// The buffer is represented as `NonNull<[u8]>`. > > + /// > > + /// # Safety > > + /// > > + /// If `ptr = Some(p)`, then `p` must point to an existing and valid memory allocation created > > I don't like the single `=` (I might have written it in haste myself), > how about `==` or we use if-let syntax? > > > + /// by this allocator. The alignment encoded in `layout` must be smaller than or equal to the > > + /// alignment requested in the previous `alloc` or `realloc` call of the same allocation. > > + /// > > + /// Additionally, `ptr` is allowed to be `None`; in this case a new memory allocation is > > + /// created. > > This Safety section does not talk about the case `layout.size() == 0`, > but it should have the same requirement as `free()`. > > Also add a `# Guarantees` section here: > > /// # Guarantees > /// > /// This function has the same guarantees as [`Allocator::alloc`]. When `ptr == Some(p)`, then it > /// additionally has the following: > /// - when `Ok(ret_ptr)` is the return value, then > /// `ret_ptr[0..min(layout.size(), old_size)] == p[0..min(layout.size(), old_size)]`, where > /// `old_size` is the size of the allocation that `p` points at. We could also say "The contents of the memory pointed to by `p` are preserved up to the lesser of the new and old size." But I'm fine with both. > /// - when the return value is `Err(AllocError)`, then `p` is still valid. > > --- > Cheers, > Benno > > > + unsafe fn realloc( > > + ptr: Option>, > > + layout: Layout, > > + flags: Flags, > > + ) -> Result, AllocError>; > > > + > > + /// Free an existing memory allocation. > > + /// > > + /// # Safety > > + /// > > + /// `ptr` must point to an existing and valid memory allocation created by this `Allocator` and > > + /// must not be a dangling pointer. > > + /// > > + /// The memory allocation at `ptr` must never again be read from or written to. > > + unsafe fn free(ptr: NonNull) { > > + // SAFETY: The caller guarantees that `ptr` points at a valid allocation created by this > > + // allocator. We are passing a `Layout` with the smallest possible alignment, so it is > > + // smaller than or equal to the alignment previously used with this allocation. > > + let _ = unsafe { Self::realloc(Some(ptr), Layout::new::<()>(), Flags(0)) }; > > + } > > +} > > -- > > 2.45.2 > > >