From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B7D2C52D6D for ; Fri, 2 Aug 2024 12:02:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9599C6B007B; Fri, 2 Aug 2024 08:02:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 908B56B0083; Fri, 2 Aug 2024 08:02:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D0006B0085; Fri, 2 Aug 2024 08:02:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5F3386B007B for ; Fri, 2 Aug 2024 08:02:44 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id BAD94140E96 for ; Fri, 2 Aug 2024 12:02:43 +0000 (UTC) X-FDA: 82407168606.12.7026952 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf09.hostedemail.com (Postfix) with ESMTP id E96AF140029 for ; Fri, 2 Aug 2024 12:02:41 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=F2XjONP9; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf09.hostedemail.com: domain of dakr@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=dakr@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722600119; a=rsa-sha256; cv=none; b=UfmS7NhUWjLJdaZrQNNcj4nvIRrmlQ2wvUnVGuP/4l9azgNwgzdFSouct8bhByJUs+vs3g Uo91eOP0CXMXx0VAO+dxy9fn8YnCe6ACkFsNnouONv2nnzdx/GdPZ61TuKwi+4pCyv4AJ7 6JrfljepcuPIFfLrFjyAiAJUiduvHt8= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=F2XjONP9; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf09.hostedemail.com: domain of dakr@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=dakr@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722600119; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4i/tCtFuaDssxoNXPazxztkVk14i+fPmQo1dxdMAtp4=; b=296uB8lWA7PfiWXOYzAKJEfY59SxHC6crsCV71Z2pfdfv5kzDxMulv0N9i9AlUo9GwUs/q idm19FfxnQH2zNAepQKkJ5WJ0iJQCWs9P/IlxCBAZxWSaZo6fZLPvwhs7M7ZLXzUitvBEE 8QaKtg/H20pMS4ah0lxg2N61OXZvB0M= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id CE4C7629F5; Fri, 2 Aug 2024 12:02:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 27B1FC4AF0B; Fri, 2 Aug 2024 12:02:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722600160; bh=Ox56jQpv10zSiCtaYQE/iYK8oMxsBahCCNxSxZdaSvI=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=F2XjONP9dcbpSo70yeYKdJah7jJNPb2awHNNwVYJWJS6nsQssTQRgweFPmk+AGIFv Ff1Om9fxnxrEM9xvDnZQ+MI4hrsHpQnslcjspObtlowtwFT7/XOeSuQTbriUNEF4zY sWfBfXwuC+x1og3wUYPBs3RskHy/ZdbWyxA1ks8aTRUyLBH/WyY1wlP0AqUP8z87z8 crk8Z+hrUW+bYWYEPyDR7uGIdZmwZSlcQBjYun+6/iUH2zRJ8dElNKfu2Zvu0uR09q rRI4Bbbb587Zv11v84YuxM/Ha64SqVA7XZyIxQWo1lVk+LdXIvmB9/1cldMOpuziR2 L6qzJZhl358hQ== Date: Fri, 2 Aug 2024 14:02:32 +0200 From: Danilo Krummrich To: Alice Ryhl Cc: ojeda@kernel.org, alex.gaynor@gmail.com, wedsonaf@gmail.com, boqun.feng@gmail.com, gary@garyguo.net, bjorn3_gh@protonmail.com, benno.lossin@proton.me, a.hindborg@samsung.com, akpm@linux-foundation.org, daniel.almeida@collabora.com, faith.ekstrand@collabora.com, boris.brezillon@collabora.com, lina@asahilina.net, mcanal@igalia.com, zhiw@nvidia.com, acurrid@nvidia.com, cjia@nvidia.com, jhubbard@nvidia.com, airlied@redhat.com, ajanulgu@redhat.com, lyude@redhat.com, linux-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v3 17/25] rust: alloc: implement `collect` for `IntoIter` Message-ID: References: <20240801000641.1882-1-dakr@kernel.org> <20240801000641.1882-18-dakr@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: E96AF140029 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: x6ha6yj59xbbndi4qb95i8i1godhqjpd X-HE-Tag: 1722600161-986507 X-HE-Meta: U2FsdGVkX1+MQ94Yh4zcynXpO87qRUyvSlfXdPPQW+NI2nkjDVany3h1rdnkOzrdXirE5a6X7+oF5xaZG1YNm9fCyqI1dShLOsTxQGPluMSZ7xX+jMb1leheyHa1wdoB2oMhiq0BKBZLppUbMvsJaUVz/x6ivhbrLm/omdJ8MW2PU8fi1O+OmzjBIeuY7wfTlhgA07o+RZsVkHLdmu1jYIKMXQPUp6Lfm3CsOUXIqhfuKOE4dQ+vNdaViq/V4ASbEQHPopC5BYCQWMN90yYRm+jDYZyHA1T7zi6xCzCC/uEsXpV/WalA2afNI3Uz8zHUQY90DqJEgUwRBOoi4IujpwqvFKML0ml6w23N3XJh5MTtBm71Y1S+p8dIHjKYH/gcYqShxLvAoarOeqs7NqDVQiSZqPGdrjEXpMxaXlc9KNyzzjnjxvFV1pv5kGvO2UhcnvjYy/8vehAj6Vic2WLDnNREyrEBnCFPijQV0V64/RHgS2L+P+PZzrMZttGDtqUm3vgbzO/XH+x8FEcJTKtV87oW6JU8+GkRdQcxqFGhM3jgtsp7sltcoynAwSAfO93LV621h2lZ9oqGXPuNNcZ+GYuQvhAdq/Bt+okN+gy/yabpvOJ9AHGrFPS2nrpnaYvfRscYVDq4wSgAZzH5u3z7JxrypraAsMRpD9xF3HB9FTT0L6nSJRA2MFGOxqvcmAEaxmgk4JfLBHvKkM9ZqQQ7JPZDPS8w4HYxKL5JhLoG8i3vOLOcpe64bDPXxqaQ4OHxl4B7d80osaefO31cjD8/PMgR59GVu99rr7j0SlfBSslqCIJTz77AlzM3HSgCbBT0r32A4mGA16YH0S71WLDjG400/TCDJCNeydPZtUn3ghQEzX04x8tq4ObdEI4hJ+6J9wF4Z4gIyY63Vw1s6qLdLdnDnRxjULv2Asjx14DuVlEsr8sVK5IsJ0i7J+e6i5htYXekCYCTv3AjXXR1sHd VGjZdt7h JomHzeNW3Th19vVGBZApOqdrXkx+4/+CarhYAdr5oU9ZQBdPXseYjRwPBDm5hviIIWItaynyHVfod1O5/qdnQqgNwZABf6F9sAeWl5T8rO+rEM12gAtLOskWCqFtSaQK7znfHZgjwWHkxfIXX/9SWSRR4PkWL8XAiQdfdK/Qu7aqDQ+Ucra+feUIKCAbYgC1kpjQJ7o0IIvTvZC83WQFuGe1zDktMNrl9MQw0N4jo2pD0B6iQHMNQJP/910jSuqOOgDGIjB2VNg0obnmkYllGx5bLSHoLOpFM1Go2H0vMGR5HaDcqrO8eMfpppdmK3ZERNEJ2y9hxXM+4c626NieEXk5HE5yt/igIdCrvffoHZcBVgXvV+nSiV0XRlO57Ld1asBf3niCpDJ98Jh/4xM8LZVLNlg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Aug 02, 2024 at 09:08:48AM +0200, Alice Ryhl wrote: > On Thu, Aug 1, 2024 at 5:37 PM Danilo Krummrich wrote: > > > > On Thu, Aug 01, 2024 at 05:10:22PM +0200, Alice Ryhl wrote: > > > On Thu, Aug 1, 2024 at 2:08 AM Danilo Krummrich wrote: > > > > > > > > Currently, we can't implement `FromIterator`. There are a couple of > > > > issues with this trait in the kernel, namely: > > > > > > > > - Rust's specialization feature is unstable. This prevents us to > > > > optimze for the special case where `I::IntoIter` equals `Vec`'s > > > > `IntoIter` type. > > > > - We also can't use `I::IntoIter`'s type ID either to work around this, > > > > since `FromIterator` doesn't require this type to be `'static`. > > > > - `FromIterator::from_iter` does return `Self` instead of > > > > `Result`, hence we can't properly handle allocation > > > > failures. > > > > - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle > > > > additional allocation flags. > > > > > > > > Instead, provide `IntoIter::collect`, such that we can at least convert > > > > `IntoIter` into a `Vec` again. > > > > > > > > Signed-off-by: Danilo Krummrich > > > > > > I'm not convinced a collect implementation specific to IntoIter is necessary? > > > > For the reasons above, we can't implement `FromIterator`. At some point we may > > want to implement our own kernel `FromIterator` trait, but that's out of scope > > for this series. > > > > For now, I just want to provide a way to get a `Vec` from `IntoIter` again, > > which without `Vec::collect` would be impossible. > > If you have a need for a collect on this particular kind of iterator, then okay. Even once we have our own `FromIterator` trait, we probably want to keep this specific one besides the generic `collect` for optimization. With the generic one we'd otherwise copy into a new `Vec`. > > > > > + > > > > + // SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be > > > > + // smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves > > > > + // it as it is. > > > > + ptr = match unsafe { A::realloc(Some(buf.cast()), layout, flags) } { > > > > > > Why would you shrink it? You can just keep the capacity. > > > > What if the vector was pretty huge and meanwhile as advanced by a lot? I think > > we don't want to waste those resources. > > > > Ideally the corresponding `Allocator` implements a proper heuristic for when to > > actually shrink. For instance, krealloc() never shrinks, and it's probably not > > worth it. For vrealloc() though we clearly want to shrink properly (i.e. unmap > > and free spare pages) at some point. > > The Rust Vec never shrinks unless explicitly requested. But I guess > it's okay either way. It actually does, see [1]. But Rust's `Vec` does it less efficiently. It either keeps the `Vec` as it is, or creates a whole new one. [1] https://github.com/rust-lang/rust/blob/master/library/alloc/src/vec/spec_from_iter.rs#L39 > > Alice >