* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
[not found] <20230609063118.24852-1-amiculas@cisco.com>
@ 2023-06-09 10:36 ` Christian Brauner
2023-06-09 11:22 ` Ariel Miculas (amiculas)
` (2 more replies)
0 siblings, 3 replies; 28+ messages in thread
From: Christian Brauner @ 2023-06-09 10:36 UTC (permalink / raw)
To: Ariel Miculas, linux-fsdevel; +Cc: rust-for-linux, linux-mm
On Fri, Jun 09, 2023 at 09:29:58AM +0300, Ariel Miculas wrote:
> Hi all!
>
> This is a proof of concept driver written for the PuzzleFS
Uhm, the PuzzleFS filesystem isn't actually sent to fsdevel? Yet I see
tons of patches in there that add wrappers to our core fs data
structures. I even see a ton of new files that all fall clearly into
fsdevel territory:
create mode 100644 rust/kernel/cred.rs
create mode 100644 rust/kernel/delay.rs
create mode 100644 rust/kernel/driver.rs
create mode 100644 rust/kernel/file.rs
create mode 100644 rust/kernel/fs.rs
create mode 100644 rust/kernel/fs/param.rs
create mode 100644 rust/kernel/io_buffer.rs
create mode 100644 rust/kernel/iov_iter.rs
create mode 100644 rust/kernel/mm.rs
create mode 100644 rust/kernel/mount.rs
create mode 100644 rust/kernel/pages.rs
There's also quite a lot of new mm/ in there, no?
Any wrappers and code for core fs should be maintained as part of fs.
Rust shouldn't become a way to avoid our reviews once you have a few
wrappers added somewhere.
> next-generation container filesystem [1]. I've included a short abstract
> about puzzlefs further below. This driver is based on the rust-next
> branch, on top of which I've backported the filesystem abstractions from
> Wedson Almeida Filho [2][3] and Miguel Ojeda's third-party crates
> support: proc-macro2, quote, syn, serde and serde_derive [4]. I've added
> the additional third-party crates serde_cbor[5] and hex [6]. Then I've
> adapted the user space puzzlefs code [1] so that the puzzlefs kernel
> module could present the directory hierarchy and implement the basic
> read functionality.
> For some additional context, puzzlefs was started by Tycho Andersen and
> it's the successor of atomfs. This FOSDEM presentation from 2019 [12]
> covers the rationale for a new oci image format and presents a higher
> level overview of our goals with puzzlefs.
> I've split the rest of the cover letter in following sections (using a
> markdown style):
> * Example: it describes a practical example of what was achieved
> * Limitations: it presents the existing limitations of this POC
> * Upstreaming steps: it describes the steps needed for upstreaming this
> driver
> * Setup: it shows how to setup the necessary environment for testing the
> puzzlefs driver
> * Puzzlefs abstract: it provides a short overview of puzzlefs
>
> # Example
> An example is provided below:
>
> ```
> ~ # cat /proc/filesystems | grep puzzlefs
> nodev puzzlefs
> ~ # mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o image_manifest="2d
> 6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b" none /mnt
> ~ # ls -lR /mnt/
> /mnt/:
> total 0
> drwxr-xr-x 2 0 0 0 Jun 8 12:26 dir-1
> drwxr-xr-x 2 0 0 0 Jun 8 12:26 dir-2
> drwxr-xr-x 2 0 0 0 Jun 8 12:26 dir-3
> drwxr-xr-x 2 0 0 0 Jun 8 12:26 dir-4
> -rw-r--r-- 1 0 0 0 Jun 8 12:26 file1
> -rw-r--r-- 1 0 0 0 Jun 8 12:26 file2
>
> /mnt/dir-1:
> total 0
>
> /mnt/dir-2:
> total 0
>
> /mnt/dir-3:
> total 0
>
> /mnt/dir-4:
> total 0
> ~ # cat /mnt/file2
> ana are mere bla bla bla
> ~ # wc /mnt/file1
> 202 202 5454 /mnt/file1
> ```
>
> In this example, /home/puzzlefs_oci is the puzzlefs oci directory:
> ```
> ~ # ls -lR /home/puzzlefs_oci/
> /home/puzzlefs_oci/:
> total 8
> drwxr-xr-x 3 1000 1000 0 Jun 8 14:33 blobs
> -rw-r--r-- 1 1000 1000 266 Jun 8 14:33 index.json
> -rw-r--r-- 1 1000 1000 37 Jun 8 14:33 oci-layout
>
> /home/puzzlefs_oci/blobs:
> total 0
> drwxr-xr-x 2 1000 1000 0 Jun 8 14:33 sha256
>
> /home/puzzlefs_oci/blobs/sha256:
> total 16
> -rw------- 1 1000 1000 89 Jun 8 14:33 2d6602d678140540dc7e96de652a76a8b16eb
> -rw------- 1 1000 1000 925 Jun 8 14:33 4df03518eea406343dbb55046720f6a478881
> -rw------- 1 1000 1000 5479 Jun 8 14:33 d86a87b19bd9a2fec0d31687c1d669cdb59eb
> ```
>
> `2d6602d678140540dc7e96de652a76a8b16eb` is the puzzlefs image manifest
> hash for the first_try tag:
> ```
> $ cat /tmp/oci-simple/index.json | jq .
> {
> "schemaVersion": -1,
> "manifests": [
> {
> "digest": "sha256:2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b",
> "size": 89,
> "media_type": "application/vnd.puzzlefs.image.rootfs.v1",
> "annotations": {
> "org.opencontainers.image.ref.name": "first_try"
> }
> }
> ],
> "annotations": {}
> }
> ```
>
> I will describe how to build a puzzlefs image in the `Setup` section, at
> step 5.
>
> # Limitations
> One limitation is that the puzzlefs driver doesn't implement any lookup
> functionality and instead it inserts every directory entry into the
> dcache during init (see the `DCACHE_BASED` constant). This is similar to
> how the sample `rust_fs` driver works, but the goal is to implement
> proper lookup functions. However, more filesystem abstractions need to
> be implemented before this can be achieved.
>
> # Upstreaming steps
> Before the puzzlefs driver can be upstreamed, the following need to be
> merged:
> * Wedson's filesystem abstractions [3]
> * the necessary third-party crates [4] (with the preliminary discussion
> about whether this is desirable)
>
> # Setup
> My setup is based on Wedson's tutorial [8]. Next, I will describe the
> necessary steps to build an initrd and run a custom kernel under qemu.
>
> 1. Get the linux rust-next branch [9] and apply this patchset
>
> 2. Follow the rust quickstart guide [10]
>
> 3. configure and build the kernel
> ```
> $ make LLVM=1 allnoconfig qemu-busybox-min.config rust.config
> $ make LLVM=1 -j$(nproc)
> ```
>
> 4. setup busybox
> ```
> git clone git://git.busybox.net/busybox
> cd busybox
> make menuconfig # enable 'Build static binary' config
> make
> make install
> ```
> This will create the `_install` directory with the rootfs inside it.
>
> 5. create a home directory in the rootfs and copy a puzzlefs oci
> directory in home/puzzlefs_oci
> To create a puzzlefs oci directory:
> * download this custom puzzlefs repository [11] (it's custom because we
> want to build an image without verity data)
> * run `make release`
> * create a simple filesystem structure with a few directories and files
> (I've created one at ../test-puzzlefs/simple_rootfs)
> * build a puzzlefs oci image at
> `~/work/busybox/_install/home/puzzlefs_oci` (replace this path with
> your busybox path) with the tag `first_try`:
> ```
> $ target/release/puzzlefs build --omit-verity \
> ../test-puzzlefs/simple_rootfs ~/work/busybox/_install/home/puzzlefs_oci \
> first_try
> ```
> * get first_try's image manifest from index.json (inside `puzzlefs_oci`)
> ```
> $ cat index.json | jq . | grep digest
> "digest": "sha256:2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b",
> ```
>
> 6. add the following 'init' script in the busybox rootfs (rootfs path
> defaults to `./_install'):
> ```
> #!/bin/sh
> mount -t devtmpfs none /dev
> mkdir -p /proc
> mount -t proc none /proc
>
> ifconfig lo up
> udhcpc -i eth0
>
> mkdir /mnt
> mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o \
> image_manifest="2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b" \
> none /mnt
>
> setsid sh -c 'exec sh -l </dev/ttyS0 >/dev/ttyS0 2>&1'
> ```
>
> Make sure to replace the `image_manifest` with your own digest. This
> init script will be passed to rdinit in the kernel command line.
>
> 7. generate the initramfs
>
> Assuming busybox is in `~/work/busybox`:
> ```
> cd ~/work/busybox/_install && find . | cpio -H newc -o | gzip > ../ramdisk.img
> ```
> This will generate a compressed ramdisk image in
> `~/work/busybox/ramdisk.img`.
>
> 8. run with qemu (this assumes the linux tree is at '../linux' and busybox
> is at '../busybox'):
> ```
> qemu-system-x86_64 \
> -accel kvm \
> -cpu host \
> -m 4G \
> -initrd ../busybox/ramdisk.img \
> -kernel ../linux/arch/x86/boot/bzImage \
> -nographic \
> -append 'console=ttyS0 nokaslr debug rdinit=/init' \
> -nic user,model=rtl8139 \
> -no-reboot
> ```
>
> 9. Check whether puzzlefs has been successfully mounted
> ```
> ~ # mount | grep puzzlefs
> none on /mnt type puzzlefs (rw,relatime)
> ~ # ls /mnt/
> dir-1 dir-2 dir-3 dir-4 file1 file2
> ```
>
>
> # Puzzlefs abstract
> Puzzlefs [1] is a container filesystem designed to address the
> limitations of the existing OCI format. The main goals of the project
> are reduced duplication, reproducible image builds, direct mounting
> support and memory safety guarantees, some inspired by the OCIv2 design
> document [7].
>
> Reduced duplication is achieved using the content defined chunking
> algorithm FastCDC. This implementation allows chunks to be shared among
> layers. Building a new layer starting from an existing one allows
> reusing most of the chunks.
>
> Another goal of the project is reproducible image builds, which is
> achieved by defining a canonical representation of the image format.
>
> Direct mounting support is a key feature of puzzlefs and, together with
> fs-verity, it provides data integrity. Currently, puzzlefs is
> implemented as a userspace filesystem (FUSE). A read-only kernel
> filesystem driver is underway.
>
> Lastly, memory safety is critical to puzzlefs, leading to the decision
> to implement it in Rust. Another goal is to share the same code between
> user space and kernel space in order to provide one secure
> implementation.
>
>
> [1] https://github.com/anuvu/puzzlefs
> [2] https://github.com/wedsonaf/linux/tree/fs
> [3] https://github.com/Rust-for-Linux/linux/issues/1004
> [4] https://github.com/Rust-for-Linux/linux/pull/1007
> [5] https://docs.rs/serde_cbor/latest/serde_cbor/
> [6] https://docs.rs/hex/0.4.3/hex/
> [7] https://hackmd.io/@cyphar/ociv2-brainstorm
> [8] https://www.youtube.com/watch?v=tPs1uRqOnlk
> [9] https://github.com/Rust-for-Linux/linux/tree/rust-next
> [10] https://docs.kernel.org/rust/quick-start.html
> [11] https://github.com/ariel-miculas/puzzlefs/tree/support-no-verity-data
> [12] https://archive.fosdem.org/2019/schedule/event/containers_atomfs/
>
> Ariel Miculas (58):
> rust: kernel: add libraries required by the filesystem abstractions
> rust: kernel: backport the delay module from the rust branch
> rust: kernel: add container_of macro
> rust: kernel: add offset_of macro
> drop: Add crate::pr_warn declaration
> rust: kernel: rename from_kernel_errno to from_errno
> rust: kernel: Rename from_pointer to from_foreing and into_pointer to
> into_foreign
> rust: kernel: add count_paren_items macro, needed by define_fs_params
> macro
> rust: helpers: add missing rust helper 'alloc_pages'
> kernel: configs: add qemu-busybox-min.config
> rust: kernel: format the rust code
> samples: puzzlefs: add initial puzzlefs sample, copied from rust_fs.rs
> kernel: configs: enable rust samples in rust.config
> Add SAMPLE_RUST_SERDE in rust.config
> rust: kernel: fix compile errors after rebase to rust-next
> rust: serde_cbor: import crate
> rust: serde_cbor: add SPDX License Identifiers
> rust: serde_cbor: add no_fp_fmt_parse support
> rust: Kbuild: enable serde_cbor
> samples: rust: add cbor serialize/deserialize example
> rust: serde_cbor: add support for serde_cbor's from_slice method by
> using a custom alloc_kernel feature
> rust: serde: add support for deserializing Vec with kernel_alloc
> feature
> rust: file: Replace UnsafeCell with Opaque for File
> rust: kernel: implement fmt::Debug for CString
> samples: puzzlefs: rename RustFs to PuzzleFs
> samples: puzzlefs: add basic deserializing support for the puzzlefs
> metadata
> rust: file: present the filesystem context to the open function
> rust: kernel: add an abstraction over vfsmount to allow cloning a new
> private mount
> rust: file: add from_path, from_path_in_root_mnt and read_with_offset
> methods to File
> samples: puzzlefs: pass the Vfsmount structure from open to read and
> return the contents of the data file inside /home/puzzlefs_oci
> rust: file: move from_path, from_path_in_root_mnt and read_with_offset
> methods to a RegularFile newtype
> rust: file: ensure RegularFile can only create regular files
> rust: file: add get_pos method to RegularFile
> rust: file: add methods read_to_end, get_file_size and update_pos to
> RegularFile
> rust: file: define a minimal Read trait and implement it for
> RegularFile
> samples: puzzlefs: add cbor_get_array_size method
> samples: puzzlefs: add KernelError to WireFormatError and implement
> From conversion
> samples: puzzlefs: implement new for MetadataBlob
> samples: puzzlefs: build puzzlefs into the kernel, thus avoiding the
> need to export rust symbols
> rust: alloc: add try_clone for Vec<T>
> rust: alloc: add from_iter_fallible for Vec<T>
> samples: puzzlefs: implement to_errno and from_errno for
> WireFormatError
> samples: puzzlefs: add TryReserveError (and from conversion) to
> WireFormatError
> samples: puzzlefs: add higher level inode related functionality
> samples: puzzlefs: populate the directory entries with the inodes from
> the puzzlefs metadata file
> rust: hex: import crate
> rust: hex: add SPDX license identifiers
> rust: Kbuild: enable `hex`
> rust: hex: implement FromHex trait and hex::decode using a custom
> kernel_alloc feature
> rust: hex: add encode_hex_iter and encode_hex_upper_iter methods
> rust: puzzlefs: add HexError to WireFormatError and implement the From
> conversion
> rust: puzzlefs: display the error value for
> WireFormatError::KernelError
> samples: puzzlefs: add Rootfs and Digest structs to types.rs
> samples: puzzlefs: implement the conversion from WireFormatError to
> kernel::error::Error
> rust: puzzlefs: read the puzzlefs image manifest instead of an
> individual metadata layer
> rust: puzzlefs: rename PuzzleFs to PuzzleFsModule to avoid confusion
> with the PuzzleFS struct
> rust: puzzlefs: add support for reading files
> rust: puzzlefs: add oci_root_dir and image_manifest filesystem
> parameters
>
> Miguel Ojeda (15):
> rust: proc-macro2: import crate
> rust: proc-macro2: add SPDX License Identifiers
> rust: proc-macro2: remove `unicode_ident` dependency
> rust: quote: import crate
> rust: quote: add SPDX License Identifiers
> rust: syn: import crate
> rust: syn: add SPDX License Identifiers
> rust: syn: remove `unicode-ident` dependency
> rust: serde: import crate
> rust: serde: add `no_fp_fmt_parse` support
> rust: serde: add SPDX License Identifiers
> rust: serde_derive: import crate
> rust: serde_derive: add SPDX License Identifiers
> rust: Kbuild: enable `proc-macro2`, `quote`, `syn`, `serde` and
> `serde_derive`
> rust: test `serde` support
>
> Wedson Almeida Filho (7):
> rust: add definitions for ref-counted inodes and dentries
> rust: add ability to register a file system
> rust: define fs context
> rust: add support for file system parameters
> rust: allow fs driver to initialise new superblocks
> rust: add `module_fs` macro
> WIP: rust: allow fs to be populated
>
> Makefile | 14 +-
> arch/x86/configs/qemu-busybox-min.config | 11 +
> kernel/configs/qemu-busybox-min.config | 56 +
> kernel/configs/rust.config | 11 +
> rust/.gitignore | 1 +
> rust/Makefile | 232 +-
> rust/alloc/vec/mod.rs | 48 +
> rust/bindings/bindings_helper.h | 14 +
> rust/bindings/lib.rs | 5 +
> rust/helpers.c | 76 +
> rust/hex/error.rs | 78 +
> rust/hex/lib.rs | 506 +++
> rust/hex/serde.rs | 104 +
> rust/kernel/cred.rs | 46 +
> rust/kernel/delay.rs | 104 +
> rust/kernel/driver.rs | 28 +
> rust/kernel/error.rs | 52 +-
> rust/kernel/file.rs | 1117 ++++++
> rust/kernel/fs.rs | 1478 ++++++++
> rust/kernel/fs/param.rs | 558 +++
> rust/kernel/io_buffer.rs | 153 +
> rust/kernel/iov_iter.rs | 81 +
> rust/kernel/lib.rs | 83 +
> rust/kernel/mm.rs | 149 +
> rust/kernel/mount.rs | 66 +
> rust/kernel/pages.rs | 144 +
> rust/kernel/str.rs | 6 +
> rust/kernel/test_serde.rs | 26 +
> rust/kernel/test_serde/de.rs | 439 +++
> rust/kernel/test_serde/error.rs | 73 +
> rust/kernel/test_serde/ser.rs | 466 +++
> rust/kernel/user_ptr.rs | 175 +
> rust/proc-macro2/detection.rs | 77 +
> rust/proc-macro2/fallback.rs | 1004 ++++++
> rust/proc-macro2/lib.rs | 1341 ++++++++
> rust/proc-macro2/marker.rs | 20 +
> rust/proc-macro2/parse.rs | 874 +++++
> rust/proc-macro2/rcvec.rs | 144 +
> rust/proc-macro2/wrapper.rs | 996 ++++++
> rust/quote/ext.rs | 112 +
> rust/quote/format.rs | 170 +
> rust/quote/ident_fragment.rs | 88 +
> rust/quote/lib.rs | 1436 ++++++++
> rust/quote/runtime.rs | 440 +++
> rust/quote/spanned.rs | 45 +
> rust/quote/to_tokens.rs | 211 ++
> rust/serde/de/format.rs | 32 +
> rust/serde/de/ignored_any.rs | 246 ++
> rust/serde/de/impls.rs | 2755 +++++++++++++++
> rust/serde/de/mod.rs | 2313 +++++++++++++
> rust/serde/de/seed.rs | 21 +
> rust/serde/de/utf8.rs | 48 +
> rust/serde/de/value.rs | 1718 ++++++++++
> rust/serde/integer128.rs | 84 +
> rust/serde/lib.rs | 351 ++
> rust/serde/macros.rs | 238 ++
> rust/serde/private/de.rs | 2997 ++++++++++++++++
> rust/serde/private/doc.rs | 161 +
> rust/serde/private/mod.rs | 52 +
> rust/serde/private/ser.rs | 1316 +++++++
> rust/serde/private/size_hint.rs | 23 +
> rust/serde/ser/fmt.rs | 180 +
> rust/serde/ser/impls.rs | 987 ++++++
> rust/serde/ser/impossible.rs | 218 ++
> rust/serde/ser/mod.rs | 1992 +++++++++++
> rust/serde/std_error.rs | 50 +
> rust/serde_cbor/de.rs | 1370 ++++++++
> rust/serde_cbor/error.rs | 320 ++
> rust/serde_cbor/lib.rs | 371 ++
> rust/serde_cbor/read.rs | 647 ++++
> rust/serde_cbor/ser.rs | 748 ++++
> rust/serde_cbor/tags.rs | 224 ++
> rust/serde_cbor/value/de.rs | 168 +
> rust/serde_cbor/value/mod.rs | 158 +
> rust/serde_cbor/value/ser.rs | 447 +++
> rust/serde_cbor/write.rs | 177 +
> rust/serde_derive/bound.rs | 408 +++
> rust/serde_derive/de.rs | 3148 +++++++++++++++++
> rust/serde_derive/dummy.rs | 46 +
> rust/serde_derive/fragment.rs | 76 +
> rust/serde_derive/internals/ast.rs | 204 ++
> rust/serde_derive/internals/attr.rs | 1908 +++++++++++
> rust/serde_derive/internals/case.rs | 199 ++
> rust/serde_derive/internals/check.rs | 445 +++
> rust/serde_derive/internals/ctxt.rs | 64 +
> rust/serde_derive/internals/mod.rs | 30 +
> rust/serde_derive/internals/receiver.rs | 287 ++
> rust/serde_derive/internals/respan.rs | 18 +
> rust/serde_derive/internals/symbol.rs | 71 +
> rust/serde_derive/lib.rs | 112 +
> rust/serde_derive/pretend.rs | 203 ++
> rust/serde_derive/ser.rs | 1342 ++++++++
> rust/serde_derive/this.rs | 34 +
> rust/serde_derive/try.rs | 26 +
> rust/syn/attr.rs | 664 ++++
> rust/syn/await.rs | 4 +
> rust/syn/bigint.rs | 68 +
> rust/syn/buffer.rs | 400 +++
> rust/syn/custom_keyword.rs | 255 ++
> rust/syn/custom_punctuation.rs | 302 ++
> rust/syn/data.rs | 495 +++
> rust/syn/derive.rs | 276 ++
> rust/syn/discouraged.rs | 196 ++
> rust/syn/error.rs | 430 +++
> rust/syn/export.rs | 41 +
> rust/syn/expr.rs | 3560 +++++++++++++++++++
> rust/syn/ext.rs | 141 +
> rust/syn/file.rs | 127 +
> rust/syn/gen/clone.rs | 2243 ++++++++++++
> rust/syn/gen/debug.rs | 3044 +++++++++++++++++
> rust/syn/gen/eq.rs | 2197 ++++++++++++
> rust/syn/gen/fold.rs | 3343 ++++++++++++++++++
> rust/syn/gen/hash.rs | 2871 ++++++++++++++++
> rust/syn/gen/visit.rs | 3788 +++++++++++++++++++++
> rust/syn/gen/visit_mut.rs | 3788 +++++++++++++++++++++
> rust/syn/gen_helper.rs | 156 +
> rust/syn/generics.rs | 1339 ++++++++
> rust/syn/group.rs | 284 ++
> rust/syn/ident.rs | 103 +
> rust/syn/item.rs | 3315 ++++++++++++++++++
> rust/syn/lib.rs | 985 ++++++
> rust/syn/lifetime.rs | 156 +
> rust/syn/lit.rs | 1602 +++++++++
> rust/syn/lookahead.rs | 171 +
> rust/syn/mac.rs | 221 ++
> rust/syn/macros.rs | 179 +
> rust/syn/op.rs | 236 ++
> rust/syn/parse.rs | 1316 +++++++
> rust/syn/parse_macro_input.rs | 181 +
> rust/syn/parse_quote.rs | 169 +
> rust/syn/pat.rs | 929 +++++
> rust/syn/path.rs | 856 +++++
> rust/syn/print.rs | 18 +
> rust/syn/punctuated.rs | 1070 ++++++
> rust/syn/reserved.rs | 46 +
> rust/syn/sealed.rs | 6 +
> rust/syn/span.rs | 69 +
> rust/syn/spanned.rs | 116 +
> rust/syn/stmt.rs | 351 ++
> rust/syn/thread.rs | 43 +
> rust/syn/token.rs | 1015 ++++++
> rust/syn/tt.rs | 109 +
> rust/syn/ty.rs | 1288 +++++++
> rust/syn/verbatim.rs | 35 +
> rust/syn/whitespace.rs | 67 +
> samples/rust/Kconfig | 28 +
> samples/rust/Makefile | 3 +
> samples/rust/local_data_format/de.rs | 422 +++
> samples/rust/local_data_format/error.rs | 73 +
> samples/rust/local_data_format/ser.rs | 443 +++
> samples/rust/puzzle.rs | 4 +
> samples/rust/puzzle/error.rs | 91 +
> samples/rust/puzzle/inode.rs | 150 +
> samples/rust/puzzle/oci.rs | 71 +
> samples/rust/puzzle/types.rs | 389 +++
> samples/rust/puzzle/types/cbor_helpers.rs | 50 +
> samples/rust/puzzlefs.rs | 220 ++
> samples/rust/rust_fs.rs | 105 +
> samples/rust/rust_serde.rs | 125 +
> scripts/Makefile.build | 4 +-
> 160 files changed, 89204 insertions(+), 29 deletions(-)
> create mode 100644 arch/x86/configs/qemu-busybox-min.config
> create mode 100644 kernel/configs/qemu-busybox-min.config
> create mode 100644 rust/hex/error.rs
> create mode 100644 rust/hex/lib.rs
> create mode 100644 rust/hex/serde.rs
> create mode 100644 rust/kernel/cred.rs
> create mode 100644 rust/kernel/delay.rs
> create mode 100644 rust/kernel/driver.rs
> create mode 100644 rust/kernel/file.rs
> create mode 100644 rust/kernel/fs.rs
> create mode 100644 rust/kernel/fs/param.rs
> create mode 100644 rust/kernel/io_buffer.rs
> create mode 100644 rust/kernel/iov_iter.rs
> create mode 100644 rust/kernel/mm.rs
> create mode 100644 rust/kernel/mount.rs
> create mode 100644 rust/kernel/pages.rs
> create mode 100644 rust/kernel/test_serde.rs
> create mode 100644 rust/kernel/test_serde/de.rs
> create mode 100644 rust/kernel/test_serde/error.rs
> create mode 100644 rust/kernel/test_serde/ser.rs
> create mode 100644 rust/kernel/user_ptr.rs
> create mode 100644 rust/proc-macro2/detection.rs
> create mode 100644 rust/proc-macro2/fallback.rs
> create mode 100644 rust/proc-macro2/lib.rs
> create mode 100644 rust/proc-macro2/marker.rs
> create mode 100644 rust/proc-macro2/parse.rs
> create mode 100644 rust/proc-macro2/rcvec.rs
> create mode 100644 rust/proc-macro2/wrapper.rs
> create mode 100644 rust/quote/ext.rs
> create mode 100644 rust/quote/format.rs
> create mode 100644 rust/quote/ident_fragment.rs
> create mode 100644 rust/quote/lib.rs
> create mode 100644 rust/quote/runtime.rs
> create mode 100644 rust/quote/spanned.rs
> create mode 100644 rust/quote/to_tokens.rs
> create mode 100644 rust/serde/de/format.rs
> create mode 100644 rust/serde/de/ignored_any.rs
> create mode 100644 rust/serde/de/impls.rs
> create mode 100644 rust/serde/de/mod.rs
> create mode 100644 rust/serde/de/seed.rs
> create mode 100644 rust/serde/de/utf8.rs
> create mode 100644 rust/serde/de/value.rs
> create mode 100644 rust/serde/integer128.rs
> create mode 100644 rust/serde/lib.rs
> create mode 100644 rust/serde/macros.rs
> create mode 100644 rust/serde/private/de.rs
> create mode 100644 rust/serde/private/doc.rs
> create mode 100644 rust/serde/private/mod.rs
> create mode 100644 rust/serde/private/ser.rs
> create mode 100644 rust/serde/private/size_hint.rs
> create mode 100644 rust/serde/ser/fmt.rs
> create mode 100644 rust/serde/ser/impls.rs
> create mode 100644 rust/serde/ser/impossible.rs
> create mode 100644 rust/serde/ser/mod.rs
> create mode 100644 rust/serde/std_error.rs
> create mode 100644 rust/serde_cbor/de.rs
> create mode 100644 rust/serde_cbor/error.rs
> create mode 100644 rust/serde_cbor/lib.rs
> create mode 100644 rust/serde_cbor/read.rs
> create mode 100644 rust/serde_cbor/ser.rs
> create mode 100644 rust/serde_cbor/tags.rs
> create mode 100644 rust/serde_cbor/value/de.rs
> create mode 100644 rust/serde_cbor/value/mod.rs
> create mode 100644 rust/serde_cbor/value/ser.rs
> create mode 100644 rust/serde_cbor/write.rs
> create mode 100644 rust/serde_derive/bound.rs
> create mode 100644 rust/serde_derive/de.rs
> create mode 100644 rust/serde_derive/dummy.rs
> create mode 100644 rust/serde_derive/fragment.rs
> create mode 100644 rust/serde_derive/internals/ast.rs
> create mode 100644 rust/serde_derive/internals/attr.rs
> create mode 100644 rust/serde_derive/internals/case.rs
> create mode 100644 rust/serde_derive/internals/check.rs
> create mode 100644 rust/serde_derive/internals/ctxt.rs
> create mode 100644 rust/serde_derive/internals/mod.rs
> create mode 100644 rust/serde_derive/internals/receiver.rs
> create mode 100644 rust/serde_derive/internals/respan.rs
> create mode 100644 rust/serde_derive/internals/symbol.rs
> create mode 100644 rust/serde_derive/lib.rs
> create mode 100644 rust/serde_derive/pretend.rs
> create mode 100644 rust/serde_derive/ser.rs
> create mode 100644 rust/serde_derive/this.rs
> create mode 100644 rust/serde_derive/try.rs
> create mode 100644 rust/syn/attr.rs
> create mode 100644 rust/syn/await.rs
> create mode 100644 rust/syn/bigint.rs
> create mode 100644 rust/syn/buffer.rs
> create mode 100644 rust/syn/custom_keyword.rs
> create mode 100644 rust/syn/custom_punctuation.rs
> create mode 100644 rust/syn/data.rs
> create mode 100644 rust/syn/derive.rs
> create mode 100644 rust/syn/discouraged.rs
> create mode 100644 rust/syn/error.rs
> create mode 100644 rust/syn/export.rs
> create mode 100644 rust/syn/expr.rs
> create mode 100644 rust/syn/ext.rs
> create mode 100644 rust/syn/file.rs
> create mode 100644 rust/syn/gen/clone.rs
> create mode 100644 rust/syn/gen/debug.rs
> create mode 100644 rust/syn/gen/eq.rs
> create mode 100644 rust/syn/gen/fold.rs
> create mode 100644 rust/syn/gen/hash.rs
> create mode 100644 rust/syn/gen/visit.rs
> create mode 100644 rust/syn/gen/visit_mut.rs
> create mode 100644 rust/syn/gen_helper.rs
> create mode 100644 rust/syn/generics.rs
> create mode 100644 rust/syn/group.rs
> create mode 100644 rust/syn/ident.rs
> create mode 100644 rust/syn/item.rs
> create mode 100644 rust/syn/lib.rs
> create mode 100644 rust/syn/lifetime.rs
> create mode 100644 rust/syn/lit.rs
> create mode 100644 rust/syn/lookahead.rs
> create mode 100644 rust/syn/mac.rs
> create mode 100644 rust/syn/macros.rs
> create mode 100644 rust/syn/op.rs
> create mode 100644 rust/syn/parse.rs
> create mode 100644 rust/syn/parse_macro_input.rs
> create mode 100644 rust/syn/parse_quote.rs
> create mode 100644 rust/syn/pat.rs
> create mode 100644 rust/syn/path.rs
> create mode 100644 rust/syn/print.rs
> create mode 100644 rust/syn/punctuated.rs
> create mode 100644 rust/syn/reserved.rs
> create mode 100644 rust/syn/sealed.rs
> create mode 100644 rust/syn/span.rs
> create mode 100644 rust/syn/spanned.rs
> create mode 100644 rust/syn/stmt.rs
> create mode 100644 rust/syn/thread.rs
> create mode 100644 rust/syn/token.rs
> create mode 100644 rust/syn/tt.rs
> create mode 100644 rust/syn/ty.rs
> create mode 100644 rust/syn/verbatim.rs
> create mode 100644 rust/syn/whitespace.rs
> create mode 100644 samples/rust/local_data_format/de.rs
> create mode 100644 samples/rust/local_data_format/error.rs
> create mode 100644 samples/rust/local_data_format/ser.rs
> create mode 100644 samples/rust/puzzle.rs
> create mode 100644 samples/rust/puzzle/error.rs
> create mode 100644 samples/rust/puzzle/inode.rs
> create mode 100644 samples/rust/puzzle/oci.rs
> create mode 100644 samples/rust/puzzle/types.rs
> create mode 100644 samples/rust/puzzle/types/cbor_helpers.rs
> create mode 100644 samples/rust/puzzlefs.rs
> create mode 100644 samples/rust/rust_fs.rs
> create mode 100644 samples/rust/rust_serde.rs
>
> --
> 2.40.1
>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 10:36 ` [RFC PATCH 00/80] Rust PuzzleFS filesystem driver Christian Brauner
@ 2023-06-09 11:22 ` Ariel Miculas (amiculas)
2023-06-09 11:45 ` Christian Brauner
2023-06-09 11:42 ` Miguel Ojeda
2023-06-09 23:52 ` Kent Overstreet
2 siblings, 1 reply; 28+ messages in thread
From: Ariel Miculas (amiculas) @ 2023-06-09 11:22 UTC (permalink / raw)
To: Christian Brauner, linux-fsdevel; +Cc: rust-for-linux, linux-mm
[-- Attachment #1: Type: text/plain, Size: 33334 bytes --]
Hello Christian,
I didn't send these patches to a wider audience because this is an initial prototype of the PuzzleFS driver, and it has a few prerequisites before it could be even considered for merging. First of all, the rust filesystem abstractions and their dependencies need to be upstreamed, then there needs to be a discussion regarding the inclusion of third-party crates in the linux kernel.
My plan was to send these patches to the rust-for-linux mailing list and then start a discussion with Miguel Ojeda regarding the upstreaming approach.
There are a lot of new files added in this patch series because I've included all the dependencies required so that my patches could be applied to the rust-next branch, but these dependencies will most likely need to be upstreamed separately.
It was never my intention to avoid your reviews, should I also send subsequent patches to linux-fsdevel, even if they're in the early stages of development?
Regards,
Ariel
________________________________
From: Christian Brauner <brauner@kernel.org>
Sent: Friday, June 9, 2023 1:36 PM
To: Ariel Miculas (amiculas) <amiculas@cisco.com>; linux-fsdevel@vger.kernel.org <linux-fsdevel@vger.kernel.org>
Cc: rust-for-linux@vger.kernel.org <rust-for-linux@vger.kernel.org>; linux-mm@kvack.org <linux-mm@kvack.org>
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
On Fri, Jun 09, 2023 at 09:29:58AM +0300, Ariel Miculas wrote:
> Hi all!
>
> This is a proof of concept driver written for the PuzzleFS
Uhm, the PuzzleFS filesystem isn't actually sent to fsdevel? Yet I see
tons of patches in there that add wrappers to our core fs data
structures. I even see a ton of new files that all fall clearly into
fsdevel territory:
create mode 100644 rust/kernel/cred.rs
create mode 100644 rust/kernel/delay.rs
create mode 100644 rust/kernel/driver.rs
create mode 100644 rust/kernel/file.rs
create mode 100644 rust/kernel/fs.rs
create mode 100644 rust/kernel/fs/param.rs
create mode 100644 rust/kernel/io_buffer.rs
create mode 100644 rust/kernel/iov_iter.rs
create mode 100644 rust/kernel/mm.rs
create mode 100644 rust/kernel/mount.rs
create mode 100644 rust/kernel/pages.rs
There's also quite a lot of new mm/ in there, no?
Any wrappers and code for core fs should be maintained as part of fs.
Rust shouldn't become a way to avoid our reviews once you have a few
wrappers added somewhere.
> next-generation container filesystem [1]. I've included a short abstract
> about puzzlefs further below. This driver is based on the rust-next
> branch, on top of which I've backported the filesystem abstractions from
> Wedson Almeida Filho [2][3] and Miguel Ojeda's third-party crates
> support: proc-macro2, quote, syn, serde and serde_derive [4]. I've added
> the additional third-party crates serde_cbor[5] and hex [6]. Then I've
> adapted the user space puzzlefs code [1] so that the puzzlefs kernel
> module could present the directory hierarchy and implement the basic
> read functionality.
> For some additional context, puzzlefs was started by Tycho Andersen and
> it's the successor of atomfs. This FOSDEM presentation from 2019 [12]
> covers the rationale for a new oci image format and presents a higher
> level overview of our goals with puzzlefs.
> I've split the rest of the cover letter in following sections (using a
> markdown style):
> * Example: it describes a practical example of what was achieved
> * Limitations: it presents the existing limitations of this POC
> * Upstreaming steps: it describes the steps needed for upstreaming this
> driver
> * Setup: it shows how to setup the necessary environment for testing the
> puzzlefs driver
> * Puzzlefs abstract: it provides a short overview of puzzlefs
>
> # Example
> An example is provided below:
>
> ```
> ~ # cat /proc/filesystems | grep puzzlefs
> nodev puzzlefs
> ~ # mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o image_manifest="2d
> 6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b" none /mnt
> ~ # ls -lR /mnt/
> /mnt/:
> total 0
> drwxr-xr-x 2 0 0 0 Jun 8 12:26 dir-1
> drwxr-xr-x 2 0 0 0 Jun 8 12:26 dir-2
> drwxr-xr-x 2 0 0 0 Jun 8 12:26 dir-3
> drwxr-xr-x 2 0 0 0 Jun 8 12:26 dir-4
> -rw-r--r-- 1 0 0 0 Jun 8 12:26 file1
> -rw-r--r-- 1 0 0 0 Jun 8 12:26 file2
>
> /mnt/dir-1:
> total 0
>
> /mnt/dir-2:
> total 0
>
> /mnt/dir-3:
> total 0
>
> /mnt/dir-4:
> total 0
> ~ # cat /mnt/file2
> ana are mere bla bla bla
> ~ # wc /mnt/file1
> 202 202 5454 /mnt/file1
> ```
>
> In this example, /home/puzzlefs_oci is the puzzlefs oci directory:
> ```
> ~ # ls -lR /home/puzzlefs_oci/
> /home/puzzlefs_oci/:
> total 8
> drwxr-xr-x 3 1000 1000 0 Jun 8 14:33 blobs
> -rw-r--r-- 1 1000 1000 266 Jun 8 14:33 index.json
> -rw-r--r-- 1 1000 1000 37 Jun 8 14:33 oci-layout
>
> /home/puzzlefs_oci/blobs:
> total 0
> drwxr-xr-x 2 1000 1000 0 Jun 8 14:33 sha256
>
> /home/puzzlefs_oci/blobs/sha256:
> total 16
> -rw------- 1 1000 1000 89 Jun 8 14:33 2d6602d678140540dc7e96de652a76a8b16eb
> -rw------- 1 1000 1000 925 Jun 8 14:33 4df03518eea406343dbb55046720f6a478881
> -rw------- 1 1000 1000 5479 Jun 8 14:33 d86a87b19bd9a2fec0d31687c1d669cdb59eb
> ```
>
> `2d6602d678140540dc7e96de652a76a8b16eb` is the puzzlefs image manifest
> hash for the first_try tag:
> ```
> $ cat /tmp/oci-simple/index.json | jq .
> {
> "schemaVersion": -1,
> "manifests": [
> {
> "digest": "sha256:2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b",
> "size": 89,
> "media_type": "application/vnd.puzzlefs.image.rootfs.v1",
> "annotations": {
> "org.opencontainers.image.ref.name": "first_try"
> }
> }
> ],
> "annotations": {}
> }
> ```
>
> I will describe how to build a puzzlefs image in the `Setup` section, at
> step 5.
>
> # Limitations
> One limitation is that the puzzlefs driver doesn't implement any lookup
> functionality and instead it inserts every directory entry into the
> dcache during init (see the `DCACHE_BASED` constant). This is similar to
> how the sample `rust_fs` driver works, but the goal is to implement
> proper lookup functions. However, more filesystem abstractions need to
> be implemented before this can be achieved.
>
> # Upstreaming steps
> Before the puzzlefs driver can be upstreamed, the following need to be
> merged:
> * Wedson's filesystem abstractions [3]
> * the necessary third-party crates [4] (with the preliminary discussion
> about whether this is desirable)
>
> # Setup
> My setup is based on Wedson's tutorial [8]. Next, I will describe the
> necessary steps to build an initrd and run a custom kernel under qemu.
>
> 1. Get the linux rust-next branch [9] and apply this patchset
>
> 2. Follow the rust quickstart guide [10]
>
> 3. configure and build the kernel
> ```
> $ make LLVM=1 allnoconfig qemu-busybox-min.config rust.config
> $ make LLVM=1 -j$(nproc)
> ```
>
> 4. setup busybox
> ```
> git clone git://git.busybox.net/busybox
> cd busybox
> make menuconfig # enable 'Build static binary' config
> make
> make install
> ```
> This will create the `_install` directory with the rootfs inside it.
>
> 5. create a home directory in the rootfs and copy a puzzlefs oci
> directory in home/puzzlefs_oci
> To create a puzzlefs oci directory:
> * download this custom puzzlefs repository [11] (it's custom because we
> want to build an image without verity data)
> * run `make release`
> * create a simple filesystem structure with a few directories and files
> (I've created one at ../test-puzzlefs/simple_rootfs)
> * build a puzzlefs oci image at
> `~/work/busybox/_install/home/puzzlefs_oci` (replace this path with
> your busybox path) with the tag `first_try`:
> ```
> $ target/release/puzzlefs build --omit-verity \
> ../test-puzzlefs/simple_rootfs ~/work/busybox/_install/home/puzzlefs_oci \
> first_try
> ```
> * get first_try's image manifest from index.json (inside `puzzlefs_oci`)
> ```
> $ cat index.json | jq . | grep digest
> "digest": "sha256:2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b",
> ```
>
> 6. add the following 'init' script in the busybox rootfs (rootfs path
> defaults to `./_install'):
> ```
> #!/bin/sh
> mount -t devtmpfs none /dev
> mkdir -p /proc
> mount -t proc none /proc
>
> ifconfig lo up
> udhcpc -i eth0
>
> mkdir /mnt
> mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o \
> image_manifest="2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b" \
> none /mnt
>
> setsid sh -c 'exec sh -l </dev/ttyS0 >/dev/ttyS0 2>&1'
> ```
>
> Make sure to replace the `image_manifest` with your own digest. This
> init script will be passed to rdinit in the kernel command line.
>
> 7. generate the initramfs
>
> Assuming busybox is in `~/work/busybox`:
> ```
> cd ~/work/busybox/_install && find . | cpio -H newc -o | gzip > ../ramdisk.img
> ```
> This will generate a compressed ramdisk image in
> `~/work/busybox/ramdisk.img`.
>
> 8. run with qemu (this assumes the linux tree is at '../linux' and busybox
> is at '../busybox'):
> ```
> qemu-system-x86_64 \
> -accel kvm \
> -cpu host \
> -m 4G \
> -initrd ../busybox/ramdisk.img \
> -kernel ../linux/arch/x86/boot/bzImage \
> -nographic \
> -append 'console=ttyS0 nokaslr debug rdinit=/init' \
> -nic user,model=rtl8139 \
> -no-reboot
> ```
>
> 9. Check whether puzzlefs has been successfully mounted
> ```
> ~ # mount | grep puzzlefs
> none on /mnt type puzzlefs (rw,relatime)
> ~ # ls /mnt/
> dir-1 dir-2 dir-3 dir-4 file1 file2
> ```
>
>
> # Puzzlefs abstract
> Puzzlefs [1] is a container filesystem designed to address the
> limitations of the existing OCI format. The main goals of the project
> are reduced duplication, reproducible image builds, direct mounting
> support and memory safety guarantees, some inspired by the OCIv2 design
> document [7].
>
> Reduced duplication is achieved using the content defined chunking
> algorithm FastCDC. This implementation allows chunks to be shared among
> layers. Building a new layer starting from an existing one allows
> reusing most of the chunks.
>
> Another goal of the project is reproducible image builds, which is
> achieved by defining a canonical representation of the image format.
>
> Direct mounting support is a key feature of puzzlefs and, together with
> fs-verity, it provides data integrity. Currently, puzzlefs is
> implemented as a userspace filesystem (FUSE). A read-only kernel
> filesystem driver is underway.
>
> Lastly, memory safety is critical to puzzlefs, leading to the decision
> to implement it in Rust. Another goal is to share the same code between
> user space and kernel space in order to provide one secure
> implementation.
>
>
> [1] https://github.com/anuvu/puzzlefs
> [2] https://github.com/wedsonaf/linux/tree/fs
> [3] https://github.com/Rust-for-Linux/linux/issues/1004
> [4] https://github.com/Rust-for-Linux/linux/pull/1007
> [5] https://docs.rs/serde_cbor/latest/serde_cbor/
> [6] https://docs.rs/hex/0.4.3/hex/
> [7] https://hackmd.io/@cyphar/ociv2-brainstorm
> [8] https://www.youtube.com/watch?v=tPs1uRqOnlk
> [9] https://github.com/Rust-for-Linux/linux/tree/rust-next
> [10] https://docs.kernel.org/rust/quick-start.html
> [11] https://github.com/ariel-miculas/puzzlefs/tree/support-no-verity-data
> [12] https://archive.fosdem.org/2019/schedule/event/containers_atomfs/
>
> Ariel Miculas (58):
> rust: kernel: add libraries required by the filesystem abstractions
> rust: kernel: backport the delay module from the rust branch
> rust: kernel: add container_of macro
> rust: kernel: add offset_of macro
> drop: Add crate::pr_warn declaration
> rust: kernel: rename from_kernel_errno to from_errno
> rust: kernel: Rename from_pointer to from_foreing and into_pointer to
> into_foreign
> rust: kernel: add count_paren_items macro, needed by define_fs_params
> macro
> rust: helpers: add missing rust helper 'alloc_pages'
> kernel: configs: add qemu-busybox-min.config
> rust: kernel: format the rust code
> samples: puzzlefs: add initial puzzlefs sample, copied from rust_fs.rs
> kernel: configs: enable rust samples in rust.config
> Add SAMPLE_RUST_SERDE in rust.config
> rust: kernel: fix compile errors after rebase to rust-next
> rust: serde_cbor: import crate
> rust: serde_cbor: add SPDX License Identifiers
> rust: serde_cbor: add no_fp_fmt_parse support
> rust: Kbuild: enable serde_cbor
> samples: rust: add cbor serialize/deserialize example
> rust: serde_cbor: add support for serde_cbor's from_slice method by
> using a custom alloc_kernel feature
> rust: serde: add support for deserializing Vec with kernel_alloc
> feature
> rust: file: Replace UnsafeCell with Opaque for File
> rust: kernel: implement fmt::Debug for CString
> samples: puzzlefs: rename RustFs to PuzzleFs
> samples: puzzlefs: add basic deserializing support for the puzzlefs
> metadata
> rust: file: present the filesystem context to the open function
> rust: kernel: add an abstraction over vfsmount to allow cloning a new
> private mount
> rust: file: add from_path, from_path_in_root_mnt and read_with_offset
> methods to File
> samples: puzzlefs: pass the Vfsmount structure from open to read and
> return the contents of the data file inside /home/puzzlefs_oci
> rust: file: move from_path, from_path_in_root_mnt and read_with_offset
> methods to a RegularFile newtype
> rust: file: ensure RegularFile can only create regular files
> rust: file: add get_pos method to RegularFile
> rust: file: add methods read_to_end, get_file_size and update_pos to
> RegularFile
> rust: file: define a minimal Read trait and implement it for
> RegularFile
> samples: puzzlefs: add cbor_get_array_size method
> samples: puzzlefs: add KernelError to WireFormatError and implement
> From conversion
> samples: puzzlefs: implement new for MetadataBlob
> samples: puzzlefs: build puzzlefs into the kernel, thus avoiding the
> need to export rust symbols
> rust: alloc: add try_clone for Vec<T>
> rust: alloc: add from_iter_fallible for Vec<T>
> samples: puzzlefs: implement to_errno and from_errno for
> WireFormatError
> samples: puzzlefs: add TryReserveError (and from conversion) to
> WireFormatError
> samples: puzzlefs: add higher level inode related functionality
> samples: puzzlefs: populate the directory entries with the inodes from
> the puzzlefs metadata file
> rust: hex: import crate
> rust: hex: add SPDX license identifiers
> rust: Kbuild: enable `hex`
> rust: hex: implement FromHex trait and hex::decode using a custom
> kernel_alloc feature
> rust: hex: add encode_hex_iter and encode_hex_upper_iter methods
> rust: puzzlefs: add HexError to WireFormatError and implement the From
> conversion
> rust: puzzlefs: display the error value for
> WireFormatError::KernelError
> samples: puzzlefs: add Rootfs and Digest structs to types.rs
> samples: puzzlefs: implement the conversion from WireFormatError to
> kernel::error::Error
> rust: puzzlefs: read the puzzlefs image manifest instead of an
> individual metadata layer
> rust: puzzlefs: rename PuzzleFs to PuzzleFsModule to avoid confusion
> with the PuzzleFS struct
> rust: puzzlefs: add support for reading files
> rust: puzzlefs: add oci_root_dir and image_manifest filesystem
> parameters
>
> Miguel Ojeda (15):
> rust: proc-macro2: import crate
> rust: proc-macro2: add SPDX License Identifiers
> rust: proc-macro2: remove `unicode_ident` dependency
> rust: quote: import crate
> rust: quote: add SPDX License Identifiers
> rust: syn: import crate
> rust: syn: add SPDX License Identifiers
> rust: syn: remove `unicode-ident` dependency
> rust: serde: import crate
> rust: serde: add `no_fp_fmt_parse` support
> rust: serde: add SPDX License Identifiers
> rust: serde_derive: import crate
> rust: serde_derive: add SPDX License Identifiers
> rust: Kbuild: enable `proc-macro2`, `quote`, `syn`, `serde` and
> `serde_derive`
> rust: test `serde` support
>
> Wedson Almeida Filho (7):
> rust: add definitions for ref-counted inodes and dentries
> rust: add ability to register a file system
> rust: define fs context
> rust: add support for file system parameters
> rust: allow fs driver to initialise new superblocks
> rust: add `module_fs` macro
> WIP: rust: allow fs to be populated
>
> Makefile | 14 +-
> arch/x86/configs/qemu-busybox-min.config | 11 +
> kernel/configs/qemu-busybox-min.config | 56 +
> kernel/configs/rust.config | 11 +
> rust/.gitignore | 1 +
> rust/Makefile | 232 +-
> rust/alloc/vec/mod.rs | 48 +
> rust/bindings/bindings_helper.h | 14 +
> rust/bindings/lib.rs | 5 +
> rust/helpers.c | 76 +
> rust/hex/error.rs | 78 +
> rust/hex/lib.rs | 506 +++
> rust/hex/serde.rs | 104 +
> rust/kernel/cred.rs | 46 +
> rust/kernel/delay.rs | 104 +
> rust/kernel/driver.rs | 28 +
> rust/kernel/error.rs | 52 +-
> rust/kernel/file.rs | 1117 ++++++
> rust/kernel/fs.rs | 1478 ++++++++
> rust/kernel/fs/param.rs | 558 +++
> rust/kernel/io_buffer.rs | 153 +
> rust/kernel/iov_iter.rs | 81 +
> rust/kernel/lib.rs | 83 +
> rust/kernel/mm.rs | 149 +
> rust/kernel/mount.rs | 66 +
> rust/kernel/pages.rs | 144 +
> rust/kernel/str.rs | 6 +
> rust/kernel/test_serde.rs | 26 +
> rust/kernel/test_serde/de.rs | 439 +++
> rust/kernel/test_serde/error.rs | 73 +
> rust/kernel/test_serde/ser.rs | 466 +++
> rust/kernel/user_ptr.rs | 175 +
> rust/proc-macro2/detection.rs | 77 +
> rust/proc-macro2/fallback.rs | 1004 ++++++
> rust/proc-macro2/lib.rs | 1341 ++++++++
> rust/proc-macro2/marker.rs | 20 +
> rust/proc-macro2/parse.rs | 874 +++++
> rust/proc-macro2/rcvec.rs | 144 +
> rust/proc-macro2/wrapper.rs | 996 ++++++
> rust/quote/ext.rs | 112 +
> rust/quote/format.rs | 170 +
> rust/quote/ident_fragment.rs | 88 +
> rust/quote/lib.rs | 1436 ++++++++
> rust/quote/runtime.rs | 440 +++
> rust/quote/spanned.rs | 45 +
> rust/quote/to_tokens.rs | 211 ++
> rust/serde/de/format.rs | 32 +
> rust/serde/de/ignored_any.rs | 246 ++
> rust/serde/de/impls.rs | 2755 +++++++++++++++
> rust/serde/de/mod.rs | 2313 +++++++++++++
> rust/serde/de/seed.rs | 21 +
> rust/serde/de/utf8.rs | 48 +
> rust/serde/de/value.rs | 1718 ++++++++++
> rust/serde/integer128.rs | 84 +
> rust/serde/lib.rs | 351 ++
> rust/serde/macros.rs | 238 ++
> rust/serde/private/de.rs | 2997 ++++++++++++++++
> rust/serde/private/doc.rs | 161 +
> rust/serde/private/mod.rs | 52 +
> rust/serde/private/ser.rs | 1316 +++++++
> rust/serde/private/size_hint.rs | 23 +
> rust/serde/ser/fmt.rs | 180 +
> rust/serde/ser/impls.rs | 987 ++++++
> rust/serde/ser/impossible.rs | 218 ++
> rust/serde/ser/mod.rs | 1992 +++++++++++
> rust/serde/std_error.rs | 50 +
> rust/serde_cbor/de.rs | 1370 ++++++++
> rust/serde_cbor/error.rs | 320 ++
> rust/serde_cbor/lib.rs | 371 ++
> rust/serde_cbor/read.rs | 647 ++++
> rust/serde_cbor/ser.rs | 748 ++++
> rust/serde_cbor/tags.rs | 224 ++
> rust/serde_cbor/value/de.rs | 168 +
> rust/serde_cbor/value/mod.rs | 158 +
> rust/serde_cbor/value/ser.rs | 447 +++
> rust/serde_cbor/write.rs | 177 +
> rust/serde_derive/bound.rs | 408 +++
> rust/serde_derive/de.rs | 3148 +++++++++++++++++
> rust/serde_derive/dummy.rs | 46 +
> rust/serde_derive/fragment.rs | 76 +
> rust/serde_derive/internals/ast.rs | 204 ++
> rust/serde_derive/internals/attr.rs | 1908 +++++++++++
> rust/serde_derive/internals/case.rs | 199 ++
> rust/serde_derive/internals/check.rs | 445 +++
> rust/serde_derive/internals/ctxt.rs | 64 +
> rust/serde_derive/internals/mod.rs | 30 +
> rust/serde_derive/internals/receiver.rs | 287 ++
> rust/serde_derive/internals/respan.rs | 18 +
> rust/serde_derive/internals/symbol.rs | 71 +
> rust/serde_derive/lib.rs | 112 +
> rust/serde_derive/pretend.rs | 203 ++
> rust/serde_derive/ser.rs | 1342 ++++++++
> rust/serde_derive/this.rs | 34 +
> rust/serde_derive/try.rs | 26 +
> rust/syn/attr.rs | 664 ++++
> rust/syn/await.rs | 4 +
> rust/syn/bigint.rs | 68 +
> rust/syn/buffer.rs | 400 +++
> rust/syn/custom_keyword.rs | 255 ++
> rust/syn/custom_punctuation.rs | 302 ++
> rust/syn/data.rs | 495 +++
> rust/syn/derive.rs | 276 ++
> rust/syn/discouraged.rs | 196 ++
> rust/syn/error.rs | 430 +++
> rust/syn/export.rs | 41 +
> rust/syn/expr.rs | 3560 +++++++++++++++++++
> rust/syn/ext.rs | 141 +
> rust/syn/file.rs | 127 +
> rust/syn/gen/clone.rs | 2243 ++++++++++++
> rust/syn/gen/debug.rs | 3044 +++++++++++++++++
> rust/syn/gen/eq.rs | 2197 ++++++++++++
> rust/syn/gen/fold.rs | 3343 ++++++++++++++++++
> rust/syn/gen/hash.rs | 2871 ++++++++++++++++
> rust/syn/gen/visit.rs | 3788 +++++++++++++++++++++
> rust/syn/gen/visit_mut.rs | 3788 +++++++++++++++++++++
> rust/syn/gen_helper.rs | 156 +
> rust/syn/generics.rs | 1339 ++++++++
> rust/syn/group.rs | 284 ++
> rust/syn/ident.rs | 103 +
> rust/syn/item.rs | 3315 ++++++++++++++++++
> rust/syn/lib.rs | 985 ++++++
> rust/syn/lifetime.rs | 156 +
> rust/syn/lit.rs | 1602 +++++++++
> rust/syn/lookahead.rs | 171 +
> rust/syn/mac.rs | 221 ++
> rust/syn/macros.rs | 179 +
> rust/syn/op.rs | 236 ++
> rust/syn/parse.rs | 1316 +++++++
> rust/syn/parse_macro_input.rs | 181 +
> rust/syn/parse_quote.rs | 169 +
> rust/syn/pat.rs | 929 +++++
> rust/syn/path.rs | 856 +++++
> rust/syn/print.rs | 18 +
> rust/syn/punctuated.rs | 1070 ++++++
> rust/syn/reserved.rs | 46 +
> rust/syn/sealed.rs | 6 +
> rust/syn/span.rs | 69 +
> rust/syn/spanned.rs | 116 +
> rust/syn/stmt.rs | 351 ++
> rust/syn/thread.rs | 43 +
> rust/syn/token.rs | 1015 ++++++
> rust/syn/tt.rs | 109 +
> rust/syn/ty.rs | 1288 +++++++
> rust/syn/verbatim.rs | 35 +
> rust/syn/whitespace.rs | 67 +
> samples/rust/Kconfig | 28 +
> samples/rust/Makefile | 3 +
> samples/rust/local_data_format/de.rs | 422 +++
> samples/rust/local_data_format/error.rs | 73 +
> samples/rust/local_data_format/ser.rs | 443 +++
> samples/rust/puzzle.rs | 4 +
> samples/rust/puzzle/error.rs | 91 +
> samples/rust/puzzle/inode.rs | 150 +
> samples/rust/puzzle/oci.rs | 71 +
> samples/rust/puzzle/types.rs | 389 +++
> samples/rust/puzzle/types/cbor_helpers.rs | 50 +
> samples/rust/puzzlefs.rs | 220 ++
> samples/rust/rust_fs.rs | 105 +
> samples/rust/rust_serde.rs | 125 +
> scripts/Makefile.build | 4 +-
> 160 files changed, 89204 insertions(+), 29 deletions(-)
> create mode 100644 arch/x86/configs/qemu-busybox-min.config
> create mode 100644 kernel/configs/qemu-busybox-min.config
> create mode 100644 rust/hex/error.rs
> create mode 100644 rust/hex/lib.rs
> create mode 100644 rust/hex/serde.rs
> create mode 100644 rust/kernel/cred.rs
> create mode 100644 rust/kernel/delay.rs
> create mode 100644 rust/kernel/driver.rs
> create mode 100644 rust/kernel/file.rs
> create mode 100644 rust/kernel/fs.rs
> create mode 100644 rust/kernel/fs/param.rs
> create mode 100644 rust/kernel/io_buffer.rs
> create mode 100644 rust/kernel/iov_iter.rs
> create mode 100644 rust/kernel/mm.rs
> create mode 100644 rust/kernel/mount.rs
> create mode 100644 rust/kernel/pages.rs
> create mode 100644 rust/kernel/test_serde.rs
> create mode 100644 rust/kernel/test_serde/de.rs
> create mode 100644 rust/kernel/test_serde/error.rs
> create mode 100644 rust/kernel/test_serde/ser.rs
> create mode 100644 rust/kernel/user_ptr.rs
> create mode 100644 rust/proc-macro2/detection.rs
> create mode 100644 rust/proc-macro2/fallback.rs
> create mode 100644 rust/proc-macro2/lib.rs
> create mode 100644 rust/proc-macro2/marker.rs
> create mode 100644 rust/proc-macro2/parse.rs
> create mode 100644 rust/proc-macro2/rcvec.rs
> create mode 100644 rust/proc-macro2/wrapper.rs
> create mode 100644 rust/quote/ext.rs
> create mode 100644 rust/quote/format.rs
> create mode 100644 rust/quote/ident_fragment.rs
> create mode 100644 rust/quote/lib.rs
> create mode 100644 rust/quote/runtime.rs
> create mode 100644 rust/quote/spanned.rs
> create mode 100644 rust/quote/to_tokens.rs
> create mode 100644 rust/serde/de/format.rs
> create mode 100644 rust/serde/de/ignored_any.rs
> create mode 100644 rust/serde/de/impls.rs
> create mode 100644 rust/serde/de/mod.rs
> create mode 100644 rust/serde/de/seed.rs
> create mode 100644 rust/serde/de/utf8.rs
> create mode 100644 rust/serde/de/value.rs
> create mode 100644 rust/serde/integer128.rs
> create mode 100644 rust/serde/lib.rs
> create mode 100644 rust/serde/macros.rs
> create mode 100644 rust/serde/private/de.rs
> create mode 100644 rust/serde/private/doc.rs
> create mode 100644 rust/serde/private/mod.rs
> create mode 100644 rust/serde/private/ser.rs
> create mode 100644 rust/serde/private/size_hint.rs
> create mode 100644 rust/serde/ser/fmt.rs
> create mode 100644 rust/serde/ser/impls.rs
> create mode 100644 rust/serde/ser/impossible.rs
> create mode 100644 rust/serde/ser/mod.rs
> create mode 100644 rust/serde/std_error.rs
> create mode 100644 rust/serde_cbor/de.rs
> create mode 100644 rust/serde_cbor/error.rs
> create mode 100644 rust/serde_cbor/lib.rs
> create mode 100644 rust/serde_cbor/read.rs
> create mode 100644 rust/serde_cbor/ser.rs
> create mode 100644 rust/serde_cbor/tags.rs
> create mode 100644 rust/serde_cbor/value/de.rs
> create mode 100644 rust/serde_cbor/value/mod.rs
> create mode 100644 rust/serde_cbor/value/ser.rs
> create mode 100644 rust/serde_cbor/write.rs
> create mode 100644 rust/serde_derive/bound.rs
> create mode 100644 rust/serde_derive/de.rs
> create mode 100644 rust/serde_derive/dummy.rs
> create mode 100644 rust/serde_derive/fragment.rs
> create mode 100644 rust/serde_derive/internals/ast.rs
> create mode 100644 rust/serde_derive/internals/attr.rs
> create mode 100644 rust/serde_derive/internals/case.rs
> create mode 100644 rust/serde_derive/internals/check.rs
> create mode 100644 rust/serde_derive/internals/ctxt.rs
> create mode 100644 rust/serde_derive/internals/mod.rs
> create mode 100644 rust/serde_derive/internals/receiver.rs
> create mode 100644 rust/serde_derive/internals/respan.rs
> create mode 100644 rust/serde_derive/internals/symbol.rs
> create mode 100644 rust/serde_derive/lib.rs
> create mode 100644 rust/serde_derive/pretend.rs
> create mode 100644 rust/serde_derive/ser.rs
> create mode 100644 rust/serde_derive/this.rs
> create mode 100644 rust/serde_derive/try.rs
> create mode 100644 rust/syn/attr.rs
> create mode 100644 rust/syn/await.rs
> create mode 100644 rust/syn/bigint.rs
> create mode 100644 rust/syn/buffer.rs
> create mode 100644 rust/syn/custom_keyword.rs
> create mode 100644 rust/syn/custom_punctuation.rs
> create mode 100644 rust/syn/data.rs
> create mode 100644 rust/syn/derive.rs
> create mode 100644 rust/syn/discouraged.rs
> create mode 100644 rust/syn/error.rs
> create mode 100644 rust/syn/export.rs
> create mode 100644 rust/syn/expr.rs
> create mode 100644 rust/syn/ext.rs
> create mode 100644 rust/syn/file.rs
> create mode 100644 rust/syn/gen/clone.rs
> create mode 100644 rust/syn/gen/debug.rs
> create mode 100644 rust/syn/gen/eq.rs
> create mode 100644 rust/syn/gen/fold.rs
> create mode 100644 rust/syn/gen/hash.rs
> create mode 100644 rust/syn/gen/visit.rs
> create mode 100644 rust/syn/gen/visit_mut.rs
> create mode 100644 rust/syn/gen_helper.rs
> create mode 100644 rust/syn/generics.rs
> create mode 100644 rust/syn/group.rs
> create mode 100644 rust/syn/ident.rs
> create mode 100644 rust/syn/item.rs
> create mode 100644 rust/syn/lib.rs
> create mode 100644 rust/syn/lifetime.rs
> create mode 100644 rust/syn/lit.rs
> create mode 100644 rust/syn/lookahead.rs
> create mode 100644 rust/syn/mac.rs
> create mode 100644 rust/syn/macros.rs
> create mode 100644 rust/syn/op.rs
> create mode 100644 rust/syn/parse.rs
> create mode 100644 rust/syn/parse_macro_input.rs
> create mode 100644 rust/syn/parse_quote.rs
> create mode 100644 rust/syn/pat.rs
> create mode 100644 rust/syn/path.rs
> create mode 100644 rust/syn/print.rs
> create mode 100644 rust/syn/punctuated.rs
> create mode 100644 rust/syn/reserved.rs
> create mode 100644 rust/syn/sealed.rs
> create mode 100644 rust/syn/span.rs
> create mode 100644 rust/syn/spanned.rs
> create mode 100644 rust/syn/stmt.rs
> create mode 100644 rust/syn/thread.rs
> create mode 100644 rust/syn/token.rs
> create mode 100644 rust/syn/tt.rs
> create mode 100644 rust/syn/ty.rs
> create mode 100644 rust/syn/verbatim.rs
> create mode 100644 rust/syn/whitespace.rs
> create mode 100644 samples/rust/local_data_format/de.rs
> create mode 100644 samples/rust/local_data_format/error.rs
> create mode 100644 samples/rust/local_data_format/ser.rs
> create mode 100644 samples/rust/puzzle.rs
> create mode 100644 samples/rust/puzzle/error.rs
> create mode 100644 samples/rust/puzzle/inode.rs
> create mode 100644 samples/rust/puzzle/oci.rs
> create mode 100644 samples/rust/puzzle/types.rs
> create mode 100644 samples/rust/puzzle/types/cbor_helpers.rs
> create mode 100644 samples/rust/puzzlefs.rs
> create mode 100644 samples/rust/rust_fs.rs
> create mode 100644 samples/rust/rust_serde.rs
>
> --
> 2.40.1
>
[-- Attachment #2: Type: text/html, Size: 61754 bytes --]
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 10:36 ` [RFC PATCH 00/80] Rust PuzzleFS filesystem driver Christian Brauner
2023-06-09 11:22 ` Ariel Miculas (amiculas)
@ 2023-06-09 11:42 ` Miguel Ojeda
2023-06-09 23:52 ` Kent Overstreet
2 siblings, 0 replies; 28+ messages in thread
From: Miguel Ojeda @ 2023-06-09 11:42 UTC (permalink / raw)
To: Christian Brauner; +Cc: Ariel Miculas, linux-fsdevel, rust-for-linux, linux-mm
On Fri, Jun 9, 2023 at 1:06 PM Christian Brauner <brauner@kernel.org> wrote:
>
> Any wrappers and code for core fs should be maintained as part of fs.
> Rust shouldn't become a way to avoid our reviews once you have a few
> wrappers added somewhere.
Definitely and, to be clear, we are strict about it (e.g.
https://rust-for-linux.com/contributing#the-rust-subsystem).
In fact, we appreciate maintainers that are willing to take patches
through their tree and take ownership of code too (e.g. KUnit and DRM
are doing so already).
I imagine Ariel sent the RFC as a way to announce his work early on,
especially given how some patches were split, the lack of commit
messages and tags, etc.
In other cases, we have reviewed patches privately first to iron out
this sort of thing, but I wasn't aware of this series coming (I knew
Ariel was working on puzzlefs and that he wanted to submit it
eventually, but not that an RFC was ready).
Also, some of the code here comes from the `rust` branch, but some of
it may not be ready yet for upstreaming.
Cheers,
Miguel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 11:22 ` Ariel Miculas (amiculas)
@ 2023-06-09 11:45 ` Christian Brauner
2023-06-09 12:03 ` Ariel Miculas (amiculas)
` (2 more replies)
0 siblings, 3 replies; 28+ messages in thread
From: Christian Brauner @ 2023-06-09 11:45 UTC (permalink / raw)
To: Ariel Miculas (amiculas); +Cc: linux-fsdevel, rust-for-linux, linux-mm
On Fri, Jun 09, 2023 at 11:22:12AM +0000, Ariel Miculas (amiculas) wrote:
> Hello Christian,
>
> I didn't send these patches to a wider audience because this is an
> initial prototype of the PuzzleFS driver, and it has a few
> prerequisites before it could be even considered for merging. First of
> all, the rust filesystem abstractions and their dependencies need to
> be upstreamed, then there needs to be a discussion regarding the
Yes.
> inclusion of third-party crates in the linux kernel.
>
> My plan was to send these patches to the rust-for-linux mailing list and then start a discussion with Miguel Ojeda regarding the upstreaming approach.
> There are a lot of new files added in this patch series because I've included all the dependencies required so that my patches could be applied to the rust-next branch, but these dependencies will most likely need to be upstreamed separately.
>
> It was never my intention to avoid your reviews, should I also send
> subsequent patches to linux-fsdevel, even if they're in the early
> stages of development?
Yeah, I think that would be great.
Because the series you sent here touches on a lot of things in terms of
infrastructure alone. That work could very well be rather interesting
independent of PuzzleFS. We might just want to get enough infrastructure
to start porting a tiny existing fs (binderfs or something similar
small) to Rust to see how feasible this is and to wet our appetite for
bigger changes such as accepting a new filesystem driver completely
written in Rust.
But aside from the infrastructure discussion:
This is yet another filesystem for solving the container image problem
in the kernel with the addition of yet another filesystem. We just went
through this excercise with another filesystem. So I'd expect some
reluctance here. Tbh, the container world keeps sending us filesystems
at an alarming rate. That's two within a few months and that leaves a
rather disorganized impression.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 11:45 ` Christian Brauner
@ 2023-06-09 12:03 ` Ariel Miculas (amiculas)
2023-06-09 12:56 ` Gao Xiang
[not found] ` <CANiq72nAcGKBVcVLrfAOkqaKsfftV6D1u97wqNxT38JnNsKp5A@mail.gmail.com>
2023-06-09 12:20 ` Colin Walters
2 siblings, 1 reply; 28+ messages in thread
From: Ariel Miculas (amiculas) @ 2023-06-09 12:03 UTC (permalink / raw)
To: Christian Brauner
Cc: linux-fsdevel, rust-for-linux, linux-mm, Serge Hallyn (shallyn)
Adding Serge Hallyn for visibility.
Regards,
Ariel
From: Christian Brauner <brauner@kernel.org>
Sent: Friday, June 9, 2023 2:45 PM
To: Ariel Miculas (amiculas) <amiculas@cisco.com>
Cc: linux-fsdevel@vger.kernel.org <linux-fsdevel@vger.kernel.org>; rust-for-linux@vger.kernel.org <rust-for-linux@vger.kernel.org>; linux-mm@kvack.org <linux-mm@kvack.org>
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
On Fri, Jun 09, 2023 at 11:22:12AM +0000, Ariel Miculas (amiculas) wrote:
> Hello Christian,
>
> I didn't send these patches to a wider audience because this is an
> initial prototype of the PuzzleFS driver, and it has a few
> prerequisites before it could be even considered for merging. First of
> all, the rust filesystem abstractions and their dependencies need to
> be upstreamed, then there needs to be a discussion regarding the
Yes.
> inclusion of third-party crates in the linux kernel.
>
> My plan was to send these patches to the rust-for-linux mailing list and then start a discussion with Miguel Ojeda regarding the upstreaming approach.
> There are a lot of new files added in this patch series because I've included all the dependencies required so that my patches could be applied to the rust-next branch, but these dependencies will most likely need to be upstreamed separately.
>
> It was never my intention to avoid your reviews, should I also send
> subsequent patches to linux-fsdevel, even if they're in the early
> stages of development?
Yeah, I think that would be great.
Because the series you sent here touches on a lot of things in terms of
infrastructure alone. That work could very well be rather interesting
independent of PuzzleFS. We might just want to get enough infrastructure
to start porting a tiny existing fs (binderfs or something similar
small) to Rust to see how feasible this is and to wet our appetite for
bigger changes such as accepting a new filesystem driver completely
written in Rust.
But aside from the infrastructure discussion:
This is yet another filesystem for solving the container image problem
in the kernel with the addition of yet another filesystem. We just went
through this excercise with another filesystem. So I'd expect some
reluctance here. Tbh, the container world keeps sending us filesystems
at an alarming rate. That's two within a few months and that leaves a
rather disorganized impression.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
[not found] ` <CANiq72nAcGKBVcVLrfAOkqaKsfftV6D1u97wqNxT38JnNsKp5A@mail.gmail.com>
@ 2023-06-09 12:11 ` Ariel Miculas (amiculas)
2023-06-09 12:21 ` Greg KH
2023-06-09 13:05 ` Alice Ryhl
1 sibling, 1 reply; 28+ messages in thread
From: Ariel Miculas (amiculas) @ 2023-06-09 12:11 UTC (permalink / raw)
To: Miguel Ojeda, Christian Brauner
Cc: linux-fsdevel, rust-for-linux, linux-mm, Alice Ryhl
Sorry about that, it seems like I need to switch to plain text mode for every reply in outlook, which is annoying.
Regards,
Ariel
From: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
Sent: Friday, June 9, 2023 3:07 PM
To: Christian Brauner <brauner@kernel.org>
Cc: Ariel Miculas (amiculas) <amiculas@cisco.com>; linux-fsdevel@vger.kernel.org <linux-fsdevel@vger.kernel.org>; rust-for-linux@vger.kernel.org <rust-for-linux@vger.kernel.org>; linux-mm@kvack.org <linux-mm@kvack.org>; Alice Ryhl <aliceryhl@google.com>
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
On Fri, Jun 9, 2023 at 1:48 PM Christian Brauner <brauner@kernel.org> wrote:
>
> Because the series you sent here touches on a lot of things in terms of
> infrastructure alone. That work could very well be rather interesting
> independent of PuzzleFS. We might just want to get enough infrastructure
> to start porting a tiny existing fs (binderfs or something similar
> small) to Rust to see how feasible this is and to wet our appetite for
> bigger changes such as accepting a new filesystem driver completely
> written in Rust.
That would be great, thanks Christian! (Cc'ing Alice for binderfs -- I
think Rust Binder is keeping binderfs in C for the moment, but if you
are willing to try things, they are probably interested :)
Ariel: sorry, we crossed messages; I didn't receive your message at
[1], the rust-for-linux list probably dropped it due to the included
HTML.
[1] https://lore.kernel.org/linux-mm/CH0PR11MB529981313ED5A1F815350E41CD51A@CH0PR11MB5299.namprd11.prod.outlook.com/
Cheers,
Miguel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 11:45 ` Christian Brauner
2023-06-09 12:03 ` Ariel Miculas (amiculas)
[not found] ` <CANiq72nAcGKBVcVLrfAOkqaKsfftV6D1u97wqNxT38JnNsKp5A@mail.gmail.com>
@ 2023-06-09 12:20 ` Colin Walters
2023-06-09 12:42 ` Christian Brauner
2023-06-09 13:45 ` Ariel Miculas (amiculas)
2 siblings, 2 replies; 28+ messages in thread
From: Colin Walters @ 2023-06-09 12:20 UTC (permalink / raw)
To: Christian Brauner, Ariel Miculas (amiculas)
Cc: linux-fsdevel, rust-for-linux, linux-mm
On Fri, Jun 9, 2023, at 7:45 AM, Christian Brauner wrote:
>
> Because the series you sent here touches on a lot of things in terms of
> infrastructure alone. That work could very well be rather interesting
> independent of PuzzleFS. We might just want to get enough infrastructure
> to start porting a tiny existing fs (binderfs or something similar
> small) to Rust to see how feasible this is and to wet our appetite for
> bigger changes such as accepting a new filesystem driver completely
> written in Rust.
(Not a kernel developer, but this argument makes sense to me)
> But aside from the infrastructure discussion:
>
> This is yet another filesystem for solving the container image problem
> in the kernel with the addition of yet another filesystem. We just went
> through this excercise with another filesystem. So I'd expect some
> reluctance here. Tbh, the container world keeps sending us filesystems
> at an alarming rate. That's two within a few months and that leaves a
> rather disorganized impression.
I am sure you are aware there's not some "container world" monoculture, there are many organizations, people and companies here with some healthy co-opetition but also some duplication inherent from that.
That said at a practical level, Ariel in the https://github.com/containers GH organization we're kind of a "big tent" place. A subset of the organization is very heavily Rust oriented now (certainly the parts I touch) and briefly skimming the puzzlefs code, there are definitely some bits of code we could consider sharing in userspace. Actually though since this isn't releated to the in-kernel discussion I'll file an issue on Github and we can discuss there.
But there is definitely a subset of the discussion that Christian is referring to here that is about the intersections/overlap with the composefs approach that is relevant for this list. Maybe we could try to collaborate on an unbiased "puzzlefs vs composefs" document? (What's in https://github.com/anuvu/puzzlefs/tree/master/doc is a bit sparse right now)
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 12:11 ` Ariel Miculas (amiculas)
@ 2023-06-09 12:21 ` Greg KH
0 siblings, 0 replies; 28+ messages in thread
From: Greg KH @ 2023-06-09 12:21 UTC (permalink / raw)
To: Ariel Miculas (amiculas)
Cc: Miguel Ojeda, Christian Brauner, linux-fsdevel, rust-for-linux,
linux-mm, Alice Ryhl
On Fri, Jun 09, 2023 at 12:11:14PM +0000, Ariel Miculas (amiculas) wrote:
> Sorry about that, it seems like I need to switch to plain text mode for every reply in outlook, which is annoying.
You should also turn off the top-posting mode, as that's not good
etiquette on the kernel mailing lists for obvious reasons :)
thanks,
greg k-h
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 12:20 ` Colin Walters
@ 2023-06-09 12:42 ` Christian Brauner
2023-06-09 17:28 ` Serge Hallyn
2023-06-09 13:45 ` Ariel Miculas (amiculas)
1 sibling, 1 reply; 28+ messages in thread
From: Christian Brauner @ 2023-06-09 12:42 UTC (permalink / raw)
To: Colin Walters
Cc: Ariel Miculas (amiculas), linux-fsdevel, rust-for-linux, linux-mm
On Fri, Jun 09, 2023 at 08:20:30AM -0400, Colin Walters wrote:
>
>
> On Fri, Jun 9, 2023, at 7:45 AM, Christian Brauner wrote:
> >
> > Because the series you sent here touches on a lot of things in terms of
> > infrastructure alone. That work could very well be rather interesting
> > independent of PuzzleFS. We might just want to get enough infrastructure
> > to start porting a tiny existing fs (binderfs or something similar
> > small) to Rust to see how feasible this is and to wet our appetite for
> > bigger changes such as accepting a new filesystem driver completely
> > written in Rust.
>
> (Not a kernel developer, but this argument makes sense to me)
>
> > But aside from the infrastructure discussion:
> >
> > This is yet another filesystem for solving the container image problem
> > in the kernel with the addition of yet another filesystem. We just went
> > through this excercise with another filesystem. So I'd expect some
> > reluctance here. Tbh, the container world keeps sending us filesystems
> > at an alarming rate. That's two within a few months and that leaves a
> > rather disorganized impression.
>
> I am sure you are aware there's not some "container world"
> monoculture, there are many organizations, people and companies here
That submission here explicitly references OCI v2. Composefs explicitly
advertises 100% compatibility with OCI. So, there's a set of OCI specs
including runtime and image. As far as I'm concerned you're all one
container world under the OCI umbrella.
We're not going to push multiple filesystems into the kernel that all do
slightly different things but all serve the OCI container world and use
some spec as an argument to move stuff into the kernel.
The OCI initiative is hailed as unifying the container ecosystem. Hence,
we can expect coordination.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 12:03 ` Ariel Miculas (amiculas)
@ 2023-06-09 12:56 ` Gao Xiang
0 siblings, 0 replies; 28+ messages in thread
From: Gao Xiang @ 2023-06-09 12:56 UTC (permalink / raw)
To: Ariel Miculas (amiculas), Christian Brauner
Cc: linux-fsdevel, rust-for-linux, linux-mm, Serge Hallyn (shallyn),
Colin Walters
On 2023/6/9 20:03, Ariel Miculas (amiculas) wrote:
...
>
> But aside from the infrastructure discussion:
>
> This is yet another filesystem for solving the container image problem
> in the kernel with the addition of yet another filesystem. We just went
> through this excercise with another filesystem. So I'd expect some
> reluctance here. Tbh, the container world keeps sending us filesystems
> at an alarming rate. That's two within a few months and that leaves a
> rather disorganized impression.
Just a head up. Since Rust kernel infrastructure is too premature,
it's impossible to handle page cache / iomap and many useful stuffs.
In the long term, at least (someday) after Rust infrastructure is
mature, I will implement EROFS ino Rust as a try as well.
As for chunk CDC, I don't see it's hard (since we already have some
CDC approach since Linux v6.1) but as an effective disk filesystem
for performance, EROFS on-disk data is all block-aligned to match
storage and page cache alignment. If it's really needed, I could
update a more complete (but ineffective and slow) index version to
implement unaligned extents (both for decoded and encoded sides).
Yet I really think the main purpose of a kernel filesystem is to
make full use of kernel infrastructure for performance (like page
cache handling) otherwise a FUSE approach is enough.
Finally, as for OCI container image stuffs, I'd like to avoid
saying this topic anymore on the list (too tired about this). I've
seen _three_ in-kernel approaches already before this one and I tend
to avoid listing the complete names (including FUSE alternatives)
here. I really suggest if you guys could sit down and plan at least
a complete OCI standard for the next image format (even you don't
want to reuse any exist filesystem for whatever reasons).
Thanks,
Gao Xiang
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
[not found] ` <CANiq72nAcGKBVcVLrfAOkqaKsfftV6D1u97wqNxT38JnNsKp5A@mail.gmail.com>
2023-06-09 12:11 ` Ariel Miculas (amiculas)
@ 2023-06-09 13:05 ` Alice Ryhl
1 sibling, 0 replies; 28+ messages in thread
From: Alice Ryhl @ 2023-06-09 13:05 UTC (permalink / raw)
To: miguel.ojeda.sandonis, brauner
Cc: amiculas, linux-fsdevel, linux-mm, rust-for-linux, aliceryhl
Miguel Ojeda writes:
> That would be great, thanks Christian! (Cc'ing Alice for binderfs -- I
> think Rust Binder is keeping binderfs in C for the moment, but if you
> are willing to try things, they are probably interested :)
Yeah, Rust binder already needs bindings to a lot of other parts of the
kernel, so I decided to not rewrite the binderfs part for now to cut
down on the number of subsystems I would need to upstream bindings for.
Upstreaming bindings has proved to be a lot of work.
If someone else wants to upstream the filesystem bindings that binderfs
would need, then we can certainly look into using them in Rust binder.
Alice
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 12:20 ` Colin Walters
2023-06-09 12:42 ` Christian Brauner
@ 2023-06-09 13:45 ` Ariel Miculas (amiculas)
2023-06-09 17:10 ` Trilok Soni
1 sibling, 1 reply; 28+ messages in thread
From: Ariel Miculas (amiculas) @ 2023-06-09 13:45 UTC (permalink / raw)
To: Colin Walters, Christian Brauner; +Cc: linux-fsdevel, rust-for-linux, linux-mm
A "puzzlefs vs composefs" document sounds like a good idea. The documentation in puzzlefs is a little outdated and could be improved.
Feel free to create a github issue and tag me in there.
PS: as soon as I figure out how to turn off the top-posting mode, I'll do it.
Regards,
Ariel
________________________________________
From: Colin Walters <walters@verbum.org>
Sent: Friday, June 9, 2023 3:20 PM
To: Christian Brauner; Ariel Miculas (amiculas)
Cc: linux-fsdevel@vger.kernel.org; rust-for-linux@vger.kernel.org; linux-mm@kvack.org
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
On Fri, Jun 9, 2023, at 7:45 AM, Christian Brauner wrote:
>
> Because the series you sent here touches on a lot of things in terms of
> infrastructure alone. That work could very well be rather interesting
> independent of PuzzleFS. We might just want to get enough infrastructure
> to start porting a tiny existing fs (binderfs or something similar
> small) to Rust to see how feasible this is and to wet our appetite for
> bigger changes such as accepting a new filesystem driver completely
> written in Rust.
(Not a kernel developer, but this argument makes sense to me)
> But aside from the infrastructure discussion:
>
> This is yet another filesystem for solving the container image problem
> in the kernel with the addition of yet another filesystem. We just went
> through this excercise with another filesystem. So I'd expect some
> reluctance here. Tbh, the container world keeps sending us filesystems
> at an alarming rate. That's two within a few months and that leaves a
> rather disorganized impression.
I am sure you are aware there's not some "container world" monoculture, there are many organizations, people and companies here with some healthy co-opetition but also some duplication inherent from that.
That said at a practical level, Ariel in the https://github.com/containers GH organization we're kind of a "big tent" place. A subset of the organization is very heavily Rust oriented now (certainly the parts I touch) and briefly skimming the puzzlefs code, there are definitely some bits of code we could consider sharing in userspace. Actually though since this isn't releated to the in-kernel discussion I'll file an issue on Github and we can discuss there.
But there is definitely a subset of the discussion that Christian is referring to here that is about the intersections/overlap with the composefs approach that is relevant for this list. Maybe we could try to collaborate on an unbiased "puzzlefs vs composefs" document? (What's in https://github.com/anuvu/puzzlefs/tree/master/doc is a bit sparse right now)
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 13:45 ` Ariel Miculas (amiculas)
@ 2023-06-09 17:10 ` Trilok Soni
2023-06-09 17:16 ` Ariel Miculas (amiculas)
0 siblings, 1 reply; 28+ messages in thread
From: Trilok Soni @ 2023-06-09 17:10 UTC (permalink / raw)
To: Ariel Miculas (amiculas), Colin Walters, Christian Brauner
Cc: linux-fsdevel, rust-for-linux, linux-mm
On 6/9/2023 6:45 AM, Ariel Miculas (amiculas) wrote:
> A "puzzlefs vs composefs" document sounds like a good idea. The documentation in puzzlefs is a little outdated and could be improved.
> Feel free to create a github issue and tag me in there.
>
> PS: as soon as I figure out how to turn off the top-posting mode, I'll do it.
>
Let me know as well if you could do w/ Outlook :). Switch to other email
clients if possible.
---Trilok Soni
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 17:10 ` Trilok Soni
@ 2023-06-09 17:16 ` Ariel Miculas (amiculas)
2023-06-09 17:41 ` Miguel Ojeda
2023-06-09 18:43 ` James Bottomley
0 siblings, 2 replies; 28+ messages in thread
From: Ariel Miculas (amiculas) @ 2023-06-09 17:16 UTC (permalink / raw)
To: Trilok Soni, Colin Walters, Christian Brauner
Cc: linux-fsdevel, rust-for-linux, linux-mm
I could switch to my personal gmail, but last time Miguel Ojeda asked me to use my cisco email when I send commits signed off by amiculas@cisco.com.
If this is not a hard requirement, then I could switch.
Regards,
Ariel
________________________________________
From: Trilok Soni <quic_tsoni@quicinc.com>
Sent: Friday, June 9, 2023 8:10 PM
To: Ariel Miculas (amiculas); Colin Walters; Christian Brauner
Cc: linux-fsdevel@vger.kernel.org; rust-for-linux@vger.kernel.org; linux-mm@kvack.org
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
On 6/9/2023 6:45 AM, Ariel Miculas (amiculas) wrote:
> A "puzzlefs vs composefs" document sounds like a good idea. The documentation in puzzlefs is a little outdated and could be improved.
> Feel free to create a github issue and tag me in there.
>
> PS: as soon as I figure out how to turn off the top-posting mode, I'll do it.
>
Let me know as well if you could do w/ Outlook :). Switch to other email
clients if possible.
---Trilok Soni
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 12:42 ` Christian Brauner
@ 2023-06-09 17:28 ` Serge Hallyn
0 siblings, 0 replies; 28+ messages in thread
From: Serge Hallyn @ 2023-06-09 17:28 UTC (permalink / raw)
To: Christian Brauner
Cc: Colin Walters, Ariel Miculas (amiculas),
linux-fsdevel, rust-for-linux, linux-mm
On Fri, Jun 09, 2023 at 02:42:47PM +0200, Christian Brauner wrote:
> On Fri, Jun 09, 2023 at 08:20:30AM -0400, Colin Walters wrote:
> >
> >
> > On Fri, Jun 9, 2023, at 7:45 AM, Christian Brauner wrote:
> > >
> > > Because the series you sent here touches on a lot of things in terms of
> > > infrastructure alone. That work could very well be rather interesting
> > > independent of PuzzleFS. We might just want to get enough infrastructure
> > > to start porting a tiny existing fs (binderfs or something similar
> > > small) to Rust to see how feasible this is and to wet our appetite for
> > > bigger changes such as accepting a new filesystem driver completely
> > > written in Rust.
> >
> > (Not a kernel developer, but this argument makes sense to me)
> >
> > > But aside from the infrastructure discussion:
> > >
> > > This is yet another filesystem for solving the container image problem
> > > in the kernel with the addition of yet another filesystem. We just went
> > > through this excercise with another filesystem. So I'd expect some
> > > reluctance here. Tbh, the container world keeps sending us filesystems
> > > at an alarming rate. That's two within a few months and that leaves a
> > > rather disorganized impression.
> >
> > I am sure you are aware there's not some "container world"
> > monoculture, there are many organizations, people and companies here
>
> That submission here explicitly references OCI v2. Composefs explicitly
> advertises 100% compatibility with OCI. So, there's a set of OCI specs
OCI v2 doesn't currently exist :) There were many design goals for
it, and near as I can tell, after everyone discussed those together,
everyone went off to work on implementing the bits the needed - which
is a right and proper step before coming back and comparing notes
about what went well, etc.
The two main things where puzzlefs is experimenting are the use of
content defined chunking (CDC), and being written in rust (and,
especially, to have the same rust code base be used for the in kernel
driver, the fuse mounter, the builder, and the userspace extractor).
(Well, and reproducible images through a canonical representation,
but...)
It requires a POC in order to really determine whether the CDC will
be worth it, or will have pitfalls. So far, it looks very promising.
Adding that functionality to composefs one day could be cool.
Likewise, it requires a user in order to push on the infrastructure
required to support a full filesystem in rust in the kernel. But that
really isn't something we can "add to composefs". :)
The main goal of this posting, then, was to show the infrastructure
pieces and work on that with the community. We're definitely not
(currently :) asking for puzzlefs to be included. However, it is
our (business and personal) justification for the rest of the work.
> including runtime and image. As far as I'm concerned you're all one
> container world under the OCI umbrella.
One big happy family!
> We're not going to push multiple filesystems into the kernel that all do
> slightly different things but all serve the OCI container world and use
> some spec as an argument to move stuff into the kernel.
>
> The OCI initiative is hailed as unifying the container ecosystem. Hence,
> we can expect coordination.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 17:16 ` Ariel Miculas (amiculas)
@ 2023-06-09 17:41 ` Miguel Ojeda
2023-06-09 18:49 ` James Bottomley
2023-06-09 18:43 ` James Bottomley
1 sibling, 1 reply; 28+ messages in thread
From: Miguel Ojeda @ 2023-06-09 17:41 UTC (permalink / raw)
To: Ariel Miculas (amiculas)
Cc: Trilok Soni, Colin Walters, Christian Brauner, linux-fsdevel,
rust-for-linux, linux-mm
On Fri, Jun 9, 2023 at 7:25 PM Ariel Miculas (amiculas)
<amiculas@cisco.com> wrote:
>
> I could switch to my personal gmail, but last time Miguel Ojeda asked me to use my cisco email when I send commits signed off by amiculas@cisco.com.
> If this is not a hard requirement, then I could switch.
For patches, yeah, that is ideal, so that it matches the Git author / `From:`.
But for the other emails, you could use your personal address, if that
makes things easier.
Hope that helps!
Cheers,
Miguel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 17:16 ` Ariel Miculas (amiculas)
2023-06-09 17:41 ` Miguel Ojeda
@ 2023-06-09 18:43 ` James Bottomley
2023-06-09 18:59 ` Ariel Miculas (amiculas)
1 sibling, 1 reply; 28+ messages in thread
From: James Bottomley @ 2023-06-09 18:43 UTC (permalink / raw)
To: Ariel Miculas (amiculas), Trilok Soni, Colin Walters, Christian Brauner
Cc: linux-fsdevel, rust-for-linux, linux-mm
On Fri, 2023-06-09 at 17:16 +0000, Ariel Miculas (amiculas) wrote:
> I could switch to my personal gmail, but last time Miguel Ojeda asked
> me to use my cisco email when I send commits signed off by
> amiculas@cisco.com.
> If this is not a hard requirement, then I could switch.
For sending patches, you can simply use git-send-email. All you need
to point it at is the outgoing email server (which should be a config
setting in whatever tool you are using now). We have a (reasonably) up
to date document with some recommendations:
https://www.kernel.org/doc/html/latest/process/email-clients.html
I've successfully used evolution with an exchange server for many
years, but the interface isn't to everyone's taste and Mozilla
Thunderbird is also known to connect to it. Basic outlook has proven
impossible to configure correctly (which is why it doesn't have an
entry).
Regards,
James
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 17:41 ` Miguel Ojeda
@ 2023-06-09 18:49 ` James Bottomley
2023-06-09 19:08 ` Miguel Ojeda
2023-06-09 19:11 ` Ariel Miculas
0 siblings, 2 replies; 28+ messages in thread
From: James Bottomley @ 2023-06-09 18:49 UTC (permalink / raw)
To: Miguel Ojeda, Ariel Miculas (amiculas)
Cc: Trilok Soni, Colin Walters, Christian Brauner, linux-fsdevel,
rust-for-linux, linux-mm
On Fri, 2023-06-09 at 19:41 +0200, Miguel Ojeda wrote:
> On Fri, Jun 9, 2023 at 7:25 PM Ariel Miculas (amiculas)
> <amiculas@cisco.com> wrote:
> >
> > I could switch to my personal gmail, but last time Miguel Ojeda
> > asked me to use my cisco email when I send commits signed off by
> > amiculas@cisco.com. If this is not a hard requirement, then I could
> > switch.
>
> For patches, yeah, that is ideal, so that it matches the Git author /
> `From:`.
>
> But for the other emails, you could use your personal address, if
> that makes things easier.
It's still not a requirement, though. You can send from your gmail
account and still have
From: Ariel Miculas <amiculas@cisco.com>
As the first line (separated from the commit message by a blank line),
which git am (or b4) will pick up as the author email. This behaviour
is specifically for people who want the author to be their corporate
email address, but have failed to persuade corporate IT to make it
possible.
Regards,
James
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 18:43 ` James Bottomley
@ 2023-06-09 18:59 ` Ariel Miculas (amiculas)
2023-06-09 19:20 ` Ariel Miculas
2023-06-09 19:53 ` Alice Ryhl
0 siblings, 2 replies; 28+ messages in thread
From: Ariel Miculas (amiculas) @ 2023-06-09 18:59 UTC (permalink / raw)
To: James Bottomley, Trilok Soni, Colin Walters, Christian Brauner
Cc: linux-fsdevel, rust-for-linux, linux-mm
I did use git send-email for sending this patch series, but I cannot find any setting in the Outlook web client for disabling "top posting" when replying to emails:
https://answers.microsoft.com/en-us/outlook_com/forum/all/eliminate-top-posting/5e1e5729-30f8-41e9-84cb-fb5e81229c7c
Regards,
Ariel
________________________________________
From: James Bottomley <James.Bottomley@HansenPartnership.com>
Sent: Friday, June 9, 2023 9:43 PM
To: Ariel Miculas (amiculas); Trilok Soni; Colin Walters; Christian Brauner
Cc: linux-fsdevel@vger.kernel.org; rust-for-linux@vger.kernel.org; linux-mm@kvack.org
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
On Fri, 2023-06-09 at 17:16 +0000, Ariel Miculas (amiculas) wrote:
> I could switch to my personal gmail, but last time Miguel Ojeda asked
> me to use my cisco email when I send commits signed off by
> amiculas@cisco.com.
> If this is not a hard requirement, then I could switch.
For sending patches, you can simply use git-send-email. All you need
to point it at is the outgoing email server (which should be a config
setting in whatever tool you are using now). We have a (reasonably) up
to date document with some recommendations:
https://www.kernel.org/doc/html/latest/process/email-clients.html
I've successfully used evolution with an exchange server for many
years, but the interface isn't to everyone's taste and Mozilla
Thunderbird is also known to connect to it. Basic outlook has proven
impossible to configure correctly (which is why it doesn't have an
entry).
Regards,
James
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 18:49 ` James Bottomley
@ 2023-06-09 19:08 ` Miguel Ojeda
2023-06-09 19:11 ` Ariel Miculas
1 sibling, 0 replies; 28+ messages in thread
From: Miguel Ojeda @ 2023-06-09 19:08 UTC (permalink / raw)
To: James Bottomley
Cc: Ariel Miculas (amiculas),
Trilok Soni, Colin Walters, Christian Brauner, linux-fsdevel,
rust-for-linux, linux-mm
On Fri, Jun 9, 2023 at 8:49 PM James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
>
> On Fri, 2023-06-09 at 19:41 +0200, Miguel Ojeda wrote:
> > For patches, yeah, that is ideal, so that it matches the Git author /
> > `From:`.
>
> It's still not a requirement, though. You can send from your gmail
> account and still have
Yeah, that is what I said "ideal". When Ariel sent the first patch, he
didn't use the `From:` tag within the email body, but attributed it to
Cisco, and I had not seen any commit/email from his Cisco address yet,
so I asked him if he could use his corporate address.
Cheers,
Miguel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 18:49 ` James Bottomley
2023-06-09 19:08 ` Miguel Ojeda
@ 2023-06-09 19:11 ` Ariel Miculas
2023-06-09 20:01 ` James Bottomley
2023-06-10 9:34 ` Miguel Ojeda
1 sibling, 2 replies; 28+ messages in thread
From: Ariel Miculas @ 2023-06-09 19:11 UTC (permalink / raw)
To: James Bottomley
Cc: Miguel Ojeda, Ariel Miculas (amiculas),
Trilok Soni, Colin Walters, Christian Brauner, linux-fsdevel,
rust-for-linux, linux-mm
Yes, but then how can you be sure that amiculas@cisco.com is the real
author of the commit? I think that's why Miguel Ojeda asked me to send
them from my business email, otherwise some random gmail account could
claim that he is "Ariel Miculas", so he's entitled to sign-off as
amiculas@cisco.com.
Regards,
Ariel
On Fri, Jun 9, 2023 at 10:03 PM James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
>
> On Fri, 2023-06-09 at 19:41 +0200, Miguel Ojeda wrote:
> > On Fri, Jun 9, 2023 at 7:25 PM Ariel Miculas (amiculas)
> > <amiculas@cisco.com> wrote:
> > >
> > > I could switch to my personal gmail, but last time Miguel Ojeda
> > > asked me to use my cisco email when I send commits signed off by
> > > amiculas@cisco.com. If this is not a hard requirement, then I could
> > > switch.
> >
> > For patches, yeah, that is ideal, so that it matches the Git author /
> > `From:`.
> >
> > But for the other emails, you could use your personal address, if
> > that makes things easier.
>
> It's still not a requirement, though. You can send from your gmail
> account and still have
>
> From: Ariel Miculas <amiculas@cisco.com>
>
> As the first line (separated from the commit message by a blank line),
> which git am (or b4) will pick up as the author email. This behaviour
> is specifically for people who want the author to be their corporate
> email address, but have failed to persuade corporate IT to make it
> possible.
>
> Regards,
>
> James
>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 18:59 ` Ariel Miculas (amiculas)
@ 2023-06-09 19:20 ` Ariel Miculas
2023-06-09 19:45 ` Trilok Soni
2023-06-09 19:53 ` Alice Ryhl
1 sibling, 1 reply; 28+ messages in thread
From: Ariel Miculas @ 2023-06-09 19:20 UTC (permalink / raw)
To: Ariel Miculas (amiculas)
Cc: James Bottomley, Trilok Soni, Colin Walters, Christian Brauner,
linux-fsdevel, rust-for-linux, linux-mm
Now if I could figure out how to disable "top posting" in gmail...
Regards,
Ariel
On Fri, Jun 9, 2023 at 10:06 PM Ariel Miculas (amiculas)
<amiculas@cisco.com> wrote:
>
> I did use git send-email for sending this patch series, but I cannot find any setting in the Outlook web client for disabling "top posting" when replying to emails:
> https://answers.microsoft.com/en-us/outlook_com/forum/all/eliminate-top-posting/5e1e5729-30f8-41e9-84cb-fb5e81229c7c
>
> Regards,
> Ariel
>
> ________________________________________
> From: James Bottomley <James.Bottomley@HansenPartnership.com>
> Sent: Friday, June 9, 2023 9:43 PM
> To: Ariel Miculas (amiculas); Trilok Soni; Colin Walters; Christian Brauner
> Cc: linux-fsdevel@vger.kernel.org; rust-for-linux@vger.kernel.org; linux-mm@kvack.org
> Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
>
> On Fri, 2023-06-09 at 17:16 +0000, Ariel Miculas (amiculas) wrote:
> > I could switch to my personal gmail, but last time Miguel Ojeda asked
> > me to use my cisco email when I send commits signed off by
> > amiculas@cisco.com.
> > If this is not a hard requirement, then I could switch.
>
> For sending patches, you can simply use git-send-email. All you need
> to point it at is the outgoing email server (which should be a config
> setting in whatever tool you are using now). We have a (reasonably) up
> to date document with some recommendations:
>
> https://www.kernel.org/doc/html/latest/process/email-clients.html
>
> I've successfully used evolution with an exchange server for many
> years, but the interface isn't to everyone's taste and Mozilla
> Thunderbird is also known to connect to it. Basic outlook has proven
> impossible to configure correctly (which is why it doesn't have an
> entry).
>
> Regards,
>
> James
>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 19:20 ` Ariel Miculas
@ 2023-06-09 19:45 ` Trilok Soni
0 siblings, 0 replies; 28+ messages in thread
From: Trilok Soni @ 2023-06-09 19:45 UTC (permalink / raw)
To: Ariel Miculas, Ariel Miculas (amiculas)
Cc: James Bottomley, Colin Walters, Christian Brauner, linux-fsdevel,
rust-for-linux, linux-mm
On 6/9/2023 12:20 PM, Ariel Miculas wrote:
> Now if I could figure out how to disable "top posting" in gmail...
>
> Regards,
> Ariel
>
> On Fri, Jun 9, 2023 at 10:06 PM Ariel Miculas (amiculas)
> <amiculas@cisco.com> wrote:
>>
>> I did use git send-email for sending this patch series, but I cannot find any setting in the Outlook web client for disabling "top posting" when replying to emails:
>> https://answers.microsoft.com/en-us/outlook_com/forum/all/eliminate-top-posting/5e1e5729-30f8-41e9-84cb-fb5e81229c7c
>>
>> Regards,
>> Ariel
>>
>> ________________________________________
>> From: James Bottomley <James.Bottomley@HansenPartnership.com>
>> Sent: Friday, June 9, 2023 9:43 PM
>> To: Ariel Miculas (amiculas); Trilok Soni; Colin Walters; Christian Brauner
>> Cc: linux-fsdevel@vger.kernel.org; rust-for-linux@vger.kernel.org; linux-mm@kvack.org
>> Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
>>
>> On Fri, 2023-06-09 at 17:16 +0000, Ariel Miculas (amiculas) wrote:
>>> I could switch to my personal gmail, but last time Miguel Ojeda asked
>>> me to use my cisco email when I send commits signed off by
>>> amiculas@cisco.com.
>>> If this is not a hard requirement, then I could switch.
>>
>> For sending patches, you can simply use git-send-email. All you need
>> to point it at is the outgoing email server (which should be a config
>> setting in whatever tool you are using now). We have a (reasonably) up
>> to date document with some recommendations:
>>
>> https://www.kernel.org/doc/html/latest/process/email-clients.html
>>
>> I've successfully used evolution with an exchange server for many
>> years, but the interface isn't to everyone's taste and Mozilla
>> Thunderbird is also known to connect to it. Basic outlook has proven
>> impossible to configure correctly (which is why it doesn't have an
>> entry).
One of the big reasons why I have quic_tsoni at quicinc dot com
(Thunderbird/Mutt/whateveryoulike except Outlook) on top of tsoni at
quicinc dot com (Outlook) to make sure that we comply w/ upstream
guidelines. Two email IDs are hard to manage but it gives
the good separation and freedom.
---Trilok Soni
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 18:59 ` Ariel Miculas (amiculas)
2023-06-09 19:20 ` Ariel Miculas
@ 2023-06-09 19:53 ` Alice Ryhl
1 sibling, 0 replies; 28+ messages in thread
From: Alice Ryhl @ 2023-06-09 19:53 UTC (permalink / raw)
To: Ariel Miculas (amiculas),
James Bottomley, Trilok Soni, Colin Walters, Christian Brauner
Cc: linux-fsdevel, rust-for-linux, linux-mm
On 6/9/23 20:59, Ariel Miculas (amiculas) wrote:
> I did use git send-email for sending this patch series, but I cannot find any setting in the Outlook web client for disabling "top posting" when replying to emails:
> https://answers.microsoft.com/en-us/outlook_com/forum/all/eliminate-top-posting/5e1e5729-30f8-41e9-84cb-fb5e81229c7c
>
> Regards,
> Ariel
You can also use git-send-email for sending replies. Just make a .txt
file with this format:
Subject: <subject goes here>
<your reply after an empty line>
The lore.kernel.org site generates the appropriate git-send-email
command for sending the reply, so you can copy-paste that to send it.
I send most of my replies like that.
> From: James Bottomley <James.Bottomley@HansenPartnership.com>
> Sent: Friday, June 9, 2023 9:43 PM
> To: Ariel Miculas (amiculas); Trilok Soni; Colin Walters; Christian Brauner
> Cc: linux-fsdevel@vger.kernel.org; rust-for-linux@vger.kernel.org; linux-mm@kvack.org
> Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
>
> On Fri, 2023-06-09 at 17:16 +0000, Ariel Miculas (amiculas) wrote:
>> I could switch to my personal gmail, but last time Miguel Ojeda asked
>> me to use my cisco email when I send commits signed off by
>> amiculas@cisco.com.
>> If this is not a hard requirement, then I could switch.
>
> For sending patches, you can simply use git-send-email. All you need
> to point it at is the outgoing email server (which should be a config
> setting in whatever tool you are using now). We have a (reasonably) up
> to date document with some recommendations:
>
> https://www.kernel.org/doc/html/latest/process/email-clients.html
>
> I've successfully used evolution with an exchange server for many
> years, but the interface isn't to everyone's taste and Mozilla
> Thunderbird is also known to connect to it. Basic outlook has proven
> impossible to configure correctly (which is why it doesn't have an
> entry).
>
> Regards,
>
> James
>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 19:11 ` Ariel Miculas
@ 2023-06-09 20:01 ` James Bottomley
2023-06-10 9:34 ` Miguel Ojeda
1 sibling, 0 replies; 28+ messages in thread
From: James Bottomley @ 2023-06-09 20:01 UTC (permalink / raw)
To: Ariel Miculas
Cc: Miguel Ojeda, Ariel Miculas (amiculas),
Trilok Soni, Colin Walters, Christian Brauner, linux-fsdevel,
rust-for-linux, linux-mm
On Fri, 2023-06-09 at 22:11 +0300, Ariel Miculas wrote:
> Yes, but then how can you be sure that amiculas@cisco.com is the real
> author of the commit? I think that's why Miguel Ojeda asked me to
> send them from my business email, otherwise some random gmail account
> could claim that he is "Ariel Miculas", so he's entitled to sign-off
> as amiculas@cisco.com.
Because it's a public list and the real one would observe and repudiate
...
The DCO is an attestation which we basically believe absent any proof
to the contrary. It's also exhausting and an incredible amount of
pointless admin not to believe anything until it's proved, which is why
we don't.
James
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 10:36 ` [RFC PATCH 00/80] Rust PuzzleFS filesystem driver Christian Brauner
2023-06-09 11:22 ` Ariel Miculas (amiculas)
2023-06-09 11:42 ` Miguel Ojeda
@ 2023-06-09 23:52 ` Kent Overstreet
2023-06-10 9:40 ` Miguel Ojeda
2 siblings, 1 reply; 28+ messages in thread
From: Kent Overstreet @ 2023-06-09 23:52 UTC (permalink / raw)
To: Christian Brauner; +Cc: Ariel Miculas, linux-fsdevel, rust-for-linux, linux-mm
On Fri, Jun 09, 2023 at 12:36:29PM +0200, Christian Brauner wrote:
> On Fri, Jun 09, 2023 at 09:29:58AM +0300, Ariel Miculas wrote:
> > Hi all!
> >
> > This is a proof of concept driver written for the PuzzleFS
>
> Uhm, the PuzzleFS filesystem isn't actually sent to fsdevel? Yet I see
> tons of patches in there that add wrappers to our core fs data
> structures. I even see a ton of new files that all fall clearly into
> fsdevel territory:
>
> create mode 100644 rust/kernel/cred.rs
> create mode 100644 rust/kernel/delay.rs
> create mode 100644 rust/kernel/driver.rs
> create mode 100644 rust/kernel/file.rs
> create mode 100644 rust/kernel/fs.rs
> create mode 100644 rust/kernel/fs/param.rs
> create mode 100644 rust/kernel/io_buffer.rs
> create mode 100644 rust/kernel/iov_iter.rs
> create mode 100644 rust/kernel/mm.rs
> create mode 100644 rust/kernel/mount.rs
> create mode 100644 rust/kernel/pages.rs
>
> There's also quite a lot of new mm/ in there, no?
>
> Any wrappers and code for core fs should be maintained as part of fs.
> Rust shouldn't become a way to avoid our reviews once you have a few
> wrappers added somewhere.
I'm of the opinion that Rust wrappers should generally live in the same
directories as the code they're wrapping. The Rust wrappers are unsafe
code that is intimately coupled with the C code they're creating safe
interfaces for, and the organizational structure should reflect that.
This'll be particularly important when mixing Rust and C in the same
module, as I intend to do in bcachefs in the not-too-distant future.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 19:11 ` Ariel Miculas
2023-06-09 20:01 ` James Bottomley
@ 2023-06-10 9:34 ` Miguel Ojeda
1 sibling, 0 replies; 28+ messages in thread
From: Miguel Ojeda @ 2023-06-10 9:34 UTC (permalink / raw)
To: Ariel Miculas
Cc: James Bottomley, Ariel Miculas (amiculas),
Trilok Soni, Colin Walters, Christian Brauner, linux-fsdevel,
rust-for-linux, linux-mm
On Fri, Jun 9, 2023 at 9:11 PM Ariel Miculas <ariel.miculas@gmail.com> wrote:
>
> Yes, but then how can you be sure that amiculas@cisco.com is the real
> author of the commit? I think that's why Miguel Ojeda asked me to send
> them from my business email, otherwise some random gmail account could
> claim that he is "Ariel Miculas", so he's entitled to sign-off as
> amiculas@cisco.com.
Yeah, that was the main thing; and by asking that, if you managed to
send that original patch from the corporate one, then there was no
need for the `From:` to begin with :)
Cheers,
Miguel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver
2023-06-09 23:52 ` Kent Overstreet
@ 2023-06-10 9:40 ` Miguel Ojeda
0 siblings, 0 replies; 28+ messages in thread
From: Miguel Ojeda @ 2023-06-10 9:40 UTC (permalink / raw)
To: Kent Overstreet
Cc: Christian Brauner, Ariel Miculas, linux-fsdevel, rust-for-linux,
linux-mm
On Sat, Jun 10, 2023 at 2:12 AM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> I'm of the opinion that Rust wrappers should generally live in the same
> directories as the code they're wrapping. The Rust wrappers are unsafe
> code that is intimately coupled with the C code they're creating safe
> interfaces for, and the organizational structure should reflect that.
+1, that is the plan, which also means `MAINTAINERS` does not need
extra `F` fields etc.
Cheers,
Miguel
^ permalink raw reply [flat|nested] 28+ messages in thread
end of thread, other threads:[~2023-06-10 9:41 UTC | newest]
Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20230609063118.24852-1-amiculas@cisco.com>
2023-06-09 10:36 ` [RFC PATCH 00/80] Rust PuzzleFS filesystem driver Christian Brauner
2023-06-09 11:22 ` Ariel Miculas (amiculas)
2023-06-09 11:45 ` Christian Brauner
2023-06-09 12:03 ` Ariel Miculas (amiculas)
2023-06-09 12:56 ` Gao Xiang
[not found] ` <CANiq72nAcGKBVcVLrfAOkqaKsfftV6D1u97wqNxT38JnNsKp5A@mail.gmail.com>
2023-06-09 12:11 ` Ariel Miculas (amiculas)
2023-06-09 12:21 ` Greg KH
2023-06-09 13:05 ` Alice Ryhl
2023-06-09 12:20 ` Colin Walters
2023-06-09 12:42 ` Christian Brauner
2023-06-09 17:28 ` Serge Hallyn
2023-06-09 13:45 ` Ariel Miculas (amiculas)
2023-06-09 17:10 ` Trilok Soni
2023-06-09 17:16 ` Ariel Miculas (amiculas)
2023-06-09 17:41 ` Miguel Ojeda
2023-06-09 18:49 ` James Bottomley
2023-06-09 19:08 ` Miguel Ojeda
2023-06-09 19:11 ` Ariel Miculas
2023-06-09 20:01 ` James Bottomley
2023-06-10 9:34 ` Miguel Ojeda
2023-06-09 18:43 ` James Bottomley
2023-06-09 18:59 ` Ariel Miculas (amiculas)
2023-06-09 19:20 ` Ariel Miculas
2023-06-09 19:45 ` Trilok Soni
2023-06-09 19:53 ` Alice Ryhl
2023-06-09 11:42 ` Miguel Ojeda
2023-06-09 23:52 ` Kent Overstreet
2023-06-10 9:40 ` Miguel Ojeda
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox