A virtual filesystem layer for WASI

Related tags

Filesystem webassembly vfs wasi
Overview

wasi-vfs

A virtual filesystem layer for WASI.

NOTICE: This project currently supports only WASI applications on the top of wasi-libc

This project provides a language and host-agnostic virtual filesystem layer for WASI.

Supported filesystems

  • Embedded file system: a read only file system embedded in the .wasm binary.
  • to be implemented more...

Installation

A typical installation from the release binaries might look like the following:

$ export WASI_VFS_VERSION=0.1.0
$ curl -LO "https://github.com/kateinoigakukun/wasi-vfs/releases/download/v${WASI_VFS_VERSION}/wasi-vfs-cli-x86_64-unknown-linux-gnu.zip"
$ unzip wasi-vfs-cli-x86_64-unknown-linux-gnu.zip
$ mv wasi-vfs /usr/local/bin/wasi-vfs

Building

To build the project, you need to install the wasi-sdk version 14.0 or later.

$ export WASI_SDK_PATH=/path/to/wasi-sdk
$ cargo build --target wasm32-unknown-unknown

Demo

Prerequisites

Set $WASI_SDK_PATH environment variable to the path of the wasi-sdk (version 14 or later).

Build and run WASI application

$ git clone https://github.com/kateinoigakukun/wasi-vfs.git
$ cd wasi-vfs

# Build libwasi_vfs.a
$ cargo build --target wasm32-unknown-unknown

# Build a WASI app with libwasi_vfs.a
$ $WASI_SDK_PATH/bin/clang -target wasm32-unknown-wasi -o getline.wasm examples/getline.c ./target/wasm32-unknown-unknown/debug/libwasi_vfs.a

# Run the WASI app with --mapdir
$ wasmtime run --mapdir /::./examples/mnt getline.wasm -- /hello.txt
Hello

# Pack ./examples/mnt directory into a WASM binary
$ cargo run -p wasi-vfs-cli -- pack getline.wasm --mapdir /::./examples/mnt -o getline.packed.wasm

# Run the WASM binary again without --mapdir
$ wasmtime run getline.packed.wasm -- /hello.txt
Hello

Testing

Unit tests

$ CARGO_TARGET_WASM32_WASI_RUNNER=wasmtime cargo test --target wasm32-wasi

End-to-end tests

$ cargo build --target wasm32-unknown-unknown
$ LIB_WASI_VFS_A=$PWD/target/wasm32-unknown-unknown/debug/libwasi_vfs.a ./tools/run-make-test.sh

How does it work?

wasi-vfs pack command is a wrapper of wizer, which is a pre-initializer for Wasm applications. The initialization process scans the mapped directories, then copies them into in-memory virtual filesystem.

Limitations

Currently, this project only supports WASI applications on the top of wasi-libc because of the following reasons:

This project depends on wasm-ld and wasi-libc's imported symbol behavior. wasi-libc declares some external symbols to import WASI functions in C like below. When __imported_wasi_snapshot_preview1_fd_read is not defined in any input object files, wasm-ld produces a (import "wasi_snapshot_preview1" "fd_read") entry. This is how wasi-libc calls WASI functions.

int32_t __imported_wasi_snapshot_preview1_fd_read(int32_t arg0, int32_t arg1, int32_t arg2, int32_t arg3) __attribute__((
    __import_module__("wasi_snapshot_preview1"),
    __import_name__("fd_read")
));

This project exploits that external symbols to hook WASI function calls by defining them in libwasi_vfs.a. If those symbols are defined, wasm-ld doesn't produce import entries, and it links symbols normally.

Therefore, this project currently doesn't support Rust application, which calls WASI functions directly without using wasi-libc.

After module-linking and interface-types will be merged, and WASI will adopt shared-nothing architecture, this project will be able to support all WASI applications.

You might also like...
Open Source Rust kernel; Runs WASM and WASI as lightweight containers.

馃槼 etheryal Kernel etheryal kernel is an Open Source capability-based Kernel written in the Rust programming language. The kernel allows implementing

null 32 Dec 4, 2022
ffmpeg libraries precompiled for WebAsembly/WASI, as a Rust crate.

FFMPEG crate for WebAssembly/WASI This crate bundles FFMPEG's libraries, precompiled for WebAssembly. No native installation required. Compatible with

Frank Denis 45 Dec 14, 2022
Let's combine wasi-nn and witx-bindgen and see how it goes!

WASI-NN Experiment (API Docs) Experiments with wasmtime, the wasi-nn proposal, and tract. Getting Started To use this experiment, you will first need

Hammer of the Gods 3 Nov 4, 2022
Todo application on Atmos (WebAssembly with WASI)

Get starting Install subo CLI at https://github.com/suborbital/subo Build todo app cd todos subo build . Run docker compose docker compose up Pl

Thang Chung 2 Dec 9, 2021
simple operating system written in Rust with wasi support.

Agave OS Agave OS is a simple operating system written in Rust with wasi support. It is designed to be lightweight and easy to use. It is also designe

Dean Srebnik 5 Jun 12, 2023
Cargo wrapper for working with Webassembly wasi(x).

cargo-wasix A cargo subcommand that wraps regular cargo commands for compiling Rust code to wasix, a superset of Websassembly wasi with additional fun

WASIX 19 Jun 12, 2023
A faster way to navigate your filesystem
A faster way to navigate your filesystem

zoxide A faster way to navigate your filesystem Table of contents Introduction Examples Getting started Step 1: Install zoxide Step 2: Install fzf (op

Ajeet D'Souza 8.7k Jan 8, 2023
Fuse filesystem that returns symlinks to executables based on the PATH of the requesting process.

Envfs Fuse filesystem that returns symlinks to executables based on the PATH of the requesting process. This is useful to execute shebangs on NixOS th

J枚rg Thalheim 98 Jan 2, 2023
The reference implementation of the Linux FUSE (Filesystem in Userspace) interface

libfuse About FUSE (Filesystem in Userspace) is an interface for userspace programs to export a filesystem to the Linux kernel. The FUSE project consi

null 4.2k Jan 4, 2023
rpg-cli 鈥攜our filesystem as a dungeon!
rpg-cli 鈥攜our filesystem as a dungeon!

rpg-cli is a bare-bones JRPG-inspired terminal game written in Rust. It can work as an alternative to cd where you randomly encounter enemies as you change directories.

Facundo Olano 1.2k Jan 4, 2023
fd is a program to find entries in your filesystem. It is a simple, fast and user-friendly alternative to find
fd is a program to find entries in your filesystem. It is a simple, fast and user-friendly alternative to find

fd is a program to find entries in your filesystem. It is a simple, fast and user-friendly alternative to find. While it does not aim to support all of find's powerful functionality, it provides sensible (opinionated) defaults for a majority of use cases.

David Peter 25.9k Jan 9, 2023
馃毀 (Alpha stage software) Binary that supports remote filesystem and process operations. 馃毀

distant Binary to connect with a remote machine to edit files and run programs. 馃毀 (Alpha stage software) This program is in rapid development and may

Chip Senkbeil 296 Dec 28, 2022
A Tauri Plugin to watch the filesystem for changes

Tauri Plugin FSWatch This plugin provides a "classical" Tauri Plugin Interface to watch changes on files and directories through notify. Architecture

Tauri 36 Jan 5, 2023
A filesystem-tracer

fstracer A filesystem-tracer. Actually this is just an experiment of mine to learn more about Rust's Foreign Function Interface (FFI), writing LD_PREL

null 1 Nov 17, 2021
Supertag is a tag-based filesystem, written in Rust, for Linux and MacOS
Supertag is a tag-based filesystem, written in Rust, for Linux and MacOS

Supertag is a tag-based filesystem, written in Rust, for Linux and MacOS. It provides a tag-based view of your files by removing the hierarchy constraints typically imposed on files and folders. In other words, it allows you to think about your files not as objects stored in folders, but as objects that can be filtered by folders.

Andrew Moffat 539 Dec 24, 2022
Cover your tracks during Linux Exploitation by leaving zero traces on system logs and filesystem timestamps. 馃懟馃悮
Cover your tracks during Linux Exploitation by leaving zero traces on system logs and filesystem timestamps. 馃懟馃悮

moonwalk Cover your tracks during Linux Exploitation / Penetration Testing by leaving zero traces on system logs and filesystem timestamps. 馃摉 Table o

Mufeed VH 1.1k Jan 6, 2023
Prints the absolute path of all regular files in an unmounted btrfs filesystem image.

btrfs-walk-tut Prints the absolute path of all regular files in an unmounted btrfs filesystem image. Learning about btrfs: Btrfs Basics Series This re

Nishal Kulkarni 2 Feb 3, 2022
Special FUSE filesystem to map /etc/resolv.conf to different files depending on Linux network namespace

Linux network namespaces allow separate networking environment for a group of processes (sharing uid or from a separate user). DNS settings (/etc/resolv.conf) are however shared between all those environments, which may be inconvenient in some setups.

Vitaly Shukela 2 May 16, 2022
This utility traverses through your filesystem looking for open-source dependencies that are seeking donations by parsing README.md and FUNDING.yml files
This utility traverses through your filesystem looking for open-source dependencies that are seeking donations by parsing README.md and FUNDING.yml files

This utility traverses through your filesystem looking for open-source dependencies that are seeking donations by parsing README.md and FUNDING.yml files

Mufeed VH 38 Dec 30, 2022
Comments
  • How to use `wasi-vfs` with Python?

    How to use `wasi-vfs` with Python?

    I'd like to use wasi-vfs with Python-Wasi, as I've done with Ruby, but it seems something has to be done with the .wasm file before wasi-vfs can add the files. How can I get started? Thank you.

    CC: @nickvidal @bstrie

    opened by rjzak 16
  • support arm

    support arm 

     => ERROR [11/11] RUN wasi-vfs pack ruby.wasm --mapdir /src::./src --mapdir /usr::./head-wasm32-unknown-wasi-full/usr -o my-ruby-app.wasm                            0.2s
------
 > [11/11] RUN wasi-vfs pack ruby.wasm --mapdir /src::./src --mapdir /usr::./head-wasm32-unknown-wasi-full/usr -o my-ruby-app.wasm:
#14 0.155 qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory

    fixed it by using docker build --platform linux/x86_64 but would be nice if it just worked

    opened by grosser 5
  • make mac installs easy

    make mac installs easy

    I stopped after the sdk clone was taking >1GB to download

    any of these would be nice:

    • darwin binaries
    • homebrew formula
    • copy-pastable clone+build instructions
    opened by grosser 2
  • Support CPython

    Support CPython 

    $ git clone https://github.com/tiran/cpython.git
$ cd cpython
$ LDFLAGS=/path/to/libwasi_vfs.a Tools/wasm/wasm_build.py wasi compile
$ cd builddir/wasi && DESTDIR=$PWD/install make install && cd ../..
$ wasi-vfs pack builddir/wasi/python.wasm --mapdir /::$(pwd)/builddir/wasi/install -o builddir/wasi/python-with-vfs.wasm
$ wasmtime run builddir/wasi/python-with-vfs.wasm
Python 3.12.0a0 (heads/main-dirty:32ac98e899, Aug 25 2022, 23:50:38) [Clang 13.0.0 (https://github.com/llvm/llvm-project fd1d8c2f04dde23bee0fb3a7d069 on wasi
Type "help", "copyright", "credits" or "license" for more information.
>>>
    opened by kateinoigakukun 0
Owner
Yuta Saito
UTC+9
Yuta Saito
GitHub
Spacedrive is an open source cross-platform file explorer, powered by a virtual distributed filesystem written in Rust.

Spacedrive A file explorer from the future. spacedrive.com 禄 Download for macOS 路 Windows 路 Linux 路 iOS 路 watchOS 路 Android ~ Links will be added once

Spacedrive 16.2k Jan 7, 2023
The reference implementation of the Linux FUSE (Filesystem in Userspace) interface

libfuse About FUSE (Filesystem in Userspace) is an interface for userspace programs to export a filesystem to the Linux kernel. The FUSE project consi

null 4.2k Jan 4, 2023
A Tauri Plugin to watch the filesystem for changes

Tauri Plugin FSWatch This plugin provides a "classical" Tauri Plugin Interface to watch changes on files and directories through notify. Architecture

Tauri 36 Jan 5, 2023
Supertag is a tag-based filesystem, written in Rust, for Linux and MacOS

Supertag is a tag-based filesystem, written in Rust, for Linux and MacOS. It provides a tag-based view of your files by removing the hierarchy constraints typically imposed on files and folders. In other words, it allows you to think about your files not as objects stored in folders, but as objects that can be filtered by folders.

Andrew Moffat 539 Dec 24, 2022
A simple tool to convert a WebAssembly module to a WASI component (component model, WASI-preview2).

A simple tool to convert a WebAssembly module (wasm32-wasi, wasm32-freestanding) to a WASI component (component model, WASI-preview2). Installation In

Frank Denis 5 Feb 18, 2024
Easy c痰蜖贪r痰虥虪o檀虉酞s潭虙泰s痰蛝汰-t潭蜐滩h潭虤摊r痰蛺毯e谭探號a檀虓胎d潭虜號 r痰虘蜋e潭虈蛶f檀虈蜋e谈蜎號r潭蜅號e潭蛣瘫n痰虙瘫c谭虃胎e谭虓蜌 s谭蛯坦h谭虊滩a痰蛬汰r谭虋太i痰虋贪n谭蛡态g痰虄苔 f潭虋袒o痰虥虦r谈蛣瘫 R痰虌虨u痰蛝毯s谭虆號t谈蜅抬!谈蛬碳

Rust S痰蛢i谈虛n痰虊 I檀n檀f潭e谈r痰n谭a檀l mutability! Howdy, friendly Rust developer! Ever had a value get m痰虆摊冒潭蛫v檀叹坍锚檀蜆碳d away right under your nose just when

null 294 Dec 23, 2022
the file filesystem: mount semi-structured data (like JSON) as a Unix filesystem

ffs: the file filesystem ffs, the file filessytem, let's you mount semi-structured data as a fileystem---a tree structure you already know how to work

Michael Greenberg 176 Dec 31, 2022
Spacedrive is an open source cross-platform file explorer, powered by a virtual distributed filesystem written in Rust.

Spacedrive A file explorer from the future. spacedrive.com 禄 Download for macOS 路 Windows 路 Linux 路 iOS 路 watchOS 路 Android ~ Links will be added once

Spacedrive 16.2k Jan 7, 2023
Nimbus - A virtual, networked filesystem with strong upfront safety guarantees

The Nimbus Filesystem Nimbus is a virtual, networked filesystem that provides upfront safety guarantees to a user, intended for personal use. In parti

Max Fan 3 Jan 10, 2023
Transforms UDP stream into (fake) TCP streams that can go through Layer 3 & Layer 4 (NAPT) firewalls/NATs.

Phantun A lightweight and fast UDP to TCP obfuscator. Table of Contents Phantun Latest release Overview Usage 1. Enable Kernel IP forwarding 2. Add re

Datong Sun 782 Dec 30, 2022