Rust programs written entirely in Rust

Overview

mustang

Programs written entirely in Rust

Github Actions CI Status zulip chat crates.io page docs.rs docs

Mustang is a system for building programs built entirely in Rust, meaning they do not depend on any part of libc or crt1.o, and do not link in any C code.

Why? For fun! And to exercise some components built for other purposes (such as rsix) but which happen to also be part of what's needed to do what Mustang is doing. And in the future, possibly also for experimenting with new kinds of platform ABIs and new forms of process argument passing.

Mustang isn't about making anything safer, for the foreseeable future. The major libc implementations are extraordinarily well tested and mature. Mustang for its part is experimental and has lots of unsafe.

This also isn't about building a complete libc. It currently includes some things with libc-compatible interfaces, just enough to allow it to slide in underneath std, however even this may not always be necessary. We'll see.

Mustang currently runs on Rust Nightly on Linux on x86-64, aarch64, and x86.

Usage

To use it, first install rust-src, which is needed by -Z build-std:

$ rustup component add rust-src --toolchain nightly

Then, set the RUST_TARGET_PATH environment variable to a path to mustang's specs directory, so that you can name mustang targets with --target=.... For example, within a mustang repo:

$ export RUST_TARGET_PATH="$PWD/specs"

Then, in your own crate, add a dependency on mustang:

[dependencies]
mustang = { git = "https://github.com/sunfishcode/mustang" }

And add an extern crate declaration for mustang to your top-level module (eg. main.rs). This is needed even in Rust 2018 Edition, to ensure that mustang is linked in even though no functions in it are explicitly called:

extern crate mustang;

Then, compile with Rust nightly, using -Z build-std and --target=<mustang-target>. For example:

$ cargo +nightly run --quiet -Z build-std --target=x86_64-mustang-linux-gnu --example hello
.。oO(This process was started by origin! 🎯)
.。oO(Environment variables initialized by c-scape! 🌱)
.。oO(I/O performed by c-scape using rsix! 🌊)
Hello, world!
.。oO(This process will be exited by c-scape using rsix! 🚪)
$

That's a Rust program built entirely from Rust saying "Hello, world!"!

Those .。oO lines are just debugging output to confirm everything is set up properly. Once mustang is more stable, we'll stop printing them.

A simple way to check for uses of libc functions is to use nm -u, since the above commands are configured to link libc dynamically. If mustang has everything covered, there should be no output:

$ nm -u target/x86_64-mustang-linux-gnu/debug/examples/hello
$

The C Runtime

C has a runtime, and if you wish to link with any C libraries, the C runtime needs to be initialized. mustang doesn't do this by default, but it does support this when the cargo feature "initialize-c-runtime" is enabled.

To compile C code with a *-mustang-* target, you may need to tell the cc crate which C compiler to use; for example, for i686-mustang-linux-gnu, set the environment variable CC_i686-mustang-linux-gnu to i686-linux-gnu-gcc.

Known Limitations

Known limitations in mustang include:

  • Networking, current_dir, spawning new processes, threads, and unwinding panics are not implemented yet.
  • No support for dynamic linking yet.

Background

Mustang is partly inspired by similar functionality in steed, but a few things are different. cargo's build-std is now available, which makes it much easier to work with custom targets. And Mustang is starting with the approach of starting by replacing libc interfaces and using std as-is, rather than reimplementing std. This is likely to evolve, but whatever we do, a high-level goal of Mustang is to avoid ever having to reimplement std.

Where does mustang go from here? Will it support feature X, platform Y, or use case Z? If origin can do program startup in Rust, and rsix can do system calls in Rust, what does it all mean?

And could mustang eventually support new ABIs that aren't limited to passing C-style argc/argv(/envp) convention, allowing new kinds of program argument passing?

Let's find out! Come say hi in the chat or an issue.

How does one port mustang to a new architecture?

  • Port rsix to the architecture, adding assembly sequences for making syscalls on the architecture.
  • Add assembly code to the _start function in src/lib.rs to call origin::rust.
  • Create a target file in specs/, by first following these instructions to generate a default target file, and then:
    • change dynamic-linking to false
    • add -nostdlib, -Wl,--require-defined=start, and -Wl,--require-defined=environ to pre-link-args
    • add "vendor": "mustang" See other targets in the specs/ directory for examples.
  • Add the architecture to example/test/test.rs.
  • Add CI testing to .github/workflows/main.yml, by copying what's done for other architectures.

How does one port mustang to a new OS?

One probably needs to do similar things as for a new architecture, and also write a new origin::rust implementation to handle the OS's convention for arguments, environment variables, and initialization functions.

Comments
  • Split up some files into modules/impl FS functions

    Split up some files into modules/impl FS functions

    The main point of this change is to make sure all LFS functions are implemented in terms of the the 64 variant, as well as move all of the fs located code outside of the single massive file. Where functions ending in at were missing, they have been added and all calls updated to reduce code duplication.

    Closes (because that's probably better than putting a comment in each issue): #88, partial #87, #71.

    opened by carbotaniuman 12
  • Remove redundant `checked_cast`, add `utimes` family

    Remove redundant `checked_cast`, add `utimes` family

    As the comment in one of the files says, I'm not sure what Rustix's story is wrt 64bit time on 32-bit systems.

    This also adds some_or_ret_einval, which I'm down to bikeshed, but it should get rid of all the unwraps for parsing flags from bits.

    opened by carbotaniuman 9
  • Please document the intended policy for minimum Linux kernel versions

    Please document the intended policy for minimum Linux kernel versions

    I'd love to see mustang have a documented policy for how it selects the minimum Linux kernel version it supports, as well as the current minimum version.

    I can imagine three possible policies here:

    1. Recent kernels: require a relatively recent kernel, so that mustang can unconditionally use new system calls without fallbacks (or missing functionality if there's no reasonable fallback). This would be appropriate if mustang is primarily demonstrating the best and simplest possible implementation path, without worrying about widespread production use outside of deployment environments people can entirely control.
    2. Stable kernels: Require a kernel within the last few years, roughly allowing for "last stable release of common Linux distributions". This would require some fallbacks for new functionality, but fewer of them. This would be appropriate for widespread production use of mustang, including use in containers and distribution of common tools (e.g. precompiled versions of command-line tools).
    3. Enterprise kernels: Support every kernel version Rust currently supports. This would require the largest number of fallbacks, but support anyone who could possibly run mustang on any Rust system.
    opened by joshtriplett 8
  • What might an official Rust target look like?

    What might an official Rust target look like?

    An official target could be much easier to use, with no mustang::can_run_here!() macro, no target json file, and no -Z build-std. And internally, it could avoid some of the linker hacks, and achieve much smaller code size, because it wouldn't need to force-link in all the libc symbols just to prevent the system libc from being linked in. And, this would be a natural moment to pick a new name for the target :smile: .

    What might this look like? This issue is some early brainstorming in this space with many open questions, and more questions are welcome!

    One big question is, should such a target use the c-scape C ABI compatibility layer?

    • If not:

      • https://github.com/bytecodealliance/rsix/issues/76
      • https://github.com/sunfishcode/mustang/issues/38
      • Figure out a plan for mutex/rwlock, DNS resolution, and possible other stuff where mustang has more dependencies than std might want.
      • Others?
    • If so:

      • I wonder if it's possible to build up c-scape and all its dependencies, including std, into a single #[crate_type = "staticlib"] library, which we'd name "libc", and then we'd have the new Rust target use that. That would mean two copies of std present in resulting programs, which might present some tricky issues, and very likely produces large binaries, but if it's not too tricky this might be a faster path to getting something working while making relatively few changes to Rust itself. And then we could work on eliminating c-scape gradually over time.
    help wanted question 
    opened by sunfishcode 8
  • Use the `unwind` crate to implement libunwind interfaces.

    Use the `unwind` crate to implement libunwind interfaces.

    The unwind crate appears to work well for this purpose, even on panics with RUST_BACKTRACE=full.

    This uses the fde-phdr-dl backend, which uses dl_iterate_phdr, so implement dl_iterate_phdr.

    Fixes #27.

    opened by sunfishcode 8
  • Move `fork` from origin to c_scape

    Move `fork` from origin to c_scape

    After a fork, the child only has one thread, and it looks like all the other threads are suspended. This may leave behind locks in locked states and cause deadlocks, however deadlocks are not considered unsafe. Consequently, fork doesn't need to be unsafe.

    This also changes the on_fork callback functions to be Box<Fn() + Send> so that it's safe to call them, and adds code to c-scape and non-mustang targets to wrap them as needed.

    opened by sunfishcode 7
  • Fill in signatures for pthread_cond*

    Fill in signatures for pthread_cond*

    This wasn't easy to figure out, but I think now I'm at a point where it wouldn't take me that long to fill in most of the remaining signatures of functions with a commented-out libc!.

    The padding bytes are most likely only correct for x86_64. What would I have to do to figure out the right padding for x86?

    opened by jplatte 7
  • c-scape: add pthread_setname_np (#139)

    c-scape: add pthread_setname_np (#139)

    As mentioned in the issue #139, any code using threads fails to link. Since this commit, rust links against a pthread_setname_np on linux. Android is unaffected. https://github.com/rust-lang/rust/commit/013986be1bcfbfd9cedddf22dfd9584ccdb7ee6e

    opened by HookedBehemoth 6
  • Mustang doesn't compile with latest nightly

    Mustang doesn't compile with latest nightly

    Compiling mustang with the latest nightly gets these errors:

    inlinable function call in a function with debug info must have a !dbg location
      call void @_Unwind_Resume(ptr %81) #15
    inlinable function call in a function with debug info must have a !dbg location
      call void @_Unwind_Resume(ptr %101) #15
    inlinable function call in a function with debug info must have a !dbg location
      call void @_Unwind_Resume(ptr %144) #15
    inlinable function call in a function with debug info must have a !dbg location
      call void @_Unwind_Resume(ptr %111) #15
    LLVM ERROR: Broken module found, compilation aborted!
    
    opened by sunfishcode 6
  • Implement `getcwd`, `chdir`

    Implement `getcwd`, `chdir`

    Rust's std::env::current_dir and set_current_dir are implemented by calls to getcwd and chdir. The first half in implementing these is to add support to rsix: these should go in rsix::process. There's already a imp/linux_raw implementation of the getcwd and chdir system calls, so this needs a imp/libc implementation which should just call the libc function, and then a public API in src/process. For getcwd, the public API should take care of allocating the buffer and growing it as needed until getcwd successfully writes the whole string, returning an OsString, similar to readlinkat.

    The second step is to implement c-scape support in terms of the rsix public API. It may look odd to do it this way, because rsix will be dynamically allocating, and we'll need to copy the (sub)string out of the dynamic allocated buffer into the fixed-size requested buffer. But one of the goals for c-scape is to help test rsix, and to help prepare for rsix itself to be the low-level API instead of c-scape.

    opened by sunfishcode 6
  • Optimize `memchr` etc.

    Optimize `memchr` etc.

    Currently c-scape's implementation of memchr and similar functions is very simple. It may be possible to optimize them using the memx crate.

    c-scape is compiled with #![no_builtins] to discourage the compiler from optimizing the definitions of C library functions into calls to C library functions, however the compiler can still generate calls to memcpy, memmove, memset, and memcmpy, so we need to be careful the compiler doesn't do that with the implementations of those functions :-).

    good first issue 
    opened by sunfishcode 6
  • Add splice and sendfile

    Add splice and sendfile

    Adds splice and sendfile, as those got added sometime in when I last looked at this and now.

    I'm very unsure of all the unsigned to signed conversions and back here, and I'm not really sure why rustix has offsets as unsigned here.

    I've uncommented the tests, but strangely enough I don't think that tests actually tests the splice or sendfile path, as those appear locked behind benches. I will add some tests for this.

    opened by carbotaniuman 2
  • Map of mustang and rustix

    Map of mustang and rustix

    I recently learned about Mermaid diagrams, and I recently had a discussion with someone who gave me the idea to make a diagram to show how some of the parts in and around Mustang relate to each other. Here's what I came up with:

    graph TD;
        ripgrep["ripgrep (and other programs)"]-- optionally -->std-unmodified;
        ripgrep-- optionally -->std-ported;
        std-unmodified["std (unmodified)"]-->c-scape;
        std-ported-->origin;
        std-ported["std (ported)"]-->rustix;
        c-scape-->rustix;
        c-scape-->origin;
        origin-. optionally .->platform-libc;
        origin-->rustix;
        origin-. optionally .->linux-syscalls;
        rustix-- optionally -->platform-libc;
        rustix-- optionally -->linux-syscalls;
    

    Mustang is the experimental project which currently includes c-scape and origin and some support tools to help compile programs to use them, allowing people to try them out on real programs such as ripgrep. This doesn't require any changes to std's code.

    A sibling project is the port of std to origin and rustix. The port is not complete, though it is mostly usable since rustix calls can coexist with the existing libc calls. Compared to the Mustang approach, this approach results in simpler and more efficient code, since it avoids Rust calling through the C ABI, translating back into idiomatic Rust, and then translating into the C-like system calls or platform libc ABI. It stays in Rust until the syscalls. It also factors out a lot of low-level unsafe blocks and error handling from the higher-level logic of std.

    Some people have asked about what an official Rust target using these components might look like. There are a variety of ways this could be done, though there's more work and reorganization needed. At this time, work toward this goal is on hold as there hasn't been sufficient community interest. If anyone has thoughts about what they'd like to see here, please reach out!

    Both Mustang and the port of std to rustix serve as interesting testcases for rustix, so I hope to continue maintaining them for that purpose.

    Rustix itself is a relatively mature project, which is used in production, and which is part of several much larger stories, including I/O safety and cap-std, which themselves are part of larger stories, so even though some of the goals are on hold, rustix will continue to have other goals, and I plan to continue to actively maintain rustix.

    opened by sunfishcode 1
  • Implement `epoll`

    Implement `epoll`

    As reported here, this is needed for cargo-watch.

    epoll has turned out to be fairly tricky, because rustix's epoll API does a significant amount of work to present a safe API, which turns out to be complex to re-expose as an unsafe C API. I started some work here toward having rustix also expose an unsafe API for epoll as well, though it's not complete yet.

    opened by sunfishcode 2
  • mustang doesn't work with lto

    mustang doesn't work with lto

    This reproduces with just adding

    [profile.release]
    lto = true
    

    to the top-level Cargo.toml in the mustang repo, and running the hello example. It gets lots of undefined references to things like core::panicking::assert_failed_inner. See also https://github.com/sunfishcode/mustang/issues/22#issuecomment-983208986.

    opened by sunfishcode 1
Owner
Dan Gohman
I'm working on Wasmtime, WASI, and lots of things related to WebAssembly.
Dan Gohman
Small programs written in Rust. Warm up for the upcoming Selenium Manager

Rust Examples This repository contains several example programs written in Rust. Selenium Manager These examples are used as warm up for the upcoming

Boni García 5 Dec 30, 2022
A additional Rust compiler pass to detect memory safe bugs of Rust programs.

SafeDrop A additional Rust compiler pass to detect memory safe bugs of Rust programs. SafeDrop performs path-sensitive and field-sensitive inter-proce

Artisan-Lab  (Fn*) 5 Nov 25, 2022
Rust library for compiling and running other programs.

Exers ?? Exers is a rust library for compiling and running code in different languages and runtimes. Usage example fn main() { // Imports...

olix3001 5 Jun 10, 2023
Visualization for Timely Dataflow and Differential Dataflow programs

DDShow Visualization for Timely Dataflow and Differential Dataflow programs Getting started with ddshow First, install ddshow via cargo. As of now dds

Chase Wilson 61 Nov 25, 2022
🕶 Assorted checks and validations for writing safer Solana programs.

vipers ?? Assorted checks and validations for writing safer Solana programs. Motivation Solana's fee mechanism is unlike Ethereum's, in that the numbe

Saber 131 Sep 14, 2022
Complete code for the larger example programs from the book.

Code Examples for Programming Rust This repository contains complete code for the larger example programs from the book “Programming Rust”, by Jim Bla

Programming Rust 670 Jan 1, 2023
Cogo is a high-performance library for programming stackful coroutines with which you can easily develop and maintain massive concurrent programs.

Cogo is a high-performance library for programming stackful coroutines with which you can easily develop and maintain massive concurrent programs.

co-rs 47 Nov 17, 2022
Write Anchor-compatible Solana programs in Python

seahorse: Write Solana programs in Python The ease of Python with the safety of Rust. Seahorse lets you write Solana programs in Python. It is a commu

✨ amelia chen ✨ 214 Dec 28, 2022
Gain intuition about the goings-on of your multithreaded/multicomponent programs

Intuition: a super simple profiler with a terminal ui based on tui-rs. Gain intuition about the goings-on of your multithreaded/multicomponent program

GenesysGo 9 Mar 2, 2023
Simple autoclicker written in Rust, to learn the Rust language.

RClicker is an autoclicker written in Rust, written to learn more about the Rust programming language. RClicker was was written by me to learn more ab

null 7 Nov 15, 2022
clone of grep cli written in Rust. From Chapter 12 of the Rust Programming Language book

minigrep is a clone of the grep cli in rust Minigrep will find a query string in a file. To test it out, clone the project and run cargo run body poem

Raunak Singh 1 Dec 14, 2021
The Rust Compiler Collection is a collection of compilers for various languages, written with The Rust Programming Language.

rcc The Rust Compiler Collection is a collection of compilers for various languages, written with The Rust Programming Language. Compilers Language Co

null 2 Jan 17, 2022
Game Boy Emulator written in Rust, as a way to fully grasp the Rust programming language

Flan's Game Boy Emulator Game Boy Emulator written in Rust, as a way to get hands-on with the Rust programming language, and creating a proper project

Flan 3 Dec 31, 2022
Nixt is an interpreted programming language written in Rust

Nixt Nixt is an interpreted lisp inspired programming language written in Rust Index About Examples Installation Build About Nixt goal is to provide a

Wafelack 17 Jul 18, 2022
Fegeya Elitebuild, small, powerful build system. Written in Rust.

Fegeya Elitebuild Small, powerful, work-in-progress build system. Written in Rust. Features: No functions (all are built-ins) All variables are global

Ferhat Geçdoğan 25 Nov 9, 2022
Raytracer tutorial for PPCA 2021, written in Rust.

Pseudo Photograph Company of ACM 工科和ACM的朋友们都已结课!看看这些了不起的艺术品: 工科 ACM ACM伪摄影公司,简称PPCA,于2021年成立 ?? 这个项目的主要工作是使用Rust语言实现一个光线追踪渲染器。以这个形式,你能通过学习一门新的(而且漂亮的)语

null 113 Dec 13, 2022
An i386 operation system written in pure rust for fun and no profit.

OrustS An i386 operation system written in pure rust (for fun and no profit). This operation system is under active developing. Checklist implement a

M4tsuri 10 Aug 12, 2022
A tool to make grocery lists written in Rust

grusterylist: makes grocery lists, written in Rust grusterylist uses and can add to local libraries of user-added recipes and grocery items to put tog

null 3 Jun 17, 2022
Tests a wide variety of N64 features, from common to hardware quirks. Written in Rust. Executes quickly.

n64-systemtest Tests a wide variety of N64 features, from common to hardware quirks. Written in Rust. Executes quickly. n64-systemtest is a test rom t

null 37 Jan 7, 2023