Rust programs written entirely in Rust

Related tags

rust
Overview

mustang

Programs written entirely in Rust

Github Actions CI Status zulip chat crates.io page docs.rs docs

Mustang is a system for building programs built entirely in Rust, meaning they do not depend on any part of libc or crt1.o, and do not link in any C code.

Why? For fun! And to exercise some components built for other purposes (such as rsix) but which happen to also be part of what's needed to do what Mustang is doing. And in the future, possibly also for experimenting with new kinds of platform ABIs and new forms of process argument passing.

Mustang isn't about making anything safer, for the foreseeable future. The major libc implementations are extraordinarily well tested and mature. Mustang for its part is experimental and has lots of unsafe.

This also isn't about building a complete libc. It currently includes some things with libc-compatible interfaces, just enough to allow it to slide in underneath std, however even this may not always be necessary. We'll see.

Mustang currently runs on Rust Nightly on Linux on x86-64, aarch64, and x86.

Usage

To use it, first install rust-src, which is needed by -Z build-std:

$ rustup component add rust-src --toolchain nightly

Then, set the RUST_TARGET_PATH environment variable to a path to mustang's specs directory, so that you can name mustang targets with --target=.... For example, within a mustang repo:

$ export RUST_TARGET_PATH="$PWD/specs"

Then, in your own crate, add a dependency on mustang:

[dependencies]
mustang = { git = "https://github.com/sunfishcode/mustang" }

And add an extern crate declaration for mustang to your top-level module (eg. main.rs). This is needed even in Rust 2018 Edition, to ensure that mustang is linked in even though no functions in it are explicitly called:

extern crate mustang;

Then, compile with Rust nightly, using -Z build-std and --target=<mustang-target>. For example:

$ cargo +nightly run --quiet -Z build-std --target=x86_64-mustang-linux-gnu --example hello
.。oO(This process was started by origin! 🎯)
.。oO(Environment variables initialized by c-scape! 🌱)
.。oO(I/O performed by c-scape using rsix! 🌊)
Hello, world!
.。oO(This process will be exited by c-scape using rsix! 🚪)
$

That's a Rust program built entirely from Rust saying "Hello, world!"!

Those .。oO lines are just debugging output to confirm everything is set up properly. Once mustang is more stable, we'll stop printing them.

A simple way to check for uses of libc functions is to use nm -u, since the above commands are configured to link libc dynamically. If mustang has everything covered, there should be no output:

$ nm -u target/x86_64-mustang-linux-gnu/debug/examples/hello
$

The C Runtime

C has a runtime, and if you wish to link with any C libraries, the C runtime needs to be initialized. mustang doesn't do this by default, but it does support this when the cargo feature "initialize-c-runtime" is enabled.

To compile C code with a *-mustang-* target, you may need to tell the cc crate which C compiler to use; for example, for i686-mustang-linux-gnu, set the environment variable CC_i686-mustang-linux-gnu to i686-linux-gnu-gcc.

Known Limitations

Known limitations in mustang include:

  • Networking, current_dir, spawning new processes, threads, and unwinding panics are not implemented yet.
  • No support for dynamic linking yet.

Background

Mustang is partly inspired by similar functionality in steed, but a few things are different. cargo's build-std is now available, which makes it much easier to work with custom targets. And Mustang is starting with the approach of starting by replacing libc interfaces and using std as-is, rather than reimplementing std. This is likely to evolve, but whatever we do, a high-level goal of Mustang is to avoid ever having to reimplement std.

Where does mustang go from here? Will it support feature X, platform Y, or use case Z? If origin can do program startup in Rust, and rsix can do system calls in Rust, what does it all mean?

And could mustang eventually support new ABIs that aren't limited to passing C-style argc/argv(/envp) convention, allowing new kinds of program argument passing?

Let's find out! Come say hi in the chat or an issue.

How does one port mustang to a new architecture?

  • Port rsix to the architecture, adding assembly sequences for making syscalls on the architecture.
  • Add assembly code to the _start function in src/lib.rs to call origin::rust.
  • Create a target file in specs/, by first following these instructions to generate a default target file, and then:
    • change dynamic-linking to false
    • add -nostdlib, -Wl,--require-defined=start, and -Wl,--require-defined=environ to pre-link-args
    • add "vendor": "mustang" See other targets in the specs/ directory for examples.
  • Add the architecture to example/test/test.rs.
  • Add CI testing to .github/workflows/main.yml, by copying what's done for other architectures.

How does one port mustang to a new OS?

One probably needs to do similar things as for a new architecture, and also write a new origin::rust implementation to handle the OS's convention for arguments, environment variables, and initialization functions.

Issues
  • Add support for linking with C libraries

    Add support for linking with C libraries

    This adds an option to initialize the C runtime, so that it's (relatively) safe to call C libraries.

    Fixes #2.

    opened by sunfishcode 3
  • Use compiler_builtins for mem{cmp,cpy,move,set}

    Use compiler_builtins for mem{cmp,cpy,move,set}

    As proposed in #5. This is just a straightforward code replacement, I'm not sure what else if anything needs to be done. I've left bcmp as-is since I can't see any reason to use compiler_builtins there.

    opened by cole-miller 3
  • Linking with C libraries

    Linking with C libraries

    It's currently not safe to link with C libraries, because Mustang doesn't initialize the C runtime. I've now added a point about this to the README.md. But is there a way Mustang could make this work?

    __libc_start_main appears to be a relatively stable interface, that looks like it might work in both glibc and musl. Mustang could call it and pass a no-op main to let libc initialize itself, but the tricky part is, it calls exit when it's done.

    Since mustang defines its own exit, one option here would be for mustang to call __libc_start_main and "catch" the exit. But then, what should it do? GLIBC's __libc_start_main is annotated with __attribute__ ((noreturn), and the C interface to exit uses __attribute__((noreturn)), so we can't have exit return. Another option would be longjmp, but we can't currently do setjmp in Rust. Another would be to panic and use catch_unwind, but a panic across an FFI boundary is UB.

    Another idea is to create a new thread, call __libc_start_main on it, and then terminate the thread once it calls exit. But, Mustang doesn't support threads yet. It's tempting to just call libc's pthread_create et al in that case, however we can't safely call C code until the C runtime is initialized, which is what we're trying to do :-}.

    I'm thinking longjmp may be the best option for now. A C-library compatibility mode would be all about having C code in the process, so having a small amount of extra C code to call setjmp would be fine.

    opened by sunfishcode 2
  • Fix typo in the readme

    Fix typo in the readme

    null

    opened by bjorn3 1
  • Call out to the memchr crate for memchr and memrchr

    Call out to the memchr crate for memchr and memrchr

    See #5.

    Note: I've changed the signatures of memchr and memrchr to match the (weird) C signatures, which take a const pointer but return a mut pointer (e.g.).

    opened by cole-miller 1
  • Implement more functions in c-scape

    Implement more functions in c-scape

    This patch sequence adds basic support for everything in std::fs, all the libm functions, getrandom, mmap/mprotect, pipe2, isatty, and clock_gettime.

    The targets are renamed from <arch>-unknown-linux-mustang to <arch>-mustang-linux-gnu, which makes them more compatible with existing crates.

    opened by sunfishcode 0
  • Implement `malloc` properly, and other fixes

    Implement `malloc` properly, and other fixes

    Also, add a system for checking the ABI against the libc crate.

    opened by sunfishcode 0
  • Networking syscalls

    Networking syscalls

    I'm currently working on basic network syscalls (socket, listen, bind, accept, recv, send, and a few others). I'm first reorganizing some of the network support in rsix, and then I'll do the hopefully obvious support in c-scape.

    opened by sunfishcode 0
  • Revamp linking.

    Revamp linking.

    Redesign how mustang ensures its libraries are linked in, to be somewhat more robust in the face of various kinds of optimizations. Use explicit symbols to pull in individual modules. This also makes it easier to split more c-scape functionality into separate modules.

    • Instead of having users declare extern crate mustang, have them invoke a macro, mustang::can_compile_this!();.

    • Don't declare c-scape functions as pub. Most code should use the libc crate declarations; c-scape uses extern "C" to intercept those at link time.

    • Put extern "C" functions in a named section; this overrides -ffunction-sections and puts all the functions in the same section, so that the linker doesn't DCE individual functions before they have a chance to intercept libc calls.

    • Similarly, add #[inline(never)] so that these functions aren't inlined, which can result in symbols being removed if an optimizer believes it has seen all of their uses.

    This helps with #7, though is not the only thing needed.

    opened by sunfishcode 0
  • Implement `getcwd`, `chdir`

    Implement `getcwd`, `chdir`

    Rust's std::env::current_dir and set_current_dir are implemented by calls to getcwd and chdir. The first half in implementing these is to add support to rsix: these should go in rsix::process. There's already a imp/linux_raw implementation of the getcwd and chdir system calls, so this needs a imp/libc implementation which should just call the libc function, and then a public API in src/process. For getcwd, the public API should take care of allocating the buffer and growing it as needed until getcwd successfully writes the whole string, returning an OsString, similar to readlinkat.

    The second step is to implement c-scape support in terms of the rsix public API. It may look odd to do it this way, because rsix will be dynamically allocating, and we'll need to copy the (sub)string out of the dynamic allocated buffer into the fixed-size requested buffer. But one of the goals for c-scape is to help test rsix, and to help prepare for rsix itself to be the low-level API instead of c-scape.

    opened by sunfishcode 4
  • Re-introduce riscv64 support.

    Re-introduce riscv64 support.

    null

    opened by sunfishcode 1
  • Optimize `memchr` etc.

    Optimize `memchr` etc.

    Currently c-scape's implementation of memchr and similar functions is very simple. It may be possible to optimize them using the memx crate.

    c-scape is compiled with #![no_builtins] to discourage the compiler from optimizing the definitions of C library functions into calls to C library functions, however the compiler can still generate calls to memcpy, memmove, memset, and memcmpy, so we need to be careful the compiler doesn't do that with the implementations of those functions :-).

    good first issue 
    opened by sunfishcode 5
  • Can I get to know how the name 'mustang' came into your mind?

    Can I get to know how the name 'mustang' came into your mind?

    Title says it all

    opened by sudipghimire533 2
Owner
Dan Gohman
I'm working on Wasmtime, WASI, and lots of things related to WebAssembly.
Dan Gohman
List of Rust books

Rust Books Books Starter Books Advanced Books Resources Books Starter Books The Rust Programming Language Free Welcome! This book will teach you about

Spiros Gerokostas 1.4k Sep 10, 2021
Rust programs written entirely in Rust

mustang Programs written entirely in Rust Mustang is a system for building programs built entirely in Rust, meaning they do not depend on any part of

Dan Gohman 290 Sep 14, 2021
The missing batteries of Rust

stdx - The missing batteries of Rust New to Rust and don't yet know what crates to use? stdx has the best crates. Current revision: stdx 0.119.0-rc, f

Brian Anderson 1.5k Sep 13, 2021
A peer-reviewed collection of articles/talks/repos which teach concise, idiomatic Rust.

This repository collects resources for writing clean, idiomatic Rust code. Please bring your own. ?? Idiomatic coding means following the conventions

Matthias 2.8k Sep 10, 2021
a cheat-sheet for mathematical notation in Rust 🦀 code form

math-as-rust ?? Based on math-as-code This is a reference to ease developers into mathematical notation by showing comparisons with Rust code.

Eduardo Pereira 9 Sep 12, 2021
:crab: Small exercises to get you used to reading and writing Rust code!

rustlings ?? ❤️ Greetings and welcome to rustlings. This project contains small exercises to get you used to reading and writing Rust code. This inclu

The Rust Programming Language 18.9k Sep 16, 2021
Rust 核心库和标准库的源码级中文翻译,可作为 IDE 工具的智能提示 (Rust core library and standard library translation. can be used as IntelliSense for IDE tools)

Rust 标准库中文版 这是翻译 Rust 库 的地方, 相关源代码来自于 https://github.com/rust-lang/rust。 如果您不会说英语,那么拥有使用中文的文档至关重要,即使您会说英语,使用母语也仍然能让您感到愉快。Rust 标准库是高质量的,不管是新手还是老手,都可以从中

wtklbm 216 Sep 13, 2021
A Collection of Rust Tips and Tricks

Table of Contents Using Nested Paths in Rust Struct Update Syntax in Rust Field Init Shorthand in Rust Shadowing in Rust Using Nested Paths in Rust So

Vandad Nahavandipoor 14 Sep 3, 2021
🕶 Assorted checks and validations for writing safer Solana programs.

vipers ?? Assorted checks and validations for writing safer Solana programs. Motivation Solana's fee mechanism is unlike Ethereum's, in that the numbe

Saber 37 Sep 9, 2021
Yet another trojan-gfw in Rust

Trojan-rust Yet another trojan-gfw implementation in Rust. Features Server mode only (for now). Supports Redis auth & flow stat. Uses OpenSSL as crypt

粒粒橙 25 Aug 11, 2021
Visualization for Timely Dataflow and Differential Dataflow programs

DDShow Visualization for Timely Dataflow and Differential Dataflow programs Getting started with ddshow First, install ddshow via cargo. As of now dds

Chase Wilson 48 Aug 10, 2021
A Quest to Find a Highly Compressed Emoji :shortcode: Lookup Function

Highly Compressed Emoji Shortcode Mapping An experiment to try and find a highly compressed representation of the entire unicode shortcodes-to-emoji m

Daniel Prilik 14 Aug 9, 2021
Take your first step in writing a compiler. Implemented in Rust.

first-step-rust Take your first step in writing a compiler, using Rust. Building from Source Make sure the Rust toolchain is installed on your compute

PKU Compiler Course 8 Sep 17, 2021
Raytracer tutorial for PPCA 2021, written in Rust.

Pseudo Photograph Company of ACM 工科和ACM的朋友们都已结课!看看这些了不起的艺术品: 工科 ACM ACM伪摄影公司,简称PPCA,于2021年成立 ?? 这个项目的主要工作是使用Rust语言实现一个光线追踪渲染器。以这个形式,你能通过学习一门新的(而且漂亮的)语

null 98 Sep 2, 2021
Leetcode Solutions in Rust, Advent of Code Solutions in Rust and more

RUST GYM Rust Solutions Leetcode Solutions in Rust AdventOfCode Solutions in Rust This project demostrates how to create Data Structures and to implem

Larry Fantasy 197 Sep 11, 2021
Write Cloudflare Workers in 100% Rust via WebAssembly

Work-in-progress ergonomic Rust bindings to Cloudflare Workers environment. Write your entire worker in Rust! Read the Notes and FAQ Example Usage use

Cloudflare 393 Sep 14, 2021
Comprehensive DSP graph and synthesis library for developing a modular synthesizer in Rust, such as HexoSynth.

HexoDSP - Comprehensive DSP graph and synthesis library for developing a modular synthesizer in Rust, such as HexoSynth. This project contains the com

Weird Constructor 13 Sep 7, 2021
The source code that accompanies Hands-on Rust: Effective Learning through 2D Game Development and Play by Herbert Wolverson

Hands-on Rust Source Code This repository contains the source code for the examples found in Hands-on Rust. These are also available from my publisher

Herbert 76 Sep 11, 2021
Simple and performant hot-reloading for Rust

reloady Simple, performant hot-reloading for Rust. Requires Rust nightly and only works on Linux for now. installing CLI To install the CLI helper car

Anirudh Balaji 13 Sep 2, 2021