The main point of this change is to make sure all LFS functions are implemented in terms of the the 64 variant, as well as move all of the fs located code outside of the single massive file. Where functions ending in at were missing, they have been added and all calls updated to reduce code duplication.

Closes (because that's probably better than putting a comment in each issue): #88, partial #87, #71.

As the comment in one of the files says, I'm not sure what Rustix's story is wrt 64bit time on 32-bit systems.

This also adds some_or_ret_einval, which I'm down to bikeshed, but it should get rid of all the unwraps for parsing flags from bits.

I'd love to see mustang have a documented policy for how it selects the minimum Linux kernel version it supports, as well as the current minimum version.

I can imagine three possible policies here:

1. Recent kernels: require a relatively recent kernel, so that mustang can unconditionally use new system calls without fallbacks (or missing functionality if there's no reasonable fallback). This would be appropriate if mustang is primarily demonstrating the best and simplest possible implementation path, without worrying about widespread production use outside of deployment environments people can entirely control.
2. Stable kernels: Require a kernel within the last few years, roughly allowing for "last stable release of common Linux distributions". This would require some fallbacks for new functionality, but fewer of them. This would be appropriate for widespread production use of mustang, including use in containers and distribution of common tools (e.g. precompiled versions of command-line tools).
3. Enterprise kernels: Support every kernel version Rust currently supports. This would require the largest number of fallbacks, but support anyone who could possibly run mustang on any Rust system.

An official target could be much easier to use, with no mustang::can_run_here!() macro, no target json file, and no -Z build-std. And internally, it could avoid some of the linker hacks, and achieve much smaller code size, because it wouldn't need to force-link in all the libc symbols just to prevent the system libc from being linked in. And, this would be a natural moment to pick a new name for the target :smile: .

What might this look like? This issue is some early brainstorming in this space with many open questions, and more questions are welcome!

One big question is, should such a target use the c-scape C ABI compatibility layer?

• If not:

  • https://github.com/bytecodealliance/rsix/issues/76
  • https://github.com/sunfishcode/mustang/issues/38
  • Figure out a plan for mutex/rwlock, DNS resolution, and possible other stuff where mustang has more dependencies than std might want.
  • Others?

• If so:

  • I wonder if it's possible to build up c-scape and all its dependencies, including std, into a single #[crate_type = "staticlib"] library, which we'd name "libc", and then we'd have the new Rust target use that. That would mean two copies of std present in resulting programs, which might present some tricky issues, and very likely produces large binaries, but if it's not too tricky this might be a faster path to getting something working while making relatively few changes to Rust itself. And then we could work on eliminating c-scape gradually over time.

The unwind crate appears to work well for this purpose, even on panics with RUST_BACKTRACE=full.

This uses the fde-phdr-dl backend, which uses dl_iterate_phdr, so implement dl_iterate_phdr.

Fixes #27.

After a fork, the child only has one thread, and it looks like all the other threads are suspended. This may leave behind locks in locked states and cause deadlocks, however deadlocks are not considered unsafe. Consequently, fork doesn't need to be unsafe.

This also changes the on_fork callback functions to be Box<Fn() + Send> so that it's safe to call them, and adds code to c-scape and non-mustang targets to wrap them as needed.

This wasn't easy to figure out, but I think now I'm at a point where it wouldn't take me that long to fill in most of the remaining signatures of functions with a commented-out libc!.

The padding bytes are most likely only correct for x86_64. What would I have to do to figure out the right padding for x86?

As mentioned in the issue #139, any code using threads fails to link. Since this commit, rust links against a pthread_setname_np on linux. Android is unaffected. https://github.com/rust-lang/rust/commit/013986be1bcfbfd9cedddf22dfd9584ccdb7ee6e

Compiling mustang with the latest nightly gets these errors:

inlinable function call in a function with debug info must have a !dbg location
  call void @_Unwind_Resume(ptr %81) #15
inlinable function call in a function with debug info must have a !dbg location
  call void @_Unwind_Resume(ptr %101) #15
inlinable function call in a function with debug info must have a !dbg location
  call void @_Unwind_Resume(ptr %144) #15
inlinable function call in a function with debug info must have a !dbg location
  call void @_Unwind_Resume(ptr %111) #15
LLVM ERROR: Broken module found, compilation aborted!

Rust's std::env::current_dir and set_current_dir are implemented by calls to getcwd and chdir. The first half in implementing these is to add support to rsix: these should go in rsix::process. There's already a imp/linux_raw implementation of the getcwd and chdir system calls, so this needs a imp/libc implementation which should just call the libc function, and then a public API in src/process. For getcwd, the public API should take care of allocating the buffer and growing it as needed until getcwd successfully writes the whole string, returning an OsString, similar to readlinkat.

The second step is to implement c-scape support in terms of the rsix public API. It may look odd to do it this way, because rsix will be dynamically allocating, and we'll need to copy the (sub)string out of the dynamic allocated buffer into the fixed-size requested buffer. But one of the goals for c-scape is to help test rsix, and to help prepare for rsix itself to be the low-level API instead of c-scape.

Currently c-scape's implementation of memchr and similar functions is very simple. It may be possible to optimize them using the memx crate.

c-scape is compiled with #![no_builtins] to discourage the compiler from optimizing the definitions of C library functions into calls to C library functions, however the compiler can still generate calls to memcpy, memmove, memset, and memcmpy, so we need to be careful the compiler doesn't do that with the implementations of those functions :-).

Adds splice and sendfile, as those got added sometime in when I last looked at this and now.

I'm very unsure of all the unsigned to signed conversions and back here, and I'm not really sure why rustix has offsets as unsigned here.

I've uncommented the tests, but strangely enough I don't think that tests actually tests the splice or sendfile path, as those appear locked behind benches. I will add some tests for this.

I recently learned about Mermaid diagrams, and I recently had a discussion with someone who gave me the idea to make a diagram to show how some of the parts in and around Mustang relate to each other. Here's what I came up with:

graph TD;
    ripgrep["ripgrep (and other programs)"]-- optionally -->std-unmodified;
    ripgrep-- optionally -->std-ported;
    std-unmodified["std (unmodified)"]-->c-scape;
    std-ported-->origin;
    std-ported["std (ported)"]-->rustix;
    c-scape-->rustix;
    c-scape-->origin;
    origin-. optionally .->platform-libc;
    origin-->rustix;
    origin-. optionally .->linux-syscalls;
    rustix-- optionally -->platform-libc;
    rustix-- optionally -->linux-syscalls;

Mustang is the experimental project which currently includes c-scape and origin and some support tools to help compile programs to use them, allowing people to try them out on real programs such as ripgrep. This doesn't require any changes to std's code.

A sibling project is the port of std to origin and rustix. The port is not complete, though it is mostly usable since rustix calls can coexist with the existing libc calls. Compared to the Mustang approach, this approach results in simpler and more efficient code, since it avoids Rust calling through the C ABI, translating back into idiomatic Rust, and then translating into the C-like system calls or platform libc ABI. It stays in Rust until the syscalls. It also factors out a lot of low-level unsafe blocks and error handling from the higher-level logic of std.

Some people have asked about what an official Rust target using these components might look like. There are a variety of ways this could be done, though there's more work and reorganization needed. At this time, work toward this goal is on hold as there hasn't been sufficient community interest. If anyone has thoughts about what they'd like to see here, please reach out!

Both Mustang and the port of std to rustix serve as interesting testcases for rustix, so I hope to continue maintaining them for that purpose.

Rustix itself is a relatively mature project, which is used in production, and which is part of several much larger stories, including I/O safety and cap-std, which themselves are part of larger stories, so even though some of the goals are on hold, rustix will continue to have other goals, and I plan to continue to actively maintain rustix.

As reported here, this is needed for cargo-watch.

epoll has turned out to be fairly tricky, because rustix's epoll API does a significant amount of work to present a safe API, which turns out to be complex to re-expose as an unsafe C API. I started some work here toward having rustix also expose an unsafe API for epoll as well, though it's not complete yet.

This reproduces with just adding

[profile.release]
lto = true

to the top-level Cargo.toml in the mustang repo, and running the hello example. It gets lots of undefined references to things like core::panicking::assert_failed_inner. See also https://github.com/sunfishcode/mustang/issues/22#issuecomment-983208986.