Migrate C code to Rust

Overview

Travis Build Status GitHub Actions Status Azure Build Status Latest Version Rustc Version

C2Rust helps you migrate C99-compliant code to Rust. The translator (or transpiler) produces unsafe Rust code that closely mirrors the input C code. The primary goal of the translator is to preserve functionality; test suites should continue to pass after translation. Generating safe and idiomatic Rust code from C ultimately requires manual effort. However, we are building a scriptable refactoring tool that reduces the tedium of doing so. You can also cross-check the translated code against the original (tutorial).

Here's the big picture:

C2Rust overview

To learn more, check out our RustConf'18 talk on YouTube and try the C2Rust translator online at www.c2rust.com.

Documentation

To learn more about using and developing C2Rust, check out the manual. The manual is still a work-in-progress, so if you can't find something please let us know.

Installation

Prerequisites

C2Rust requires LLVM 6 or later with its corresponding clang compiler and libraries. Python 3.4 or later, CMake 3.4.3 or later, and openssl (1.0) are also required. These prerequisites may be installed with the following commands, depending on your platform:

  • Ubuntu 16.04, 18.04 & 18.10:

      apt install build-essential llvm-6.0 clang-6.0 libclang-6.0-dev cmake libssl-dev pkg-config python3
    
  • Arch Linux:

      pacman -S base-devel llvm clang cmake openssl python
    
  • NixOS / nix:

      nix-shell
    
  • OS X: XCode command-line tools and recent LLVM (we recommend the Homebrew version) are required.

      xcode-select --install
      brew install llvm python3 cmake openssl
    

Finally, installing the correct nightly Rust compiler with Rustup is required on all platforms. You will also need to add rustfmt and rustc-dev:

rustup install nightly-2019-12-05
rustup component add --toolchain nightly-2019-12-05 rustfmt rustc-dev

Installing from crates.io

cargo +nightly-2019-12-05 install c2rust

On OS X with Homebrew LLVM, you need to point the build system at the LLVM installation as follows:

LLVM_CONFIG_PATH=/usr/local/opt/llvm/bin/llvm-config cargo +nightly-2019-12-05 install c2rust

On Linux with Linuxbrew LLVM, you need to point the build system at the LLVM installation as follows:

LLVM_CONFIG_PATH=/home/linuxbrew/.linuxbrew/opt/llvm/bin/llvm-config cargo +nightly-2019-12-05 install c2rust    

Note: adjust LLVM_CONFIG_PATH accordingly if Linuxbrew was installed to your home directory.

On Gentoo, you need to point the build system to the location of libclang.so and llvm-config as follows:

LLVM_CONFIG_PATH=/path/to/llvm-config LIBCLANG_PATH=/path/to/libclang.so cargo +nightly-2019-12-05 install c2rust 

If you have trouble with building and installing, or want to build from the latest master, the developer docs provide more details on the build system.

Installing from Git

If you'd like to check our recently developed features or you urgently require a bugfixed version of c2rust you can install it directly from Git:

cargo +nightly-2019-12-05 install --git https://github.com/immunant/c2rust.git c2rust

Please note that the master branch is under constant development and you may expirience issues or crashes.

You should also set LLVM_CONFIG_PATH accordingly if required as described above.

Translating C to Rust

To translate C files specified in compile_commands.json (see below), run the c2rust tool with the transpile subcommand:

c2rust transpile compile_commands.json

(The c2rust refactor tool is also available for refactoring Rust code, see refactoring).

The translator requires the exact compiler commands used to build the C code. This information is provided via a compilation database file named compile_commands.json. (Read more about compilation databases here and here). Many build systems can automatically generate this file; we show a few examples below.

Once you have a compile_commands.json file describing the C build, translate the C code to Rust with the following command:

c2rust transpile path/to/compile_commands.json

To generate a Cargo.toml template for a Rust library, add the -e option:

c2rust transpile --emit-build-files path/to/compile_commands.json

To generate a Cargo.toml template for a Rust binary, do this:

c2rust transpile --binary myprog path/to/compile_commands.json

Where --binary myprog tells the transpiler to use the main method from myprog.rs as the entry point for a binary.

The translated Rust files will not depend directly on each other like normal Rust modules. They will export and import functions through the C API. These modules can be compiled together into a single static Rust library or binary.

There are several known limitations in this translator. The translator will emit a warning and attempt to skip function definitions that cannot be translated.

Generating compile_commands.json files

The compile_commands.json file can be automatically created using either cmake, intercept-build, or bear.

It may be a good idea to remove optimizations(-OX) from the compile commands file, as there are optimization builtins which we do not support translating.

... with cmake

When creating the initial build directory with cmake specify -DCMAKE_EXPORT_COMPILE_COMMANDS=1. This only works on projects configured to be built by cmake. This works on Linux and MacOS.

cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 ...

... with intercept-build

intercept-build (part of the scan-build tool) is recommended for non-cmake projects. intercept-build is bundled with clang under tools/scan-build-py but a standalone version can be easily installed via PIP with:

pip install scan-build

Usage:

intercept-build <build command>

You can also use intercept-build to generate a compilation database for compiling a single C file, for example:

intercept-build sh -c "cc program.c"

... with bear (linux only)

If you have bear installed, it can be used similarly to intercept-build:

bear <build command>

Contact

To report issues with translation or refactoring, please use our Issue Tracker.

To reach the development team, join our discord channel or email us at [email protected].

FAQ

I translated code on platform X but it didn't work correctly on platform Y

We run the C preprocessor before translation to Rust. This specializes the code to the host platform. For this reason, we do not support cross compiling translated code at the moment.

What platforms can C2Rust be run on?

The translator and refactoring tool support both macOS and Linux. Other features, such as cross checking the functionality between C and Rust code, are currently limited to Linux hosts.

Acknowledgements and Licensing

This material is available under the BSD-3 style license as found in the LICENSE file.

The C2Rust translator is inspired by Jamey Sharp's Corrode translator. We rely on Emscripten's Relooper algorithm to translate arbitrary C control flows.

This material is based upon work supported by the United States Air Force and DARPA under Contract No. FA8750-15-C-0124. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Air Force and DARPA. Distribution Statement A, “Approved for Public Release, Distribution Unlimited.”

Comments
  • `tinycbor` dependency doesn't build on macOS/BSD when `cjson` is installed

    `tinycbor` dependency doesn't build on macOS/BSD when `cjson` is installed

    Not having any luck because aarch64-apple-darwin doesn't exist for 2019-12-05.

    $ rustup install nightly-2019-12-05
    info: syncing channel updates for 'nightly-2019-12-05-aarch64-apple-darwin'
    info: latest update on 2019-12-05, rust version 1.41.0-nightly (6d77e45f0 2019-12-04)
    error: target 'aarch64-apple-darwin' not found in channel.  Perhaps check https://doc.rust-lang.org/nightly/rustc/platform-support.html for available targets
    

    Attempting with a current nightly:

    $ rustup component add --toolchain nightly rustfmt rustc-dev
    info: component 'rustfmt' for target 'aarch64-apple-darwin' is up to date
    info: component 'rustc-dev' for target 'aarch64-apple-darwin' is up to date
    $ cargo +nightly install c2rust
    ...
    WARN rustc_metadata::locator no metadata found: incompatible metadata version found: '/Users/n8henrie/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.asan.dylib'
    WARN rustc_metadata::locator no metadata found: incompatible metadata version found: '/Users/n8henrie/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.lsan.dylib'
    WARN rustc_metadata::locator no metadata found: incompatible metadata version found: '/Users/n8henrie/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.tsan.dylib'
    WARN rustc_metadata::locator no metadata found: incompatible metadata version found: '/Users/n8henrie/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.asan.dylib'
    WARN rustc_metadata::locator no metadata found: incompatible metadata version found: '/Users/n8henrie/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.lsan.dylib'
    WARN rustc_metadata::locator no metadata found: incompatible metadata version found: '/Users/n8henrie/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.tsan.dylib'
    error[E0463]: can't find crate for `rustc`
    

    Unsurprisingly, same error trying to build locally:

    $ git tag --sort=creatordate | tail -n 1
    0.15.1
    $ git checkout 0.15.1
    HEAD is now at 63848a29 Update c2rust-asm-casts to 0.2.0
    $ cargo +nightly build --release
       Compiling c2rust-ast-builder v0.15.0 (/Users/n8henrie/git/c2rust/c2rust-ast-builder)
       Compiling pathdiff v0.1.0
       Compiling json v0.12.0
       Compiling strum v0.16.0
       Compiling pulldown-cmark v0.6.1
       Compiling strum v0.15.0
       Compiling glob v0.2.11
       Compiling maplit v1.0.2
       Compiling is-match v0.1.0
       Compiling open v1.3.2
       Compiling c2rust-asm-casts v0.2.0 (/Users/n8henrie/git/c2rust/c2rust-asm-casts)
       Compiling libc v0.2.66
    WARN rustc_metadata::locator no metadata found: incompatible metadata version found: '/Users/n8henrie/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.asan.dylib'
    WARN rustc_metadata::locator no metadata found: incompatible metadata version found: '/Users/n8henrie/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.lsan.dylib'
       Compiling proc-macro2 v1.0.6
    WARN rustc_metadata::locator no metadata found: incompatible metadata version found: '/Users/n8henrie/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.tsan.dylib'
       Compiling syn v1.0.11
    WARN rustc_metadata::locator no metadata found: incompatible metadata version found: '/Users/n8henrie/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.asan.dylib'
    WARN rustc_metadata::locator no metadata found: incompatible metadata version found: '/Users/n8henrie/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.lsan.dylib'
    WARN rustc_metadata::locator no metadata found: incompatible metadata version found: '/Users/n8henrie/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.tsan.dylib'
    error[E0463]: can't find crate for `rustc`
     --> c2rust-ast-builder/src/lib.rs:2:1
      |
    2 | extern crate rustc;
      | ^^^^^^^^^^^^^^^^^^^ can't find crate
    
    For more information about this error, try `rustc --explain E0463`.
    error: could not compile `c2rust-ast-builder` due to previous error
    warning: build failed, waiting for other jobs to finish...
    error: build failed
    

    Is there a toolchain available for aarch64-apple-darwin that I can use to build / install c2rust?

    opened by n8henrie 31
  • Update `rustc` to `nightly-2022-08-08`

    Update `rustc` to `nightly-2022-08-08`

    Supercedes #513.

    I kept behavior identical (at least I meant to) during this update, even if it may make more sense to relax a match or avoid an .unwrap() now. We can fix those later if we determine it is correct to do so. These places I marked with TODOs.

    I fixed almost all of the errors that came up with upgrading rustc; many were similar/the same to #513, but there are a few left in c2rust-analyze that I was unsure how to handle. Some enums got more variants, and I'm not sure how we should handle those in matches, as that's new behavior. @spernsteiner, if you can help with those, that'd be great!

    opened by kkysen 29
  • Translation of C's `va_copy` into Rust's `std::ffi::VaList::copy`

    Translation of C's `va_copy` into Rust's `std::ffi::VaList::copy`

    There is no good way to perform syntax-directed translation of all possible C functions using va_list into a corresponding Rust function using std::ffi::VaList::copy. VaLists copy has the following signature:

    pub unsafe fn copy<F, R>(&self, f: F) -> R
                where F: for<'copy> FnOnce(VaList<'copy>) -> R;
    

    copy takes a closure whose input VaList cannot outlive the closure (e.g. it cannot be passed back as a result of the closure). This is so that va_end can be called on the input VaList once the closure returns.

    Since copy takes an immutable reference to the original va_list and std::ffi::VaList::arg requires a mutable reference, it is not possible to call std::ffi::VaList::arg inside the closure passed to copy. Therefore, a C function that calls va_arg on two va_lists (with one being a va_copy of the other) in the same basic block, cannot be expressed using the current Rust variadic function API

    blocked 
    opened by thedataking 23
  • Instrument using `$RUSTC_WRAPPER` and `cargo` as a subcommand

    Instrument using `$RUSTC_WRAPPER` and `cargo` as a subcommand

    Fixes https://github.com/immunant/c2rust/issues/448 and should fix https://github.com/immunant/c2rust/issues/488.

    Overview

    This reimplements c2rust-instrument by invoking cargo as a subcommand, overriding $RUSTC_WRAPPER.

    Implementation

    More specifically, instead of using cargo as a dependency, which is enormous (very long compile times) and causes some compilation issues (https://github.com/immunant/c2rust/issues/488), we invoke cargo as a subcommand. This also has the advantage of using the correct rustup cargo wrapper so that it resolves correctly. See https://github.com/immunant/c2rust/issues/448 for a more detailed explanation of why this is a better approach and why other tools like clippy and miri do it this way.

    Then we set $RUSTC_WRAPPER to our own binary. We use $RUSTC_WRAPPER to detect if we're being invoked as a cargo or rustc wrapper.

    If we're the cargo wrapper, we:

    • parse args using clap derive (https://github.com/immunant/c2rust/issues/395)
    • run cargo as a subcommand, passing:
      • the cargo args as is (so all normal cargo invocations all work)
      • our own executable as $RUSTC_WRAPPER
      • the metadata path in $C2RUST_INSTRUMENT_METADATA

    If we're the rustc wrapper, we:

    • determine if we should instrument the current rustc command if $CARGO_PRIMARY_PACKAGE is set
    • run RunCompiler::new(...).run() with our MirTransformCallbacks, or TimePassesCallbacks::default() (rustc's default) if not instrumenting
    • then save the metadata to the metadata file

    Note that we have to pass the --sysroot to RunCompiler as normally rustc looks this up relative to its executable's location, which works when the rustup rustc wrapper resolves to the toolchain-specific rustc, but since we're using RunCompiler directly and being rustc, we need to explicitly set the sysroot.

    Usage Changes

    Instead of passing c2rust_analysis_rt as an --extern and running cargo separately to compile it, we just add it as a normal, optional dependency to the crate we're instrumenting. In this way, it's similar to libc, which can be externed as it's in the sysroot, but the preferred way is to specify it as a normal dependency. Thus, we can also drop the runtime argument that seemed odd to have to specify to c2rust instrument.

    The existing c2rust-instrument binary in the c2rust package is now gone (along with its --features dynamic-instrumentation), replaced by the new c2rust-instrument binary in dynamic-instrumentation/src/bin.rs (the crate has also been renamed to c2rust-instrument). Now, c2rust_instrument no longer exposes a library crate, and is purely a binary crate. This is because there's no way to use c2rust-instrument not as a binary, as it invokes itself and thus needs control over main to function correctly.

    This does cause a slight change in that c2rust no longer sees the instrument.yaml; only c2rust-instrument does. This could potentially hurt shell autocompletion in the future (not currently implemented), but the drawbacks seems slim. Instead, we don't need to remember --features dynamic-instrumentation as a simple cargo build builds everything. And we've upgraded to a superior clap 3 derive parser, not the deprecated YAML parser.

    Fast!

    Furthermore, now that we use cargo normally and include c2rust_analysis_rt as a normal dependency, we can delete only the main binary (with cargo clean --package), forcing it to be rebuilt and re-instrumented. Most of the time building used to be in building c2rust_analysis_rt, and on the fast donna, was ~15 seconds for c2rust instrument. Now it takes only ~0.5 seconds, 30x faster.

    Also enabling this is using a separate $CARGO_TARGET_DIR: instrument.target instead of the default target. This allows builds not to conflict with normal, non-instrumented builds using cargo build, so we don't need to cargo clean everything every time.

    Thus, we now also unconditionally instrument (in pdg.sh). Before, we only did when the c2rust-instrument binary changed, but this missed changes to the instrumented crate. Now that it's fast enough, we just always instrument.

    Previously, I was working on getting incremental instrumentation to work, but that proved harder than I thought for the corner cases (which did pop up). With this change, we remove the largest block to fast iteration times, so I think we don't need incremental instrumentation at all until we get to iterating on small edits in large instrumented crates.

    Fixes and Improvements

    • https://github.com/immunant/c2rust/issues/448: the primary fix. Now we can specify any cargo args, such as build, run, test, --release, etc.
    • https://github.com/immunant/c2rust/issues/488: should be fixed as the offending cargo dependency is removed, but needs to be tested by @boyland-pf.
    • We can remove pretty-instrument-err.mjs and get-binary-names-from-cargo-metadata.mjs, the temporary node scripts.
      Thus, we can remove all node dependencies.
    • With enough polishing, we may be able to entirely get rid of pdg.sh.
    • The cargo dependency is dropped, saving a lot on compile time, as it is a huge dependency.
    • The --features dynamic-instrumentation is removed, so a simple cargo build works for everything now.
    opened by kkysen 21
  • Add `--all-features`, `cargo fmt --check`, `cargo check`, `cargo test`, `cargo doc`, and `RUSTFLAGS=-Dwarnings` to CI

    Add `--all-features`, `cargo fmt --check`, `cargo check`, `cargo test`, `cargo doc`, and `RUSTFLAGS=-Dwarnings` to CI

    Starts to f ix https://github.com/immunant/c2rust/issues/477.

    This adds --all-features/--features dynamic-instrumentation, cargo fmt --check, cargo check, cargo test, cargo doc, and RUSTFLAGS=-Dwarnings to CI. Only cargo clippy is left.

    This adds about 5 minutes to CI, going from 10-15 minutes to 15-25 minutes. This is mainly from:

    • cargo check --all-features: ~5 minutes
    • cargo test: ~3 minutes (mostly pdg snapshot and static analysis tests)

    On the other hand, these take practically no time:

    • cargo fmt --check: 1 second
    • cargo doc --no-deps: 6 seconds

    Previous we only had:

    • [x] cargo build
    • [x] ./scripts/test_translator.py tests/

    Now we have added:

    • [x] cargo fmt --check
    • [x] RUSTFLAGS='-D warnings' cargo check --all-features
    • [x] RUSTFLAGS='-D warnings' cargo build
    • [x] RUSTFLAGS='-D warnings' cargo test
    • [x] RUSTDOCFLAGS='-D warnings' cargo doc --all-features --document-private-items

    We still have these to go:

    • [ ] RUSTFLAGS='-D warnings' cargo clippy --tests --all --all-features (includes cargo check): https://github.com/immunant/c2rust/issues/474

    For commands that check only and do not run (cargo check, cargo doc), we specify --all-features, which is equivalent to --features llvm-static. But for commands that do build/run (cargo build, cargo test), we don't specify --all-features as we don't want to test --features llvm-static by default, and it doesn't work on the Arch and Fedora docker images (see https://github.com/immunant/c2rust/issues/500).

    Currently, cargo test -p c2rust-analyze is excluded as it fails on some OSes in CI (https://github.com/immunant/c2rust/issues/593). It will be re-enabled once that's fixed, but I want to get cargo test into CI as soon as possible.

    opened by kkysen 20
  • StmtExpr dropped during translation (was lseek not translated)

    StmtExpr dropped during translation (was lseek not translated)

    example:

    #include <unistd.h>
    #include <fcntl.h>
    #include <sys/stat.h>
    
    uintptr_t virt_to_phys(void* virt) {
    	long pagesize = sysconf(_SC_PAGESIZE);
    	int fd = open("/proc/self/pagemap", O_RDONLY);
    	lseek(fd, (uintptr_t) virt / pagesize * sizeof(uintptr_t), SEEK_SET);
    	uintptr_t phy = 0;
    	read(fd, &phy, sizeof(phy));
    	return (phy & 0x7fffffffffffffULL) * pagesize + ((uintptr_t) virt) % pagesize;
    }
    

    is translated to

    #![allow(dead_code,
             mutable_transmutes,
             non_camel_case_types,
             non_snake_case,
             non_upper_case_globals,
             unused_assignments,
             unused_mut)]
    extern crate libc;
    extern "C" {
        #[no_mangle]
        fn read(__fd: libc::c_int, __buf: *mut libc::c_void, __nbytes: size_t)
         -> ssize_t;
        #[no_mangle]
        fn sysconf(__name: libc::c_int) -> libc::c_long;
        #[no_mangle]
        fn open(__file: *const libc::c_char, __oflag: libc::c_int, _: ...)
         -> libc::c_int;
    }
    (...)
    #[no_mangle]
    pub unsafe extern "C" fn virt_to_phys(mut virt: *mut libc::c_void)
     -> intptr_t {
        let mut pagesize: libc::c_long = sysconf(_SC_PAGESIZE as libc::c_int);
        let mut fd: libc::c_int =
            open(b"/proc/self/pagemap\x00" as *const u8 as *const libc::c_char,
                 0i32);
        let mut phy: intptr_t = 0i32 as intptr_t;
        read(fd, &mut phy as *mut intptr_t as *mut libc::c_void,
             ::std::mem::size_of::<intptr_t>() as libc::c_ulong);
        panic!("Reached end of non-void function without returning");
    }
    
    bug 
    opened by emmericp 18
  • Add info needed for static analysis to PDG `Node`s

    Add info needed for static analysis to PDG `Node`s

    Checks whether nodes flow to loads/stores/offsets, and whether they are unique.

    Uniqueness of node X is determined based on whether there is a node Y which

    • is X's ancestor through copies, fields, and offsets
    • is a node Z's ancestor through copies, fields, offsets, where the fields are the same between X and Z
    • Z is chronologically between X and X's last descendent
    opened by boyland-pf 16
  • make build.rs scripts work without rustup

    make build.rs scripts work without rustup

    Fixes https://github.com/immunant/c2rust/issues/516.

    Rustup is useful to install our dependencies at the versions we need, but we don't need to explicitly depend on it; users may install dependencies other ways, too.

    opened by fw-immunant 16
  • Introduce a `path!` macro for creating items from simple paths

    Introduce a `path!` macro for creating items from simple paths

    This PR introduces the path! macro which can be used like

    let p: syn::Path = path![::core::ptr::null_mut];
    let e: syn::Expr = path![Vec::new];
    let t: syn::Type = path![::libc::c_char];
    

    It can output any type T which has impl Make<T> for Path. Currently this means Path, Type and Expr. Currently the macro supports only simple raw paths, i.e. no variable interpolation (only literal identifiers), no qualified self and no generic parameters. The restriction on simple interpolation of variables may be lifted in a later PR. The restriction on qualified self and generics will likely no be lifted, since they seem far beyond the capabilities of reasonable declarative macros.

    This PR also adds impl Make<Path> for slices and arrays. This allows to avoid unnecessary allocations when creating paths (almost all paths created by the transpiler have a statically known size). As an extra benefit, switching from vec![..] to [..] construction looks more lightweight, and allows processing of the array by rustfmt (vec! calls are not formatted properly in general).

    This PR also specializes the methods of Builder with respect to most generic parameters. Those generics were redundant since each position could be occupied only by a single specific type. E.g. Make<Box<Expr>> is implemented only by Box<Expr>, so that is always the type of impl Make<Box<Expr>> arguments.

    Specializing those generic parameters has no restriction on the practical usability of the API, and it in fact makes it more flexible: generic parameters break type inference. If I have fn foo<T: Trait>(t: T) and fn bar<R>() -> R, then foo(bar()) will result in a compile error since the type of R is generally impossible to infer. The type inference is heavily used by the path! macro to overload on the return type.

    As a bonus, specializing the parameters allows to remove a significant number of redundant explicit type casts. It also results in better compile speed (generics aren't fully compiled in the defining crate, and are instantiated separately in each codegen unit). The speed benefit is minor, but a nice touch.

    opened by afetisov 15
  • Refactoring, cleanup, and generic testing script

    Refactoring, cleanup, and generic testing script

    I added a script, ./scripts/pdg.sh <test crate root> <args...>, that instruments and runs a test crate, such as analysis/test or lighttpd. This is a more generic version of @aneksteind's saved command.

    • It only re-builds/instruments if the c2rust-instrument binary is newer than metadata.bc.
    • It saves the c2rust instrument output to instrument.{out,err}.log and the pdg output to pdg.log.
    • It supresses the many warnings from the test crate with RUSTFLAGS="-Awarnings".
    • It uses ./scripts/get-get-binary-names-from-cargo-metadata.mjs to parse cargo metadata to know what default binary targets to run (because cargo run doesn't work with the instrumented binary).

    Then I cleaned up a bunch of things:

    • Removed unused imports.
    • _-prefixed unused variables, functions, types, etc.
    • Ran rustfmt. Now everything builds without warnings.

    And then I refactored some things in c2rust_pdg:

    • Switched to using the RAII c2rust_analysis_rt::Runtime instead of initialize and finalize calls.
    • Added anyhow for error-handling instead of .unwrap()ing everywhere.
    • Made EVENT_TRACE_FILE_PATH a normal variable in fn main instead of a lazy_static!, since it's only used once.
    • Gave some variables more expressive names.
    • Used new inline formatting strings (e.x. "{x:?}").
    • Skipped some String allocs that were unneeded.
    • Switched an Into impl to a From one.
    • Fixed doc comments on newtype_index! types.
    • #[derive(Default)]ed some structs that had similar new methods.
    • Refactored read_event_log to use iter::from_fn, which is way cleaner.
    • Added use EventKind::* wildcard imports right before large matches to reduce boilerplate.
    • Added a trait EventKindExt impled for EventKind for all the functions that took an &EventKind that were method-like.
    • Refactored has_parent out of parent (previously get_parent_object), since it only returned None or Some(obj).
    • Replaced uses of usize that were really Pointer (c2rust_analysis_rt::Pointer) with Pointer for clarity.
    • Removed some unnecessary .clone()s (as in they were no-ops).
    • Switched &Vec<_> args to &[_].
    opened by kkysen 15
  • Add address-taken local support in dynamic analysis

    Add address-taken local support in dynamic analysis

    We want to track usage of the memory of address-taken locals in dynamic analysis and in the pointer derivation graph for static analysis. This adds support for that. This also tracks operations on the local's memory even before the local becomes address-taken. Additionally, support for address-taken arguments have been added. Taking references of locals with & and &mut are also considered taking their address.

    Support for address-taken locals is done in three stages: identifying where address-taken locals get their address taken, a rewriting stage, and post-processing. The first two stages are accomplished in dynamic analysis with the introduction of two new MIR visitors and the last occurs during construction of the pointer derivation graph.

    The first traverses the AST and identifies where locals have their addresses taken and stores that information in a data structure for reference in the next stage. The next stage is another MIR visit that rewrites all address-taken locals in terms of their address. For example, if _x is an address-taken local and _y stores its address, _x is rewritten as *_y. These two stages are conducted before instrumentation points are collected in the third and final MIR visitor.

    The final stage ensures that all duplicate instances of taking the address of a particular local are treated as copies of its address.

    Below are a few examples of what is now supported:

    pub fn test_addr_taken() {
        let x = 2;
        let y = 2 + x;
        let px = std::ptr::addr_of!(x);
        let z = x + y;
    }
    
    pub fn test_addr_taken_cond(cond: bool) {
        // let x = if cond { ... } else { ... }; ... &x ...
        let x = if cond { 1 } else { 2 };
        let y = &x;
        let a = x;
    }
    
    pub fn test_addr_taken_init_cond(cond: bool) {
        // let mut x; if cond { x = 1; ... &x ... }; x = 2; ... &x ...
        let mut x;
        let mut y;
        if cond {
            x = 1;
            y = &x;
        }
        x = 2;
        let z = x;
    }
    
    #[no_mangle]
    pub fn test_addr_taken_loop() {
        // loop { let x = ...; ... &x ... }
        let mut count = 0;
        loop {
            let x = 2;
            let z = if count % 2 == 0 { &x } else { &1 };
            if count >= 3 {
                break;
            }
            count += *z;
        }
    }
    
    #[no_mangle]
    pub fn test_addr_taken_arg(mut t: T) {
        t.field3 = 0 as *const S;
        let z = &t;
    }
    
    opened by aneksteind 14
  •  Support struct fields in static analysis

    Support struct fields in static analysis

    This adds support for generating concrete Origins to pass to polonius for each ADT lifetime parameter and resolving lifetime parameters to those origins when visiting the MIR body. It also adds support for ADT field pointer permissions.

    There are three stages to making this possible:

    1. gather hypothetical (i.e. pointer-based) and actual (reference-based) lifetimes
    2. generating concrete origins
    3. resolving lifetime parameters when visiting the MIR body

    Stage 1 runs in a loop and gathers ADT metadata. The loop terminates when a fixed point is reached, as determined by changes in the ADT metadata. This approach is to handle ADTs that are recursive and/or mutually recursive.

    Stage 2 generates one concrete Origin per ADT lifetime parameter, per MIR local. When grabbing the Origin of a given struct field, even nested ones, it is one of these concrete origins that will be returned.

    Stage 3 traces the concrete Origins generated for each ADT lifetime parameter all the way to the last projection / field access. It does this by updating a mapping for each projection element visited.

    TODO

    • [ ] add snapshot testing to origin-labeled output
    opened by aneksteind 0
  • Skip or Add `NonAtomicToAtomic` Support Within C99

    Skip or Add `NonAtomicToAtomic` Support Within C99

    A request to add support or skip support for the functionality of casting regular types to a atomic type feature.

    Error:

    thread 'main' panicked at 'Unsupported implicit cast: NonAtomicToAtomic', /.../c2rust-transpile-0.16.0/src/c_ast/conversion.rs:167:14
    

    Clang Version: Ubuntu clang version 14.0.0-1ubuntu1 Target: x86_64-pc-linux-gnu Thread model: posix

    File Link: https://github.com/immunant/c2rust/blob/master/c2rust-transpile/src/c_ast/conversion.rs

    opened by plutoup 5
  • C11 `_Generic` is not supported

    C11 `_Generic` is not supported

    When transpiled code contains C11's _Generic macros, this warns:

    warning: c2rust: Encountered unsupported generic selection expression
    

    Please consider adding support for this.


    As I assume it's not easy to add, and RIOT-OS might be one of the few users (for many other use cases of C2Rust are about converting legacy code bases that don't use modern C features), I'll perfectly understand if this is a low priority issue -- in that case, the issue still serves tracking purposes (I can reference it in workarounds, and see that the workarounds can be removed when this is closed), and to collect any workaround pointers.

    enhancement 
    opened by chrysn 0
  • Incorrect translation of `alloca` in block scope

    Incorrect translation of `alloca` in block scope

    Tested with c2rust 0.16.0

    #include <stdio.h>
    #include <string.h>
    
    int one = 1;
    
    int main(void) {
      char *p;
      if (one) {
        p = __builtin_alloca(100);
      }
      strcpy(p, "Hello, world!");
      puts(p);
    }
    

    This is a program that has no memory errors. The if (one) is guaranteed to be entered, though the compiler cannot see that, and the result of __builtin_alloca persists until the function ends, even after the block has ended.

    It is translated to the following Rust code:

    #![allow(dead_code, mutable_transmutes, non_camel_case_types, non_snake_case, non_upper_case_globals, unused_assignments, unused_mut)]
    #![register_tool(c2rust)]
    #![feature(register_tool)]
    use ::c2rust_out::*;
    extern "C" {
        fn puts(__s: *const libc::c_char) -> libc::c_int;
        fn strcpy(_: *mut libc::c_char, _: *const libc::c_char) -> *mut libc::c_char;
    }
    #[no_mangle]
    pub static mut one: libc::c_int = 1 as libc::c_int;
    unsafe fn main_0() -> libc::c_int {
        let mut p: *mut libc::c_char = 0 as *mut libc::c_char;
        if one != 0 {
            let mut fresh0 = ::std::vec::from_elem(
                0,
                100 as libc::c_int as libc::c_uint as usize,
            );
            p = fresh0.as_mut_ptr() as *mut libc::c_char;
        }
        strcpy(p, b"Hello, world!\0" as *const u8 as *const libc::c_char);
        puts(p);
        return 0;
    }
    pub fn main() {
        unsafe { ::std::process::exit(main_0() as i32) }
    }
    

    Note here the let mut fresh0 = ::std::vec::from_elem(0, 100 as libc::c_int as libc::c_uint as usize); in block scope. Here, unlike in the C program, the vec will be dropped when the block is exited, and p is left a dangling pointer.

    A possible corrected translation would be to make fresh0 a function-scope variable.

    bug 
    opened by hvdijk 2
  • Incorrect translation of post-modification of bitfield

    Incorrect translation of post-modification of bitfield

    Tested using c2rust 0.16.0

    struct S {
      int i : 4;
    } s;
    
    int main(void) {
      int x = s.i++;
      int y = --s.i;
      return x + y;
    }
    

    This program returns zero. It is translated to a Rust program that returns -1.

    The generated Rust code is

    #![allow(dead_code, mutable_transmutes, non_camel_case_types, non_snake_case, non_upper_case_globals, unused_assignments, unused_mut)]
    #![register_tool(c2rust)]
    #![feature(register_tool)]
    #[macro_use]
    extern crate c2rust_bitfields;
    use ::c2rust_out::*;
    #[derive(Copy, Clone, BitfieldStruct)]
    #[repr(C)]
    pub struct S {
        #[bitfield(name = "i", ty = "libc::c_int", bits = "0..=3")]
        pub i: [u8; 1],
        #[bitfield(padding)]
        pub c2rust_padding: [u8; 3],
    }
    #[no_mangle]
    pub static mut s: S = S {
        i: [0; 1],
        c2rust_padding: [0; 3],
    };
    unsafe fn main_0() -> libc::c_int {
        let ref mut fresh0 = s.i();
        let fresh1 = *fresh0;
        *fresh0 = *fresh0 + 1;
        let mut x: libc::c_int = fresh1;
        s.set_i(s.i() - 1);
        let mut y: libc::c_int = s.i();
        return x + y;
    }
    pub fn main() {
        unsafe { ::std::process::exit(main_0() as i32) }
    }
    

    Note here the *fresh0 = *fresh0 + 1;. This does not have the desired effect of modifying s.i. Instead, it modifies the temporary returned by s.i(), a temporary which is never again used after being assigned to.

    The pre-decrement, on the other hand, is correctly translated into a call to s.set_i.

    One possible corrected translation would be

    unsafe fn main_0() -> libc::c_int {
        let fresh0 = s.i();
        s.set_i(fresh0 + 1);
        let mut x: libc::c_int = fresh0;
        s.set_i(s.i() - 1);
        let mut y: libc::c_int = s.i();
        return x + y;
    }
    
    bug 
    opened by hvdijk 0
Releases(v0.16.0)
  • v0.16.0(May 12, 2022)

    C2Rust now uses the syn crate for Rust AST construction, which means we no longer depend on rustc compiler internals and can build with a stable Rust toolchain! Huge thanks to @fw-immunant for all the hard work involved in decoupling the transpiler from Rust internals.

    In the process, we've added support for transpiling C inline assembly to stable Rust asm! blocks. C2Rust also now supports building against newer LLVM versions (up to version 13). Unfortunately we have not yet re-implemented translation of C comments into Rust using the new AST framework, but expect that to be fixed soon!

    As part of moving away from compiler internals, we have deprecated the refactoring engine and cross-checks. Source code for the refactoring engine is still available in tree, but you may encounter issues trying to use it and we're not planning to put any substantial effort in that direction in the near future.

    Other changes:

    Source code(tar.gz)
    Source code(zip)
  • 0.5.3(Feb 14, 2022)

  • 0.15.0(Oct 21, 2020)

    This release rolls up various maintenance and adds support for a few new language features. C2Rust now supports LLVM 10.

    Other changes:

    • Improved refactoring: reorg now reorganizes (approximately) 100% better, lua script support is now more robust, and refactoring correctly handles #[cfg] annotations in the input Rust code.
    • --preserve-unused-functions added to c2rust transpile. This option preserves all functions, regardless of usage in the program. Only turn this on if you know you need it! Thanks to @chrysn for this new feature.
    • @chrysn added support for AAPCSABIBuiltinVaList. Thanks again!
    Source code(tar.gz)
    Source code(zip)
  • 0.14.0(Dec 19, 2019)

    C2Rust now merges header declarations and imports them with proper Rust use statements instead of duplicating header items in every file. To take advantage of this new feature, you will need to transpile with --emit-build-files. See our blog post for a deep dive into how this works.

    This release updates C2Rust to use rustc nightly-2019-12-05, so be sure to install with cargo +nightly-2019-12-05 install c2rust.

    Other changes:

    • Updating to nightly-2019-12-05 fixes a compiler bug that was sometimes triggered by transpiled code.
    • Added support for AArch64 variadic function definitions.
    • Added experimental refactoring support for rewriting parameters and locals from raw pointers into safe Rust types.
    • Fixed infinite loop in const C macro expansion.
    • Significant improvements to the Lua refactoring API: Reading and writing fields is now supported for all AST nodes through auto-generated Lua bindings.
    • LLVM 9 support
    Source code(tar.gz)
    Source code(zip)
  • 0.12.0(Aug 23, 2019)

    This release improves support for translating variadic functions. C2Rust now supports translation of va_start, va_end, and va_copy into equivalent Rust APIs.

    Source code(tar.gz)
    Source code(zip)
  • 0.11.0(Aug 15, 2019)

    We've made some significant improvements, so it's past time for a new crates.io release. C2Rust now requires nightly 2019-06-22, so install with cargo +nightly-2019-06-22 install c2rust.

    Changes:

    • Update to nightly 2019-06-22 (needed for va_copy support)
    • Improved support for flexible array members
    • Better error messages
    • Translate constant expression C macros to Rust const globals
    • Improved support for inline assembly
    • Support GNU __builtin_choose_expr
    • Support K&R style C function pointers (unspecified arguments)
    • Improve module reorganization (consolidating duplicate header items across modules)
    • Speed up refactoring
    • Expanded Lua refactoring script API
    • Update to using c2rust-bitfields 0.2.0
    • Preserve comments during translation
    • Add support for translating multiple binaries in the same project
    • More refactoring capabilities
    Source code(tar.gz)
    Source code(zip)
  • 0.10.0(Apr 15, 2019)

    You can now install C2Rust directly from crates.io: cargo +nightly-2019-04-12 install c2rust.

    Variadic argument function definitions are now supported (if va_copy is not used). Other improvements:

    • Lua refactoring script interface
    • New example project translations: tinycc
    • Improved support for generating Cargo.toml build files
    • Support for translating to no_std Rust
    • Improved header and declaration handling
    • More robust for loop reconstruction refactoring
    • Refactoring support for printf to println! conversion
    • LLVM 8 support
    • Support for translating more C features (and extensions):
      • long double type
      • GCC atomic builtins
      • thread-local variables
      • inline attributes
      • more SIMD intrinsics
    Source code(tar.gz)
    Source code(zip)
Owner
Immunant
Information Security R&D
Immunant
Minimal, flexible framework for implementing solutions to Advent of Code in Rust

This is advent_of_code_traits, a minimal, flexible framework for implementing solutions to Advent of Code in Rust.

David 8 Apr 17, 2022
Simplified glue code generation for Deno FFI libraries written in Rust.

deno_bindgen This tool aims to simplify glue code generation for Deno FFI libraries written in Rust. Quickstart # install CLI deno install -Afq -n den

Divy Srivastava 173 Dec 17, 2022
Easy to use Rust i18n library based on code generation

rosetta-i18n rosetta-i18n is an easy-to-use and opinionated Rust internationalization (i18n) library powered by code generation. rosetta_i18n::include

null 38 Dec 18, 2022
A simple code boilerplate generator written in Rust.

?? Cgen What is Cgen? A modern, cross-platform, multi-language boilerplate generator aimed to make your code generation less hectic! If you wish to su

Rithul Kamesh 1 Dec 25, 2021
A code coverage tool for Rust projects

Tarpaulin Tarpaulin is a code coverage reporting tool for the Cargo build system, named for a waterproof cloth used to cover cargo on a ship. Currentl

null 1.8k Jan 2, 2023
Rust macro that uses GPT3 codex to generate code at compiletime

gpt3_macro Rust macro that uses GPT3 codex to generate code at compiletime. Just describe what you want the function to do and (optionally) define a f

Maximilian von Gaisberg 59 Dec 18, 2022
Generate bindings to use Rust code in Qt and QML

Rust Qt Binding Generator This code generator gets you started quickly to use Rust code from Qt and QML. In other words, it helps to create a Qt based

KDE GitHub Mirror 768 Dec 24, 2022
A rollup plugin that compile Rust code into WebAssembly modules

rollup-plugin-rust tl;dr -- see examples This is a rollup plugin that loads Rust code so it can be interop with Javascript base project. Currently, th

Fahmi Akbar Wildana 37 Aug 1, 2022
Common utilities code used across Fulcrum Genomics Rust projects

fgoxide Common utilities code used across Fulcrum Genomics Rust projects. Why? There are many helper functions that are used repeatedly across project

Fulcrum Genomics 2 Nov 2, 2022
A library for generating TypeScript definitions from rust code.

Tsify is a library for generating TypeScript definitions from rust code. Using this with wasm-bindgen will automatically output the types to .d.

Madono Haru 60 Dec 30, 2022
Rust library to scan files and expand multi-file crates source code as a single tree

syn-file-expand This library allows you to load full source code of multi-file crates into a single syn::File. Features: Based on syn crate. Handling

Vitaly Shukela 11 Jul 27, 2022
A simple quote-based code generator for Rust

flexgen A flexible, yet simple quote-based code generator for creating beautiful Rust code Why? Rust has two types of macros, and they are both very p

Scott Meeuwsen 11 Oct 13, 2022
A cargo subcommand that displays the assembly generated for Rust source code

cargo-show-asm A cargo subcommand that displays the assembly generated for Rust source code.

null 193 Dec 29, 2022
Reload Rust code without app restarts. For faster feedback cycles.

hot-lib-reloader hot-lib-reloader is a development tool that allows you to reload functions of a running Rust program. This allows to do "live program

Robert Krahn 274 Jan 1, 2023
Code examples, data structures, and links from my book, Rust Atomics and Locks.

This repository contains the code examples, data structures, and links from Rust Atomics and Locks. The examples from chapters 1, 2, 3, and 8 can be f

Mara Bos 338 Jan 6, 2023
A code generator to reduce repetitive tasks and build high-quality Rust libraries. 🦀

LibMake A code generator to reduce repetitive tasks and build high-quality Rust libraries Welcome to libmake ?? Website • Documentation • Report Bug •

Sebastien Rousseau 27 Mar 12, 2023
Mix async code with CPU-heavy thread pools using Tokio + Rayon

tokio-rayon Mix async code with CPU-heavy thread pools using Tokio + Rayon Resources Documentation crates.io TL;DR Sometimes, you're doing async stuff

Andy Barron 74 Jan 2, 2023
Waits until the exit code of a program is zero

Waitz A rust utility to wait that a program exits with 0. You need to wait for something to start up and don't know when it finishes?

Max Strübing 15 Apr 10, 2022
Sample code for compute shader 101 training

Sample code for Compute Shader 101 This repo contains sample code to help you get started writing applications using compute shaders.

Google Fonts 332 Jan 1, 2023