A safe-against-invalid-input version of wren.io, written in Rust.

Overview

safe_wren

A nearly-complete implementation of the Wren language (wren.io) in Rust.

The original https://github.com/wren-lang/wren from wren.io is refered to as wren_c to disambiguate from this safe_wren.

This should be considered a alpha release. safe_wren has only been used locally for test cases and command line, we've not yet seen it integrated with larger programs. Feedback welcome: https://github.com/RubberDuckEng/safe_wren/issues

Similaries to wren_c

  • Passes ~90% of wren_c tests
  • Exposes ~90% of wren_c C API

Differences from wren_c

  • Never crashes to bad input (but can currently still be timed-out via infinite loops)
  • Reference-counted, not (yet) garbage collected (in progress).
  • Stops at first compile error (instead of continuing)
  • Requires utf8 input and wren Strings are strings of utf8, not byte-buffers (safe_wren does not allow invalid utf8 bytes)
  • Does not allow overriding allocator (yet)
  • Still missing opt-meta, and class attributes
  • Currently about 2x slower than wren_c on some microbenchmarks (should be comparable after GC work completes)

Usage

From an existing C project:

cargo build --release produces target/release/libsafe_wren.a and target/release/libsafe_wren.so which are drop-in replacements for wren.a and wren.so and compatible with wren.h found at (wren_c/src/include/wren.h).

From Rust:

Add a dependency to your cargo.toml, e.g.

[dependencies]
safe_wren = "0.1.0"

Development

Two binaries are provided for development:

  • wren_test -- used for testing, uses only public API
  • wren_debug -- used for debugging vm, uses private calls.

cargo run FILENAME_OR_STRING will run wren_test against a file or string.

cargo run --bin=wren_debug COMMAND FILENAME_OR_STRING will run wren_debug

wren_debug commands:

  • tokenize Dumps token stream.
  • compile Dumps compiler bytecode.
  • interpret Similar to wren_test, excepts prints VM state after run.

python3 util/test.py will run the tests, including updating test_results/* with error text from any failed tests. test.py will also update test_results/passes.txt with the list of passing tests.

python3 util/to_fix.py will summarize test_results/*.

test_results/test_expectations.txt lists all currently skipped tests and why.

python3 utils/compare.py will compare output for wren_test from safe_wren vs. wren_c.

cargo fuzz works, follow instructions at https://rust-fuzz.github.io/book/cargo-fuzz.html will require using nightly rust toolchains as of Oct 2021. I haven't seen crashes in months mostly it finds timeouts due to lack of a timeout mechanism in safe_wren (yet).

@eseidel's normal development loop:

  1. Figure out what the next issue to tackle (this README.md or test_results/test_expectations.txt).
  2. Implement it, fight the rust compiler until it compiles.
  3. Write a small test in tests/bringup.
  4. Use cargo run on the test, if fails go back to debugging.
  5. Use cargo run --bin=wren_debug interpret to see what interpreter is doing.
  6. Use cargo build && python3 util/test.py; python3 util/to_fix.py to run all tests and update common_test_errors.txt.
  7. Repeat.

Work yet to do

Launch list?

  • Example using rust API
  • Example using C API
  • Announce to wren-lang
  • Generate and publish Rust docs.

Ordered goals?

  • Garbage Collection, in progress: https://github.com/RubberDuckEng/vmgc
  • fix local_outside_method.wren
  • Time the tests / make faster (next is vec::alloc from method calls)
  • Fancier test_expectations system ** Config / Expectation pairs (c | FAIL, RUST | TIMEOUT)
  • Teach utils/test.py how to easily switch between rust and c_rust and c
  • remove all uses of 'as' (use into() instead).
  • validate_superclass could now use ClassSource to validate internal, etc.
  • String codepoint APIs (including String.iterate)
  • wrong line numbers for foreign method runtime errors.
  • Class attributes: https://wren.io/classes.html#attributes
  • rust implementation of meta package.
  • continue after failure during compiling?
  • \x should not round-trip through char.
  • Sort methods to match wren_c order?
  • Variable should use a different type for each scope type.
  • fuzz both wren_c and safe_wren and compare output?
  • Add a timeout mechanism for slow/infinite loops/scripts.
  • Look at some of the slow-unit fuzz results ** fuzz/artifacts/fuzz_target_1/slow-unit-63ea01d2d5ba869bdb889c3b51b21350d5a4ffea (lookup_symbol should be a hash) ** fuzz/artifacts/fuzz_target_1/slow-unit-355b25c3fc10bfd14a363cf737abf3a07bab4a1e (needless stack resizing)
  • wren_debug interpret wren_c/test/language/static_field/nested_class.wren does out of bound lookup in line_for_pc.

Leads to pursue

Benchmarking notes

  • Current numbers show safe_wren to be about 2.5x-9x slower than wren_c across the various microbenchmarks. Unclear what real-world effect this would have.
  • Value::clone is apparent in many benchmarks.
  • Using move/drain semantics when calling args from the stack could help avoid needing to clone values when converting them with try_into_X. Or at least make try_into_X use move semenatics and the clone explicit in the caller.
  • class_for_value under call_method shows up at ~5% on several benchmarks.
  • map_numeric heavily tests Value::PartialEq
  • binary_trees leans heavily (at least 20% of time) on ptr::drop_in_place (deallocation) of RefCell. GC would reduce this.
  • map_string spends 57% of time in core::string_plus, and 20% of time truncating the stack.

wren_c bugs

  • closures/functions defined in wren_core.wren end up with a null class pointer?
  • If you yield from the root, it gets set to state=OTHER, presumably later you might be able to call things on it?
  • WrenConfiguration likely leaked for each WrenVM constructed/destructed?
You might also like...
Totally Speedy Transmute (TST) is a library providing a small, performance oriented, safe version of std::mem::transmute

Totally Speedy Transmute An evil spiritual successor to Totally Safe Transmute What is it? Totally Speedy Transmute (TST) is a library providing a sma

compare gdnative rust based physics against Godot built-in physics
compare gdnative rust based physics against Godot built-in physics

Godot vs. Rapier Rapier is an open source physics framework written in Rust. This project pits godots built-in physics against Rapier. It uses godot-r

Fastest and safest Rust implementation of parquet. `unsafe` free. Integration-tested against pyarrow

Parquet2 This is a re-write of the official parquet crate with performance, parallelism and safety in mind. The five main differentiators in compariso

service_policy_kit is a Rust based toolkit for verifying HTTP services against policies.
service_policy_kit is a Rust based toolkit for verifying HTTP services against policies.

Service Policy Kit service_policy_kit is a Rust based toolkit for verifying HTTP services against policies. You can: Build a complete testing framewor

Rust crate for linking against the RDKit C++ API

RDKit-Sys Rust code that binds to the C++ rdkit library! How does it work? RDKit is a C++ mega-library, full of cheminformatics wisdom. We don't want

Use your computer as a cosmic ray detector! One of the memory errors Rust does not protect against.

Your computer can double up as a cosmic ray detector. Yes, really! Cosmic rays hit your computer all the time. If they hit the RAM, this can sometimes

Library for scripting analyses against crates.io's database dumps

crates.io database dumps Library for scripting analyses against crates.io's database dumps. These database dumps contain all information exposed by th

Android Device Pool - A tool to run device tests against a pool of devices.

adp (Android Device Pool) What is this? A tool to run device tests against a pool of devices. It will 'checkout' a device to run your tests against an

secmem-proc is a crate designed to harden a process against low-privileged attackers running on the same system trying to obtain secret memory contents of the current process.

secmem-proc is a crate designed to harden a process against low-privileged attackers running on the same system trying to obtain secret memory contents of the current process. More specifically, the crate disables core dumps and tries to disable tracing on unix-like OSes.

An experimental implementation of Arc against Apache Datafusion

box This is an experimental repository to perform a proof of concept replacement of the Apache Spark executor for Arc with Apache DataFusion. This is

Nym provides strong network-level privacy against sophisticated end-to-end attackers, and anonymous transactions using blinded, re-randomizable, decentralized credentials.

The Nym Privacy Platform The platform is composed of multiple Rust crates. Top-level executable binary crates include: nym-mixnode - shuffles Sphinx p

Twenty48 - an implementation of the 2048 where users can compete against each other

Twenty48 is an implementation of the 2048 game to demonstrate WebAssembly, Progressive Web Apps and Wasmcloud.

Librarian runs pre-configured commands against a group of files that match a set of filters

Filesystem Librarian Librarian runs pre-configured commands against a group of files that match a set of filters. The group of files is called a libra

Parser and test runner for testing compatable common Ethereum full node tests against Polygon Zero's EVM.

EVM Test Parses and runs compatible common Ethereum tests from ethereum/tests against Polygon Zero's EVM. Note: This repo is currently very early in d

CLI tool for deterministically building and verifying executable against on-chain programs or buffer accounts

Solana Verify CLI A command line tool to build and verify solana programs. Users can ensure that the hash of the on-chain program matches the hash of

A top-down arena shooter roguelite in which you're a mythical marshmallow god fighting against peasant munchies such as chocolates, jellies, or candies!

Mythmellow A top-down arena shooter roguelite in which you're a mythical marshmallow god fighting against peasant munchies such as chocolates, jellies

A top-down arena shooter roguelite in which you're a mythical marshmallow god fighting against peasant munchies such as chocolates, jellies, or candies!

Mythmallow A top-down arena shooter roguelite in which you're a mythical marshmallow god fighting against peasant munchies such as chocolates, jellies

Fuzzy Index for Python, written in Rust. Works like error-tolerant dict, keyed by a human input.

FuzzDex FuzzDex is a fast Python library, written in Rust. It implements an in-memory fuzzy index that works like an error-tolerant dictionary keyed b

Grimsby is an Erlang Port written in Rust that can close its standard input while retaining standard output (and error)

Grimsby An Erlang Port provides the basic mechanism for communication from Erlang with the external world. From the Ports and Port Drivers: Erlang Ref

Owner
Rubber Duck Engineering
Code from our weekly live web show!
Rubber Duck Engineering
Implementation of Immix Mark-Region Garbage collector written in Rust Programming Language.

libimmixcons Implementation of Immix Mark-Region Garbage collector written in Rust Programming Language. Status This is mostly usable library. You can

playX 34 Dec 7, 2022
A computer programming language interpreter written in Rust

Ella lang Welcome to Ella lang! Ella lang is a computer programming language implemented in Rust.

Luke Chu 64 May 27, 2022
Lisp dialect scripting and extension language for Rust programs

Ketos Ketos is a Lisp dialect functional programming language. The primary goal of Ketos is to serve as a scripting and extension language for program

Murarth 721 Dec 12, 2022
Rhai - An embedded scripting language for Rust.

Rhai - Embedded Scripting for Rust Rhai is an embedded scripting language and evaluation engine for Rust that gives a safe and easy way to add scripti

Rhai - Embedded scripting language and engine for Rust 2.4k Dec 29, 2022
Interpreted language developed in Rust

Xelis VM Xelis is an interpreted language developed in Rust. It supports constants, functions, while/for loops, arrays and structures. The syntax is s

null 8 Jun 21, 2022
A Python compiler targeting JS, implemented in Rust.

A Python compiler targeting JavaScript, implemented in Rust.

Gideon Grinberg 5 Jun 17, 2021
Diplo is a script runner and dependency manager made in rust mainly for Deno.

Diplo is a script runner and dependency manager made in rust mainly for Deno. Documentation Tricked.pro/diplo Installing - windows installer Features

Tricked 23 May 9, 2022
REPL for the Rust programming language

Rusti A REPL for the Rust programming language. The rusti project is deprecated. It is not recommended for regular use. Dependencies On Unix systems,

Murarth 1.3k Dec 20, 2022
Multi-threaded Padding Oracle attacks against any service. Written in Rust.

rustpad is a multi-threaded successor to the classic padbuster, written in Rust. It abuses a Padding Oracle vulnerability to decrypt any cypher text or encrypt arbitrary plain text without knowing the encryption key!

Kibouo 76 Dec 16, 2022
'apk-yara-checker' is a little CLI tool written in Rust to check Yara rules against a folder of APK files.

apk-yara-checker 'apk-yara-checker' is a little CLI tool written in Rust to check Yara rules against a folder of APK files. You have to pass the folde

alberto__segura 15 Oct 5, 2022