Native Ruby extensions without fear

Related tags

ruby rust helix
Overview

Travis Build Status AppVeyor Build Status

Helix

⚠️ Deprecated ⚠️

Sadly, we have made the decision to deprecate this project. While we had hoped to bootstrap the project to a point where it could flourish as a community project, unfortunately we ran into a number of roadblocks along the way, along with the need for significant architectural overhaul. While these issues are solvable on a technical level, doing so correctly requires more resources than we have been able to provide and progress has stalled.

One of our goals was also to integrate our own Skylight agent with Helix, aligning the company's priorities with the project. While the Skylight agent is still written in Rust with a thin layer of C bindings (which is the part Helix would replace), we were not able to get the project to the point where we felt comfortable running it on our customer's servers. We did not identify any specific blockers that would prevent us from doing this, but ultimate, we did not have the necessary time and resources to realize this.

Since we are a small team, it is unlikely that we will be able to provide the necessary investment in the foreseeable future to achieve our ambitions for the project. At this point, we believe it is in everyone's best interest to formally deprecate the project, accurately reflecting its effective state. Meanwhile, others in the Ruby and Rust communities have continued to explore in the adjacent research areas. Some of them have made great progress and brought new ideas and innovations to the table. We look forward to seeing these new ideas come to fruition and fill the void we are leaving.

Helix allows you to write Ruby classes in Rust without having to write the glue code yourself.

ruby! {
    class Console {
        def log(string: String) {
            println!("LOG: {}", string);
        }
    }
}
$ rake build
$ bundle exec irb
>> require "console"
>> Console.log("I'm in your Rust")
LOG: I'm in your Rust
 => nil

Why Helix?

Read the Introducing Helix blog post for a quick introduction to the project!

Getting Started

https://usehelix.com/getting_started

Demos

https://usehelix.com/demos

Roadmap

https://usehelix.com/roadmap

Compatibility

Helix has been tested with the following, though other combinations may also work.

  • cargo 0.18.0 (fe7b0cdcf 2017-04-24)
  • rustc 1.17.0 (56124baa9 2017-04-24)
  • ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin16]
  • Bundler version 1.14.6

Contributing

If you'd like to experiment with Helix, you can start with some of the examples in this repository.

Clone and download the Helix repository:

$ git clone https://github.com/tildeio/helix
$ cd helix

Navigate to the console example folder and bundle your Gemfile:

$ cd examples/console
$ bundle install

Run rake irb to build and start irb:

$ bundle exec rake irb

Try running some of the methods defined in examples/console/src/lib.rs:

> c = Console.new
Console { helix: VALUE(0x7fdacc19a6a0) }
 =>
> c.hello
hello
 => nil
> c.loglog('hello', 'world')
hello world
 => nil
Issues
  • Error on rake task (it looks like the cargo bundle builds fine)

    Error on rake task (it looks like the cargo bundle builds fine)

    I've already started here: https://github.com/NullVoxPopuli/case_transform/pull/2/files

    opened by NullVoxPopuli 23
  • gem/bundler error when I run the 'console' workflow in the README

    gem/bundler error when I run the 'console' workflow in the README

    Really looking forward to incorporating Rust into my Ruby codebase :)

    I took a look at this section: https://github.com/tildeio/helix#getting-started-with-helix-examples

    I followed the instructions using Ruby 2.4.0 and here's how far I got:

    > bundle exec rake build
    
    
    Building helix runtime
    
    bundle exec rake compile:native
    Could not find gem 'rake-compiler (~> 0.9.7)' in any of the gem sources listed in your Gemfile.
    Run `bundle install` to install missing gems.
    rake aborted!
    Command failed with status (7): [bundle exec rake compile:native...]
    /private/tmp/helix/examples/console/Rakefile:15:in `block (4 levels) in <top (required)>'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler.rb:286:in `block in with_clean_env'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler.rb:521:in `with_env'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler.rb:286:in `with_clean_env'
    /private/tmp/helix/examples/console/Rakefile:14:in `block (3 levels) in <top (required)>'
    /private/tmp/helix/examples/console/Rakefile:12:in `chdir'
    /private/tmp/helix/examples/console/Rakefile:12:in `block (2 levels) in <top (required)>'
    /private/tmp/helix/ruby/lib/helix_runtime/build_task.rb:39:in `block in define'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/cli/exec.rb:74:in `load'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/cli/exec.rb:74:in `kernel_load'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/cli/exec.rb:27:in `run'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/cli.rb:335:in `exec'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/cli.rb:20:in `dispatch'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/cli.rb:11:in `start'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/exe/bundle:32:in `block in <top (required)>'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/friendly_errors.rb:121:in `with_friendly_errors'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/exe/bundle:24:in `<top (required)>'
    /usr/local/bin/bundle:22:in `load'
    /usr/local/bin/bundle:22:in `<main>'
    Tasks: TOP => build => helix:pre_build
    (See full trace by running task with --trace)
    
    Bug PR Pending 
    opened by frewsxcv 14
  • example : cannot load such file -- helix_runtime/native

    example : cannot load such file -- helix_runtime/native

    rust nightly 1.10.0

    $ cd apps/helix/example/console
     cargo build
       Compiling libc v0.2.9
       Compiling libcruby-sys v0.1.0 (file:///vagrant_data/apps/helix/crates/libcruby-sys)
       Compiling cslice v0.1.0 (https://github.com/rustbridge/neon#f28e32c7)
       Compiling helix v0.1.0 (file:///vagrant_data/apps/helix)
       Compiling console v0.1.0 (file:///vagrant_data/apps/helix/examples/console)
    note: link against the following native artifacts when linking against this static library
    note: the order and any duplication can be significant on some platforms, and so may need to be preserved
    note: library: dl
    note: library: pthread
    note: library: gcc_s
    note: library: c
    note: library: m
    note: library: rt
    note: library: util
    
    helix/examples/console$   bundle exec rake irb
    cargo build --release
    gcc -Wl,-force_load,target/release/libconsole.a --shared -Wl,-undefined,dynamic_lookup -o lib/console/native.bundle
    /vagrant_data/apps/helix/ruby/lib/helix_runtime.rb:2:in `require':LoadError: cannot load such file -- helix_runtime/native
    
    opened by ZhangHanDong 12
  • Fix compat with serde_json

    Fix compat with serde_json

    The previous code was relying on the accidental fact that only a single impl of PartialEq existed. serde_json defines another impl of PartialEq, which causes the constraint to be insufficient, which results in a compile error when serde_json is added to a project with Helix.

    This commit forces the VALUE into a usize eagerly, avoiding this inference issue.

    Fixes #93

    h/t @alyssais

    Thanks to @nikomatsakis for helping me figure out the root cause.

    opened by chancancode 12
  • windows build not finding helix_runtime

    windows build not finding helix_runtime

    OS Name: Microsoft Windows 7 Enterprise OS Version: 6.1.7601 Service Pack 1 Build 7601

    c:\Dominic\programming\rails\flipper>gem list helix

    *** LOCAL GEMS ***

    helix-rails (0.5.0) helix_runtime (0.5.0)

    c:\Dominic\programming\rails\flipper>rake build Unable to load text_transform/native. Please run rake build Building text_transform rake aborted! HelixRuntime::MissingDllError: Unable to find helix-runtime-0-5-0.dll in $PATH.

    Bug Needs Bug Verification 
    opened by dsisnero 7
  • Fewer allocations by making function signatures take `&CStr`s.

    Fewer allocations by making function signatures take `&CStr`s.

    As discussed in https://github.com/tildeio/helix/pull/76.

    [breaking-change]

    opened by frewsxcv 6
  • A few cleanup/refactoring commits.

    A few cleanup/refactoring commits.

    null

    Cleanup Needs Submitter Response 
    opened by frewsxcv 6
  • Doesn't build with 1.8.0

    Doesn't build with 1.8.0

    $ rustc --version
    rustc 1.8.0 (db2939409 2016-04-11)
    

    errors:

    src/macros.rs:99:127: 99:138 error: `$ret:ty` is followed by `$body:block`, which is not allowed for `ty` fragments
    src/macros.rs:99     { $cls:ident; ($($mimpl:tt)*) ; ($($mdef:tt)*) ; def $name:ident( $self_arg:tt , $($arg:ident : $argty:ty),* ) -> $ret:ty $body:block $($rest:tt)* } => {
                                                                                                                                                   ^~~~~~~~~~~
    

    Any ideas?

    opened by steveklabnik 5
  • Improve Windows Scripting

    Improve Windows Scripting

    @wagenet did a great job with #29; it's great for now.

    Before shipping 1.0, we should review the PR again to see if there are any ways to simplify the way it works. @wagenet: would you mind jotting down any ideas you have for future improvements here?

    opened by chancancode 5
  • impl `Sync` for `ID`

    impl `Sync` for `ID`

    While we may be treating the representation of this type as opaque, I think we can reasonably assume that it will never change to anything that isn't Sync.

    I think it may be worth considering making VALUE be Sync as well. Even though it is effectively a pointer, it's a pointer to a Ruby object, and therefore subject to the GVL.

    opened by sgrif 5
  • What is Helix doing that affects performance so much?

    What is Helix doing that affects performance so much?

    Hi, I'm exploring Rust as an extension for Ruby for a lot of expensive calculations. I made a project that implements a simple financial algorithm in six ways (Ruby, C, Helix, ruru, FFI). They are all implemented with the minimum viable code to allow Ruby to be able to call a Rust function cash_flow.

    The benchmark for Helix was surprising, and I'm curious what is unique about Helix that causes the function calls to return Ruby so slowly in comparison with the other methods.

    As you can see from the numbers below, helix's iterations per second when called from Ruby are almost half of ruru, C, and Ruby for a simple function:

    Warming up --------------------------------------
             ruby method    203479 i/100ms
     rust helix instance    120885 i/100ms
        rust helix class    121661 i/100ms
          rust ffi class    161558 i/100ms
         rust ruru class    199846 i/100ms
                 c class    221703 i/100ms
    Calculating -------------------------------------
                          iterations per second     total iterations    time
             ruby method  4966573.4 (±6.8%) i/s -   24824438          in 5.022462s
     rust helix instance  1875397.8 (±6.1%) i/s -    9429030          in 5.046921s
        rust helix class  1852779.7 (±5.9%) i/s -    9246236          in 5.008588s
          rust ffi class  3082134.8 (±8.1%) i/s -   15348010          in 5.019943s
         rust ruru class  4275527.6 (±6.2%) i/s -   21383522          in 5.021156s
                 c class  5483016.5 (±6.0%) i/s -   27491172          in 5.032074s
    

    However, when running a criterion benchmark for the function within the Rust repository, the performance is superb:

    Benchmarking cash_flow
    Benchmarking cash_flow: Warming up for 3.0000 s
    Benchmarking cash_flow: Collecting 100 samples in estimated 5.0000 s (955,252,950 iterations)
    Benchmarking cash_flow: Analyzing
    cash_flow               time:   [5.2014 ns 5.2626 ns 5.3245 ns]
    Found 5 outliers among 100 measurements (5.00%)
      5 (5.00%) high mild
    slope  [5.2014 ns 5.3245 ns] R^2            [0.8278649 0.8276527]
    mean   [5.2079 ns 5.3387 ns] std. dev.      [268.40 ps 395.58 ps]
    median [5.1706 ns 5.3213 ns] med. abs. dev. [208.08 ps 356.25 ps]
    

    This is a significant difference between the actual function and whatever Helix is doing to connect Ruby to Rust. Obviously with interop there's going to be some performance drop, but as you can see the other methods were approximately comparable.

    I want to dig deeper into it because Helix was the best API and usability of all of the methods I tried, but I want to know exactly why the performance is inhibited before we implement critical code with it. Any ideas? Thank you!

    opened by bbugh 9
  • Re-try Changes in #148

    Re-try Changes in #148

    Unfortunately, #148 broke specs when it was merged so I had to revert it. @konstin we'd love to have you take another go at it.

    opened by wagenet 0
  • Activity Status

    Activity Status

    Hey guys,

    I've been playing around with Helix tonight and so far it's been an awesome experience, thank you! I'm concerned about using it for a project however because it doesn't appear to be very actively maintained or worked on. The last commit was 10 months ago and a number of issues and pull requests have been languishing for a while too. Is Helix still going places?

    opened by camertron 4
  • Coercions for Rust structs and Ruby hashes

    Coercions for Rust structs and Ruby hashes

    A common pattern in Ruby is to use a hash for temporary structured data. Manually implementing such coercions results in fairly mechanical, repetitive code that would be ideally produced by a proc-macro.

    opened by moosingin3space 2
  • Forum or IRC channel

    Forum or IRC channel

    Does Helix have a discussion community anywhere? I'm unable to find any references to one on the website.

    opened by moosingin3space 2
  • helix_runtime/libcruby-sys 1.0

    helix_runtime/libcruby-sys 1.0

    • Move to new repo
      • CI matrix
    • API surface audit
      • Too little?
      • Too much?
      • Private APIs?
      • Version-specific APIs? Polyfills?
      • Naming convention
    • Ruby support matrix
    • Platform support matrix
      • Windows? GNU cross / MSVC (supported config)
      • Compilation/cross-compilation
    • Better testing?
    • Loosen version requirements
    • Semver policy
    opened by chancancode 0
  • Unclear how to use rust based gems in other projects

    Unclear how to use rust based gems in other projects

    Hello,

    I downloaded helix, built gem 'turbo_blank' and tried to use it in irb console, which is failed. Can you add some documentation to explain how it should be done properly?

    Building gem

    [helix/examples/turbo_blank] at  master ✔
    → gem build turbo_blank.gemspec                                                                                       [239f1de]
    WARNING:  open-ended dependency on helix_runtime (>= 0) is not recommended
      if helix_runtime is semantically versioned, use:
        add_runtime_dependency 'helix_runtime', '~> 0'
    WARNING:  See http://guides.rubygems.org/specification-reference/ for help
      Successfully built RubyGem
      Name: turbo_blank
      Version: 0.1.0
      File: turbo_blank-0.1.0.gem
    

    Installing:

    [helix/examples/turbo_blank] at  master ✔
    → gem install turbo_blank                                                                                             [239f1de]
    Fetching: helix_runtime-0.7.5.gem (100%)
    Building native extensions.  This could take a while...
    Successfully installed helix_runtime-0.7.5
    Successfully installed turbo_blank-0.1.0
    Parsing documentation for helix_runtime-0.7.5
    Installing ri documentation for helix_runtime-0.7.5
    Parsing documentation for turbo_blank-0.1.0
    Installing ri documentation for turbo_blank-0.1.0
    Done installing documentation for helix_runtime, turbo_blank after 0 seconds
    2 gems installed
    

    Trying to use it in irb:

    [helix/examples/turbo_blank] at  master ✔
    → irb                                                                                                                 [239f1de]
    cannot load such file -- irbtools
    2.4.4 :001 > require 'turbo_blank'
    LoadError: cannot load such file -- turbo_blank/native
      from /Users/xxx/.rvm/rubies/ruby-2.4.4/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:120:in `require'
      from /Users/xxx/.rvm/rubies/ruby-2.4.4/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:120:in `require'
      from /Users/xxx/.rvm/gems/ruby-2.4.4/gems/turbo_blank-0.1.0/lib/turbo_blank.rb:2:in `<top (required)>'
      from /Users/xxx/.rvm/rubies/ruby-2.4.4/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:133:in `require'
      from /Users/xxx/.rvm/rubies/ruby-2.4.4/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:133:in `rescue in require'
      from /Users/xxx/.rvm/rubies/ruby-2.4.4/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:39:in `require'
      from (irb):1
      from /Users/xxx/.rvm/rubies/ruby-2.4.4/bin/irb:11:in `<main>'
    

    I tried the same with duration gem with the same results. What am I doing wrong? Can you help?

    opened by kirhgoff 0
  • Attempting to build a project when Rust isn't installed fails silently

    Attempting to build a project when Rust isn't installed fails silently

    You can run rake build but there is no error or indication that Rust isn't installed.

    Bug Ready for PR 
    opened by wagenet 0
  • Consider implicit coercions, e.g. `to_str` for String

    Consider implicit coercions, e.g. `to_str` for String

    For instance, instead of requiring that String values are actually T_STRING we could use implicit coercions (to_str) on the provided object. This would definitely be more Ruby-like in behavior.

    Is this something we'd want to do? What are the drawbacks, if any?

    Feature Needs Team Discussion 
    opened by wagenet 2
Native Ruby extensions written in Rust

Ruru (Rust + Ruby) Native Ruby extensions in Rust Documentation Website Have you ever considered rewriting some parts of your slow Ruby application? J

Dmitry Gritsay 776 Jun 14, 2021
Native Ruby extensions without fear

Helix ⚠️ Deprecated ⚠️ Sadly, we have made the decision to deprecate this project. While we had hoped to bootstrap the project to a point where it cou

Tilde 2k Jun 15, 2021
Easy way to write Node.js module using Rust

node-bindgen Easy way to write native Node.js module using idiomatic Rust Features Easy: Just write idiomatic Rust code, node-bindgen take care of gen

InfinyOn 185 Jun 12, 2021
mruby safe bindings for Rust

mrusty. mruby safe bindings for Rust mrusty lets you: run Ruby 1.9 files with a very restricted API (without having to install Ruby) reflect Rust stru

Anima 191 May 30, 2021
Rust bindings for the Python interpreter

PyO3 Rust bindings for Python. This includes running and interacting with Python code from a Rust binary, as well as writing native Python modules. Us

PyO3 4.3k Jun 13, 2021
Run Java code from Rust!

Java Native Interface Bindings for Rust This library provides complete FFI bindings to the Java Native Interface, as well as a safe and intuitive wrap

Ben Anderson 59 Dec 25, 2020
Bridge the gap between Haskell and Rust

Curryrs Curryrs (a play on the name of Haskell Curry, rs for Rust libraries, and it's pronunciation couriers) is a library for providing easy to use b

Michael Gattozzi 260 May 31, 2021
A project for generating C bindings from Rust code

cbindgen   Read the full user docs here! cbindgen creates C/C++11 headers for Rust libraries which expose a public C API. While you could do this by h

Ryan Hunt 1k Jun 8, 2021
Safe Rust <---> GraalVM Polyglot bindings using procedural macros

The class macro is the primary way to generate bindings to Java types; it will generate a struct (with generics if specified) that implements Pass and Receive and has all the methods you give stubs for. The methods generated can be used like normal rust methods, however mutability is not enforced. The fully-qualified type name should precede a block containing method and constructor stubs. Java primitives like char, int, and byte are aliased to corresponding Rust types.

Alec Petridis 20 May 11, 2021
Safe interop between Rust and C++

CXX — safe FFI between Rust and C++ This library provides a safe mechanism for calling C++ code from Rust and Rust code from C++, not subject to the m

David Tolnay 2.9k Jun 14, 2021
WebAssembly implementation from scratch in Safe Rust with zero dependencies

wain wain is a WebAssembly INterpreter written in Rust from scratch with zero dependencies. An implementation of WebAssembly. Features: No unsafe code

Linda_pp 171 Jun 6, 2021
Facilitating high-level interactions between Wasm modules and JavaScript

wasm-bindgen Facilitating high-level interactions between Wasm modules and JavaScript. Guide | API Docs | Contributing | Chat Built with ?? ?? by The

Rust and WebAssembly 4.3k Jun 13, 2021
Rust bindings for writing safe and fast native Node.js modules.

Rust bindings for writing safe and fast native Node.js modules. Getting started Once you have the platform dependencies installed, getting started is

The Neon Project 5.5k Jun 12, 2021
A setuptools/wheel/cffi extension to embed a binary data in wheels

Milksnake Milksnake is an extension for setuptools that allows you to distribute dynamic linked libraries in Python wheels in the most portable way im

Sentry 692 May 28, 2021