Native Ruby extensions without fear

Overview

Travis Build Status AppVeyor Build Status

Helix

⚠️ Deprecated ⚠️

Sadly, we have made the decision to deprecate this project. While we had hoped to bootstrap the project to a point where it could flourish as a community project, unfortunately we ran into a number of roadblocks along the way, along with the need for significant architectural overhaul. While these issues are solvable on a technical level, doing so correctly requires more resources than we have been able to provide and progress has stalled.

One of our goals was also to integrate our own Skylight agent with Helix, aligning the company's priorities with the project. While the Skylight agent is still written in Rust with a thin layer of C bindings (which is the part Helix would replace), we were not able to get the project to the point where we felt comfortable running it on our customer's servers. We did not identify any specific blockers that would prevent us from doing this, but ultimate, we did not have the necessary time and resources to realize this.

Since we are a small team, it is unlikely that we will be able to provide the necessary investment in the foreseeable future to achieve our ambitions for the project. At this point, we believe it is in everyone's best interest to formally deprecate the project, accurately reflecting its effective state. Meanwhile, others in the Ruby and Rust communities have continued to explore in the adjacent research areas. Some of them have made great progress and brought new ideas and innovations to the table. We look forward to seeing these new ideas come to fruition and fill the void we are leaving.

Helix allows you to write Ruby classes in Rust without having to write the glue code yourself.

ruby! {
    class Console {
        def log(string: String) {
            println!("LOG: {}", string);
        }
    }
}
$ rake build
$ bundle exec irb
>> require "console"
>> Console.log("I'm in your Rust")
LOG: I'm in your Rust
 => nil

Why Helix?

Read the Introducing Helix blog post for a quick introduction to the project!

Getting Started

https://usehelix.com/getting_started

Demos

https://usehelix.com/demos

Roadmap

https://usehelix.com/roadmap

Compatibility

Helix has been tested with the following, though other combinations may also work.

  • cargo 0.18.0 (fe7b0cdcf 2017-04-24)
  • rustc 1.17.0 (56124baa9 2017-04-24)
  • ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin16]
  • Bundler version 1.14.6

Contributing

If you'd like to experiment with Helix, you can start with some of the examples in this repository.

Clone and download the Helix repository:

$ git clone https://github.com/tildeio/helix
$ cd helix

Navigate to the console example folder and bundle your Gemfile:

$ cd examples/console
$ bundle install

Run rake irb to build and start irb:

$ bundle exec rake irb

Try running some of the methods defined in examples/console/src/lib.rs:

> c = Console.new
Console { helix: VALUE(0x7fdacc19a6a0) }
 =>
> c.hello
hello
 => nil
> c.loglog('hello', 'world')
hello world
 => nil
Comments
  • gem/bundler error when I run the 'console' workflow in the README

    gem/bundler error when I run the 'console' workflow in the README

    Really looking forward to incorporating Rust into my Ruby codebase :)

    I took a look at this section: https://github.com/tildeio/helix#getting-started-with-helix-examples

    I followed the instructions using Ruby 2.4.0 and here's how far I got:

    > bundle exec rake build
    
    
    Building helix runtime
    
    bundle exec rake compile:native
    Could not find gem 'rake-compiler (~> 0.9.7)' in any of the gem sources listed in your Gemfile.
    Run `bundle install` to install missing gems.
    rake aborted!
    Command failed with status (7): [bundle exec rake compile:native...]
    /private/tmp/helix/examples/console/Rakefile:15:in `block (4 levels) in <top (required)>'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler.rb:286:in `block in with_clean_env'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler.rb:521:in `with_env'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler.rb:286:in `with_clean_env'
    /private/tmp/helix/examples/console/Rakefile:14:in `block (3 levels) in <top (required)>'
    /private/tmp/helix/examples/console/Rakefile:12:in `chdir'
    /private/tmp/helix/examples/console/Rakefile:12:in `block (2 levels) in <top (required)>'
    /private/tmp/helix/ruby/lib/helix_runtime/build_task.rb:39:in `block in define'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/cli/exec.rb:74:in `load'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/cli/exec.rb:74:in `kernel_load'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/cli/exec.rb:27:in `run'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/cli.rb:335:in `exec'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/cli.rb:20:in `dispatch'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/cli.rb:11:in `start'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/exe/bundle:32:in `block in <top (required)>'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/friendly_errors.rb:121:in `with_friendly_errors'
    /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/exe/bundle:24:in `<top (required)>'
    /usr/local/bin/bundle:22:in `load'
    /usr/local/bin/bundle:22:in `<main>'
    Tasks: TOP => build => helix:pre_build
    (See full trace by running task with --trace)
    
    Bug PR Pending 
    opened by frewsxcv 14
  • Fix compat with serde_json

    Fix compat with serde_json

    The previous code was relying on the accidental fact that only a single impl of PartialEq existed. serde_json defines another impl of PartialEq, which causes the constraint to be insufficient, which results in a compile error when serde_json is added to a project with Helix.

    This commit forces the VALUE into a usize eagerly, avoiding this inference issue.

    Fixes #93

    h/t @alyssais

    Thanks to @nikomatsakis for helping me figure out the root cause.

    opened by chancancode 12
  • example : cannot load such file -- helix_runtime/native

    example : cannot load such file -- helix_runtime/native

    rust nightly 1.10.0

    $ cd apps/helix/example/console
     cargo build
       Compiling libc v0.2.9
       Compiling libcruby-sys v0.1.0 (file:///vagrant_data/apps/helix/crates/libcruby-sys)
       Compiling cslice v0.1.0 (https://github.com/rustbridge/neon#f28e32c7)
       Compiling helix v0.1.0 (file:///vagrant_data/apps/helix)
       Compiling console v0.1.0 (file:///vagrant_data/apps/helix/examples/console)
    note: link against the following native artifacts when linking against this static library
    note: the order and any duplication can be significant on some platforms, and so may need to be preserved
    note: library: dl
    note: library: pthread
    note: library: gcc_s
    note: library: c
    note: library: m
    note: library: rt
    note: library: util
    
    helix/examples/console$   bundle exec rake irb
    cargo build --release
    gcc -Wl,-force_load,target/release/libconsole.a --shared -Wl,-undefined,dynamic_lookup -o lib/console/native.bundle
    /vagrant_data/apps/helix/ruby/lib/helix_runtime.rb:2:in `require':LoadError: cannot load such file -- helix_runtime/native
    
    opened by ZhangHanDong 12
  • windows build not finding helix_runtime

    windows build not finding helix_runtime

    OS Name: Microsoft Windows 7 Enterprise OS Version: 6.1.7601 Service Pack 1 Build 7601

    c:\Dominic\programming\rails\flipper>gem list helix

    *** LOCAL GEMS ***

    helix-rails (0.5.0) helix_runtime (0.5.0)

    c:\Dominic\programming\rails\flipper>rake build Unable to load text_transform/native. Please run rake build Building text_transform rake aborted! HelixRuntime::MissingDllError: Unable to find helix-runtime-0-5-0.dll in $PATH.

    Bug Needs Bug Verification 
    opened by dsisnero 7
  • Expose `RB_NIL_P` and `RTEST` in libcruby

    Expose `RB_NIL_P` and `RTEST` in libcruby

    These seem to be the appropriate way to check for nil and truthiness of an object using the C API. I would assume that additional changes have to happen since this touches the runtime extension, but I'm not sure what to do.

    opened by sgrif 5
  • impl `Sync` for `ID`

    impl `Sync` for `ID`

    While we may be treating the representation of this type as opaque, I think we can reasonably assume that it will never change to anything that isn't Sync.

    I think it may be worth considering making VALUE be Sync as well. Even though it is effectively a pointer, it's a pointer to a Ruby object, and therefore subject to the GVL.

    opened by sgrif 5
  • Building gems on Windows?

    Building gems on Windows?

    Hello,

    I use Ruby quite a lot, but exclusively for scripting and without Rails (I use it for scripting scientific computing and calling other programs)... so I basically know nothing about Rails and all its conventions.

    Is it possible to build a gem using Helix without all the rails part? I started a new rust project, added the Helix crate, and when running 'cargo build' it obviously tells me that 'helix-runtime-0.6.0' must exist.

    I understand this file is generated when 'bundle install' is executed on a gem... how can I generate it in my case?

    THANKS IN ADVANCE!

    opened by germolinal 5
  • Better function calls

    Better function calls

    1. Wraps dangerous calls from Rust into Ruby in rb_protect to ensure that we correctly unwind in Rust before re-raising the error.
    2. Provides a ruby_funcall! macro to safely call Ruby functions from Rust code.
    Feature Needs Code Review 
    opened by wagenet 5
  • Improve Windows Scripting

    Improve Windows Scripting

    @wagenet did a great job with #29; it's great for now.

    Before shipping 1.0, we should review the PR again to see if there are any ways to simplify the way it works. @wagenet: would you mind jotting down any ideas you have for future improvements here?

    opened by chancancode 5
  • Doesn't build with 1.8.0

    Doesn't build with 1.8.0

    $ rustc --version
    rustc 1.8.0 (db2939409 2016-04-11)
    

    errors:

    src/macros.rs:99:127: 99:138 error: `$ret:ty` is followed by `$body:block`, which is not allowed for `ty` fragments
    src/macros.rs:99     { $cls:ident; ($($mimpl:tt)*) ; ($($mdef:tt)*) ; def $name:ident( $self_arg:tt , $($arg:ident : $argty:ty),* ) -> $ret:ty $body:block $($rest:tt)* } => {
                                                                                                                                                   ^~~~~~~~~~~
    

    Any ideas?

    opened by steveklabnik 5
  • Encoding error when requiring native.so

    Encoding error when requiring native.so

    On my computer (i686-linux-gnu) after successfully compiling any helix project, when the resulting native.so file is required I get this error:

    `require_relative': invalid symbol in encoding US-ASCII :"\xA8\xBF\x98\xBFt\xBF\xEB\xB7\x98\xAB\x9C\x01\x1C\xBF\x98\xBF\x18N\x86\xB7\xC6\xE0\x82\xB7\r\x12\x85\xB7" (EncodingError)
    

    Tested with rust 1.30.0 toolchain, and 1.17 toolchain, ruby 2.5.1 and 2.5.3.

    example

    Steps taken to reproduce:

    helix bootstrap example
    cd example
    bundle exec rake build
    bundle exec ruby lib/example.rb
    

    final output:

    lib/example.rb:4:in `require': invalid symbol in encoding US-ASCII :"\xA8\xB0\xEF\xBFt\xBF\xE8\xB7\x90k0\x01\x1C\xB0\xEF\xBFx\xCE\x86\xB7U\xD0\x83\xB7T\xCD\x85\xB7`\xC3\x90\x01x\xCE\x86\xB7F\xD0\x83\xB7P\xB0\xEF\xBF]-`" (EncodingError)
    

    Interestingly, the binary string from the error message is different each time you invoke it, even with the same file.

    opened by annacrombie 4
  • What is Helix doing that affects performance so much?

    What is Helix doing that affects performance so much?

    Hi, I'm exploring Rust as an extension for Ruby for a lot of expensive calculations. I made a project that implements a simple financial algorithm in six ways (Ruby, C, Helix, ruru, FFI). They are all implemented with the minimum viable code to allow Ruby to be able to call a Rust function cash_flow.

    The benchmark for Helix was surprising, and I'm curious what is unique about Helix that causes the function calls to return Ruby so slowly in comparison with the other methods.

    As you can see from the numbers below, helix's iterations per second when called from Ruby are almost half of ruru, C, and Ruby for a simple function:

    Warming up --------------------------------------
             ruby method    203479 i/100ms
     rust helix instance    120885 i/100ms
        rust helix class    121661 i/100ms
          rust ffi class    161558 i/100ms
         rust ruru class    199846 i/100ms
                 c class    221703 i/100ms
    Calculating -------------------------------------
                          iterations per second     total iterations    time
             ruby method  4966573.4 (±6.8%) i/s -   24824438          in 5.022462s
     rust helix instance  1875397.8 (±6.1%) i/s -    9429030          in 5.046921s
        rust helix class  1852779.7 (±5.9%) i/s -    9246236          in 5.008588s
          rust ffi class  3082134.8 (±8.1%) i/s -   15348010          in 5.019943s
         rust ruru class  4275527.6 (±6.2%) i/s -   21383522          in 5.021156s
                 c class  5483016.5 (±6.0%) i/s -   27491172          in 5.032074s
    

    However, when running a criterion benchmark for the function within the Rust repository, the performance is superb:

    Benchmarking cash_flow
    Benchmarking cash_flow: Warming up for 3.0000 s
    Benchmarking cash_flow: Collecting 100 samples in estimated 5.0000 s (955,252,950 iterations)
    Benchmarking cash_flow: Analyzing
    cash_flow               time:   [5.2014 ns 5.2626 ns 5.3245 ns]
    Found 5 outliers among 100 measurements (5.00%)
      5 (5.00%) high mild
    slope  [5.2014 ns 5.3245 ns] R^2            [0.8278649 0.8276527]
    mean   [5.2079 ns 5.3387 ns] std. dev.      [268.40 ps 395.58 ps]
    median [5.1706 ns 5.3213 ns] med. abs. dev. [208.08 ps 356.25 ps]
    

    This is a significant difference between the actual function and whatever Helix is doing to connect Ruby to Rust. Obviously with interop there's going to be some performance drop, but as you can see the other methods were approximately comparable.

    I want to dig deeper into it because Helix was the best API and usability of all of the methods I tried, but I want to know exactly why the performance is inhibited before we implement critical code with it. Any ideas? Thank you!

    opened by bbugh 9
  • Activity Status

    Activity Status

    Hey guys,

    I've been playing around with Helix tonight and so far it's been an awesome experience, thank you! I'm concerned about using it for a project however because it doesn't appear to be very actively maintained or worked on. The last commit was 10 months ago and a number of issues and pull requests have been languishing for a while too. Is Helix still going places?

    opened by camertron 4
  • Coercions for Rust structs and Ruby hashes

    Coercions for Rust structs and Ruby hashes

    A common pattern in Ruby is to use a hash for temporary structured data. Manually implementing such coercions results in fairly mechanical, repetitive code that would be ideally produced by a proc-macro.

    opened by moosingin3space 2
  • helix_runtime/libcruby-sys 1.0

    helix_runtime/libcruby-sys 1.0

    • Move to new repo
      • CI matrix
    • API surface audit
      • Too little?
      • Too much?
      • Private APIs?
      • Version-specific APIs? Polyfills?
      • Naming convention
    • Ruby support matrix
    • Platform support matrix
      • Windows? GNU cross / MSVC (supported config)
      • Compilation/cross-compilation
    • Better testing?
    • Loosen version requirements
    • Semver policy
    opened by chancancode 0
ruby-build is a command-line utility that makes it easy to install virtually any version of Ruby, from source.

ruby-build ruby-build is a command-line utility that makes it easy to install virtually any version of Ruby, from source. It is available as a plugin

null 3.7k Jan 5, 2023
“The Tie Between Ruby and Rust.”

Rutie Rutie — /ro͞oˈˌtī/rOOˈˌtI/rüˈˌtaI/ Integrate Ruby with your Rust application. Or integrate Rust with your Ruby application. This project allows

Daniel P. Clark 726 Jan 2, 2023
Fast state-of-the-art tokenizers for Ruby

Fast state-of-the-art tokenizers for Ruby

Andrew Kane 34 Dec 15, 2022
Rust bindings for writing safe and fast native Node.js modules.

Rust bindings for writing safe and fast native Node.js modules. Getting started Once you have the platform dependencies installed, getting started is

The Neon Project 7k Jan 4, 2023
Native webview bindings for Node.js

webview-native Native webview bindings for Node.js Installing webview-native Installing webview-native requires a supported version of Node and Rust.

SnowflakeDev Community ❄️ 7 Nov 16, 2022
Rust bindings to the Java Native Interface — JNI

JNI Bindings for Rust This project provides complete JNI bindings for Rust, allowing to: Implement native Java methods for JVM and Android in Rust Cal

null 768 Dec 30, 2022
Node.js bindings to the ripgrep library, for fast file searching in JavaScript without child processes!

ripgrepjs ripgrepjs: Node.js bindings to the ripgrep library, for direct integration with JS programs without spawning an extra subprocess! This proje

Annika 1 May 10, 2022
Native Ruby extensions written in Rust

Ruru (Rust + Ruby) Native Ruby extensions in Rust Documentation Website Have you ever considered rewriting some parts of your slow Ruby application? J

Dmitry Gritsay 812 Dec 26, 2022
ruby-build is a command-line utility that makes it easy to install virtually any version of Ruby, from source.

ruby-build ruby-build is a command-line utility that makes it easy to install virtually any version of Ruby, from source. It is available as a plugin

null 3.7k Jan 5, 2023
rust-native-tls — Bindings for native TLS libraries

rust-native-tls Documentation An abstraction over platform-specific TLS implementations. Specifically, this crate uses SChannel on Windows (via the sc

Steven Fackler 371 Jan 8, 2023
weggli-native a "native" C API for Google's weggli

weggli-native a "native" C API for Google's weggli

Trail of Bits 3 Jul 13, 2022
TMM is a Linux native game modding tool. it allows to install and depoly mods for Linux native and wine games.

Tux Mod Manager TMM is a Linux native mod manager made with the Tauri toolkit. It can install, load, remove and deploy mods for both Linux native and

Mathiew May 119 Dec 27, 2022
“The Tie Between Ruby and Rust.”

Rutie Rutie — /ro͞oˈˌtī/rOOˈˌtI/rüˈˌtaI/ Integrate Ruby with your Rust application. Or integrate Rust with your Ruby application. This project allows

Daniel P. Clark 726 Jan 2, 2023
Grape is a REST-like API framework for Ruby

Grape is a REST-like API framework for Ruby. It's designed to run on Rack or complement existing web application frameworks such as Rails and Sinatra by providing a simple DSL to easily develop RESTful APIs. It has built-in support for common conventions, including multiple formats, subdomain/prefix restriction, content negotiation, versioning and much more.

Ruby Grape 9.7k Jan 2, 2023
A little bit fast and modern Ruby version manager written in Rust

A little bit fast and modern Ruby version manager written in Rust Features Pure Rust implementation not using ruby-build Cross-platform support (macOS

Takayuki Maeda 510 Jan 5, 2023
Simple project to test grpc between ruby (client) and rust (server)

grpc-example Simple project to test grpc between ruby (client) and rust (server). Usage To simplify a lot this project uses docker and docker compose

Bruno Arueira 2 Oct 14, 2021
List of Persian Colors and hex colors for CSS, SCSS, PHP, JS, Python, and Ruby.

Persian Colors (Iranian colors) List of Persian Colors and hex colors for CSS, SCSS, PHP, C++, QML, JS, Python, Ruby and CSharp. Persian colors Name H

Max Base 12 Sep 3, 2022
Rbenv - Manage your app's Ruby environment

Seamlessly manage your app’s Ruby environment with rbenv. Use rbenv to pick a Ruby version for your application and guarantee that your development en

null 14.7k Jan 7, 2023
Fast state-of-the-art tokenizers for Ruby

Fast state-of-the-art tokenizers for Ruby

Andrew Kane 34 Dec 15, 2022
Sampling profiler and tracer for Ruby (CRuby) which runs in BPF

rbperf rbperf is a low-overhead sampling profiler and tracer for Ruby (CRuby) which runs in BPF Build To build rbperf you would need a Linux machine w

Javier Honduvilla Coto 75 Dec 19, 2022