Prepare, once and future Neon contributors, for our noblest quest yet!
We are going to port Neon to Node's new N-API!
I'll explain. N-API brings to Neon the promise of a stable, backwards-compatible ABI—binary compatibility across all future versions of Node.
This is a big deal.
Portability across Node versions means Neon will finally be practical for publishing libraries, not just apps: a few prebuilt binaries should be sufficient for all downstream customers to use your native library without ever knowing the difference.
The stuff of legend, no?
Step 1. Create the feature flag
- [x] Create a cargo feature flag to allow us to concurrently maintain the main Neon codebase along with the experimental N-API support in the same master branch. (Merged!)
- [x] Set up a test suite specifically for the N-API backend so each task can easily include adding tests (#449)
Step 2. Implement the port
- [x] Module contexts and initialization: Implement the
neon::context::ModuleContext type and pass it to the module initialization function inside the
register_module! macro defined in
/src/lib.rs. The context struct will likely need to encapsulate the underlying
napi_value as private fields. This can be implemented before we implement functions, with an unimplemented
export_function() method for now.
- [x] Functions: This is probably one of the subtler tasks. See the implementation of
neon::types::JsFunction::new(). The Rust callback can be stored as the extra
void* data passed to
- [x] Function arguments: Implement
- [x] Function returns: Implement function return values.
- [x] Call kinds: Implement
- [x] Function exports: Once we have module contexts and functions implemented, we can implement the
ModuleContext::export_function() shorthand method.
- [x] Objects: See
neon::types::JsObject::new() and the
- [x] Arrays: See
- [x] ArrayBuffers and Buffers: See
neon::types::binary and the N-API functions for working with binary data, such as
- [x] Uninitialized and null: These should be pretty bite-sized. See
- [x] Booleans: See
- [x] Numbers: See
- [x] Strings: See
neon::types::JsString. We'll need to explore what binary string representations can be used between the NAN vs N-API runtimes for constructing JS strings.
- [x] ~Classes: This will require us to figure out how to do unique branding with N-API, but I believe
napi_define_class supports this. (Here is one pure C example we can look to for inspiration.)~ <== not needed for functional completeness; see #596
- [x] Errors: See
neon::types::error. We'll need to explore how N-API does throwing and catching errors. - @anshulrgoyal 🔒
- [x] Conversions: See the uses of
neon_runtime::convert::* and the
- [x] Scopes: Luckily, the N-API HandleScope mechanim matches V8's mechanism very closely. See
neon::context and the uses of various HandleScope internal types.
- [x] Tag checks: See uses of
- [x] ~Task scheduling: See
neon::context::TaskContext, and the N-API "simply asynchronous operations" API, which uses the same underlying libuv thread pool as Neon's current backend, but with N-API's stable ABI.~ <== not needed for functional completeness; see #596
- [x] ~Thread-safe callbacks: This can be implemented for N-API once we've merged an implementation for RFC 25, using
napi_make_callback.~ <== not needed for functional completeness; see #596
- [x] Windows Support: Windows requires linking against
win_delay_load_hook. Create a custom build script to link these on windows.
We have just a couple remaining items to finish up:
- [x] Equality comparison of handles - see #666
JsBuffer::uninitialized - see #664
Step 3. Deprecate the legacy runtime
Once we finish the complete port, we can switch the default feature flags to use the new runtime and publish a new 0.x minor version. Eventually after a few releases we can remove the old runtime completely.
How to Contribute
Building N-API-based projects
To experiment with the N-API runtime or do manual testing, you can create a Neon project that uses the right feature flags. To try it out, you can run:
neon new --no-default-features --features=napi-latest --neon=path/to/neon my-project
path/to/neon is the path on your local filesystem to a local clone of the Neon repo.
The output of
neon new executed above will produce a project that fails to build. When using the
neon backend, either
neon-build should be used with a simple
cargo build or
neon-cli should be used and
neon-build should be removed. If both are used, the project will fail to build.
There is an RFC (https://github.com/neon-bindings/rfcs/pull/36) to replace
neon new which will correctly generate a project. The simplest change is to edit
- Remove the
build = "build.rs"
Note: If you create a Neon project nested inside the directory tree of a clone of the Neon repo, you'll need to add the line
to your Neon project's
native/Cargo.toml manifest in order to build the project.
Adding an N-API primitive
To add an N-API primitive, you should implement it in pure Rust (using
unsafe as necessary, but only as necessary!) in
crates/neon-runtime/napi, and call out to the N-API backend exposed through
When the Neon runtime needs to pass around a data structure, you can make two different definitions of the type, separated by testing the feature flag with
#[cfg(feature = "...")]. You may sometimes need to refactor the types in the Neon runtime to accommodate differences between the legacy and N-API runtimes.
Adding a test
test/napi directory is the space for adding N-API acceptance tests. You can add native Rust logic to
test/napi/native/src and JS logic to
test/napi/lib. You can get examples of existing acceptance tests in our existing backend in
test/dynamic, which has the same structure.
Will You Join Us?
As you can see, the quest ahead of us will be no small feat!
Indeed, but fear not: we're here to help you if you get stuck. And many of these tasks can be a great way to get started with contributing to Neon and even learning Rust.
Claim one of the tasks today by leaving a comment below or pinging @dherman or @kjvalencik on Slack!