Hi, this adds a little helper to call a JavaScript function from any rust thread as requested in #197. ~~I'm not sure if this will need a RFC, but I can write one if requested.~~ RFC

Feedback welcome, especially about the v8 scope setup.

Inspired by mika-fischer/napi-thread-safe-callback

This PR adds a neon::thread::LocalKey API for storing thread-local data:

use neon::thread::LocalKey;

static THREAD_ID: LocalKey<u32> = LocalKey::new();

THREAD_ID.get_or_init(&mut cx, || x);

let thread_id: u32 = THREAD_ID.get(&mut cx).unwrap();

Closes #728.

Hi, i just created a new neon project, but it doesn't seems to build ... Here's the log.

Updating registry `https://github.com/rust-lang/crates.io-index`
Compiling nod v0.1.0 (file:///Users/jchaput/dev/rust/nod/native)
Compiling cslice v0.1.1
Compiling gcc v0.3.43
error[E0463]: can't find crate for `neon_build`
 --> build.rs:1:1
  |
1 | extern crate neon_build;
  | ^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate

error: aborting due to previous error

error: Could not compile `nod`.
Build failed, waiting for other jobs to finish...
error: build failed
neon ERR! cargo build failed
error Command failed with exit code 1.

thanks

Prepare, once and future Neon contributors, for our noblest quest yet!

We are going to port Neon to Node's new N-API!

I'll explain. N-API brings to Neon the promise of a stable, backwards-compatible ABI—binary compatibility across all future versions of Node.

This is a big deal.

Portability across Node versions means Neon will finally be practical for publishing libraries, not just apps: a few prebuilt binaries should be sufficient for all downstream customers to use your native library without ever knowing the difference.

The stuff of legend, no?

Our Quest

Step 1. Create the feature flag

• [x] Create a cargo feature flag to allow us to concurrently maintain the main Neon codebase along with the experimental N-API support in the same master branch. (Merged!)
• [x] Set up a test suite specifically for the N-API backend so each task can easily include adding tests (#449)

Step 2. Implement the port

• [x] Module contexts and initialization: Implement the neon::context::ModuleContext type and pass it to the module initialization function inside the register_module! macro defined in /src/lib.rs. The context struct will likely need to encapsulate the underlying napi_env and napi_value as private fields. This can be implemented before we implement functions, with an unimplemented export_function() method for now.
• [x] Functions: This is probably one of the subtler tasks. See the implementation of neon::types::JsFunction::new(). The Rust callback can be stored as the extra void* data passed to napi_create_function.
• [x] Function arguments: Implement CallContext::len() and CallContext::argument().
• [x] Function returns: Implement function return values.
• [x] this: Implement CallContext::this().
• [x] Call kinds: Implement CallContext::kind().
• [x] Function exports: Once we have module contexts and functions implemented, we can implement the ModuleContext::export_function() shorthand method.
• [x] Objects: See neon::types::JsObject::new() and the neon::object::Object methods.
• [x] Arrays: See neon::types::JsArray.
• [x] ArrayBuffers and Buffers: See neon::types::binary and the N-API functions for working with binary data, such as napi_create_arraybuffer, napi_create_buffer, etc.
• [x] Uninitialized and null: These should be pretty bite-sized. See neon::types::JsUndefined and neon::types::JsNull. @goto-bus-stop
• [x] Booleans: See neon::types::JsBoolean. @goto-bus-stop
• [x] Numbers: See neon::types::JsNumber.
• [x] Strings: See neon::types::JsString. We'll need to explore what binary string representations can be used between the NAN vs N-API runtimes for constructing JS strings.
• [x] ~Classes: This will require us to figure out how to do unique branding with N-API, but I believe napi_define_class supports this. (Here is one pure C example we can look to for inspiration.)~ <== not needed for functional completeness; see #596
• [x] Errors: See neon::types::error. We'll need to explore how N-API does throwing and catching errors. - @anshulrgoyal 🔒
• [x] Conversions: See the uses of neon_runtime::convert::* and the napi_coerce_* functions.
• [x] Scopes: Luckily, the N-API HandleScope mechanim matches V8's mechanism very closely. See neon::context and the uses of various HandleScope internal types.
• [x] Tag checks: See uses of neon_runtime::tag::*.
• [x] ~Task scheduling: See neon::task and neon::context::TaskContext, and the N-API "simply asynchronous operations" API, which uses the same underlying libuv thread pool as Neon's current backend, but with N-API's stable ABI.~ <== not needed for functional completeness; see #596
• [x] ~Thread-safe callbacks: This can be implemented for N-API once we've merged an implementation for RFC 25, using napi_make_callback.~ <== not needed for functional completeness; see #596
• [x] Windows Support: Windows requires linking against node.lib and win_delay_load_hook. Create a custom build script to link these on windows.

We have just a couple remaining items to finish up:

• [x] Equality comparison of handles - see #666
• [x] JsBuffer::uninitialized - see #664

Step 3. Deprecate the legacy runtime

Once we finish the complete port, we can switch the default feature flags to use the new runtime and publish a new 0.x minor version. Eventually after a few releases we can remove the old runtime completely.

How to Contribute

Building N-API-based projects

To experiment with the N-API runtime or do manual testing, you can create a Neon project that uses the right feature flags. To try it out, you can run:

neon new --no-default-features --features=napi-latest --neon=path/to/neon my-project

where path/to/neon is the path on your local filesystem to a local clone of the Neon repo.

Manual Steps

The output of neon new executed above will produce a project that fails to build. When using the neon backend, either neon-build should be used with a simple cargo build or neon-cli should be used and neon-build should be removed. If both are used, the project will fail to build.

There is an RFC (https://github.com/neon-bindings/rfcs/pull/36) to replace neon new which will correctly generate a project. The simplest change is to edit native/Cargo.toml:

• Remove the neon-build dependency
• Remove build = "build.rs"
• delete native/build.rs

Note: If you create a Neon project nested inside the directory tree of a clone of the Neon repo, you'll need to add the line

[workspace]

to your Neon project's native/Cargo.toml manifest in order to build the project.

Adding an N-API primitive

To add an N-API primitive, you should implement it in pure Rust (using unsafe as necessary, but only as necessary!) in crates/neon-runtime/napi, and call out to the N-API backend exposed through nodejs-sys.

When the Neon runtime needs to pass around a data structure, you can make two different definitions of the type, separated by testing the feature flag with #[cfg(feature = "...")]. You may sometimes need to refactor the types in the Neon runtime to accommodate differences between the legacy and N-API runtimes.

Adding a test

The test/napi directory is the space for adding N-API acceptance tests. You can add native Rust logic to test/napi/native/src and JS logic to test/napi/lib. You can get examples of existing acceptance tests in our existing backend in test/dynamic, which has the same structure.

Will You Join Us?

As you can see, the quest ahead of us will be no small feat!

Indeed, but fear not: we're here to help you if you get stuck. And many of these tasks can be a great way to get started with contributing to Neon and even learning Rust.

Claim one of the tasks today by leaving a comment below or pinging @dherman or @kjvalencik on Slack!

Resolves #528

• [x] Run cargo test with a NodeJS version matrix and Rust toolchain matrix
• [x] Specific environment dependencies (apt dependencies etc)
• [x] Electron settings
• [x] Generate Documentation
• [x] Windows build
• [x] MacOS build

Hey, I'm learning how to set up neon here, and I'm running into an issue where, upon launching electron, I get the error: undefined symbol: __cxa_pure_virtual.

FWIW, I followed the documentation here. I found #194, but I've interpreted it as the ticket for integrating with electron-rebuild, and that the electron-build-env process was already functional? :thinking: :smile:

To reproduce:

1. git clone https://github.com/electron/electron-quick-start
2. cd electron-quick-start
3. npm i neon-hello
4. npm i electron-build-env neon-cli --save-dev
5. $(npm bin)/neon build neon-hello
6. Edit main.js, adding require('neon-hello') to the top
7. npm start

$ npm start                                         

> [email protected] start /home/mitch/dev/electron-quick-start
> electron .

App threw an error during load
Error: /home/mitch/dev/electron-quick-start/node_modules/neon-hello/native/index.node: undefined symbol: __cxa_pure_virtual
    at process.module.(anonymous function) [as dlopen] (ELECTRON_ASAR.js:160:31)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:722:18)
    at Object.module.(anonymous function) [as .node] (ELECTRON_ASAR.js:160:31)
    at Module.load (internal/modules/cjs/loader.js:602:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:541:12)
    at Function.Module._load (internal/modules/cjs/loader.js:533:3)
    at Module.require (internal/modules/cjs/loader.js:640:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (/home/mitch/dev/electron-quick-start/node_modules/neon-hello/lib/index.js:1:168)
    at Object.<anonymous> (/home/mitch/dev/electron-quick-start/node_modules/neon-hello/lib/index.js:5:3)
A JavaScript error occurred in the main process
Uncaught Exception:
Error: /home/mitch/dev/electron-quick-start/node_modules/neon-hello/native/index.node: undefined symbol: __cxa_pure_virtual
    at process.module.(anonymous function) [as dlopen] (ELECTRON_ASAR.js:160:31)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:722:18)
    at Object.module.(anonymous function) [as .node] (ELECTRON_ASAR.js:160:31)
    at Module.load (internal/modules/cjs/loader.js:602:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:541:12)
    at Function.Module._load (internal/modules/cjs/loader.js:533:3)
    at Module.require (internal/modules/cjs/loader.js:640:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (/home/mitch/dev/electron-quick-start/node_modules/neon-hello/lib/index.js:1:168)
    at Object.<anonymous> (/home/mitch/dev/electron-quick-start/node_modules/neon-hello/lib/index.js:5:3)

N-API provides the rather limited napi_set_instance_data to allow associating a single word of data with an instance. Neon already uses this API to store its own per-instance data, but it would be great if that were exposed to users in a convenient way. A possible shape for the API:

pub struct PerInstanceKey<T: 'static>(PhantomData<&'static mut T>);
impl<T: 'static> PerInstanceKey<T> {
  pub const fn new() {
    Self(PhantomData)
  }
}

impl Context {
  pub fn set_instance_data<T: 'static>(&mut self, key: &'static PerInstanceKey<T>, value: T) -> Option<T> {
    let mut user_data: HashMap = &mut InstanceData::get(self).user_data;
    let previous_value = user_data.insert(key as *const (), Box::new(value as dyn Any));
    previous_value.map(|old_value| *old_value.downcast().expect("type-safe API"))
  }

  pub fn get_instance_data_mut<T: 'static>(&mut self, key: &'static PerInstanceKey<T>) -> Option<&mut T> {
    unimplemented!() // same as set_* simplified
  }
}

static MY_DATA: PerInstanceKey<String>;

cx.set_instance_data(&MY_DATA, "hello".to_string());

An alternate version of the API would include a function to construct the initial value if not present, more like thread_local!, but this version is the simplest I could think of.

Node.js optimizes subsequent ThreadsafeFunction invocations to happen during the same event loop tick, but only if the same instance of ThreadsafeFunction is used. The performance improvement is most noticeable when used in Electron, because scheduling a new UV tick in Electron is very costly.

With this change EventQueue will use an existing instance of ThreadsafeTrampoline (wrapper around ThreadsafeFunction) if compiled with napi-6 feature, or it will fallback to creating a new ThreadsafeFunction per EventQueue instance.

Fix: #727

When I try to build with neon-cli 0.3.2 on macOS it fails with:

ld: warning: cannot export hidden symbol compiler_builtins::mem::memcpy::h89bf2c29275db9b6 from /var/folders/09/3ccx9hvj499_767y34mtcr900000gn/T/rustc5SpE5s/libcompiler_builtins-4c0e14a54ecf951d.rlib(compiler_builtins-4c0e14a54ecf951d.compiler_builtins.dcjs62au-cgu.0.rcgu.o)
          ld: warning: cannot export hidden symbol compiler_builtins::mem::memmove::hb03f562604dc3076 from /var/folders/09/3ccx9hvj499_767y34mtcr900000gn/T/rustc5SpE5s/libcompiler_builtins-4c0e14a54ecf951d.rlib(compiler_builtins-4c0e14a54ecf951d.compiler_builtins.dcjs62au-cgu.0.rcgu.o)
          ld: warning: cannot export hidden symbol compiler_builtins::mem::memset::h1f53dc51033b23d1 from /var/folders/09/3ccx9hvj499_767y34mtcr900000gn/T/rustc5SpE5s/libcompiler_builtins-4c0e14a54ecf951d.rlib(compiler_builtins-4c0e14a54ecf951d.compiler_builtins.dcjs62au-cgu.0.rcgu.o)
          ld: warning: cannot export hidden symbol compiler_builtins::mem::memcmp::hc5ebea53cd23af68 from /var/folders/09/3ccx9hvj499_767y34mtcr900000gn/T/rustc5SpE5s/libcompiler_builtins-4c0e14a54ecf951d.rlib(compiler_builtins-4c0e14a54ecf951d.compiler_builtins.dcjs62au-cgu.0.rcgu.o)
[...]
Undefined symbols for architecture x86_64:
            "v8::Isolate::ThrowException(v8::Local<v8::Value>)", referenced from:
                _Neon_Error_Throw in libneon_runtime-0d1b50c0de5edd79.rlib(neon.o)
                _Neon_Error_ThrowErrorFromUtf8 in libneon_runtime-0d1b50c0de5edd79.rlib(neon.o)
            "v8::Value::IsNumber() const", referenced from:
                _Neon_Tag_IsNumber in libneon_runtime-0d1b50c0de5edd79.rlib(neon.o)
            "v8::HandleScope::CreateHandle(v8::internal::Isolate*, v8::internal::Object*)", referenced from:
                _Neon_Class_HasInstance in libneon_runtime-0d1b50c0de5edd79.rlib(neon.o)
[...]
          ld: symbol(s) not found for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

with [...] I skipped similar lines, but I can provide full trace.

Switching back to 0.3.1 (npm install -g [email protected]) fixes the problem.

Tried with

• NodeJS v10.15.3, v10.16.3 and v12.12.0.
• rustc 1.38.0 (625451e37 2019-09-23)

Hi,

I was evaluating the new N-API codebase, and found quite fast greatly leaking piece of code. Here's the test repo that will show the leak:

https://github.com/pimeys/hello-neon

Now, I did some digging with Valgrind:

As you can see the culprit is in napi_create_reference, called from Root::new. You can dig into the results from the included massif dump! I was not able to collect this data, even by forcing global.gc(), meaning we probably do something wrong in the native side.

I'd expect this loop to have a steady flat memory profile, the data should be collected in the young-space already...

Hello,

I don't have a lot of experience with all things JS, nor am I particularly well versed in all things Windows, but we need the Windows build and although it did build at one point it now doesn't do and I've been fighting with it for hours, so if there's anything you can tell me about the following error I'd be really thankful for it.

Build environment is Win10 Education N Version 20H2 OS build 19042.572. Everything is correctly pathed from what I can see, I'm using the git bash that comes with git on Windows. That said, the error is the same on the Windows Powershell.

Environment

$ rustup show
Default host: x86_64-pc-windows-msvc
rustup home:  C:\Users\flrn\.rustup

stable-x86_64-pc-windows-msvc (default)
rustc 1.47.0 (18bf6b4f0 2020-10-07)
$ node --version
v15.0.1
$ npm --version
7.0.3
$ yarn --version
1.22.10
$ python --version
Python 3.9.0

Error

$ yarn build_native
yarn run v1.22.10
$ electron-build-env neon build checkmate --release
neon info forcing rebuild for new build settings
neon info running cargo
   Compiling autocfg v1.0.1
   Compiling memchr v2.3.4
   Compiling lazy_static v1.4.0
   Compiling regex-syntax v0.6.21
   Compiling cfg-if v0.1.10
   Compiling cc v1.0.61
   Compiling version_check v0.9.2
   Compiling proc-macro2 v1.0.24
   Compiling unicode-xid v0.2.1
   Compiling cfg-if v1.0.0
   Compiling syn v1.0.48
   Compiling ryu v1.0.5
   Compiling bitflags v1.2.1
   Compiling libc v0.2.80
   Compiling serde_derive v1.0.117
   Compiling lexical-core v0.7.4
   Compiling gimli v0.23.0
   Compiling crc32fast v1.2.1
   Compiling unicode-xid v0.0.4
   Compiling adler v0.2.3
   Compiling semver-parser v0.7.0
   Compiling static_assertions v1.1.0
   Compiling object v0.22.0
   Compiling arrayvec v0.5.2
   Compiling serde v1.0.117
   Compiling quote v0.3.15
   Compiling rustc-demangle v0.1.18
   Compiling rle-decode-fast v1.0.1
   Compiling cslice v0.2.0
   Compiling adler32 v1.2.0
   Compiling libflate_lz77 v1.0.0
   Compiling thread_local v1.0.1
   Compiling neon-build v0.4.0
   Compiling neon-build v0.5.1
   Compiling num-traits v0.2.14
   Compiling num-integer v0.1.44
   Compiling miniz_oxide v0.4.3
   Compiling num-rational v0.2.4
   Compiling num-iter v0.1.42
   Compiling num-complex v0.2.4
   Compiling error-chain v0.12.4
   Compiling nom v5.1.2
   Compiling synom v0.11.3
   Compiling semver v0.9.0
   Compiling addr2line v0.14.0
   Compiling syn v0.11.11
   Compiling aho-corasick v0.7.15
   Compiling quote v1.0.7
   Compiling libflate v1.0.3
   Compiling libmate v0.1.0 (C:\Users\flrn\workbench\datalyzer\node_modules\checkmate\native)
   Compiling regex v1.4.2
   Compiling num-traits v0.1.43
   Compiling backtrace v0.3.54
   Compiling enum-primitive-derive v0.1.2
   Compiling neon-sys v0.4.0
   Compiling num v0.2.1
   Compiling matfile v0.2.1
error: failed to run custom build command for `neon-sys v0.4.0`

Caused by:
  process didn't exit successfully: `C:\Users\flrn\workbench\datalyzer\node_modules\checkmate\native\target\release\build\neon-sys-1096345010ae9bf1\build-script-build` (exit code: 1)
  --- stdout

  > preinstall
  > echo 'Skipping node-gyp installation as part of npm install.'

  'Skipping node-gyp installation as part of npm install.'
  cargo:node_arch=x64
  cargo:node_root_dir=C:\\Users\\flrn\\.electron-gyp\\10.1.4
  cargo:node_lib_file=C:\\Users\\flrn\\.electron-gyp\\10.1.4\\<(target_arch)\\node.lib

  > build-release
  > node-gyp build

  TARGET = Some("x86_64-pc-windows-msvc")
  HOST = Some("x86_64-pc-windows-msvc")
  AR_x86_64-pc-windows-msvc = None
  AR_x86_64_pc_windows_msvc = None
  HOST_AR = None
  AR = None
  running: "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX64\\x64\\lib.exe" "-out:C:\\Users\\flrn\\workbench\\datalyzer\\node_modules\\checkmate\\native\\target\\release\\build\\neon-sys-2c4c217e95a9ae54\\out\\libneon.a" "-nologo" "C:\\Users\\flrn\\workbench\\datalyzer\\node_modules\\checkmate\\native\\target\\release\\build\\neon-sys-2c4c217e95a9ae54\\out\\native\\build\\Release\\obj\\neon\\neon.obj"
  LINK : fatal error LNK1181: cannot open input file 'C:\Users\flrn\workbench\datalyzer\node_modules\checkmate\native\target\release\build\neon-sys-2c4c217e95a9ae54\out\native\build\Release\obj\neon\neon.obj'
  exit code: 1181

  --- stderr
  gyp info it worked if it ends with ok
  gyp info using [email protected]
  gyp info using [email protected] | win32 | x64
  gyp info spawn C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\15.0\Bin\MSBuild.exe
  gyp info spawn args [
  gyp info spawn args   'build/binding.sln',
  gyp info spawn args   '/clp:Verbosity=minimal',
  gyp info spawn args   '/nologo',
  gyp info spawn args   '/p:Configuration=Release;Platform=x64'
  gyp info spawn args ]
  gyp ERR! UNCAUGHT EXCEPTION
  gyp ERR! stack Error: spawn C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\15.0\Bin\MSBuild.exe ENOENT
  gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:269:19)
  gyp ERR! stack     at onErrorNT (node:internal/child_process:465:16)
  gyp ERR! stack     at processTicksAndRejections (node:internal/process/task_queues:80:21)
  gyp ERR! System Windows_NT 10.0.19042
  gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\flrn\\workbench\\datalyzer\\node_modules\\checkmate\\native\\target\\release\\build\\neon-sys-2c4c217e95a9ae54\\out\\native\\node_modules\\node-gyp\\bin\\node-gyp.js" "build"
  gyp ERR! cwd C:\Users\flrn\workbench\datalyzer\node_modules\checkmate\native\target\release\build\neon-sys-2c4c217e95a9ae54\out\native
  gyp ERR! node -v v15.0.1
  gyp ERR! node-gyp -v v3.6.2
  gyp ERR! This is a bug in `node-gyp`.
  gyp ERR! Try to update node-gyp and file an Issue if it does not help:
  gyp ERR!     <https://github.com/nodejs/node-gyp/issues>
  npm ERR! code 7
  npm ERR! path C:\Users\flrn\workbench\datalyzer\node_modules\checkmate\native\target\release\build\neon-sys-2c4c217e95a9ae54\out\native
  npm ERR! command failed
  npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c "node-gyp build"

  npm ERR! A complete log of this run can be found in:
  npm ERR!     C:\Users\flrn\AppData\Local\npm-cache\_logs\2020-11-03T23_44_10_722Z-debug.log


  error occurred: Command "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX64\\x64\\lib.exe" "-out:C:\\Users\\flrn\\workbench\\datalyzer\\node_modules\\checkmate\\native\\target\\release\\build\\neon-sys-2c4c217e95a9ae54\\out\\libneon.a" "-nologo" "C:\\Users\\flrn\\workbench\\datalyzer\\node_modules\\checkmate\\native\\target\\release\\build\\neon-sys-2c4c217e95a9ae54\\out\\native\\build\\Release\\obj\\neon\\neon.obj" with args "lib.exe" did not execute successfully (status code exit code: 1181).


warning: build failed, waiting for other jobs to finish...
error: build failed
neon ERR! cargo build failed

Error: cargo build failed
    at Target.<anonymous> (C:\Users\flrn\workbench\datalyzer\node_modules\neon-cli\lib\target.js:98:27)
    at Generator.next (<anonymous>)
    at fulfilled (C:\Users\flrn\workbench\datalyzer\node_modules\neon-cli\lib\target.js:24:58)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
electron-build-env error
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

As I've said, any help is welcome. Thanks.

Bumps tokio from 1.21.1 to 1.23.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Based on https://github.com/neon-bindings/neon/pull/701, but with the following changes:

• Defers to JSON.stringify, JSON.parse and serde_json on non-primitive values
• Adds cx.deserialize_args for getting a tuple from arguments

Currently, this creates a new object and a new hasRef function on it for every newly created Channel.

Should we optimize this? We could create a single hasRef cached on InstanceData since it uses this. This could possibly be further optimized by creating a class with hasRef that is instantiated. Creating the async resource would look like:

• napi_get_instance_data. Get the cached NeonAsyncResource class.
• napi_new_instance. Create an instance of the class.
• napi_wrap. Wrap the class with the Arc<Mutex<State>> data.
• napi_type_tag. Type tag for safety when unwrapping.

It might be worth using #[track_caller] and track the line number in the resource name. That might help users find out which TSFN is keeping things up.

Resolves https://github.com/neon-bindings/neon/issues/948

Description

In Napi-6 mode, InstanceData registers a drop_queue threadsafe function, which ensures that leaked resources correctly get disposed. However, that TSFN itself is being reported by Jest, why-is-node-running and others (simply "Jest" thereafter) as leaked resources, since that function is never destroyed.

How to reproduce

InstanceData is lazily initialized. The issue can be observed by running any Jest test that forces initialization of InstanceData, notably by creating a Root object or calling cx.channel(). For example:

In rust:

pub fn test_inner_data_drop_queue_leak(mut cx: FunctionContext) -> JsResult<JsObject> {
    let some_object = cx.empty_object().root(&mut cx).into_inner(&mut cx);
    Ok(some_object)
}

In JavaScript:

test('httpWorkflow with mock activity', () => {
      (native as any).testInnerDataDropQueueLeak();
});

Then running using:

npx jest --detectOpenHandles

Results in this Jest warning message:

Jest has detected the following 1 open handle potentially keeping Jest from exiting:

  ●  neon threadsafe function

Discussion and Proposed solution

This is technically a false-positive, since the TSFN is .unref() so that it will not keep the process alive. Unfortunately, Node's async hook callbacks API doesn't provide an official way to know that an async resource has been unref(). There is therefore no way for Jest to know that the TSFN can safely be ignored, by relying on Node's API alone.

Instead, Jest looks at the async resource for the presence of a hasRef() function. If that function exists on the async resource object, then Jest will call that function to determine if the resource is still active. why-is-node-running also support that function, and Node itself has added this function on some of its own resources specifically for the purpose of helping Jest and the like properly detect leaks.

It is therefore suggested that a hasRef() function should be added to the async resource associated TSFNs. That function would simply return either true or false, depending on either the TSFN is referenced() or unref().

I would like to discuss to eventually bump the MSRV of neon to 1.56, as part of the 1.0 release. I used cargo-msrv to find out that the current MSRV is 1.53. This would allow for 2 specific changes (that come to mind now, maybe there are some convenient APIs that we'd get access to):

• Cargo.toml's rust-version setting, to signal the minimum version.
  • Makes it easier for users to find out the current MSRV, and helps the tools in detecting the version as well.
  • Not sure about this, but may help clippy to avoid lints, where the change would require a higher MSRV.
• Change the Rust edition to 2021.
  • Nothing too specific about this version, as I haven't had a deep look in the source code yet, but may allow doing some code simplifications here and there.
  • More than actual language features, likely to signal that the project stays up-to-date with recent Rust versions.

The bump is totally optional. I personally always use the latest Rust compiler, but I understand that's not what everyone does (or can).

Improvements to API docs:

• [x] Align wording for all neon::types doc summaries.
• [x] Examples for all neon::types types.
  • [x] JsArray
  • [x] JsBoolean
  • [x] JsDate
  • [x] JsError
  • [x] JsFunction
  • [x] JsFuture
  • [x] JsNull
  • [x] JsNumber
  • [x] JsObject
  • [x] JsPromise
  • [x] JsString
  • [x] JsUndefined
  • [x] JsValue

Main neon::types page before:

Main neon::types page after: