Simplified glue code generation for Deno FFI libraries written in Rust.

• [ ] When deno_bindgen errors out it creates a empty bindings.json which should be fixed
• [ ] Marko should check before if the file is a valid json if not recreate it

&str is already accepted as a parameter type, but it would be nice 🌠 to be able to return a &str. (Thanks to Andreu and divy for help in the discord.)

Here's an example of a &str param from the 0.2 release announcement:

#[deno_bindgen]
pub fn say_hello(message: &str)  {
    println!("{}", message);
}

If it's changed to return a &str instead of printing it...

#[deno_bindgen]
pub fn say_hello(message: &str) -> &str {
    format!("{}", message)
}

The result is:

error: custom attribute panicked  
 --> src/lib.rs:5:1               
  |                               
5 | #[deno_bindgen]               
  | ^^^^^^^^^^^^^^^               
  |                               
  = help: message: not implemented

That's from the current version of deno_bindgen, 0.5.

This would be a really nice feature to have and would allow easily passing text across the FFI boundary.

At present, the compiled library can only be used for a single platform. If you want to support cross platform, you need to build multiple sets of code. This adds a lot of tedious work.

// Auto-generated with deno_bindgen
import { CachePolicy, prepare } from "https://deno.land/x/[email protected]/plug.ts"
const opts = {
  name: "teyda_core",
  url: (new URL("../target/release", import.meta.url)).toString(),
  policy: undefined,
}
const _lib = await prepare(opts, {
  core_run: { parameters: [], result: "void", nonblocking: false },
  mh: { parameters: ["pointer", "usize"], result: "void", nonblocking: false },
})

export function test() {
  let rawResult = _lib.symbols.core_run()
  const result = rawResult
  return result
}

use deno_bindgen::deno_bindgen;
use image::{load_from_memory, EncodableLayout};
use webp::Encoder;

#[deno_bindgen]
pub struct Img {
    bin: Option<Vec<u8>>,
}

#[deno_bindgen]
pub fn toWebp(img: &[u8], quality: u8) -> Img {
    if let Ok(img) = load_from_memory(img) {
        let encoder = Encoder::from_image(&img).unwrap();
        let encoded_webp = encoder.encode((quality as f32) / 100.0);
        return Img {
            bin: Some(encoded_webp.as_bytes().into()),
        };
    }

    Img { bin: None }
}

export function toWebp(a0: Uint8Array, a1: number) {
  const a0_buf = encode(a0)

deno_bindgen doesn't work cause https://github.com/denoland/deno/pull/15518

$ deno test -A --unstable
Check file:///Users/skanehira/dev/github.com/denoland/deno_bindgen/example/bindings_test.ts
error: TS2345 [ERROR]: Argument of type 'Uint8Array' is not assignable to parameter of type 'ToNativeType<"usize" | "pointer">'.
  let rawResult = _lib.symbols.add2(a0_buf, a0_buf.byteLength)
                                    ~~~~~~
    at file:///Users/skanehira/dev/github.com/denoland/deno_bindgen/example/bindings/bindings.ts:173:37

TS2345 [ERROR]: Argument of type 'Uint8Array' is not assignable to parameter of type 'ToNativeType<"usize" | "pointer">'.
  let rawResult = _lib.symbols.test_buf(a0_buf, a0_buf.byteLength)
                                        ~~~~~~
    at file:///Users/skanehira/dev/github.com/denoland/deno_bindgen/example/bindings/bindings.ts:184:41

TS2345 [ERROR]: Argument of type 'Uint8Array' is not assignable to parameter of type 'ToNativeType<"usize" | "pointer">'.
  let rawResult = _lib.symbols.test_buffer_return(a0_buf, a0_buf.byteLength)
                                                  ~~~~~~
    at file:///Users/skanehira/dev/github.com/denoland/deno_bindgen/example/bindings/bindings.ts:190:51

TS2345 [ERROR]: Argument of type 'Uint8Array' is not assignable to parameter of type 'ToNativeType<"usize" | "pointer">'.
    a0_buf,
    ~~~~~~
    at file:///Users/skanehira/dev/github.com/denoland/deno_bindgen/example/bindings/bindings.ts:197:5

TS2345 [ERROR]: Argument of type 'Uint8Array' is not assignable to parameter of type 'ToNativeType<"usize" | "pointer">'.
  let rawResult = _lib.symbols.test_lifetime(a0_buf, a0_buf.byteLength)
                                             ~~~~~~
    at file:///Users/skanehira/dev/github.com/denoland/deno_bindgen/example/bindings/bindings.ts:210:46

TS2345 [ERROR]: Argument of type 'Uint8Array' is not assignable to parameter of type 'ToNativeType<"usize" | "isize" | "pointer">'.
  let rawResult = _lib.symbols.test_mixed(a0, a1_buf, a1_buf.byteLength)
                                              ~~~~~~
    at file:///Users/skanehira/dev/github.com/denoland/deno_bindgen/example/bindings/bindings.ts:226:47

TS2345 [ERROR]: Argument of type 'Uint8Array' is not assignable to parameter of type 'ToNativeType<"i32" | "usize" | "pointer">'.
    a1_buf,
    ~~~~~~
    at file:///Users/skanehira/dev/github.com/denoland/deno_bindgen/example/bindings/bindings.ts:234:5

TS2345 [ERROR]: Argument of type 'Uint8Array' is not assignable to parameter of type 'ToNativeType<"usize" | "pointer">'.
  let rawResult = _lib.symbols.test_mut_buf(a0_buf, a0_buf.byteLength)
                                            ~~~~~~
    at file:///Users/skanehira/dev/github.com/denoland/deno_bindgen/example/bindings/bindings.ts:243:45

TS2345 [ERROR]: Argument of type 'Uint8Array' is not assignable to parameter of type 'ToNativeType<"usize" | "pointer">'.
  let rawResult = _lib.symbols.test_serde(a0_buf, a0_buf.byteLength)
                                          ~~~~~~
    at file:///Users/skanehira/dev/github.com/denoland/deno_bindgen/example/bindings/bindings.ts:264:43

TS2345 [ERROR]: Argument of type 'Uint8Array' is not assignable to parameter of type 'ToNativeType<"usize" | "pointer">'.
  let rawResult = _lib.symbols.test_str(a0_buf, a0_buf.byteLength)
                                        ~~~~~~
    at file:///Users/skanehira/dev/github.com/denoland/deno_bindgen/example/bindings/bindings.ts:270:41

TS2345 [ERROR]: Argument of type 'Uint8Array' is not assignable to parameter of type 'ToNativeType<"usize" | "pointer">'.
  let rawResult = _lib.symbols.test_tag_and_content(a0_buf, a0_buf.byteLength)
                                                    ~~~~~~
    at file:///Users/skanehira/dev/github.com/denoland/deno_bindgen/example/bindings/bindings.ts:281:53

Found 11 errors.

Currently encode encodes as u8 array and then gives the ptr and array len.

I suggest to use cstrings instead so encoding as an u8 array that ends with a null byte.

The why is the current use method requires Deno.UnsafePointer.of(), which requires unbounded --allow-ffi (similar issue https://github.com/denoland/deno/issues/15511)

With the second method, there is no issue, and I can use --allow-ffi=lib

error: custom attribute panicked --> src/lib.rs:5:1 | 5 | #[deno_bindgen] | ^^^^^^^^^^^^^^^ | = help: message: Type definition not found for f32 identifier

fix https://github.com/denoland/deno_bindgen/issues/91

When deno_bindgen run on Windows, the fetchPrefix should be ..\target\debug. But \ escapes the next characters, so the final fetchPrefix will be \..argetdebug.

Hello, I know this is a rust based codegen but I hope it's ok to ask this. What could be the best way to access structs that contains strings in Deno from C? Is it advisable just to stringify the object in C so that I can do new Deno.UnsafePointerView(ptr).getCString() in Deno?

Example

typedef struct { char name; int age; } SampleObject;

First pass, authors will be able to pass the target triplet and the corresponding download URL (maybe using a JSON file). The CLI will generate code to download and cache the library.

After that is implemented, authors should also be able to provide a git repository URL or source (zip/tar/...) or crates.io publish and deno_bindgen will generate relevant glue code to build the crate from source using the available Rust toolchain on the user's machine.

Using deno_bindgen==0.7.0 I'm facing some issues with non_blocking / async codegen.

Given the following defs:

#[deno_bindgen(non_blocking)]
pub fn test_str_async() -> String {
    "test".to_string()
}

#[deno_bindgen(non_blocking)]
pub fn test_u32_async() -> u32 {
    1
}

#[deno_bindgen(non_blocking)]
pub fn test_void_async() {}

Generates the following TS:

...

const _lib = await prepare(opts, {
  test_str_async: { parameters: [], result: "pointer", nonblocking: true },
  test_u32_async: { parameters: [], result: "u32", nonblocking: true },
  test_void_async: { parameters: [], result: "void", nonblocking: true },
})

export function test_str_async() {
  let rawResult = _lib.symbols.test_str_async()
  const result = rawResult.then(readPointer)
  return result.then(decode)
}
export function test_u32_async() {
  let rawResult = _lib.symbols.test_u32_async()
  const result = rawResult
  return result
}
export function test_void_async() {
  let rawResult = _lib.symbols.test_void_async()
  const result = rawResult
  return result
}

Some issues:

• Function type declarations are missing and inferred as any, for test_str_async I would expect a return type of Promise<string> or the function to be marked as async function ...
• test_u32_async & test_void_async are not resolving the promise result like test_str_async - there's no indication in the function declaration it returns a promise

deno 1.26.0 (release, x86_64-unknown-linux-gnu) v8 10.7.193.3 typescript 4.8.3

code is there https://github.com/usrtax/deno_bindgen_test

use deno_bindgen::deno_bindgen;

#[deno_bindgen]
fn add3() -> &'static [u8] {
    &[
        1, 2, 3, 4, 5, 6, 7, 9, 0, 1, 2, 3, 4, 5, 6, 7, 9, 0, 1, 2, 3, 4, 5, 6, 7, 9, 0, 1, 2, 3,
    ]
}

test code https://github.com/usrtax/deno_bindgen_test/blob/main/lib/test.js

#!/usr/bin/env -S node --es-module-specifier-resolution=node --trace-uncaught --expose-gc --unhandled-rejections=strict
var main, n, sleep;

import {
  add3
} from '../bindings/bindings.ts';

sleep = () => {
  return new Promise((resolve) => {
    return setTimeout(resolve, 10);
  });
};

n = 0;

while (true) {
  main();
  if (n++ % 10000 === 0) {
    gc();
    await sleep();
    console.log(n, Deno.memoryUsage());
  }
}

run as begin

run 3mintues RES 1040M

run 10minutes RES 2177M

run 1hour RES 14.9G

Deno.memoryUsage doesn't seem to be very accurate, pay more attention to the memory usage in top

i created a pull request for Vec https://github.com/denoland/deno_bindgen/pull/106

but i real need return type is Option<Vec<u8>>

i don't kown how to impl this

can anyone help me ?

I originally provided a PR https://github.com/denoland/deno_bindgen/pull/82 (merged) for this issue: https://github.com/denoland/deno_bindgen/issues/72

I have since discovered more about Rust compilation and procedural macros and realised my "fix" doesn't work as per this: https://github.com/rust-lang/docs.rs/issues/1823

The two issues with attempting to use OUT_DIR:

• OUT_DIR is not defined when running any type of cargo build except when a build.rs exists.
• if OUT_DIR is defined (because a build.rs exists), the deno_bindgen_macro will output to ${OUT_DIR}/bindings.json as expected, but the cli.ts does not have OUT_DIR defined and will default to looking for bindings.json in the current working folder.

I also see that my "fix" introduced another issue which was fixed here: https://github.com/denoland/deno_bindgen/pull/89

After learning more and understanding more... I humbly submit a PR which:

• rolls back the non-working fix: https://github.com/denoland/deno_bindgen/pull/82
• rolls back the additional fix required for my non-working fix: https://github.com/denoland/deno_bindgen/pull/89
• fixes a typo in the docs

On my journey, I have also discovered that the issue of procedural macros having side-effects has been discussed elsewhere: https://github.com/rust-lang/cargo/issues/9084

It would seem this issue is a general issue with using procedural macros for outputting auxiliary data and out of my league in terms of resolving.

I am not sure what the solution is, but for now I will have to put up with the fact that when my project makes use of deno_bindgen I can't refer to the generated Rust docs on docs.rs...

Found this to be an issue when switching between building release and debug. It wouldn't let you go from release to debug because the rust source hadn't changed, so now you can do a clean build through the cli.