Hi,

I've been writing a dotnet target for diplomat-tool in order to expose a C# API to picky. I actually used it to generate fully functional bindings to picky in my experiment repository. The only missing piece is a way to return a buffer of bytes (basically something like the DiplomatWriteable, but using std::io::Write instead of std::fmt::Write I guess).

I still have a few corner cases to fix, but I figured I could open this PR as a draft.

Todo before I consider the MVP complete

• [x] Fix generated code when bool type is in return position.
• [x] Special case to handle results using the unit () type as error.

Future improvements

• Proper handling of Option of enum in parameters
• Support custom types in managed non-opaque struct's fields (aka: avoid double freeing). Currently the only way to access custom types of non-opaque structs is to use the "raw" API with pointers.

#172 introduces the DiplomatBuf class in diplomat_runtime.js, which basically handles conversion from JS strings and primitive slices to arrays in wasm memory that Rust can read, which allows us to reduce boilerplate in the codegen.

Currently, we use TextEncoder.encode to encode strings into a Uint8Array, and then copy that into a view of wasm memory. We should be using TextEncoder.encodeInto to avoid this intermediate array.

For slices, we use the Uint8Array constructor to convert an iterable of JS numeric objects into an array, and then copy that into wasm memory. We should look into if there are any alternatives like how strings haveencodeInto that would allow us to skip this intermediate array, and then use those if possible.

Currently when we return a struct containing opaques we generate a struct with a getter:

https://github.com/rust-diplomat/diplomat/blob/517f1a4423fb8a88d5af9dcd1465efe8fb9987ec/example/js/api.mjs#L166-L171

and then the method being generated immediately calls that getter and replaces it with the value:

https://github.com/rust-diplomat/diplomat/blob/517f1a4423fb8a88d5af9dcd1465efe8fb9987ec/example/js/api.mjs#L99-L105

We should probably not have this burden be on the method and instead handle this automagically in struct getter code.

I tried to run latest diplomat-tool on my FFI wrapper as well as the icu4x one, and got the following errors:

An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
An alias or submodule was found instead of a custom type with the name ICU4XError.
thread 'main' panicked at 'explicit panic', tool/src/main.rs:98:9

Apparently, some recent change (soon after the last pinned commit for icu4x) added some verification that is a bit too strict, probably this: https://github.com/rust-diplomat/diplomat/blob/main/core/src/ast/types.rs#L742 I'm not exactly sure of the intention behind it, but it probably needs to be relaxed or changed? This prevents usage of types from other modules (especially common for error types).

This is a prerequisite for making the C++ headers self contained

You can see this in action at https://github.com/unicode-org/icu4x/pull/2823

This is a breaking change for people using the C headers in a way where they relied on the exact import order to get additional types. That is considered bad practice and these are the C headers so I'm fine breaking it.

I'll have to do something similar in C++ though, unsure if it's as kosher there.

This PR builds off of #208 by adding a basic webpack demo that bundles everything up and lets you use the demo ICU4X bindings in example/js/lib/ on a webpage. To use, navigate to example/js/app-webpack/, run npm install, and then npm run build to run the bundler. Then index.html can be viewed through the browser (I use LiveServer. Additionally, npm run clean removes all the auto generated files.

Additionally, I changed example/js/app to instead offer a similar web interface to app-webpack, and can be set up in the same way, where the only difference is that it doesn't bundle everything together nicely. I also replaced the ava tests in app/ with a toy CLI tool that lets you format numbers. The ava tests are still in lib/.

Finally, this fixes a few small bugs encountered along the way, like how DiplomatBuf finalization code was broken and untested (as also found in https://github.com/unicode-org/icu4x/pull/2216).

This is the baby fix for https://github.com/unicode-org/icu4x/issues/2182, where, at the very least, our capi namespacing is handled by the C headers instead of doing the very unidiomatic namespace capi { #include ... }. Ideally we can have all namespaces managed well, but for now this ought to suffice.

Currently, we use bool to indicate mutability, and we end up with things like

match &param.ty {
    ast::TypeName::StrReference(/* mutable */ true) => { ... },
    // ...
}

This is bad because without the comment, it's not clear what true refers to. And the comment looks gross.

This PR introduces the ast::Mutability enum type so that we can express intent through the type system. It's definition is very simple:

pub enum Mutability {
    Mut,
    Const,
}

This makes examples like above much more readable:

match &param.ty {
    ast::TypeName::StrReference(ast::Mutability::Mut) => { ... },
    // ...
}

struct Foo<'a>(&'a str);
struct Bar<'b, 'a>(&'b Foo<'a>);

#[diplomat::opaque]
struct ICU4XFoo<'a>(Foo<'a>);


#[diplomat::opaque]
struct ICU4XBar<'b, 'a>(Bar<'b, 'a>)

impl<'a> ICU4XFoo<'a> {
   pub fn new(x: &str) -> Box<ICU4XFoo<'a>> {
       .....
   }

   pub fn get_bar<'b>(&'b self) -> Box<ICU4XBar<'b, 'a>> {
       ....
   }
}

The generated Rust code will fail to compile because get_bar generates a lifetimeless ICU4XFoo_get_bar() function. When there's a single lifetime in play this is fine, but this rapidly breaks down as there are multiple.

This is probably something to fix as a part of https://github.com/rust-diplomat/diplomat/issues/12.

My tests are triggering the UseTree::Glob(_) => todo!("Glob imports are not yet supported {:?}", base_path), since I'm doing:

#[cfg(test)]
mod test {
    use super::*;
    ...
}

I put a print statement in Module::from_syn and the Item doesn't seem to have the context of where it is in the AST to filter them out. I'm not sure how generalized this kind of behavior could be, but at least at this point it would be nice to not analyze tests.

extract_imports ItemUse {
    attrs: [],
    vis: Inherited,
    use_token: Use,
    leading_colon: None,
    tree: Path(
        UsePath {
            ident: Ident {
                sym: super,
                span: bytes(14966..14971),
            },
            colon2_token: Colon2,
            tree: Glob(
                UseGlob {
                    star_token: Star,
                },
            ),
        },
    ),
    semi_token: Semi,
}

We should be able to accept Option<&Opaque> arguments. In languages that support nullable values, the signature can be the same as the non-optional version.

A few options on the table:

1. std::optional<std::reference_wrapper<T>>
2. T*
3. A custom type

It is easy to change this; we just need to finalize the decision.

• [ ] Refactor C2 to have two entrypoints and only one header field, removing the is_decl parameter of gen_ty_name: https://github.com/rust-diplomat/diplomat/pull/301#discussion_r1045086396
• [ ] Support LibraryConfig: https://github.com/rust-diplomat/diplomat/pull/301#discussion_r1045087835
• [ ] Have the CppContext first invoke C generation, and use a separate main.rs code path https://github.com/rust-diplomat/diplomat/pull/301#discussion_r1045088601
• [ ] Support custom Writeable templates: https://github.com/rust-diplomat/diplomat/pull/301#discussion_r1045095068

"Slice" is the unsized datatype [T], which we don't support.

In the future we want an OwnedSlice (a.k.a. Box<[T]>), so we should be clear about what type of slice this is.

Related: https://github.com/rust-diplomat/diplomat/issues/103

In most places we just splat typ.name() etc into the source, which isn't always safe in the case of reserved keywords.

In many cases this type will be used as a variable name, or something. Ideally the tool backends will have some kind of special formatter for types (etc) that displays them correctly with additional underscores/etc. In general, every place an AST identifier is printed, we should be using the backend-specific formatter instead.

The C++ output of Diplomat was primarily designed with an intention of getting it working. Now that we have it, we should try to make it good

Some things that need to happen:

• https://github.com/rust-diplomat/diplomat/issues/278
• https://github.com/rust-diplomat/diplomat/issues/277
• https://github.com/rust-diplomat/diplomat/issues/242
• https://github.com/rust-diplomat/diplomat/issues/271
• In general going through and fixing all warnings. We have a couple

We could wati for the CPP2 backend, but I think these might be critical to good adoption, so we should try to do them early.

We should have a C++ expert vet our final design before we release it.