Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

As a library author, how do I dual publish to both node and deno? #3196

Closed
brainkim opened this issue Oct 24, 2019 · 59 comments
Closed

As a library author, how do I dual publish to both node and deno? #3196

brainkim opened this issue Oct 24, 2019 · 59 comments

Comments

@brainkim
Copy link

brainkim commented Oct 24, 2019

Let’s say that I have a library which is published on npm. If that library

  1. is written 100% in typescript
  2. has no dependencies
  3. does not use node.js built-in modules
  4. does not use deno.js built-in modules

what is the best way to add support for deno? Is there a good example repository which does this?

Additionally, if I have library which depends only on modules which dual support both node and deno, does this story change?

Related: #2644

@kitsonk
Copy link
Contributor

kitsonk commented Oct 24, 2019

For Deno, you don't need npm at all. Having it located somewhere on a CDN which the .ts code is supported directly. An example of a rich library would be something like oak.

The biggest "problem" with dual support would be that modules have to end in the extension in Deno. Also Node.js doesn't support TypeScript directly, and doesn't support ESM (without enabling experimental modules). So you would have to have a build a version that would work frictionlessly in Node.js and have that be your distributable to npm. I'm personally not aware of anyone who has created a good build script for Deno .ts to CommonJS .js, but it should/could be possible fairly straight forward. The builder would need to provide a Compiler Host that resolved modules in line with Deno's module resolution and also strip out the .ts extensions on emit.

If you are using Deno namespace APIs, almost the reverse of #2644 will be needed, almost a "browserify" layer that translates Deno APIs to Node.js ones.

@brainkim
Copy link
Author

brainkim commented Oct 25, 2019

Ahhh yeah the big issue is the requirement of file extensions for deno. I like your suggestion of moduleResolution: "literal" (preserve?) in the related typescript issue. If the typescript team is gonna stick with following the node module resolution algorithm, maybe deno can offer a competitive alternative to tsc which can compile to node compatible code?

Regardless of how successful deno becomes, I see dual-supporting both node and deno as pretty much necessary for most library authors.

Related: microsoft/TypeScript#27481

@kitsonk
Copy link
Contributor

kitsonk commented Oct 25, 2019

@brainkim regarding the modules resolution, there is also: microsoft/TypeScript#33437 that would be more of an ideal, but getting Node.js to support it... Though Node.js is moving towards extension required modules under the experimental ES support, there would still be a need to change the extensions on the emitted code, and if TypeScript (tsc) touches it, they will get it right for some and wrong for a lot of others.

@beshanoe
Copy link

beshanoe commented Oct 30, 2019

I think for the purpose of creating "isomorphic" Deno libs and CLIs it makes sense to implement sort of Node.js polyfill for all the Deno's built-in APIs, then bundle your tool using deno bundle and after that prepend it with the require from the polyfill, like

const {Deno, TextDecoder, TextEncoder, ...} = require('deno-polyfill');

/// deno generated bundle

probably it also makes sense to convert AMD modules in the original bundle to commonjs

@hayd
Copy link
Contributor

hayd commented Oct 30, 2019

allegedly this is what @YounGoat's deno package on npm is supposed to do...
https://www.npmjs.com/package/deno

@brainkim
Copy link
Author

Okay so deno bundle is a piece of the puzzle that I wasn’t aware of. According to the website, deno bundle is a tool which compiles your deno code to AMD (😬). Does this mean I can use typescript or webpack to compile to amd as well and have deno modules consume this?

it makes sense to implement sort of Node.js polyfill for all the Deno's built-in APIs,

This is interesting, but I’m also interested in a simpler use-case where I use 0 deno builtins and 0 node builtins (pretty much vanilla typescript).

@ry
Copy link
Member

ry commented Oct 30, 2019

@brainkim Deno can consume a single JS file or ES modules if you need imports. It doesn’t know about AMD except as an output from the bundle command.

(The AMD bit is really throwing people off - we should modify deno bundle to use https://github.com/denoland/deno/blob/master/deno_typescript/amd_runtime.js to execute the main module.)

@ry ry closed this as completed Oct 30, 2019
@ry ry reopened this Oct 30, 2019
@kitsonk
Copy link
Contributor

kitsonk commented Oct 31, 2019

@ry we talked about it for Bundling v2 (#2475), but it does feel like embedding the runner directly in the bundle so it is a totally self contained script is warranted and knock a few features off the v2 list. That way we can just say "it generates a standalone script" and let people forget about the implementation.

I can start working on this.

@kitsonk kitsonk mentioned this issue Oct 31, 2019
@tunnckoCore
Copy link

The whole thing would be easier if deno bundle is outputting a better format. I'm curious why AMD is chosen instead of UMD? Or at least it would be pretty trivial to allow users to output different format, through flag?

@kitsonk
Copy link
Contributor

kitsonk commented Oct 31, 2019

@tunnckoCore again, people are obsessing about something they shouldn't obsess about. 😁

UMD specifically would be useless. That is, depending on the implementation, 2 different module formats and an ability to load as a global script, but when you only need one module format that is loaded into an environment that doesn't support CommonJS for good reason.

If you really want to know the reason though, it goes something like this:

  • ES Modules are not concatenatable. All module bundlers that support consuming ESM have to manipulate the modules to some intermediary format, usually proprietary.
  • CommonJS modules are not concatenatable. Again bundles rewrite them to some intermediary format.
  • UMD modules are not typically concatenatable. UMD is great if you don't know if you are going to load individual modules under Node.js or if you are going to load individual modules in a browser. It isn't so great if you know your final target is a single .js file.
  • AMD modules are always concatenatable without any manipulation. It was part of the module design, because it was designed specifically for the web, knowing that you might have a single file that defines lots of modules.
  • TypeScript supports outputting both JavaScript and TypeScript (irrespective of source module format) as AMD JavaScript modules into a single file without any additional code.

So we were left with the scenarios of:

  • Creating our own emit logic to some non-standard bundle format and dealing with all the edge cases around default exports, imports, dynamic imports and re-exports of ES modules.
  • Integrating some 3rd party bundler into the Deno binary that uses a proprietary output format.
  • Leverage something that "just works" but needs a minimal module loader added in.

We opted for the last. The only value in the first two is to be able to say we weren't using AMD, which seems like a very odd thing to do.

AMD came into being for some valid good and valid reasons, and ESM doesn't solve all of them. There are very good reasons why ESM isn't concatenatable, and I respect that, but it does mean in a lot of cases, we will be stuck with some form of bundlers in perpetuity (and likely AMD).

@brainkim
Copy link
Author

brainkim commented Oct 31, 2019

@kitsonk I would consider System.js as a bundle output as well. I think typescript supports it as an output (though last I checked support is not great and System.js can be tricky to set up), it’s more closely aligned with ESmodules, and System.register calls can be robustly concatenated. It just feels more future-proof/standards-aligned IMO.

But most of this is unrelated to this issue, which is, given a pure typescript codebase without dependencies, how do I add support for deno consumers? The two approaches would be:

  1. Write for deno, compile for node.
    This can’t be done because there is no stand-alone compiler which understands deno’s explicit module resolution at present. Maybe babel can help us here?
  2. Write for node, compile for both node and deno.
    This is doable at present but non-ideal because deno supports typescript natively and having to compile to deno means we need another explicit d.ts pragma to find the typings. Also I need some guidance about what deno as a compilation target looks like. Do I need to compile to amd require modules? Can I use UMD or System.js? Should modules be concatenated/minified?

@tunnckoCore
Copy link

@brainkim , last week or two I'm thinking about the second too.

Write Nodejs (js/ts), so need for deno vscode/typescript plugins. Output esm and cjs formats - e.g. dist/index.js and dist/esm/index.js, and put the typings files on the dist/esm (e.g. dist/esm/index.d.ts), so later when Deno users want to use your module then will use some CDN (preferable UNPKG), like https://unpkg.com/@scope/pkg?module. Not sure if Deno will pick up the separate index.d.ts file?

import koko from 'https://unpkg.com/@tunnckocore/kokoko?module';

koko('sasas', 200);
// should error, because both arguments must be `number`

I have two dummy modules, that I'm playing with: @tunnckocore/kokoko and @tunnckocore/qwqwqw check them on unpkg.

The interesting thing is that UNPKG is helping us when using the ?module query param. The only problem, I think... (cc @mjackson) is that it strips the sourceMapping comment when using this query param. I'm not sure if this is a mistake or it should be that way, but anyway.

@tunnckoCore
Copy link

tunnckoCore commented Nov 21, 2019

Ha... I already wrote to that issue, haha.

Just thought about another idea that spawned after looking around the other "compat" issues and discussions, mainly triggered to look on them today because just seen that denolib/node was archieved with saying it's merged into Deno. So, first, what's that, what that mean?

And second, the another possible idea:

/**
 * ## Deno <-> Node.js Compat (a CLI)
 *
 * - Deno source, two build targets: nodejs (esm & cjs) & browser (esm)
 * - For nodejs target:
 *   + ** rollup babel as very first plugin (with babelrc: false)
 *     to transform url (deno apis) to bare specifier (nodejs apis)
 *   + output cjs and esm rollup "format"s
 * - For browsers target:
 *   + detect if URL specifier ends with `.ts`
 *   + then rollup url import/resolve plugin (as very first),
 *   + then rollup babel (as very first) transform (with babelrc: false)
 *     the typescript to javascript, then allow user-land (allow babelrc)
 *
 */

** another variant of that step is to directly URL import/resolve from Deno and transform that TS to JS, BUT that way consumers of that nodejs code source may have problems if there are any significant differences or bugs or behaviors between what Deno gives, for example, for the path.join and what Node gives for path.join

Write mainly in Deno. And if you want to support both Browsers and Node (and Deno, hence the source) you can use a small smart CLI that can be created, nothing fancy, just a thin translation layer using @rollup so we can output whatever format we want. It's would probably be best to be written in Node.js and then bundled as single file binary.

@brainkim
Copy link
Author

I think the best course of action is just to convince typescript peeps to allow explicit file names.

@dubiousjim
Copy link
Contributor

dubiousjim commented Jan 17, 2020

Don't know if this is the best place to post this, but I wanted to share the pattern I came up with for running code on both node and deno. I append this to the end of a .mjs file, which I can run in Node 12+ (using --experimental-modules) or in Deno.

function main (script, args, exit, host, handler) {
  if (args.length !== 1) {
    console.log('Usage: '+script+' FILE');
    exit(1);
  }
  const filename = args[0];
  if (host === 'deno') {
    const decoder = new TextDecoder('utf-8');
    const source = decoder.decode(Deno.readFileSync(filename));
    return handler(source);
  } else {
    return (async () => {
      const fs = await import('fs');
      const path = await import('path');
      const source = fs.readFileSync(path.normalize(filename), 'utf8');
      return handler(source);
    })();
  }
};

if (typeof process === 'object' && process.argv[0].endsWith('/node') && import.meta.url == 'file://' + process.argv[1])
  main(process.argv[1], process.argv.slice(2), process.exit, 'node', MYFILEHANDLER);
else if (typeof Deno === 'object' && import.meta.main)
  main(location && location.pathname, Deno.args, Deno.exit, 'deno', MYFILEHANDLER);

Not up to the needs of library authors, I know, but it's useful to have such a recipe for quick scripts that might be run on either host.

@dubiousjim
Copy link
Contributor

I expanded that recipe a bit, folding in the options parsing from std/flags, to be usable on both node and deno. Result is here.

@brainkim
Copy link
Author

brainkim commented Feb 6, 2020

I’ve finally made my first foray with Deno. The unpkg ?module strategy almost works, except now I have a bunch of TypeScript errors showing up for ES5 transpiled code.

Screen Shot 2020-02-06 at 2 37 36 PM

If I transpile to ESNext I worry that I will mess with dependents who don’t transpile their sources (for things like async generators). And ideally I would be able to reference my modules locally for local development, but this second library I’m working on has a single dependency (also owned by me). Anyone have any updates on best practices?

@brainkim
Copy link
Author

brainkim commented Feb 6, 2020

Coming up with a concrete story for dual publishing to npm and deno would get me to switch. unpkg might work for a while but it would suck to have to publish for local development.

@kitsonk
Copy link
Contributor

kitsonk commented Feb 6, 2020

@brainkim take a look at your package on Pika.dev: https://www.pika.dev/npm/@bikeshaving/crank

Try loading it from there... It "should" serve up the .d.ts files to Deno using the X-TypeScript-Types header which then Deno will use when type checking the code, instead of trying to type check the JavaScript. If it doesn't work, they would very much like to work with you I suspect to get it to work.

@brainkim
Copy link
Author

brainkim commented Feb 7, 2020

@kitsonk Tried out the link and got a 404. The module which pika points to doesn’t seem to exist. If you’re saying pika can allow me to dual publish to npm and deno then I’ll check it out and see if I can get it to work. I was initially turned off by references to some all-in-one editing solution but I’m willing to give it a shot.

Figuring out local development might still be a pain though.

Screen Shot 2020-02-06 at 7 48 02 PM

@kitsonk
Copy link
Contributor

kitsonk commented Feb 7, 2020

Go to that page, there is a different URL listed on the page. Pika CDN is a good CDN for both browsers and Deno for packages that contain ESM modules and types.

Hopefully @axetroy can help with the development environment end for Vscode. The type information merging is a relatively new feature and I don't know if his plugin understands it well. The information is in the Deno cache, so the plugin could support it.

@brainkim
Copy link
Author

brainkim commented Feb 7, 2020

https://cdn.pika.dev/@bikeshaving/crank@^0.1.0-beta.4 responds with a file like this:

/*
  Pika CDN - @bikeshaving/crank@0.1.0-beta.5
  https://www.pika.dev/npm/@bikeshaving/crank

  How it works:
    1. Import this package to your site using this file/URL (see examples).
    2. Your web browser will fetch the browser-optimized code from the export statements below.
    3. Don't directly import the export URLs below: they're optimized to your browser specifically and may break other users.

  Examples:
    - import {Component, render} from 'https://cdn.pika.dev/preact@^10.0.0';
    - import {Component, render} from 'https://cdn.pika.dev/preact@10.0.2';
    - import {Component, render} from 'https://cdn.pika.dev/preact@next';
    - import {Component, render} from 'https://cdn.pika.dev/preact';
    - const {Component, render} = await import('https://cdn.pika.dev/preact@^10.0.0');

  Learn more: https://www.pika.dev/cdn
*/

export * from '/-/@bikeshaving/crank@v0.1.0-beta.5/dist=es2019/crank.js';
export {default} from '/-/@bikeshaving/crank@v0.1.0-beta.5/dist=es2019/crank.js';

https://cdn.pika.dev/-/@bikeshaving/crank@v0.1.0-beta.5/dist=es2019/crank.js responds with a 404 with the following message:

Not Found: This package doesn't match hash "beta.5".

This probably has something to do with the dash or something.

This is a maze with lots of possible paths, and I’m not sure how pika helps over something like unpkg, but I’m happy for any assistance. Like I said, creating a concrete story for devs who want to dual publish is of paramount importance. Especially if you want to pick up authors who author “universal” modules, deno offers so much in its reproduction of standard DOM apis.

@hayd
Copy link
Contributor

hayd commented Feb 7, 2020

Try without the caret: https://cdn.pika.dev/@bikeshaving/crank@0.1.0-beta.4

@brainkim
Copy link
Author

brainkim commented Feb 7, 2020

Same thing:
Screen Shot 2020-02-07 at 1 48 20 AM
Screen Shot 2020-02-07 at 1 48 32 AM
https://cdn.pika.dev/-/@bikeshaving/crank@v0.1.0-beta.5/dist=es2019/crank.js

The problem probably is the dash is being parsed weirdly by pika.

@brainkim
Copy link
Author

brainkim commented Feb 7, 2020

For what it’s worth, Deno finding types via an X-TypeScript-Types header seems no better to me than package.json types fields or d.ts sibling files. I want module resolution to be obvious, and having to inspect responses for a header seems worse (to me).

@axetroy
Copy link
Contributor

axetroy commented Feb 7, 2020

@brainkim

What strange module is this?

export * from '/-/@bikeshaving/crank@v0.1.0-beta.4/dist=es2019/crank.js';

@balupton
Copy link
Contributor

balupton commented Jul 22, 2020

I've created make-deno-edition to make npm packages written in typescript compatible with deno - is working on badges - usage proof of this here https://repl.it/@balupton/badges-deno - has been used now to make 32 node packages compatible with deno - you can use project to automatically generate the readme instructions for the deno edition - and can use boundation to automatically scaffold your projects to automate the entire process - start-of-week is an example where different entries are used for node, deno, and web browsers

@brainkim
Copy link
Author

brainkim commented Jul 30, 2020

Here are some techniques I’ve discovered for node -> deno publishing while trying to make my frontend framework deno compatible.

  1. @ts-ignore the import module specifier.
import {
  Children,
  Context,
  Element as CrankElement,
  ElementValue,
  Portal,
  Renderer,
  // @ts-ignore: explicit ts
} from "./crank.ts";

You can add a @ts-ignore directive on the line directly above the module specifier. This technique works surprisingly well, and if you do this, your source typescript files will be directly importable from deno. This messes up d.ts files but I have another hack later on to address this.

  1. @deno-types comments and triple-slash directives
// @deno-types="https://unpkg.com/@bikeshaving/crank@0.3.0/index.d.ts"
import {createElement} from "https://unpkg.com/@bikeshaving/crank@0.3.0/index.js";
// @deno-types="https://unpkg.com/@bikeshaving/crank@0.3.0/html.d.ts"
import {renderer} from "https://unpkg.com/@bikeshaving/crank@0.3.0/html.js";

You can import modules without cooperation from the package author by importing the JS equivalents, and referencing the d.ts types using a @deno-types directive. Many complex libraries have d.ts files which do not actually work with deno, so your mileage may vary.

If you are the package author, you can use the following rollup plugin snippet using the package "magic-string" to prepend a triple-slash reference (/// <reference-types="index.d.ts" />) to your build artifacts.

rollup.config.js

import MagicString from "magic-string";

/**
 * A hack to add triple-slash references to sibling d.ts files for deno.
 */
function dts() {
  return {
    name: "dts",
    renderChunk(code, info) {
      if (info.isEntry) {
        const dts = "./" + info.fileName.replace(/js$/, "d.ts");
        const ms = new MagicString(code);
        ms.prepend(`/// <reference types="${dts}" />\n`);
        code = ms.toString();
        const map = ms.generateMap({hires: true});
        return {code, map};
      }
      return code;
    },
  };
}

export default {
  /* other rollup options */
  plugins: [/* plugins */, dts()],
};

This allows Deno consumers to import your rollup build artifacts and have them be typed, without any extra effort on their part. This again assumes the d.ts files work out of box, which is not guaranteed.

Between @ts-ignore-ing of module specifiers and this method, I’m not sure which technique is better. The former technique allows you to use your ts source files directly, but you might want to funnel both deno and node users to use the same build output for consistency, especially if you have source transforms or other compilation magic going on in your codebase. Additionally, you have to add @ts-ignore directives to every import in your source code, which can be a hassle.

The latter technique of relying on rollup or similar is nice, but it precludes serving your modules from raw.githubusercontent.com or deno.land/x because we typically don’t check in build artifacts into source control. I found that this technique works okay when using unpkg, although I am getting weird type errors when trying to use semver redirects.

  1. Fixing d.ts files

Method 1 causes typescript to produce d.ts files with module specifiers that have file extensions (export * from "./crank.ts"), which will throw off typescript when used from node. Method 2 will produce d.ts files with bare module specifiers (export * from "./crank"), which will work depending on server configuration. For instance, unpkg will redirect that bare specifier to the js file, which according to method 2 should have a triple slash reference, so everything will work out fine. However, if you reference the build artifact locally, you will get errors from deno about the missing file extension.

You can use the typescript transform ts-transform-import-path-rewrite to rewrite module specifiers in d.ts files. The typescript custom transformer API is basically undocumented, and works differently according to the build tools you use, but with a bit of trial and error I got the above package working with rollup-plugin-typescript2 as follows:

rollup.config.js

import typescript from "rollup-plugin-typescript2";
import {transform} from "ts-transform-import-path-rewrite";

/**
 * A hack to rewrite import paths in d.ts files for deno.
 */
function transformer() {
  const rewritePath = transform({
    rewrite(importPath) {
      // if you use method 1, you need to replace the `.ts` with `.js`
      return importPath + ".js";
    },
  });
  return {afterDeclarations: [rewritePath]};
}

export default {
  /* config options */
  plugins: [ts({transformers: [transformer]}), /* other plugins */],
}

You can refer to the full rollup.config.js file to see what I did in context, which is a combination of methods 2 and 3. I might add method 1 later, except I found that unpkg works okay. Like I mentioned I’ve discovered weird errors when unpkg redirects urls for semver reasons, but then again deno.land doesn’t seem to have semver features yet, so I don’t really care.

Needless to say, anyone who is even partly responsible for this omnishambles should feel bad. I will not be turning these techniques into a library/module because I don’t want to maintain it, and in an ideal world, the TypeScript team will stop stonewalling on explicit file extensions in module specifiers, and all these techniques will be obsoleted. In that respect, I want to say good job to all the people writing constant “Any progress on this” comments in the related TypeScript issues and pinging the TypeScript maintainers directly with rambling, bad-faith arguments. With persistence we shall win.

In the meantime, these techniques seem to work for smaller libraries, and you should feel free to copy-paste the code in my rollup config into your own libraries.

@tunnckoCore

The thing I'm bouncing around constantly is, should we concentrate on writing Deno and compile to Node or the opposite 😆 Which seems easier? Hm 🤔

I think writing for node and compiling to deno is unfortunately much easier than writing for deno and compiling for node as of today (July 30th). The big difference is that node tooling is just much further advanced (bundlers, testing). I think at the end of the day, a tool which turns deno modules into real packages would be great, but I don’t think the deno ecosystem is there yet for dual publishers.

@davidcallanan
Copy link

My solution: YouTube - Integrating Deno and Node.js

@reggi
Copy link

reggi commented Sep 7, 2020

I just came across this:

https://github.com/garronej/denoify

It a tool to author node and convert to deno.

@marcushultman
Copy link
Contributor

marcushultman commented Oct 15, 2020

I'm made another tool, inspired by denoify (that didn't meet my requirements):

https://github.com/marcushultman/denoc

Smaller (single dependency), more explicit but with simpler setup.

@reggi
Copy link

reggi commented Oct 18, 2020

I'm still very interested in creating authoring deno and a way to convert it to node (deno -> node). ❤️❤️❤️

@ebebbington
Copy link
Contributor

Maybe seeing how https://github.com/ebebbington/context-finder is setup might help

@hardy925
Copy link

hardy925 commented Oct 18, 2020

@reggi 👋

Hello, I have written a simple project in deno called ts_serialize, however, it is all vanilla Typescript with no dependencies (from either deno or anything else). And in our CI I generate a build for NPM (npm package) and run some tests on it.

We manually keep a .d.ts file that we copy over for types. But everything else is automated.

These are the files I need to create the npm build

First, we use babel_ts_serialize.ts which uses deno bundle to make the UMD file then we pass that into bable.transformSync.

the build file contains the steps to make the folder and the publishing happens in the release CI.

To test the node package before deploying, we set up an example node project that uses * the lastest in its package file then we just npm link the recent build and run the standard test command npm test.

create_npm_package_file.sh does just that, but takes input for the version number in the package file.

The release is triggered via pushing a new tag that starts with v*.

@shadowtime2000
Copy link

shadowtime2000 commented Nov 2, 2020

Came across esm.sh a CDN that polyfills Node core packages with the Deno Node compatability layer and converts to ES.

@shadowtime2000
Copy link

Looking through the yargs it seems they take a simpler approach without converters like Denoify. It seems like they are writing Typescript using the appropriate tsconfig.json options to use ES5 modules with extensions. They use TS to build the native browser ES version, Rollup to generate the CJS build from that, and for Deno they just have a mod.ts or something to import stuff from the source code. For Deno/Node native stuff they use a pattern of injecting shims into the core.

@brillout
Copy link

Has anyone tried to do this with WASI?

@caspervonb
Copy link
Contributor

caspervonb commented Dec 20, 2020

Has anyone tried to do this with WASI?

The code to initialize the WASI context is slightly different between Node and Deno, otherwise should be fine as we're fairly compliant.

@brillout
Copy link

Neat. Although Node's WASI support is still behind an experimental flag so I guess we need a couple of months/years until we can reasonably ask library users to use a WASI supporting Node version.

@caspervonb
Copy link
Contributor

Neat. Although Node's WASI support is still behind an experimental flag so I guess we need a couple of months/years until we can reasonably ask library users to use a WASI supporting Node version.

It's not completely unreasonable to ask that today depending on what the library does, e.g is it impractical to do the same thing in JavaScript. WebAssembly is a lot simpler than dealing with whatever incarnation of NAPI we're at now.

@stale
Copy link

stale bot commented Feb 20, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Feb 20, 2021
@shadowtime2000
Copy link

Not stale I guess - I think this could be converted to a GH discussion

@stale stale bot removed the stale label Feb 20, 2021
@eemeli
Copy link

eemeli commented Feb 20, 2021

I'm still a bit boggled about how little documentation there's anywhere about writing libraries for Deno, and the existence of this issue keeps reminding me about it.

There's plenty about writing applications, but for libraries, the only thing I can find on deno.land is the "Add a module" button under "Third Party Modules". I just did that for yaml. And no, I don't actually know what a "module" is supposed to look like because there doesn't appear to be any documentation on that. Guess I'll find out when someone complains about it.

@rivy
Copy link
Contributor

rivy commented Feb 21, 2021

TLDR; design pattern for single-source to CJS/ESM/TypeScript + Deno.

Being somewhat frustrated by tooling in this space, and inspired by the yargs solution, I created a design pattern for module creation that is single-source and directly generates both JS and Deno compatible modules, without the use of any post-production conversion tools.

TypeScript is used as the fundamental source language for the module source code. Importantly, all internal module imports have fully specified relative paths (with extensions). All source mode files are written in TypeScript, with '.ts' extensions. But, all internal imports use the '.js' extension. TypeScript's resolution algorithm will find the source file correctly. Yes, this is definitely confusing, but it is the officially supported method available since at least TypeScript-v2.0.

The TypeScript source is transpiled into an ESM which, with some platform abstraction, can be utilized by both NodeJS and Deno. Importantly, for Deno operation, the target transpiled ESM code must be stored back into the repository (eg, in a 'dist' folder).

The platform abstraction (Platform.Adapter) contains all external dependencies and platform-specific built-in variations. Platform-dependent definitions of Platform.Adapter contain the necessary implementation code. For example, from 'os-paths':

// src/platform-adapters/_base.ts
export namespace Platform {
    export type Adapter = {
        readonly env: { readonly get: (_: string) => string | undefined };
        readonly os: { readonly homedir?: () => string; readonly tmpdir?: () => string };
        readonly path: {
            readonly join: (..._: readonly string[]) => string;
            readonly normalize: (_: string) => string;
        };
        readonly process: {
            readonly platform: string;
        };
    };
}
// src/platform-adapters/node.ts
import * as os from 'os';
import * as path from 'path';

import { Platform } from './_base.js';

export const adapter: Platform.Adapter = {
    env: {
        get: (s) => {
            return process.env[s];
        },
    },
    os,
    path,
    process,
};

From working TypeScript code, an Adapt(adapter: Platform.Adapter) factory function can be constructed, with relative ease, as a closure around all platform-dependent code.

Then, when the module is imported, the Adapt() factory function is used which injects an instance of the required platform-specific adapter into the module object, invisible to the consumer. For example, again from 'os-paths':

// src/mod.esm.ts
import { Adapt, OSPaths } from './lib/OSPaths.js';
import { adapter } from './platform-adapters/node.js';

const default_: OSPaths = Adapt(adapter).OSPaths;

export type { OSPaths };
export default default_;

After creating a module, coded in TypeScript, producing ESM-compatible code (which is not simple), the platform abstraction code and Deno implementation is a small, straight-forward changeset (from xdg-app-paths).

For detailed/working examples, I've just finished converting three small modules:

To be sure, I'm not overly happy with the necessary npm/yarn dev scripts and long list of dev dependencies, but such is the state of JS/TypeScript author tooling for cross-platform projects.

Ultimately, all of the modules are available for use by CJS/ESM/TypeScript (via NPMjs, github repo, or jsdelivr CDN) and Deno (via 'deno.land/x', github repo, or jsdelivr CDN). See the respective repositories for further implementation details and READMEs for further usage details.

Of important note, avoiding CJS/TypeScript, and coding the module just to ESM and Deno would still allow the same platform-adapter pattern but with a huge drop in dev complexity, scripting, and tooling.

@ebebbington
Copy link
Contributor

@eemeli that isn’t necessarily related to this issue, but you can find all info in the manual, everything else is knowledge from Js and Ts

@eemeli
Copy link

eemeli commented Feb 21, 2021

@ebebbington Huh, maybe I just missed it then? I'm looking for answers to the following sorts of things, if they're in the Deno manual maybe you could give me a link?

  • What is expected of a well-behaved Deno library module? Should I e.g. have a separate README for Deno users?
  • Is there an example somewhere of such a module?
  • If my library has some external dependency, can I provide an import map or something for Deno users?
  • The "Add a module" flow asks for an optional "subdirectory in your repository that the module to be published is located in." If I want to change my choice for that later, what should I do?

@Soremwar
Copy link
Contributor

Soremwar commented Feb 21, 2021

@eemeli

What is expected of a well-behaved Deno library module?

There isn't a standard for creating a Deno module. JS libraries are a collection of methods, classes and objects that are made available through the user as they see fit. However well documented libraries tend to contain the following:

If my library has some external dependency, can I provide an import map or something for Deno users?

Import maps are not mean for libraries, but for end users. It's encouraged for library authors to use deps.ts convention or a similar approach

@kitsonk kitsonk closed this as completed Feb 21, 2021
@denoland denoland locked and limited conversation to collaborators Feb 21, 2021

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
None yet
Development

No branches or pull requests