https://github.com/denoland/deno/issues/10861 and https://github.com/denoland/deno/issues/12532 have been fixed by https://github.com/swc-project/swc/pull/2530, but swc_ecmascript hasn't been updated in a while.

cc @dsherret

My guess is there are some swc feature flags that deno-ast is depending on implicitly, but I have not debugged yet.

$ rustc --version    
rustc 1.61.0 (fe5b13d68 2022-05-18)

I created a new cargo project and added the following dependency:

deno_ast = "0.15.0"

The body of the code is:

use deno_ast::parse_module;
use deno_ast::MediaType;
use deno_ast::ParseParams;
use deno_ast::SourceTextInfo;
use std::sync::Arc;

fn main() {
    let source_text = Arc::new("class MyClass {}");
    let text_info = SourceTextInfo::new(source_text);
    let parsed_source = parse_module(ParseParams {
      specifier: "file:///my_file.ts".to_string(),
      media_type: MediaType::TypeScript,
      text_info,
      capture_tokens: true,
      maybe_syntax: None,
      scope_analysis: false,
    }).expect("should parse");
    
    // returns the comments
    parsed_source.comments();
    // returns the tokens if captured
    parsed_source.tokens();
    // returns the module (AST)
    parsed_source.module();
    // returns the `SourceTextInfo`
    parsed_source.source();
}

Compiling:

   Compiling swc_ecma_parser v0.104.2
error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/class_and_fn.rs:323:36
    |
323 |             span: span!(self, expr.span_lo()),
    |                                    ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
  --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr/ops.rs:96:30
   |
96 |             let start = left.span_lo();
   |                              ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr/ops.rs:215:34
    |
215 |             span: Span::new(left.span_lo(), right.span_hi(), Default::default()),
    |                                  ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_hi` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr/ops.rs:215:51
    |
215 |             span: Span::new(left.span_lo(), right.span_hi(), Default::default()),
    |                                                   ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_hi` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr/ops.rs:256:45
    |
256 |             let span = Span::new(start, arg.span_hi(), Default::default());
    |                                             ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_hi` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr/ops.rs:314:44
    |
314 |                 span: Span::new(start, arg.span_hi(), Default::default()),
    |                                            ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr/ops.rs:347:40
    |
347 |                 span: span!(self, expr.span_lo()),
    |                                        ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
  --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:22:26
   |
22 |         let start = expr.span_lo();
   |                          ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_hi` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:220:45
    |
220 |             let span = Span::new(start, alt.span_hi(), Default::default());
    |                                             ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for reference `&Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:918:44
    |
918 |                     exprs.first().unwrap().span_lo(),
    |                                            ^^^^^^^ method not found in `&Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_hi` found for reference `&Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:919:43
    |
919 |                     exprs.last().unwrap().span_hi(),
    |                                           ^^^^^^^ method not found in `&Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:960:36
    |
960 |         let tagged_tpl_start = tag.span_lo();
    |                                    ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for enum `swc_ecma_ast::Callee` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1171:38
     |
1171 |             let span = Span::new(obj.span_lo(), self.input.last_pos(), Default::default());
     |                                      ^^^^^^^ method not found in `swc_ecma_ast::Callee`

error[E0599]: no method named `span_lo` found for enum `swc_ecma_ast::Callee` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1172:34
     |
1172 |             debug_assert_eq!(obj.span_lo(), span.lo());
     |                                  ^^^^^^^ method not found in `swc_ecma_ast::Callee`

error[E0599]: no method named `span_lo` found for enum `swc_ecma_ast::Callee` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1293:40
     |
1293 |             let span = span!(self, obj.span_lo());
     |                                        ^^^^^^^ method not found in `swc_ecma_ast::Callee`

error[E0599]: no method named `span_lo` found for enum `swc_ecma_ast::Callee` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1294:34
     |
1294 |             debug_assert_eq!(obj.span_lo(), span.lo());
     |                                  ^^^^^^^ method not found in `swc_ecma_ast::Callee`

error[E0599]: no method named `span_hi` found for enum `swc_ecma_ast::MemberProp` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1295:35
     |
1295 |             debug_assert_eq!(prop.span_hi(), span.hi());
     |                                   ^^^^^^^ method not found in `swc_ecma_ast::MemberProp`

error[E0599]: no method named `span_hi` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1634:60
     |
1634 | ...                   span: Span::new(start, alt.span_hi(), Default::default()),
     |                                                  ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for reference `&swc_ecma_ast::Expr` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1841:58
     |
1841 |         Ok(self.state.potential_arrow_start == Some(expr.span_lo())
     |                                                          ^^^^^^^ method not found in `&swc_ecma_ast::Expr`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::TsType>` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/typescript.rs:2152:42
     |
2152 |                     span: span!(self, ty.span_lo()),
     |                                          ^^^^^^^ method not found in `Box<swc_ecma_ast::TsType>`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::TsType>` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/typescript.rs:2159:42
     |
2159 |                     span: span!(self, ty.span_lo()),
     |                                          ^^^^^^^ method not found in `Box<swc_ecma_ast::TsType>`

error[E0599]: no method named `span_lo` found for struct `swc_ecma_ast::Ident` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/typescript.rs:2262:26
     |
2262 |         let start = expr.span_lo();
     |                          ^^^^^^^ method not found in `swc_ecma_ast::Ident`

For more information about this error, try `rustc --explain E0599`.
error: could not compile `swc_ecma_parser` due to 22 previous errors

Temporarily reverting this one because it causes source maps to have blank source file names and there's too much to deal with WRT the latest swc upgrade (so let's do a follow up PR).

Reverts denoland/deno_ast#87

cc @kitsonk

Some mime type registries associate this content type with the .jsx extension.

Examples:

• https://github.com/samuelneff/MimeTypeMap/blob/master/MimeTypeMap.cs#L273
• https://github.com/abonander/mime_guess/blob/492bece6d9b3253aeeb42d78d9358352d66ee935/src/mime_types.rs#L568

@dsherret I thought about the issue you pointed out, and I think I had it wrong, in that if we have a content type of application/javascript we should respect that, assuming the remote did the transpile already and is clearly overriding automatic content types, except for situations where it would be a type only file anyways (.d.ts, .d.mts, .d.cts) where clearly something is "wrong" since those don't emit.

I don't know any specific behaviours in the wild, but best to keep the behaviour and document it. Based on that, I also aligned it so that a application/javascript with a .mts file becomes MediaType::Mjs and application/javascript with .cts file becomes MediaType::Cjs.

@bartlomieju @ry while I still have my reservations about it, this whole thing we are having to do for TypeScript 4.5 should also allow us to identify and handle CJS files through the module "pipeline". I guess if we are going to do it, I would rather do it properly. 😒

While trying to run the example:

use deno_ast::parse_module;
use deno_ast::MediaType;
use deno_ast::ParseParams;
use deno_ast::SourceTextInfo;
use std::sync::Arc;

let source_text = Arc::new("class MyClass {}");
let text_info = SourceTextInfo::new(source_text);
let parsed_source = parse_module(ParseParams {
  specifier: "file:///my_file.ts".to_string(),
  media_type: MediaType::TypeScript,
  text_info,
  capture_tokens: true,
  maybe_syntax: None,
  scope_analysis: false,
}).expect("should parse");

// returns the comments
parsed_source.comments();
// returns the tokens if captured
parsed_source.tokens();
// returns the module (AST)
parsed_source.module();
// returns the `SourceTextInfo`
parsed_source.source();

I get an error saying:

error[E0308]: mismatched types
  --> src/parsers/javascript/parser.rs:32:45
   |
32 |         let text_info = SourceTextInfo::new(source_text);
   |                                             ^^^^^^^^^^^ expected `str`, found `&str`
   |

I've tried casting the string, but I can only ever get it to be a &str. Any help is appreciated, I'm not sure what I'm going wrong.

The new OptCall in swc makes a lot of our code more error prone. We will upgrade once this issue is resolved or alternatively change dprint-swc-ecma-ast-view to combine these two nodes together in the wrapper: https://github.com/swc-project/swc/issues/4010

---- ast::tests::test_analyze_dependencies_import_assertions stdout ----
thread 'ast::tests::test_analyze_dependencies_import_assertions' panicked at 'assertion failed: `(left == right)`
  left: `None`,
 right: `Some("json")`', src\ast.rs:395:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- tests::test_create_graph_import_assertions stdout ----
thread 'tests::test_create_graph_import_assertions' panicked at 'assertion failed: `(left == right)`

Diff < left / right > :
 Object({
     "roots": Array([
         String(
             "file:///a/test01.ts",
         ),
     ]),
     "modules": Array([
         Object({
             "size": Number(
                 9,
             ),
             "mediaType": String(
                 "Json",
             ),
             "specifier": String(
                 "file:///a/a.json",
             ),
         }),
         Object({
             "size": Number(
                 7,
             ),
             "mediaType": String(
                 "Json",
             ),
             "specifier": String(
                 "file:///a/b.json",
             ),
         }),
         Object({
             "size": Number(
                 9,
             ),
             "mediaType": String(
                 "Json",
             ),
             "specifier": String(
                 "file:///a/c.json",
             ),
         }),
         Object({
             "size": Number(
                 7,
             ),
             "mediaType": String(
                 "Json",
             ),
             "specifier": String(
                 "file:///a/d.json",
             ),
         }),
         Object({
             "dependencies": Array([
                 Object({
                     "specifier": String(
                         "./a.json",
                     ),
                     "code": Object({
                         "specifier": String(
                             "file:///a/a.json",
                         ),
                         "span": Object({
                             "start": Object({
                                 "line": Number(
                                     1,
                                 ),
                                 "character": Number(
                                     26,
                                 ),
                             }),
                             "end": Object({
                                 "line": Number(
                                     1,
                                 ),
                                 "character": Number(
                                     36,
                                 ),
                             }),
                         }),
                     }),
                     "assertionType": String(
                         "json",
                     ),
                 }),
                 Object({
                     "specifier": String(
                         "./b.json",
                     ),
                     "code": Object({
                         "specifier": String(
                             "file:///a/b.json",
                         ),
                         "span": Object({
                             "start": Object({
                                 "line": Number(
                                     2,
                                 ),
                                 "character": Number(
                                     35,
                                 ),
                             }),
                             "end": Object({
                                 "line": Number(
                                     2,
                                 ),
                                 "character": Number(
                                     45,
                                 ),
                             }),
                         }),
                     }),
                     "isDynamic": Bool(
                         true,
                     ),
>                    "assertionType": String(
>                        "json",
>                    ),
                 }),
                 Object({
                     "specifier": String(
                         "./c.json",
                     ),
                     "code": Object({
                         "specifier": String(
                             "file:///a/c.json",
                         ),
                         "span": Object({
                             "start": Object({
                                 "line": Number(
                                     3,
                                 ),
                                 "character": Number(
                                     31,
                                 ),
                             }),
                             "end": Object({
                                 "line": Number(
                                     3,
                                 ),
                                 "character": Number(
                                     41,
                                 ),
                             }),
                         }),
                     }),
                     "assertionType": String(
                         "json",
                     ),
                 }),
                 Object({
                     "specifier": String(
                         "./d.json",
                     ),
                     "code": Object({
                         "specifier": String(
                             "file:///a/d.json",
                         ),
                         "span": Object({
                             "start": Object({
                                 "line": Number(
                                     5,
                                 ),
                                 "character": Number(
                                     35,
                                 ),
                             }),
                             "end": Object({
                                 "line": Number(
                                     5,
                                 ),
                                 "character": Number(
                                     45,
                                 ),
                             }),
                         }),
                     }),
                     "isDynamic": Bool(
                         true,
                     ),
                 }),
             ]),
             "mediaType": String(
                 "TypeScript",
             ),
             "size": Number(
                 329,
             ),
             "specifier": String(
                 "file:///a/test01.ts",
             ),
         }),
     ]),
     "redirects": Object({}),
 })

', src\lib.rs:1491:5

---- tests::test_parse_module_import_assertions stdout ----
thread 'tests::test_parse_module_import_assertions' panicked at 'assertion failed: `(left == right)`

Diff < left / right > :
 Object({
     "dependencies": Array([
         Object({
             "specifier": String(
                 "./a.json",
             ),
             "code": Object({
                 "specifier": String(
                     "file:///a/a.json",
                 ),
                 "span": Object({
                     "start": Object({
                         "line": Number(
                             1,
                         ),
                         "character": Number(
                             18,
                         ),
                     }),
                     "end": Object({
                         "line": Number(
                             1,
                         ),
                         "character": Number(
                             28,
                         ),
                     }),
                 }),
             }),
             "assertionType": String(
                 "json",
             ),
         }),
         Object({
             "specifier": String(
                 "./b.json",
             ),
             "code": Object({
                 "specifier": String(
                     "file:///a/b.json",
                 ),
                 "span": Object({
                     "start": Object({
                         "line": Number(
                             2,
                         ),
                         "character": Number(
                             17,
                         ),
                     }),
                     "end": Object({
                         "line": Number(
                             2,
                         ),
                         "character": Number(
                             27,
                         ),
                     }),
                 }),
             }),
             "isDynamic": Bool(
                 true,
             ),
>            "assertionType": String(
>                "json",
>            ),
         }),
     ]),
     "mediaType": String(
         "TypeScript",
     ),
     "size": Number(
         119,
     ),
     "specifier": String(
         "file:///a/test01.ts",
     ),
 })

', src\lib.rs:1896:5


failures:
    ast::tests::test_analyze_dependencies_import_assertions
    tests::test_create_graph_import_assertions
    tests::test_parse_module_import_assertions

From my code https://github.com/dprint/dprint-plugin-typescript/blob/6ec15e921a836835b8f4cad9ae48c834788afabe/src/swc/parse_swc_ast.rs#L95 -- Going to move it down here into deno_ast.

Will fix https://github.com/denoland/deno/issues/12454 and probably some other --no-check issues.

cc @lucacasonato

I think it would be beneficial to:

1. Move all transpiling and bundling out of the CLI and to deno_ast. It would be in a feature flag. Obviously this would be bundling without deno_graph so there would be some adapter that deno_graph would be plugged into.
2. Add a ton of bundling and transpiling tests in here.
3. Pin all swc deps to a specific patch version so everything upstream gets tested on the same swc deps (might be good to do this anyway)

I think this would allow us to:

1. Ensure everything is in a good state before we upgrade all the upstream crates. (We don't find something is wrong after integrating into the cli).
2. Help us refactor out Deno.emit to a Wasm module that could connect this with deno_graph.

@kitsonk thoughts? I think this would be quick to do. I could start soon if you agree.

We should move in this code from deno_lint to this crate so it can be used in the deno lsp. I'm not sure exactly what that entails...

https://github.com/denoland/deno_lint/blob/7825cb95115bfdffc62c076c8f458b3424f7190c/src/ast_parser.rs#L114

Once this is done, we should open up a PR in deno to reuse the ast for linting.

Currently returns an anyhow::Error. Should really be returning a proper structured enum based error (for ease of use thiserror can be used to build this error struct).

https://github.com/esm-dev/esm.sh/blob/master/packages/cjs-esm-exports/src/cjs.rs

We should at the minimum add more tests from here: https://github.com/esm-dev/esm.sh/blob/master/packages/cjs-esm-exports/src/test.rs

For https://github.com/denoland/deno/issues/8529

This would allow taking a Diagnostic and then formatting it showing the position in the source file text the diagnostic occurred for.

We should add more unit tests that run this transpile using input and output files.

https://github.com/denoland/deno/blob/43a63530acb16e57cbb190eacedbd097c536a775/cli/ast/mod.rs#L208