I had a gut feeling this will make it faster but I don't know the precise reason.

Something along the lines of LLVM / branch prediction / cache friendliness ...

This uses SIMD for speeding up the multiline comment lexer.

e.g.

babylon.max.js (10M) has 14241 occurrences of multiline comments. typescript.js (9.6M) 22183 occurrences of multiline comments.

But I have yet to figure out

• [x] how to detect newlines properly in this PR.
• [x] The current hiccup is due to the multi-byte newline character LS and PS.

Speeding up single line comment parsing should save a few millisecond.

On the other hand, skipping multi-line comments in chunks will give us a huge improvement.

u32 is enough for spans.

https://github.com/Boshen/oxc/blob/b6912513ba93ebb01ebc8882598951e26a79ea6c/crates/oxc_ast/src/node.rs#L11-L18

This is currently using usize because miette only accepts Range in its miette::Diagnostic macro.

https://github.com/Boshen/oxc/blob/b6912513ba93ebb01ebc8882598951e26a79ea6c/crates/oxc_diagnostics/src/lib.rs#L29-L33

There should be a workaround but I haven't investigated in detail yet.

Description

It seems that there should exist a # symbol to make Panics be an title when user see the document comment.

Additional Information

I'm highly interested in this project, but I'm a newbie in Rust and compiler. Currently trying to understand the compiler concepts from the guide you wrote and trying to understand this great project :)

I'm also not good at english writing, so if I have written anything that offend you, please accept my apologies.

From Twitter:

• MiniJinja author said avoiding ? improved both the compile time and the runtime performances.
• dtolnay said they don't use ? operators in serde and serde-json "because it is too slow".
• serde has their own try! macro that avoids type conversion https://github.com/rust-lang/rust/issues/37939
• mitsuhiko says its still not fix

Sometime ago I drafted a version to use SIMD for skipping whitespaces:

In this good first issue, you should be able to replicate the code from above, and add it to the lexer.

You should refactor the code above into a function and call the function just above the while loop:

https://github.com/Boshen/oxc/blob/93993978de969589f1fb91a689594a5e5c4b8f02/crates/oxc_parser/src/lexer/mod.rs#L320-L330

You will also need to add #![feature(portable_simd)] to the top of crates/oxc_parser/src/lib.rs

To the run parser, you can run the example binary:

https://github.com/Boshen/oxc/blob/3daf5bcea95f6fe2c88a7956948f4cb22711f99e/crates/oxc_parser/examples/simple.rs#L7-L10

Background reading:

• https://rapidjson.org/md_doc_internals.html#SkipwhitespaceWithSIMD
• https://github.com/Tencent/rapidjson/blob/bdc49ad80a3cff89a25839dcdcd6a68b457ec9f1/include/rapidjson/reader.h#L258-L517
• https://lemire.me/blog/2017/01/20/how-quickly-can-you-remove-spaces-from-a-string

The global allocator is slower.

The final configuration should follow swc once we have a binary.

NOTE: Replacing the allocator has not effect at the moment because all of the parser's memory is allocated by bumpalo and dropped at wholesale.

On my 2.6 GHz 6-Core Intel Core i7 laptop.

group                    base
-----                    ----
parser/babylon.max.js    1.00    104.9±3.14ms    98.4 MB/sec
parser/d3.js             1.00     13.1±1.51ms    41.7 MB/sec
parser/lodash.js         1.00      4.5±0.34ms   113.0 MB/sec
parser/pdf.js            1.00      7.4±0.45ms    54.0 MB/sec
parser/typescript.js     1.00    104.3±2.99ms    92.3 MB/sec

It seems I need a way to verify the code is correct.

Basically a parse -> print -> parse routine will sufficiently indicate if parsing is correct.

Running cargo coverage prints

Test262 Summary:
AST Parsed     : 43934/43934 (100.00%)

The current parser parses all of test262.

"Positive Passed" and "Negative Passed" are commented out in the code because they not meaningful right now. They will become meaningful once semantic analysis is in place.

The conformance task now snapshots all diagnostics:

• test262.snap
• babel.snap
• typescript.snap

What you can do is review these diagnostics and improve them, you can checkout babel or tsc for reference, by using https://astexplorer.net/

To begin with, it should be relative easy to fix some of the Unexpected token errors.