Vue Compiler in Rust
https://github.com/vuejs/rfcs/discussions/369#discussioncomment-1192421
Maybe in the long run we want the whole transform (and even the Vue compiler!) to be implemented in native Go/Rust so performance would no longer be a concern ;)
Future is now!
Architecture
The original design in vue-next mixes code generation and ast parsing in the same data structure. As we can see, the transform pass will in-place mutate ast nodes, leaving the node with both code generation node and ssr code generation node.
This is typically a sign of leaky abstraction.
So in the Rust version I decided to take another approach.
The design targets at three different levels of developers in Vue ecosystem:
- Lib/App creator: every Vue developers who write component library or application code.
- Platform developer: Devs who write compiler implementation for DOM/SSR/Native platform.
- Framework author: Vue core lib author a.k.a Evan.
The core framework targets multiple platforms and can be extended to support more. Core framework components span all platforms and are hardwired to the core lib runtime.
Platforms are usually DOM or SSR environment. Hosts are browser and node, respectively. Developing a platform needs to write code for both vue-compiler and vue-runtime. Optionally platform developer can write code in host, e.g. in hybrid app or mini-program.
And finally lib or app creators can write component library, business code or application components targeted to certain platforms.
The compilation has several phases:
- Scan (output: Tokens): Hardwired in the compiler at framework level.
- Parse (output: template AST): Hardwired in the compiler at framework level.
- Convert (output: intermediate representation): Customizable for platform developers with sensible default.
- Transform (input/output: customizable IR): Customizable with default by using generic/traits.
- Code Generate (customizable output: e.g. JS/TS): Customizable with default.
Other Design different from the original compiler
- Directive parsing is implemented manually instead of by regexp.
nodeTransformsis not supported. It's too hard for app creator to use and maintain IR invariants. Platform devs can still customize by implementing converter/transformer.directiveTransformsnow can returns not onlyPropsbut alsoSimpleExpression. The extra flexibility makes a more cohesive v-bind/v-on conversion: the logic for processing the directives now resides in one single file without regard to the presence of an argument.- Runtime helper collection
context.helper/helperStringis moved out from convert and tracked in transform phase, avoiding several methods and reducing HashMap to a bitflag.
Intended Usage
- Rust library
- CLI binary
- napi based nodejs library
- wasm based npm package: a fallback if napi fails to work and a toy for browser.
- No Browser build No support since most features in full build are additional except for browser based expression checking or HTML escaping. Browser build removed them for size. But template compiler in browser is already for toy project. For browser specific summary see this google sheet.
Implementation Detail
- Plenty of
debug_asserts to maintain compiler state invariants. - The library seeks minimal allocation by using
&str,Cow<'_, str>andsmallvec. - A customized
VStris used to minimize string manipulation. Fxhashis preferred over default hasher since hash collision is not a concern.- The
bitflagscrate is used to represent runtime helper and vnode patch flags. - Use heavily optimized routines for string search primitives. (Perf reference)
- Benchmark with criterion.rs.
- Test compiler output by snapshot test.
- Use alternative allocator like wee_alloc or mi_malloc.
- Use
Box<[T]>instead ofVecto reduce type size. - Use Arean to minimize allocation.
- A
Futurelike stack-allocated transformationPasscomposition. - Use
Rcto manage error handler. Don't optimize wrong code. - Parallelized conversion with Rayon.
Reference
- vue-next: ご本家様
- html spec is the definitive guide for parsing HTML-like files.
- Vue Template Explorer gives instant results for code generation and error reporting.
- Nu html checker is the official html validator from W3C. This is the canonical error reporter for html parsing, when there is a discrepancy between the framework and the spec.
- AST explorer can inspect AST nodes interactively.
Roadmap
Todo tasks grouped by scopes.
[util]
- VStr
- string intern
- camel/pascal cache
- str ops
[core]
- scanner
- UTF8 support
- parser
- IR converter
- v-if
- v-for
- v-slot
- v-model
- slot outlet
- element
- build props
- transformer
-
SWCRSLint integration - Rewrite MergePass struct
-
- code generator
- module preamble
- wrap error handler in Rc
- compile option
- Arena allocation
- Parallelization
[dom]
- IR converter
- v-on
- v-once
- v-memo
- transformer
- code generator
[ssr]
- TODO
[sfc]
- TODO
[test]
- scanner test
- parser test
- dir parser test
- Add insta snapshot
- Move snapshot outside of src
[bench]
- Add benchmark framework
- Micro benchmarks for scanner
- Micro benchmarks for parser
- Micro benchmarks for converter
- Micro benchmarks for transformer
- Micro benchmarks for codegen
- Integrated benchmarks using repos like Element-Plus
[infra]
- Add pre-commit hooks.
- Add Github Actions for various checks.
- Change single lib to cargo workspaces.
[community]
- TODO. not ready for contribution for now.
25.4k Dec 31, 2022
3 Nov 2, 2022
146 Dec 16, 2022
31 Dec 22, 2022
25 Dec 24, 2022
133 Dec 5, 2022
190 Nov 12, 2022
261 Dec 1, 2022
1.3k Dec 21, 2022
5 Feb 28, 2024
1.7k Jan 3, 2023
3.2k Jan 4, 2023
4.4k Jan 7, 2023
3.5k Jan 7, 2023
296 Oct 18, 2022
66 Nov 28, 2022
318 Jan 1, 2023
18 Feb 9, 2022