This is a work in progress to address the issue discussed here: https://github.com/php-rust-tools/parser/issues/258

Some notes:

1) Currently, die() and exit() are both expressions. I have kept them that way, but I am wondering whether it would be more concise to construct them as statements instead. This is tricky because they behave like functions, but they are not.

I have coded the spans start and end using an Option<Span> for die() and exit(), as parentheses are optional for them: https://github.com/php-rust-tools/parser/commit/1a0fd301818da1db7e05a3d2236bdb893de8f5b8

print() handles this differently with value: Parenthesized {:

https://github.com/php-rust-tools/parser/blob/main/tests/fixtures/0269/ast.txt#L349-L355

2) print is treated as an expression, but it was coded differently: https://github.com/php-rust-tools/parser/blob/main/src/parser/expressions.rs#L1099

(The Precedence::Prefix shouldn´t be Precedence::Print?)

I am wondering whether die() and exit() should follow the same ideas used in print() or if we need to change print() to follow die() and exit().

3) isset() and unset() are special because they cannot accept all kind of argument. For example, isset(null) is not valid. Therefore, we need to create a new issue to describe the need for a new function to parse their parameters and throw parser errors when necessary. I think this could be work for another PR.

Not trivial to decide those things. :sweat_smile:

When running the test suite on my computer (Linux, Intel I5 6 cores) I got this error:

running 31 tests
test composer ... ok
test chubbyphp_framework ... ok
test doctrine_dbal ... ok
test api_platform ... ok

thread 'mezzio-framework' has overflowed its stack
fatal runtime error: stack overflow
error: test failed, to rerun pass `--test third_party_tests`

But, changing the stack size from:

.stack_size(16 * 1024 * 1024)

To:

.stack_size(24 * 1024 * 1024)

Fixes the issue.

Are you guys planning some changes on this part or want me to make a PR for it (I mean, to change the stack size)?

Hi everyone!

I'm excited to make my first contribution to the project. I recently started learning about parsers and interpreters, and I've been really interested in exploring the subject further. As a beginner in both the subject and Rust, working on this project is definitely a great way to learn.

I noticed that there were some areas of the codebase where the style and formatting could be improved, so I took the time to make some changes.

Are you guys open to accepting this kind of change?

Cheers!

first commit: add support for more type nodes, don't allow combining standalone types, proper union and intersection type representations, add proper support for try block caught types

second commit: add a small integration test runner, each test folder must contain code.php, if the code is will result in lexer errors, lexer-error.txt must be created next to code.php containing the formatted error, if the code is will result in parser errors, parser-error.txt must be created next to code.php containing the formatted error.

Readonly properties, which are required to have type declarations:

<?php

class C {
    readonly int $a;
    readonly protected string $b;
    readonly static stdClass $c;

    function __construct(readonly bool $d) {}
}

Readonly classes: (coming in PHP 8.2)

<?php

readonly class C {}

Fixes #26.

I've tried to do as straightforward of conversion as possible so this might be non-optimal in some aspects, but I figure we can do further refactoring later. All tests pass.

One thing that's worse now is test failures. The Debug impl for Vec<u8> prints things like [50, 51, 52] so it's not very human-readable.

We could look at pulling in a dependency like bstr, or otherwise adding a custom Debug impl. Let me know what you think and I can either add it onto this PR or do it as a separate one.

This PR:

• Let clap handle the command line arguments and options

I'm pretty new with rust, please let me know if I'm doing something wrong, I'm here to learn :)

• [x] class templates class Foo<T> {}
• [x] interface templates interface Foo<T> {}
• [x] method templates public function bar<T>(T $s): T {}
• [x] function templates function bar<T>(T $s): T {}
• [x] function call generics foo::<string>()
• [x] method call generics Foo::bar::<string>()
• [x] static method call generics $bar->foo::<string>()
• [x] instance class with generics new Foo::<string>()
• [x] generic types function foo(Bar<string> $s) {}
• [x] covariance templates +T
• [x] contravariance templates -T
• [x] template sub-type constraint T as Foo
• [x] template super-type constraint T super Foo
• [x] template equal-type constraint T = Foo

Something I've experimented with a little bit is a reflection API provided by the parser.

You provide it a generated AST (or multiple ASTs) and the struct is able to traverse the AST to search for function declarations, classes, etc.

You would then be able to call methods on the struct to find a particular class, function, get type information, params etc.

This would do a lot of heavy lifting for static analysis tools, dead code finders, interpreters etc.

This implements most of double-quoted string interpolation in the lexer. The parser hasn't been modified yet.

A couple things may need further tweaking:

The particular tokens that are produced in certain cases. The PHP interpreter has some very specialized tokens (T_STRING, T_NUM_STRING, T_STRING_VARNAME) for string interpolation that I'm going to see if we can do without, and instead push the logic into the parser. If I'm able to do that, it might be possible to get rid of one or two of the extra lexer states added in this PR. I also may end up finding out that there's a good reason for wanting to do this work in the lexer.

One other difference is that in the PHP interpreter, an interpolated string always begins and ends with the token ". In my implementation here, it always begins with a (possibly empty) TokenKind::StringPart, and always ends with TokenKind::DoubleQuote. Beginning with double quote means having to backtrack on what we've consumed, and beginning with a string part still seem unambiguous.

The number handling in var_offset is also not quite right. See the TODO. I'll revisit once we've implemented all integer literals.

Partially addresses #97.

Closes #71.

Introduces a new LexerState::Halted variant. This will tell the lexer to ensure that the token is followed by (); same as PHP itself.

It then just collects the rest of the tokens into a single InlineHtml token (probably could have done this differently but it works for now) and breaks out of the tokenization loop.

Could you please provide a short example for how to use traverser::Visitor? I'm not sure how to match for &mut dyn Node cases when traversing the AST

currently comments are skipped everywhere, however, we should instead:

1. collect comments on every state.next() call
2. retrieve comments and place them on the next statement.

An AST printer should be able to take an AST generated by the parser and reconstruct the original PHP source code.

This will be especially useful for some tools I have planned.