This PR introduces control statements for conditionals and iteration, and improves the handling of patterns with suffixes indicating list or optional outputs.

Can be reviewed commit by commit.

• [x] Fixes #76.
• ~~How should types other than String be handled?~~ See #86.
• ~~String::split_once() seems not to match when the rhs (after the delimiter) is empty. How should we handle empty globals, e.g. empty strings?~~ See #87.

I tried piecing together a python tsg file from https://docs.rs/tree-sitter-graph/0.5.1/tree_sitter_graph/reference/ but I get errors:

Caused by:
    0: Invalid query pattern: [0, "string", 0, #true, @id]
        ^
    1: Query error at 7:2. Invalid syntax:
       [0, "string", 0, #true, @id]```

Is there a full tsg file available for any languages to make it a bit easier to get started testing out functionality?

Adds support for if statements and a prefix operator ? that can be used to test for the presence of a given capture.

Thus, a stanza like

(module . (expression_statement)? @first) @root
{
  node @root.node
  if ?@first {
    attr (@root.node) first_expr = @first
  }
}

will set the first_expr attribute only in the case where the first statement in the input is in fact an expression.

In addition to this, this PR also adds functions for the usual boolean operators and, or, and not.

Can be reviewed commit-by-commit.

This is an alternative version of #1 that doesn't use the "tag name" mechanism to implicitly attach each graph node to exactly one syntax node. Instead you can just create graph nodes whenever you want, and use the existing "scoped variables" mechanism to attach them to syntax nodes whenever you want.

The .tsg below has two bugs:

1. Typo on line 3 (@ehre should be @here)
2. Use of @path.context on line 10 which is never defined on that syntax node

[
  (struct_item name: (_) @here)
  (enum_item name: (_) @ehre) ;; oops, typo
] {
  let @here.context = "%pattern"
}

(scoped_type_identifier) @path {
  node s
  attr (s) symbol = @path.context ;; oops, not defined
}

I ran it on this test case.

pub enum A {
    M(std::string::String),
}

Expected behavior: Either some error message for bug 1, or the usual message for bug 2 (Undefined scoped variable [syntax node scoped_identifier (2, 7)].context)

Actual behavior:

$ ./bin/tree-sitter-stack-graphs test --grammar . --tsg ./bug.tsg ./testcase.rs
Error: Error running test ./testcase.rs

Caused by:
    0: Executing attr ((load 1)) symbol = (scoped [syntax node scoped_type_identifier (2, 7)] 'context) at (10, 3)
    1: Expected a syntax node  got #null

This seems like a bug to me because I can't think of any legit way for #null to get into the computation at line 10.

To reproduce:

git clone [email protected]:jorendorff/tree-sitter-rust.git
cd tree-sitter-rust
git checkout jorendorff/internal-error
script/setup
./bin/tree-sitter-stack-graphs test --grammar . --tsg ./bug.tsg ./testcase.rs

• Improve display of parse errors
• Move parse error check to CLI
• Add flag to allow parse errors
• Skip tree traversal unless there are errors

Example verbose error:

Unexpected syntax on line 3 column 17:

| type U = { X: { f number } };
                  ^

Example compact error:

Unexpected syntax on line 3 column 17: f

Can be reviewed commit-by-commit

When I build the project,

$ cargo install --features cli tree-sitter-graph
error[E0658]: unions with non-`Copy` fields are unstable
   --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/smallvec-1.8.0/src/lib.rs:406:1
    |
406 | / union SmallVecData<A: Array> {
407 | |     inline: core::mem::ManuallyDrop<MaybeUninit<A>>,
408 | |     heap: (*mut A::Item, usize),
409 | | }
    | |_^
    |
    = note: see issue #55149 <https://github.com/rust-lang/rust/issues/55149> for more information

error: aborting due to previous error

How can i do for it?

Thanks!

This variant allows you to execute a TSG file into an existing Graph instance, which is useful if you need to create predefined nodes and/or edges before executing.

Include statement location in error messages to improve debugging.

Resulting error messages look as follows:

Error: Could not execute TSG file ../../../tsg-python/python.tsg

Caused by:
    0: Executing attr (@assign.node) value = @value.node at (541, 5)
    1: Undefined variable @value.node on node [syntax node attribute (7, 15)]

Error: Could not execute TSG file ../../../tsg-python/python.tsg

Caused by:
    0: Executing edge @parent.node -> @stmt.node at (178, 5)
    1: Undefined variable @stmt.node on node [syntax node function_definition (43, 0)]

Depends on

• [x] #10
• [x] #11

I've got an error in my rules indicating a duplicate variable @block.after_scope where @block is (of course) a syntax node. It would be very helpful to know what kind of syntax tree node it is, and where it occurs in the source, just like we show for undefined variables.

I think we're going to want to be able to run the union of multiple .tsg programs at a time, and to do that successfully and not have them clobber each other I think their variable and attr names will need to be namespaced.

I also think we're going to want to be able to explicitly use another namespace to e.g. extend one tsg program with another.

Like #85, but for attributes: using something like propagators, allow redundant attributes &c. if and only if their values are compatible (for most purposes we can think of this as "equal-valued").

I'd like to be able to provide some sort of schema for the kinds of nodes I'm constructing within a graph, and where they can be found in relation to the syntax tree.

(This would give us a nice way to describe stack graph or eval rules for example, in a way that's machine-checkable.)

Talked with @dcreager about this, and he suggested testing against such as a midway point to actually verifying.

In addition to verification/validation, this could also be used to give us constructors for subgraphs, which could be quite convenient when building stack graphs for example.

In the old DSL, scoped variables were available within the lexical scope of the node they were attached to—its entire subtree.

This is no longer the case, leading me to believe that the name is a misnomer, and that we should try to come up with something more indicative of what they mean.

As of #78, we can pass --global var=value arguments on the CLI to specify globals. However, the parser rejects var= as a value, which should correctly specify a value of the empty string. Either way, there is currently no way to supply an empty-valued string global.

#78 allows us to provide --global var=value arguments on the CLI. This is limited to passing strings, however. It would be nice to support other, non-string globals.

One possibility would be type-specific flags like --global-int. I'm not sure how we'd want to handle collections.