This PR does two things:

1. support INTERVAL && TIMESTAMPTZ type in postgres-extended
2. refactor run() in postgres-extended: use a macro to generate process

Support extended query mode. It's compatible with past versions, extended query mode only takes effect when using the flag "--extend"

for now, this pr has the following limit:

• we only support most common type in binary format (but not all the type.
• rust-postgres only support one-dimensional array type.

Hence we can run all original e2e test now.

Signed-off-by: Runji Wang [email protected]

If the engine returns a value with leading spaces, it will affect the result of rowsort, but won't be recognized in error message since we trim the string before reporting error but after the rowsort. This PR trims the string before rowsort to avoid this problem. Despite this, engines should not return a value with leading or tailing space.

In distributed environment, the order of query result set is uncertain, which is also not required in SQL without "ORDER BY".

To simplify the test script, we can support syntax like SET DEFAULT [GLOBAL] $SortMode to set the default sort mode required by query statement in the current file or all files, instead of specify sort mode after each. query statement.

statement error string::interval: context-dependent operators are not allowed in computed column\nHINT: STRING to INTERVAL casts depend on session IntervalStyle; use parse_interval\(string\) instead
CREATE TABLE invalid_table (
  invalid_col interval AS ('1 hour'::string::interval) STORED
)

Statement error is also a part of the testing. In some circumstances, we may expect the system to produce errors as expected.

In some cases, we may want to test whether concurrent sessions from different stateless database frontends are working correctly. This PR allows users to specify multiple (host, port)s and make sqllogictest choose one randomly per session.

Related: https://github.com/singularity-data/risingwave/issues/4720

It's becoming more and more complex, so it would be good to have a separate crate.

cc @xxchan do you want to work on this? If so feel free to assign yourself. Otherwise I'll do the refactor maybe this month.

Sometimes, the test files are maintained by different people and they may use the same name for a table or view. This is reasonable as we may view these test files as independent units.

When some test case in one test file failed, sqllogictest proceeds to execute another test file. Without dropping all of the tables and views, it can cause table exists failure in the coming tests.

Therefore, this issue proposes that we may have a finally statement like mechanism to clean up things. Statement in the finally code block would be always executed at last.

Is this mechanism worth implementing? Not necessarily design and implement it in this way, though.

~Draft as it builds on https://github.com/risinglightdb/sqllogictest-rs/pull/129~

Rationale

I want to be able to update Records given the output of a particular run (see https://github.com/apache/arrow-datafusion/issues/4570)

Changes

1. Add update_record_with_output function
2. Tests

I think this is my last planned PR to sqllogictest for a bit (I want to go try and integrated this into DataFusion upstream first)

First of all, thank you again for this library -- it is really awesome.

Rationale

I am trying to implement "test script completion mode" in DataFusion. In order to do so, I would like to be able to update parsed Records with new output and then write them back to a file. Ideally the only changes in the file would be the new results and the original file will have all the comments, whitespace, etc.

DataFusion ticket: https://github.com/apache/arrow-datafusion/issues/4570, https://github.com/apache/arrow-datafusion/issues/4570#issuecomment-1345252492

Changes

1. Add impl Display for Record (refactor unparse)
2. Add tests for roundtrip parsing / unparsing
3. Add Record::Whitespace so the whitespace in the original files can be reconstructed (found during testing)

Questions

1. I had a question about whitespace after Record::Statement and RecordQuery which I will note inline
2. If you accept this change, I would be happy to add more round trip parsing tests for the other example files

This PR allows users to add hook functions after each query or statement completes. For example, to test fault recovery, I'd like to kill nodes and wait for a while between queries. In this case I can register a custom function using these API.

As a testing framework, we need to ensure our correctness. Currently we can be relatively confident whether previously passed tests will fail after upgrading sqllogictest by testing agains real usage in e.g., risingwave. But it would definitely be better to add a test suite in our repo. https://github.com/risinglightdb/sqllogictest-rs/pull/123

A more dangerous problem which is also harder to detect is that we won't pass previously failed tests. Imagine that after some changes, we allow all test cases to pass. 😇 We need to test errors.

Here's an example of error that won't be run:

https://github.com/risinglightdb/sqllogictest-rs/blob/d753e4c77fbf00cdaab69477df2e9c3dba3f5fcc/examples/basic/basic.slt#L32-L34

• [ ] test failure https://github.com/risinglightdb/sqllogictest-rs/issues/122
• fix https://github.com/risinglightdb/sqllogictest-rs/issues/108
• [ ] files too large. use git submodule or sth? 🤪 Currently only added some files

It's used in sqlite's test suite e.g.,

https://github.com/risingwavelabs/sqlite-sqllogictest/blob/aa3e4345dff63b65ef0d0969872ffa2b4a685225/test/evidence/in1.test#L39-L43

Because users generally will return Result in tests, and there's no change to call display(true) to get colorful error message. If it's printed, it will be duplicate with the Err.

Maybe we can refer to https://docs.rs/pretty_assertions/latest/pretty_assertions/ about how can we opt-in colorful message in library usage.

so that it can be used in a pipeline's fasion, or multiple globs (expanded by shell).

Currently it accepts a single glob string argument, so it also need to be quoted.

e.g.,

❯ sqllogictest ./**/*.slt  
error: Found argument './examples/condition/condition.slt' which wasn't expected, or isn't valid in this context