PartiQL libraries and tools in Rust.

Related tags

rust query sql
Overview

PartiQL Rust

Crate Docs License CI Build codecov

This is a collection of crates to provide Rust support for the PartiQL query language.

The crates in this repository are considered experimental, under active/early development, and APIs are subject to change.

This project uses workspaces to manage the crates in this repository. The partiql crate is intended to be the crate that exports all the relevant partiql-* sub-crate functionality. It is factored in this way to make applications needing only some sub-component of the PartiQL implementation possible (e.g. an application that only requires the PartiQL parser can depend on partiql-parser directly).

Due to the lack of namespacing in crates.io, we have published 0.0.0 versions for the sub-crates we know we will need. A crate with a version 0.1.0 or higher, should have real, albeit potentially very experimental and/or early implementations.

Security

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.

Issues
  • Add COMMENT to Scanner grammar rule

    Add COMMENT to Scanner grammar rule

    Issue: #17

    Description of changes: I added COMMENT rule to the pest grammar. I was hesitant to edit the Rule::Query as well. Should I?

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

    opened by tbmreza 5
  • Adds skeleton for the initial crates for PartiQL.

    Adds skeleton for the initial crates for PartiQL.

    Also adds ignore file.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by almann 2
  • Adds Pest Grammar to Ion Conversion

    Adds Pest Grammar to Ion Conversion

    The purpose of this is the beginning of a tool to generate Ion ASTs for Pest grammars. This leverages the Pest AST in the pest_meta crate to construct a reasonable Ion representation. The idea is that this could be bridged to a CLI and then piped into a script to generate things like TextMate grammars and the LaTeX fragments for our specification.

    Adds the pestion crate to provide a simple trait PestToElement and implementations over the Pest AST and &str syntax definitions to create Ion Element serialized forms.

    Resolves #35.

    Here is a rendering of the doc comment example:

    image

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

    opened by almann 2
  • Adds the scanner rules for punctuation

    Adds the scanner rules for punctuation

    Adds the Pest grammar rules for these tokens and particularly adds the lookahead assertions for the various edge cases around the punctuation tokens that interact with each other and comments.

    Adds special Content enum variants for ./*/? and a basic variant for Operator and Delimiter.

    This should the final pre-requisite for all of the terminal parse rules in the PEG and allow us to start adding the parser rules for expressions.

    An explicit TODO is around modeling the various operators as their own enum or enum variants, right now they are returned as normalized string content.

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

    opened by almann 2
  • First issue

    First issue

    Issue #, if available: #23

    Description of changes: Adds a unit variant IllegalState to ParserError enum. Users of the enum would call illegal_state() convenience function.

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

    opened by tbmreza 2
  • Initial API for `Scanner`.

    Initial API for `Scanner`.

    Provides an interface for lexing PartiQL. Traditionally we would factor this as the low-level interface for the parser, but with the PEG based implementation, we just surface the relevant parts of the parser as a rule to parse with.

    This commit helps explore how to effectively integrate with the somewhat low-level APIs for parsing that Pest provides.

    • Adds scanner module.
    • Adds Scanner trait to the prelude.
    • Makes PartiQLParser public to crate.
    • Refactors LineAndColumn as a tuple for Position::At.
      • Adds this to the prelude.
    • Changes entry point for PEG to Query and added Scanner as the entry point rules for implementing the Scanner API.
    • Adds PairsExt/PairExt trait/impl to add utility methods for working with Pest Pairs/Pair.
    • Adds LineAndColumn::position_from and cleans up some doc/doc tests.

    Resolves #13.

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

    opened by almann 2
  • Adds support for string literals and identifiers in `Scanner`.

    Adds support for string literals and identifiers in `Scanner`.

    Refactors the PEG to make each keyword and explicit rule that has the form KeywordName_. This allows us to easily refer to them in rules without having to have magic strings. The rationale for the _ suffix is to make sure it doesn't collide with the other grammar rules and not have a wordy suffix like Keyword on every rule.

    Also:

    • Adds case folding to keywords.
    • Makes error cases in the scanner/recognizer tests less brittle.
    • Adds Token to the Query rule and updates tests for recognize_partiql.

    Depends on #8 and #12--as such is targeting a staging branch but I did not want to prevent review.

    Rendered update of the doc test that exercises some of this:

    image

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

    opened by almann 2
  • Initial API for `Scanner`.

    Initial API for `Scanner`.

    Provides an interface for lexing PartiQL. Traditionally we would factor this as the low-level interface for the parser, but with the PEG based implementation, we just surface the relevant parts of the parser as a rule to parse with.

    This commit helps explore how to effectively integrate with the somewhat low-level APIs for parsing that Pest provides.

    Notes:

    • Adds scanner module.
    • Adds Scanner trait to the prelude.
    • Makes PartiQLParser public to crate.
    • Refactors LineAndColumn as a tuple for Position::At.
      • Adds this to the prelude.
    • Changes entry point for PEG to Query and added Scanner as the entry point rules for implementing the Scanner API.
    • Adds PairsExt/PairExt trait/impl to add utility methods for working with Pest Pairs/Pair.
    • Adds LineAndColumn::move_relative and cleans up some doc/doc tests.

    Pest links for reference:

    • https://pest.rs/book/parser_api.html - API description
    • https://docs.rs/pest/2.1.3/pest/index.html - API Reference

    This PR is dependent on #8 and as such is targeting a staging branch but I did not want to prevent review.

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

    opened by almann 2
  • Adds Github Action Workflows for CI/Coverage.

    Adds Github Action Workflows for CI/Coverage.

    Also adds pull request template.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by almann 1
  • Adds `pest2ion` CLI test for PartiQL Grammar

    Adds `pest2ion` CLI test for PartiQL Grammar

    Adds a TestCase struct to encapsulate the test case parameters for testing the CLI. Makes a simple conversion from &'static str pair which is for when you have simple in/out type of tests. Adds function to generate a test case out of the PartiQL grammar file which doesn't really validate anything about the grammar serialization other than we get something the library would produce.

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

    opened by almann 1
  • `pest2ion` Tool Cannot Generate Pretty/Compact Text for PartiQL Grammar

    `pest2ion` Tool Cannot Generate Pretty/Compact Text for PartiQL Grammar

    This is probably an upstream bug with ion-rs/ion-c-sys, but when serializing the PartiQL grammar, we get an unexpected EOF:

    Binary:

    ❯ cargo run --package pest-ion --bin pest2ion -- partiql-parser/src/peg/partiql.pest -b | wc -c
        Finished dev [unoptimized + debuginfo] target(s) in 0.02s
         Running `target/debug/pest2ion partiql-parser/src/peg/partiql.pest -b`
    14897
    

    Text:

    ❯ RUST_BACKTRACE=1 cargo run --package pest-ion --bin pest2ion -- partiql-parser/src/peg/partiql.pest -t | wc -c
        Finished dev [unoptimized + debuginfo] target(s) in 0.02s
         Running `target/debug/pest2ion partiql-parser/src/peg/partiql.pest -t`
    thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: IonCError { code: 20, message: "IERR_UNEXPECTED_EOF", additional: "iERR Result" }', /home/ANT.AMAZON.COM/almann/.cargo/registry/src/github.com-1ecc6299db9ec823/ion-c-sys-0.4.10/src/writer.rs:612:50
    stack backtrace:
       0: rust_begin_unwind
                 at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:493:5
       1: core::panicking::panic_fmt
                 at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/panicking.rs:92:14
       2: core::option::expect_none_failed
                 at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/option.rs:1329:5
       3: core::result::Result<T,E>::unwrap
                 at /home/ANT.AMAZON.COM/almann/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/result.rs:1037:23
       4: <ion_c_sys::writer::IonCWriterHandle as core::ops::drop::Drop>::drop
                 at /home/ANT.AMAZON.COM/almann/.cargo/registry/src/github.com-1ecc6299db9ec823/ion-c-sys-0.4.10/src/writer.rs:612:13
       5: core::ptr::drop_in_place<ion_c_sys::writer::IonCWriterHandle>
                 at /home/ANT.AMAZON.COM/almann/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:187:1
       6: core::ptr::drop_in_place<ion_rs::value::writer::IonCSliceElementWriter>
                 at /home/ANT.AMAZON.COM/almann/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:187:1
       7: pest2ion::main
                 at ./pest-ion/src/bin/pest2ion/main.rs:82:1
       8: core::ops::function::FnOnce::call_once
                 at /home/ANT.AMAZON.COM/almann/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    0
    

    The problem seems to be around the cleanup of the writer handle (the panic is happening during the cleanup of the Ion C writer.

    bug 
    opened by almann 1
  • Promote Delimiters/Operators in `scanner::Content` to Enum/Enum Variants

    Promote Delimiters/Operators in `scanner::Content` to Enum/Enum Variants

    #30 adds scanner tokens for various punctuation. We have first-class variants for Dot/Star/Parameter, but Delimiter and Operator have the string of the token as their content. It probably makes sense to lift this into either their own nested enums, or their own top-level variants.

    enhancement 
    opened by almann 0
  • Leverage `Cow<'val, str>` in `scanner::Content`

    Leverage `Cow<'val, str>` in `scanner::Content`

    In #15, we added some new tokens that manipulate the text to be put into the content. We should leverage the borrowed form of the Cow when possible. There are at least three places where we should return the borrowed variant when there is no replacement:

    https://github.com/partiql/partiql-lang-rust/blob/d309746f642c8850cdc412daed5a44b845bf79e9/partiql-parser/src/scanner.rs#L151-L155

    https://github.com/partiql/partiql-lang-rust/blob/d309746f642c8850cdc412daed5a44b845bf79e9/partiql-parser/src/scanner.rs#L157-L163

    https://github.com/partiql/partiql-lang-rust/blob/d309746f642c8850cdc412daed5a44b845bf79e9/partiql-parser/src/scanner.rs#L189

    enhancement good first issue 
    opened by almann 1
  • Refactor `ParserError` to be a `struct`.

    Refactor `ParserError` to be a `struct`.

    Currently our ParserError is an enum with a single SyntaxError variant:

    https://github.com/partiql/partiql-lang-rust/blob/4aa104c65ee7b69eca967c10e823016e00c91493/partiql-parser/src/result.rs#L130-L136

    More likely than not, we all ParserErrors will have a message with optional position information, we should refactor the variant into some kind of ParserErrorType or something that could be as simple as just the tag (e.g. SyntaxError), but could also have error specific data.

    See also #16, no matter what we do here, we'll like a utility method to do non-message equality.

    enhancement good first issue 
    opened by almann 0
  • Extract `SyntaxError` position comparison as a utility.

    Extract `SyntaxError` position comparison as a utility.

    We have code that wants to assert that a some error happened at some position, we should write a function to do this versus forcing the pattern matching boilerplate at the call site.


    Similar to the other code--this probably suggests we should extract this out now that I think about it.

    Originally posted by @almann in https://github.com/partiql/partiql-lang-rust/pull/15#discussion_r630733972

    enhancement good first issue 
    opened by almann 1
  • Consider adding a `Span` API

    Consider adding a `Span` API

    @dlurton points out that we probably want to consider encapsulating location information into a Span API.

    I think this is a reasonable idea and is conceptually similar to the underlying Pest's [Span][1].

    Besides the start/end for line/column, I think adding the byte offset (UTF-8 code unit) information as well would be good here.

    [1]: https://docs.rs/pest/2.1.3/pest/struct.Span.html

    Should these be in a struct by themselves?

    struct Span {
        /// Start location for this token.
        start_location: LineAndColumn,
        /// End location for this token.
        end_location: LineAndColumn
    }
    

    ??

    Originally posted by @dlurton in https://github.com/partiql/partiql-lang-rust/pull/12#discussion_r629547393

    enhancement good first issue 
    opened by almann 3
  • Extract/Inject Useful Constants to/from PEG

    Extract/Inject Useful Constants to/from PEG

    Having a declarative grammar is very desirable, but we should also be able to either extract the constants into the library or use code generation to stitch together the Pest PEG definition from code in the library. Either way is fine, so long as there exists a declarative grammar that is pure Pest (or whatever we end up with) and the various bits of the grammar that we want constants for in the library.

    This can be useful for helping support things like code completion and syntax highlighting for downstream users.

    enhancement 
    opened by almann 0
This is an implementation defining standard for client-side-validation

Client-side-validation Foundation Libraries This is an implementation defining standard of client-side-validation representing a set of its Foundation

LNP/BP Association 3 May 14, 2021
Czkawka is a simple, fast and easy to use app to remove unnecessary files from your computer.

Multi functional app to find duplicates, empty folders, similar images etc.

Rafał Mikrut 4.2k Jun 13, 2021
Migrate C code to Rust

C2Rust helps you migrate C99-compliant code to Rust. The translator (or transpiler) produces unsafe Rust code that closely mirrors the input C code. T

Immunant 1.9k Jun 13, 2021
C to Rust translator

Corrode: Automatic semantics-preserving translation from C to Rust This program reads a C source file and prints an equivalent module in Rust syntax.

Jamey Sharp 2.1k Jun 11, 2021
Utilities and tools based around Amazon S3 to provide convenience APIs in a CLI

s3-utils Utilities and tools based around Amazon S3 to provide convenience APIs in a CLI. This tool contains a small set of command line utilities for

Isaac Whitfield 33 May 18, 2021
Modeling is a tools to analysis different languages by Ctags

Modeling Modeling is a tools to analysis different languages by Ctags process: generate to opt call ctags with opt analysis ctags logs output resulse

Inherd OS Team (硬核开源小组) 4 Jun 10, 2021
Simple ray tracer written in Rust

Simple ray tracer written in Rust from scratch I've just finished my first semester at the Faculty of Applied Mathematics and Computer Science at the

Vladislav 190 Jun 12, 2021
A library to compile USDT probes into a Rust library

sonde sonde is a library to compile USDT probes into a Rust library, and to generate a friendly Rust idiomatic API around it. Userland Statically Defi

Ivan Enderlin 25 Mar 28, 2021
Language Integrated Query in Rust.

Linq in Rust Language Integrated Query in Rust (created by declarative macros). Inspired by LINQ in .NET. What's LINQ This project is under developmen

StardustDL 56 Jun 9, 2021
Rust experiments involving Haskell-esque do notation, state, failure and Nom parsers!

Introduction As a long time Haskell developer recently delving into Rust, something I've really missed is monads and do notation. One thing monadic do

Kerfuffle 21 May 23, 2021
Cross-platform Window library in Rust for Tauri. [WIP]

Cross-platform application window creation library in Rust that supports all major platforms like Windows, macOS, Linux, iOS and Android. Built for you, maintained for Tauri.

Tauri 35 Jun 10, 2021
A lean, minimal, and stable set of types for color interoperation between crates in Rust.

This library provides a lean, minimal, and stable set of types for color interoperation between crates in Rust. Its goal is to serve the same function that mint provides for (linear algebra) math types.

Gray Olson 10 May 31, 2021
Simple color picker that lets the user create harmonic palettes with ease.

epick Simple color picker that lets the user create harmonic palettes with ease. Get it You can checkout the web demo over here or get a native binary

Wojciech Kępka 11 Jun 2, 2021
A tiling window manager for Windows 10 based on binary space partitioning

yatta BSP Tiling Window Manager for Windows 10 Getting Started This project is still heavily under development and there are no prebuilt binaries avai

Jade 125 Jun 14, 2021