Lexer and parser collections.

MaxXing

Last update: Jan 27, 2023

Related tags

Overview

laps

Lexer and parser collections.

With laps, you can build parsers by just defining ASTs and deriving Parse trait for them.

Usage

Add laps to your project by running cargo add:

cargo add laps

Example

Implement a lexer for S-expression:

use laps::prelude::*;

#[token_kind]
enum TokenKind {
  /// Atom.
  Atom(String),
  /// Parentheses.
  Paren(char),
  /// End-of-file.
  Eof,
}

type Token = laps::token::Token<TokenKind>;

struct Lexer<T>(laps::reader::Reader<T>);

impl<T: std::io::Read> Tokenizer for Lexer<T> {
  type Token = Token;

  fn next_token(&mut self) -> laps::span::Result<Self::Token> {
    // skip spaces
    self.0.skip_until(|c| !c.is_whitespace())?;
    // check the current character
    Ok(match self.0.peek()? {
      // parentheses
      Some(c) if c == '(' || c == ')' => Token::new(c, self.0.next_span()?.clone()),
      // atom
      Some(_) => {
        let (atom, span) = self
          .0
          .collect_with_span_until(|c| c.is_whitespace() || c == '(' || c == ')')?;
        Token::new(atom, span)
      }
      // end-of-file
      None => Token::new(TokenKind::Eof, self.0.next_span()?.clone()),
    })
  }
}

And the parser and ASTs (or actually CSTs):

token_ast! {
  macro Token(mod = crate, Kind = TokenKind) {
    [atom] => (TokenKind::Atom(_), "atom"),
    [lpr] => (TokenKind::Paren('('), _),
    [rpr] => (TokenKind::Paren(')'), _),
    [eof] => (TokenKind::Eof, _),
  }
}

#[derive(Parse)]
#[token(Token)]
enum Statement {
  Elem(Elem),
  End(Token![eof]),
}

#[derive(Parse)]
#[token(Token)]
struct SExp(Token![lpr], Vec<Elem>, Token![rpr]);

#[derive(Parse)]
#[token(Token)]
enum Elem {
  Atom(Token![atom]),
  SExp(SExp),
}

The above implementation is very close in form to the corresponding EBNF representation of the S-expression:

Statement ::= Elem | EOF;
SExp      ::= "(" {Elem} ")";
Elem      ::= ATOM | SExp;

More Examples

See the examples directory, which contains the following examples:

sexp: a S-expression parser.
calc: a simple expression calculator.
json: a simple JSON parser.
clike: interpreter for a C-like programming language.

Changelog

See CHANGELOG.md.

License

You might also like...

A CSS parser, transformer, and minifier written in Rust.

@parcel/css A CSS parser, transformer, and minifier written in Rust. Features Extremely fast – Parsing and minifying large files is completed in milli

3.1k Jan 9, 2023

An IRC (RFC1459) parser and formatter, built in Rust.

ircparser An IRC (RFC1459) parser and formatter, built in Rust. ircparser should work on basically any Rust version, but the earliest version checked

2 Oct 18, 2022

A WIP svelte parser written in rust. Designed with error recovery and reporting in mind

Svelte(rs) A WIP parser for svelte files that is designed with error recovery and reporting in mind. This is mostly a toy project for now, with some v

3 Apr 19, 2023

Rust parser combinator framework

nom, eating data byte by byte nom is a parser combinators library written in Rust. Its goal is to provide tools to build safe parsers without compromi

7.6k Jan 7, 2023

url parameter parser for rest filter inquiry

inquerest Inquerest can parse complex url query into a SQL abstract syntax tree. Example this url: /person?age=lt.42&(student=eq.true|gender=eq.'M')&

25 Nov 2, 2020

Parsing Expression Grammar (PEG) parser generator for Rust

Parsing Expression Grammars in Rust Documentation | Release Notes rust-peg is a simple yet flexible parser generator that makes it easy to write robus

1.2k Dec 30, 2022

A fast monadic-style parser combinator designed to work on stable Rust.

Chomp Chomp is a fast monadic-style parser combinator library designed to work on stable Rust. It was written as the culmination of the experiments de

228 Oct 31, 2022

A parser combinator library for Rust

combine An implementation of parser combinators for Rust, inspired by the Haskell library Parsec. As in Parsec the parsers are LL(1) by default but th

1.1k Dec 28, 2022

LR(1) parser generator for Rust

LALRPOP LALRPOP is a Rust parser generator framework with usability as its primary goal. You should be able to write compact, DRY, readable grammars.

2.4k Jan 7, 2023

Releases(v0.0.2)

v0.0.2(Jan 13, 2023)
Added

derive syntax for macro token_ast.

More examples, including sexp, calc and json.

More documentation comments.

Module prelude for some common traits and macros.

token_kind now implements Clone, 1ryFrom<Kind> and 1ryFrom<&Kind> for token kinds.

token_ast now implements unwrap and unwrap_ref methods for token ASTs.

Changed

Updated version of some dependencies.

Feature no-logger to default feature logger.

License to either Apache 2.0 or MIT.

Fixed

Fault about byte buffer offset (byte_buf_offset) in Reader.

Fault about namespace of some Rust preludes in procedure macros.

Fault about error message in method TokenStream::expect.

Fault about string width calculation in Span's error logging related methods.

Source code(tar.gz)
Source code(zip)