A procedural macro for defining nom combinators in simple DSL

Related tags

Parsing nom-rule
Overview

nom-rule

Documentation Crates.io LICENSE

A procedural macro for defining nom combinators in simple DSL. Requires nom v5.0+.

Dependencies

[dependencies]
nom = "7"
nom-rule = "0.2"

Syntax

The procedural macro rule! provided by this crate is designed for the ease of writing grammar spec as well as to improve maintainability, it follows these simple rules:

  1. TOKEN: match the token by token kind. You should provide a parser to eat the next token if the token kind matched. it will get expanded into match_token(TOKEN).
  2. ";": match the token by token text. You should provide a parser to eat the next token if the token text matched. it will get expanded into match_text(";") in this example.
  3. #fn_name: an external nom parser function. In the example above, ident is a predefined parser for identifiers.
  4. a ~ b ~ c: a sequence of parsers to take one by one. It'll get expanded into nom::sequence::tuple.
  5. (...)+: one or more repeated patterns. It'll get expanded into nom::multi::many1.
  6. (...)*: zero or more repeated patterns. It'll get expanded into nom::multi::many0.
  7. (...)?: Optional parser. It'll get expanded into nom::combinator::opt.
  8. a | b | c: Choices between a, b, and c. It'll get expanded into nom::branch::alt.
  9. &a: Positive predicate. It'll get expanded into nom::combinator::map(nom::combinator::peek(a), |_| ()). Note that it doesn't consume the input.
  10. !a: Negative predicate. It'll get expanded into nom::combinator::not. Note that it doesn't consume the input.
  11. ... : "description": Context description for error reporting. It'll get expanded into nom::error::context.

Example

Define match_text parser and match_token parser for your custom token type. You can use nom::combinator::fail as match_token if your parser use &str or &[u8] as input because you won't match on token kinds.

#[derive(Clone, Debug, PartialEq)]
struct Token<'a> {
    kind: TokenKind,
    text: &'a str,
    span: Span,
}

#[derive(Clone, Copy, Debug, PartialEq)]
enum TokenKind {
    Whitespace,

    // Keywords
    CREATE,
    TABLE,

    // Symbols
    LParen,
    RParen,
    Semicolon,
    Comma,

    Ident,
}

fn match_text<'a, Error: ParseError<'a>>>(
    text: &'a str,
) -> impl FnMut(Input<'a>) -> IResult<'a>, &'a Token<'a>, Error> {
    move |i| satisfy(|token: &Token<'a>| token.text == text)(i)
}

fn match_token<'a, Error: ParseError<'a>>>(
    kind: TokenKind,
) -> impl FnMut(Input<'a>) -> IResult<'a>, &'a Token<'a>, Error> {
    move |i| satisfy(|token: &Token<'a>| token.kind == kind)(i)
}

Then give the two parser to nom_rule::rule! by wrapping it into a custom macro:

macro_rules! rule {
    ($($tt:tt)*) => { 
        nom_rule::rule!($crate::match_text, $crate::match_token, $($tt)*)
    }
}

To define a parser for the SQL of creating table:

let mut rule = rule!(
    CREATE ~ TABLE ~ #ident ~ "(" ~ (#ident ~ #ident ~ ","?)* ~ ")" ~ ";" : "CREATE TABLE statement"
);

It will get expanded into:

let mut rule = 
    nom::error::context(
        "CREATE TABLE statement",
        nom::sequence::tuple((
            (crate::match_token)(CREATE),
            (crate::match_token)(TABLE),
            ident,
            (crate::match_text)("("),
            nom::multi::many0(nom::sequence::tuple((
                ident,
                ident,
                nom::combinator::opt((crate::match_text)(",")),
            ))),
            (crate::match_text)(")"),
            (crate::match_text)(";"),
        ))
    );

Auto Sequence (nightly only)

nom-rule is able to automatically insert ~ in the rule when necessary so that you get the example above working the same as the following:

let mut rule = rule!(
    CREATE TABLE #ident "(" (#ident #ident ","?)* ")" ";" : "CREATE TABLE statement"
);

To enable this feature, you need to use a nightly channel rust complier, and add this to the Cargo.toml:

nom-rule = { version = "0.2", features = ["auto-sequence"] }
You might also like...
A procedural macro for configuring constant values across crates

toml-cfg Rough ideas: Crates can declare variables that can be overridden Anything const, e.g. usize, strings, etc. (Only) The "root crate" can overri

Procedural macro to derive Serde serializer-deserializer for Prost

prost-serde-derive prost-serde-derive is a procedural macro to generate Serde serializers and deserializers for Prost-generated structs. Rationale Cur

A procedural macro to generate a new function implementation for your struct.

Impl New 🦀 A procedural macro to generate a new function implementation for your struct. 🚀 Add to your project Add this to your Cargo.toml: [depende

A procedural macro that copy-pastes match arms for new type variant enums.

All the same! If you ever had code that looks like this: use std::io; use std::pin::Pin; use std::task::{Context, Poll}; use tokio::io::AsyncWrite; us

Parse command line arguments by defining a struct.

StructOpt Parse command line arguments by defining a struct. It combines clap with custom derive. Documentation Find it on Docs.rs. You can also check

This is an implementation defining standard for client-side-validation

Client-side-validation Foundation Libraries This is an implementation defining standard of client-side-validation representing a set of its Foundation

Attribute for defining `macro_rules!` macros with proper visibility and scoping

macro-vis This crate provides an attribute for defining macro_rules! macros that have proper visibility and scoping. The default scoping and publicity

A tool for defining and running multi-container Docker applications
A tool for defining and running multi-container Docker applications

Ikki Ikki is a tool for defining and running multi-container Docker applications. It is similar to Docker Compose but comes with some differences. Goa

AI-TOML Workflow Specification (aiTWS), a comprehensive and flexible specification for defining arbitrary Ai centric workflows.

AI-TOML Workflow Specification (aiTWS) The AI-TOML Workflow Specification (aiTWS) is a flexible and extensible specification for defining arbitrary wo

Simple rust asset handling derive macro for enums, and a proc-macro learning resource!

asset-derive Summary • Todos • Docs Summary Simple Rust asset loading derive macro for Enums, and a resource for learning proc-macros! Please feel fre

A Rust crate for LL(k) parser combinators.

oni-comb-rs (鬼昆布,おにこんぶ) A Rust crate for LL(k) parser combinators. Main project oni-comb-parser-rs Sub projects The following is projects implemented

Composable n-gram combinators that are ergonomic and bare-metal fast
Composable n-gram combinators that are ergonomic and bare-metal fast

CREATURE FEATUR(ization) A crate for polymorphic ML & NLP featurization that leverages zero-cost abstraction. It provides composable n-gram combinator

PEG parser combinators using operator overloading without macros.

pom PEG parser combinators created using operator overloading without macros. Document Tutorial API Reference Learning Parser Combinators With Rust -

ShakeFlow: Functional Hardware Description with Latency-Insensitive Interface Combinators

ShakeFlow: Functional Hardware Description with Latency-Insensitive Interface Combinators This repository contains the artifact for the following pape

Crate extending futures stream combinators, that is adding precise rate limiter
Crate extending futures stream combinators, that is adding precise rate limiter

stream-rate-limiter Stream combinator .rate_limiter(opt: RateLimitOptions) Provides way to limit stream element rate with constant intervals. It adds

Djotters is a Djot parser and translater written via parser combinators, in rust.

🎭 Djotters Turning your Djot (markdown) into lovely HTML! Djotters is here to let you draft up a document and watch it render in real time. If you wa

Rust experiments involving Haskell-esque do notation, state, failure and Nom parsers!

Introduction As a long time Haskell developer recently delving into Rust, something I've really missed is monads and do notation. One thing monadic do

a nom parser combinator that matches a psql statement.

psql_splitter a nom parser combinator that matches a psql statement. Postgres has a dialect of SQL that I'm going to call pgsql. Postgres also has a c

Tutorial for parsing with nom 5.

Nom Tutorial Nom is a wonderful parser combinators library written in Rust. It can handle binary and text files. Consider it where you would otherwise

Comments
  • Complete example

    Complete example

    Is there any way that you could put the example into a separate module? I can't get either the example in the readme or the tests module to compile.

    error: proc macro panicked
      --> src/main.rs:71:9
       |
    71 |           nom_rule::rule!($crate::match_text, $crate::match_token, $($tt)*)
       |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...
    76 |       let mut rule = rule!(
       |  ____________________-
    77 | |         CREATE ~ TABLE ~ #ident ~ ^"(" ~ (#ident ~ #ident ~ ","?)* ~ ")" ~ ";" : "CREATE TABLE statement"
    78 | |     );
       | |_____- in this macro invocation
    
       = help: message: called `Option::unwrap()` on a `None` value
       = note: this error originates in the macro `rule` (in Nightly builds, run with -Z macro-backtrace for more info)
    
    

    Also is it possible to use rules inside of other rules?

    opened by slyons 3
Owner
Andy Lok
Andy Lok
A Rust crate for LL(k) parser combinators.

oni-comb-rs (鬼昆布,おにこんぶ) A Rust crate for LL(k) parser combinators. Main project oni-comb-parser-rs Sub projects The following is projects implemented

Junichi Kato 24 Nov 3, 2022
PEG parser combinators using operator overloading without macros.

pom PEG parser combinators created using operator overloading without macros. Document Tutorial API Reference Learning Parser Combinators With Rust -

Junfeng Liu 412 Dec 31, 2022
A repo which serves as a learning ground for the Nom crate.

Fun with Nom This repo is created as a learning resource for the Nom crate for Rust. Nom is a parser combinators library. It can be used to parse just

Jeff Mitchell 3 Feb 18, 2024
An LR parser generator, implemented as a proc macro

parsegen parsegen is an LR parser generator, similar to happy, ocamlyacc, and lalrpop. It currently generates canonical LR(1) parsers, but LALR(1) and

Ömer Sinan Ağacan 5 Feb 28, 2022
LR(1) grammar parser of simple expression

LR(1)语法分析程序 实验内容 编写LR(1)语法分析程序,实现对算术表达式的语法分析。要求所分析算数表达式由如下的文法产生: E -> E+T | E-T | T T -> T*F | T/F | F F -> (E) | num 程序设计与实现 使用方式:运行.\lr1-parser.exe

Gao Keyong 1 Nov 24, 2021
Pure, simple and elegant HTML parser and editor.

HTML Editor Pure, simple and elegant HTML parser and editor. Examples Parse HTML segment/document let document = parse("<!doctype html><html><head></h

Lomirus 16 Nov 8, 2022
unFlow is a Design as Code implementation, a DSL for UX & backend modeling. DSL to Sketch file, Sketch to DSL, DSL to code.

unflow 是一个低代码、无代码设计语言。unFlow is a Design as Code implementation, a DSL for UX & backend modeling. DSL to Sketch file, Sketch to DSL, DSL to code.

Inherd OS Team (硬核开源小组) 70 Nov 27, 2022
Easy c̵̰͠r̵̛̠ö̴̪s̶̩̒s̵̭̀-t̶̲͝h̶̯̚r̵̺͐e̷̖̽ḁ̴̍d̶̖̔ ȓ̵͙ė̶͎ḟ̴͙e̸̖͛r̶̖͗ë̶̱́ṉ̵̒ĉ̷̥e̷͚̍ s̷̹͌h̷̲̉a̵̭͋r̷̫̊ḭ̵̊n̷̬͂g̵̦̃ f̶̻̊ơ̵̜ṟ̸̈́ R̵̞̋ù̵̺s̷̖̅ţ̸͗!̸̼͋

Rust S̵̓i̸̓n̵̉ I̴n̴f̶e̸r̵n̷a̴l mutability! Howdy, friendly Rust developer! Ever had a value get m̵̯̅ð̶͊v̴̮̾ê̴̼͘d away right under your nose just when

null 294 Dec 23, 2022
Example of using nom parsers from a proc macro

Example of using nom parsers from a proc macro This project is organised as 3 crates: nom_macro is the main project, exposing the proc macro and the g

Geoffroy Couprie 1 Mar 20, 2022
Obake is a procedural macro for declaring and maintaining versioned data-structures.

Obake is a procedural macro for declaring and maintaining versioned data-structures. The name 'obake' is taken from the Japanese 'お化け (おばけ)', a class of supernatural beings in Japanese folklore that shapeshift.

Nathan Corbyn 174 Dec 26, 2022