A procedural macro for defining nom combinators in simple DSL

Andy Lok

Last update: Dec 12, 2022

Related tags

Parsing nom-rule

Overview

nom-rule

A procedural macro for defining nom combinators in simple DSL. Requires nom v5.0+.

Dependencies

[dependencies]
nom = "7"
nom-rule = "0.2"

Syntax

The procedural macro rule! provided by this crate is designed for the ease of writing grammar spec as well as to improve maintainability, it follows these simple rules:

TOKEN: match the token by token kind. You should provide a parser to eat the next token if the token kind matched. it will get expanded into match_token(TOKEN).
";": match the token by token text. You should provide a parser to eat the next token if the token text matched. it will get expanded into match_text(";") in this example.
#fn_name: an external nom parser function. In the example above, ident is a predefined parser for identifiers.
a ~ b ~ c: a sequence of parsers to take one by one. It'll get expanded into nom::sequence::tuple.
(...)+: one or more repeated patterns. It'll get expanded into nom::multi::many1.
(...)*: zero or more repeated patterns. It'll get expanded into nom::multi::many0.
(...)?: Optional parser. It'll get expanded into nom::combinator::opt.
a | b | c: Choices between a, b, and c. It'll get expanded into nom::branch::alt.
&a: Positive predicate. It'll get expanded into nom::combinator::map(nom::combinator::peek(a), |_| ()). Note that it doesn't consume the input.
!a: Negative predicate. It'll get expanded into nom::combinator::not. Note that it doesn't consume the input.
... : "description": Context description for error reporting. It'll get expanded into nom::error::context.

Example

Define match_text parser and match_token parser for your custom token type. You can use nom::combinator::fail as match_token if your parser use &str or &[u8] as input because you won't match on token kinds.

#[derive(Clone, Debug, PartialEq)]
struct Token<'a> {
    kind: TokenKind,
    text: &'a str,
    span: Span,
}

#[derive(Clone, Copy, Debug, PartialEq)]
enum TokenKind {
    Whitespace,

    // Keywords
    CREATE,
    TABLE,

    // Symbols
    LParen,
    RParen,
    Semicolon,
    Comma,

    Ident,
}

fn match_text<'a, Error: ParseError<'a>>>(
    text: &'a str,
) -> impl FnMut(Input<'a>) -> IResult<'a>, &'a Token<'a>, Error> {
    move |i| satisfy(|token: &Token<'a>| token.text == text)(i)
}

fn match_token<'a, Error: ParseError<'a>>>(
    kind: TokenKind,
) -> impl FnMut(Input<'a>) -> IResult<'a>, &'a Token<'a>, Error> {
    move |i| satisfy(|token: &Token<'a>| token.kind == kind)(i)
}

Then give the two parser to nom_rule::rule! by wrapping it into a custom macro:

macro_rules! rule {
    ($($tt:tt)*) => { 
        nom_rule::rule!($crate::match_text, $crate::match_token, $($tt)*)
    }
}

To define a parser for the SQL of creating table:

let mut rule = rule!(
    CREATE ~ TABLE ~ #ident ~ "(" ~ (#ident ~ #ident ~ ","?)* ~ ")" ~ ";" : "CREATE TABLE statement"
);

It will get expanded into:

let mut rule = 
    nom::error::context(
        "CREATE TABLE statement",
        nom::sequence::tuple((
            (crate::match_token)(CREATE),
            (crate::match_token)(TABLE),
            ident,
            (crate::match_text)("("),
            nom::multi::many0(nom::sequence::tuple((
                ident,
                ident,
                nom::combinator::opt((crate::match_text)(",")),
            ))),
            (crate::match_text)(")"),
            (crate::match_text)(";"),
        ))
    );

Auto Sequence (nightly only)

nom-rule is able to automatically insert ~ in the rule when necessary so that you get the example above working the same as the following:

let mut rule = rule!(
    CREATE TABLE #ident "(" (#ident #ident ","?)* ")" ";" : "CREATE TABLE statement"
);

To enable this feature, you need to use a nightly channel rust complier, and add this to the Cargo.toml:

nom-rule = { version = "0.2", features = ["auto-sequence"] }

A procedural macro for configuring constant values across crates

toml-cfg Rough ideas: Crates can declare variables that can be overridden Anything const, e.g. usize, strings, etc. (Only) The "root crate" can overri

43 Dec 24, 2022

Procedural macro to derive Serde serializer-deserializer for Prost

prost-serde-derive prost-serde-derive is a procedural macro to generate Serde serializers and deserializers for Prost-generated structs. Rationale Cur

4 Dec 15, 2022

A procedural macro to generate a new function implementation for your struct.

Impl New 🦀 A procedural macro to generate a new function implementation for your struct. 🚀 Add to your project Add this to your Cargo.toml: [depende

4 Sep 8, 2023

A procedural macro that copy-pastes match arms for new type variant enums.

All the same! If you ever had code that looks like this: use std::io; use std::pin::Pin; use std::task::{Context, Poll}; use tokio::io::AsyncWrite; us

15 Feb 20, 2024

Parse command line arguments by defining a struct.

StructOpt Parse command line arguments by defining a struct. It combines clap with custom derive. Documentation Find it on Docs.rs. You can also check

2.6k Jan 5, 2023

This is an implementation defining standard for client-side-validation

Client-side-validation Foundation Libraries This is an implementation defining standard of client-side-validation representing a set of its Foundation

8 Dec 3, 2022

Attribute for defining `macro_rules!` macros with proper visibility and scoping

macro-vis This crate provides an attribute for defining macro_rules! macros that have proper visibility and scoping. The default scoping and publicity

2 Aug 29, 2022

A tool for defining and running multi-container Docker applications

Ikki Ikki is a tool for defining and running multi-container Docker applications. It is similar to Docker Compose but comes with some differences. Goa

39 Dec 21, 2022

AI-TOML Workflow Specification (aiTWS), a comprehensive and flexible specification for defining arbitrary Ai centric workflows.

AI-TOML Workflow Specification (aiTWS) The AI-TOML Workflow Specification (aiTWS) is a flexible and extensible specification for defining arbitrary wo

20 Apr 8, 2023

Comments

Complete example

Is there any way that you could put the example into a separate module? I can't get either the example in the readme or the tests module to compile.

error: proc macro panicked
  --> src/main.rs:71:9
   |
71 |           nom_rule::rule!($crate::match_text, $crate::match_token, $($tt)*)
   |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
76 |       let mut rule = rule!(
   |  ____________________-
77 | |         CREATE ~ TABLE ~ #ident ~ ^"(" ~ (#ident ~ #ident ~ ","?)* ~ ")" ~ ";" : "CREATE TABLE statement"
78 | |     );
   | |_____- in this macro invocation

   = help: message: called `Option::unwrap()` on a `None` value
   = note: this error originates in the macro `rule` (in Nightly builds, run with -Z macro-backtrace for more info)

Also is it possible to use rules inside of other rules?

opened by slyons 3