Parsing and inspecting Rust literals (particularly useful for proc macros)

Overview

litrs: parsing and inspecting Rust literals

CI status of master Crates.io Version docs.rs

litrs offers functionality to parse Rust literals, i.e. tokens in the Rust programming language that represent fixed values. This is particularly useful for proc macros, but can also be used outside of a proc-macro context.

Why this library? Unfortunately, the proc_macro API shipped with the compiler offers no easy way to inspect literals. There are mainly two libraries for this purpose: syn and literalext. The latter is deprecated. And syn is oftentimes overkill for the task at hand, especially when developing function like proc-macros (e.g. foo!(..)). This crate is a lightweight alternative. Also, when it comes to literals, litrs offers a bit more flexibility and a few more features compared to syn.

While this library is fairly young, it is extensively tested and I think the number of parsing bugs should already be very low. I'm interested in community feedback! If you consider using this, please speak your mind in this issue.

Example

In proc macro

use std::convert::TryFrom;
use proc_macro::TokenStream;
use litrs::Literal;

#[proc_macro]
pub fn foo(input: TokenStream) -> TokenStream {
    // Please do proper error handling in your real code!
    let first_token = input.into_iter().next().expect("no input");

    // `try_from` will return an error if the token is not a literal.
    match Literal::try_from(first_token) {
        // Convenient methods to produce decent errors via `compile_error!`.
        Err(e) => return e.to_compile_error(),

        // You can now inspect your literal!
        Ok(Literal::Integer(i)) => {
            println!("Got an integer specified in base {:?}", i.base());

            let value = i.value::<u64>().expect("integer literal too large");
            println!("Is your integer even? {}", value % 2 == 0);
        }
        Ok(other) => {
            println!("Got a non-integer literal");
        }
    }

    TokenStream::new() // dummy output
}

If you are expecting a specific kind of literal, you can also use this, which will return an error if the token is not a float literal.

FloatLit::try_from(first_token)

Parsing from a &str

Outside of a proc macro context you might want to parse a string directly.

use litrs::{FloatLit, Literal};

let lit = Literal::parse("'🦀'").expect("failed to parse literal");
let float_lit = FloatLit::parse("2.7e3").expect("failed to parse as float literal");

See the documentation or the examples/ directory for more examples and information.



License

Licensed under either of Apache License, Version 2.0 or MIT license at your option. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Comments
  • Conversion to proc_macro2::Literal

    Conversion to proc_macro2::Literal

    I've started writing a conversion from (litrs::Literal, Span) to proc_macro2::Literal, and was wondering if there was a trick for doing it (given that Literal only has a ton of constructors for each sub-kind of numeric literal). If not, I was wondering if you'd take the conversion as a PR (behind a Cargo feature, of course).

    opened by wycats 6
  • Example for proc macro

    Example for proc macro

    Your readme states:

    This is particularly useful for proc macros

    but yet there's no example for a proc macro. I can understand that it would be overkill for the readme, but consider creating an examples directory and put it there for a full-fledged example.

    opened by hellow554 2
  • doc: fix link to reference and clarify bool literals

    doc: fix link to reference and clarify bool literals

    This fixes #6 by providing good links to the Rust reference and adds the remark that in the Rust syntax false and true are considered keywords and not actual literals.

    opened by FedericoStra 1
  • Pull out majority of parsing code into non-generic functions

    Pull out majority of parsing code into non-generic functions

    Most parsing code only operates on &str and does not need the generic type B: Buffer. So for compile times, it would be preferable to pull those parts into an internal non-generic function.

    opened by LukasKalbertodt 1
  • Broken external link in documentation

    Broken external link in documentation

    Hello, magnificent crate! :+1:

    I just wanted to point out that the link in the documentation of BoolLit is broken. The target https://doc.rust-lang.org/reference/tokens.html#boolean-literals does not exist anymore in the reference and should probably be replaced by https://doc.rust-lang.org/reference/keywords.html#strict-keywords and/or https://doc.rust-lang.org/reference/expressions/literal-expr.html#boolean-literal-expressions.

    opened by FedericoStra 0
  • Support non-number suffixes

    Support non-number suffixes

    I randomly stumbled upon this section of the reference talking about how all literal tokens can have suffixes, not only number literals. I wasn't aware of that at all. Rust errors if such a token ever ends up in actual Rust code, but they are valid as input to proc macros.

    This library should support those.

    opened by LukasKalbertodt 2
  • Community Feedback

    Community Feedback

    Do you have any opinions, complaints, suggestions, ideas, ... about litrs? Let me know! This issue is less formal and more relaxed than "normal issues", so feel free to just dump your thoughts here.

    opened by LukasKalbertodt 7
Releases(v0.3.0)
Owner
Lukas Kalbertodt
I like teaching and coding.
Lukas Kalbertodt
Example of using nom parsers from a proc macro

Example of using nom parsers from a proc macro This project is organised as 3 crates: nom_macro is the main project, exposing the proc macro and the g

Geoffroy Couprie 1 Mar 20, 2022
PEG parser combinators using operator overloading without macros.

pom PEG parser combinators created using operator overloading without macros. Document Tutorial API Reference Learning Parser Combinators With Rust -

Junfeng Liu 412 Dec 31, 2022
A Rust crate for RDF parsing and inferencing.

RDF-rs This crate provides the tools necessary to parse RDF graphs. It currently contains a full (with very few exceptions) Turtle parser that can par

null 2 May 29, 2022
Yet Another Parser library for Rust. A lightweight, dependency free, parser combinator inspired set of utility methods to help with parsing strings and slices.

Yap: Yet another (rust) parsing library A lightweight, dependency free, parser combinator inspired set of utility methods to help with parsing input.

James Wilson 117 Dec 14, 2022
Parsing Expression Grammar (PEG) parser generator for Rust

Parsing Expression Grammars in Rust Documentation | Release Notes rust-peg is a simple yet flexible parser generator that makes it easy to write robus

Kevin Mehall 1.2k Dec 30, 2022
A Rust library for zero-allocation parsing of binary data.

Zero A Rust library for zero-allocation parsing of binary data. Requires Rust version 1.6 or later (requires stable libcore for no_std). See docs for

Nick Cameron 45 Nov 27, 2022
A typed parser generator embedded in Rust code for Parsing Expression Grammars

Oak Compiled on the nightly channel of Rust. Use rustup for managing compiler channels. You can download and set up the exact same version of the comp

Pierre Talbot 138 Nov 25, 2022
Rust library for parsing configuration files

configster Rust library for parsing configuration files Config file format The 'option' can be any string with no whitespace. arbitrary_option = false

The Impossible Astronaut 19 Jan 5, 2022
rbdt is a python library (written in rust) for parsing robots.txt files for large scale batch processing.

rbdt ?? ?? ?? ?? rbdt is a work in progress, currently being extracted out of another (private) project for the purpose of open sourcing and better so

Knuckleheads' Club 0 Nov 9, 2021
A Rust crate for hassle-free Corosync's configuration file parsing

corosync-config-parser A Rust crate for hassle-free Corosync's configuration file parsing. Inspired by Kilobyte22/config-parser. Usage extern crate co

Alessio Biancalana 2 Jun 10, 2022
This crate provide parsing fontconfig file but not yet complete all features

This crate provide parsing fontconfig file but not yet complete all features

null 4 Dec 27, 2022
Extensible inline parser engine, the backend parsing engine for Lavendeux.

Lavendeux Parser - Extensible inline parser engine lavendeux-parser is an exensible parsing engine for mathematical expressions. It supports variable

Richard Carson 10 Nov 3, 2022
A parser combinator for parsing &[Token].

PickTok A parser combinator like nom but specialized in parsing &[Token]. It has similar combinators as nom, but also provides convenient parser gener

Mikuto Matsuo 6 Feb 24, 2023
Pure, simple and elegant HTML parser and editor.

HTML Editor Pure, simple and elegant HTML parser and editor. Examples Parse HTML segment/document let document = parse("<!doctype html><html><head></h

Lomirus 16 Nov 8, 2022
A native Rust port of Google's robots.txt parser and matcher C++ library.

robotstxt A native Rust port of Google's robots.txt parser and matcher C++ library. Native Rust port, no third-part crate dependency Zero unsafe code

Folyd 72 Dec 11, 2022
JsonPath engine written in Rust. Webassembly and Javascript support too

jsonpath_lib Rust 버전 JsonPath 구현으로 Webassembly와 Javascript에서도 유사한 API 인터페이스를 제공 한다. It is JsonPath JsonPath engine written in Rust. it provide a simil

Changseok Han 95 Dec 29, 2022
🕑 A personal git log and MacJournal output parser, written in rust.

?? git log and MacJournal export parser A personal project, written in rust. WORK IN PROGRESS; NOT READY This repo consolidates daily activity from tw

Steven Black 4 Aug 17, 2022
Sqllogictest parser and runner in Rust.

Sqllogictest-rs Sqllogictest parser and runner in Rust. License Licensed under either of Apache License, Version 2.0 (LICENSE-APACHE or http://www.apa

Singularity Data Inc. 101 Dec 21, 2022
A library to display rich (Markdown) snippets and texts in a rust terminal application

A CLI utilities library leveraging Markdown to format terminal rendering, allowing separation of structure, data and skin. Based on crossterm so works

Canop 614 Dec 29, 2022