Rust port of gjson，get JSON value by dotpath syntax

Chen Jiaju

Last update: Dec 6, 2022

Related tags

Encoding JSON rust json json-parser json-path serde-json dotpath json-rs ajson

Overview

A-JSON

Read JSON values quickly - Rust JSON Parser

change name to AJSON, see issue

Inspiration comes from gjson in golang

Installation

Add it to your Cargo.toml file:

[dependencies]
ajson = "0.2"

Then add it to your code:

extern crate ajson;

Todo

Add documentation
Follow api-guidelines
Update benchmark
Optimize

A simple example

AJSON get json value with specified path, such as project.name or project.version. When the path matches, it returns immediately!

let data = r#"
{
  "project": {
    "name": "ajson",
    "maintainer": "importcjj",
    "version": 0.1,
    "rusts": ["stable", "nightly"]
  }
}
"#;

let name = ajson::get(data, "project.name").unwrap();
println!("{}", name.as_str()); // ajson

Path Syntax

JSON example

{
    "name": {"first": "Tom", "last": "Anderson"},
    "age":37,
    "children": ["Sara","Alex","Jack"],
    "fav.movie": "Deer Hunter",
    "friends": [
        {"first": "Dale", "last": "Murphy", "age": 44, "nets": ["ig", "fb", "tw"]},
        {"first": "Roger", "last": "Craig", "age": 68, "nets": ["fb", "tw"]},
        {"first": "Jane", "last": "Murphy", "age": 47, "nets": ["ig", "tw"]}
    ]
}

basic

Below is a quick overview of the path syntax, for more complete information please check out GJSON Syntax.

A path is a series of keys separated by a dot. A key may contain special wildcard characters '*' and '?'. To access an array value use the index as the key. To get the number of elements in an array or to access a child path, use the '#' character. The dot and wildcard characters can be escaped with ''.

name.last        >> "Anderson"
age              >> 37
children         >> ["Sara","Alex","Jack"]
children.#       >> 3
children.1       >> "Alex"
child*.2         >> "Jack"
c?ildren.0       >> "Sara"
fav\.movie       >> "Deer Hunter"
friends.#.first  >> ["Dale","Roger","Jane"]
friends.1.last   >> "Craig"

Escape character

Special purpose characters, such as ., *, and ? can be escaped with .

fav\.movie             "Deer Hunter"

Arrays

The # character allows for digging into JSON Arrays.To get the length of an array you'll just use the # all by itself.

friends.#              3
friends.#.age         [44,68,47]

queries

You can also query an array for the first match by using #(...), or find all matches with #(...)#. Queries support the ==, !=, <, <=, >, >= comparison operators and the simple pattern matching % (like) and !% (not like) operators.

friends.#(last=="Murphy").first   >> "Dale"
friends.#(last=="Murphy")#.first  >> ["Dale","Jane"]
friends.#(age>45)#.last           >> ["Craig","Murphy"]
friends.#(first%"D*").last        >> "Murphy"
friends.#(nets.#(=="fb"))#.first  >> ["Dale","Roger"]

construct

Basically, you can use selectors to assemble whatever you want, and of course, the result is still a json ;)

{name.first,age,"murphys":friends.#(last="Murphy")#.first}
[name.first,age,children.0]

ajson::get(json, "name.[first,last]").unwrap().to_vec();
ajson::get(json, "name.first").unwrap(); 
ajson::get(json, "name.last").unwrap();

Value

Value types.

enum Value {
    String(String),
    Number(Number),
    Object(String),
    Array(String),
    Boolean(bool),
    Null,
}

Value has a number of methods that meet your different needs.

value.get(&str) -> Option<Value>
value.as_str() -> &str
value.to_u64() -> u64
value.to_i64() -> i64
value.to_f64() -> f64
value.to_bool() -> bool
value.to_vec() -> Vec<Value>
value.to_object() -> HashMap<String, Value>

value.is_number() -> bool
value.is_string() -> bool
value.is_bool() -> bool
value.is_object() -> bool
value.is_array() -> bool
value.is_null() -> bool

get or parse?

Parse needs to read a complete json element, but get returns the result immediately, so get is recommended if you want to simply get a value

io::Read

Not only string, AJSON also can parse JSON from io::Read.

use std::fs::File;

let f = file::Open("path/to/json").unwrap();
let json = ajson::parse_from_read(f).unwrap();
let value = json.get("a.b").unwrap();
println!("{}", value.as_str());

Validate

AJSON can help you get the desired value from flawed JSON, but it's worth being more careful because of its looseness.

be careful!!!

Maybe need a validate function 🤔

Performance

$ cargo bench

ajson benchmark         time:   [6.7000 us 6.8023 us 6.9081 us]                             
                        change: [-1.8368% -0.4152% +1.0466%] (p = 0.58 > 0.05)
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

serde_json benchmark    time:   [48.196 us 48.543 us 48.947 us]                                  
                        change: [+2.9073% +4.4909% +6.3532%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

json-rust benchmark     time:   [24.540 us 24.773 us 25.061 us]                                 
                        change: [+4.8288% +6.0452% +7.4633%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

MacBook Pro (13-inch, 2018, Four Thunderbolt 3 Ports)
2.7 GHz Intel Core i7
16 GB 2133 MHz LPDDR3

problems

AJSON has just been finished, there may be some bugs and shortcomings, please feel free to issue. Also, Rust is a new language for me, and maybe ajson isn't rust enough, so I hope you have some suggestions.

License

MIT License.

Comments

Crate name on Crates.io

Hi, nice work on porting gjson from Go to Rust.

I'm the creator and maintainer of the official gjson project and the gjson specification.

I noticed that you chose to use the crate name gjson in crates.io. It so happens that the official Rust version of gjson is currently in development.

I'm asking you if you could assign the ownership of the name to me as I believe the Rust community would benefit from having the gjson crate name point to the official project. https://crates.io/policies#package-ownership

Opening this issue to raise the concern as early as possible, because changing the crate name is easier at the early stage of the project.

opened by tidwall 5

performance comparison between rust-ajson with gjson.rs

I forked your benches to test the speed of ajson and gjson.rs. Below is the result::

ajson benchmark         time:   [9.1979 us 9.2489 us 9.3066 us]
                        change: [-10.706% -9.1719% -7.6992%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

gjson benchmark         time:   [3.6221 us 3.6413 us 3.6623 us]
                        change: [-15.350% -12.966% -10.873%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe

serde_json benchmark    time:   [59.616 us 59.876 us 60.162 us]
                        change: [-14.793% -12.771% -10.900%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild
  6 (6.00%) high severe

json-rust benchmark     time:   [29.568 us 29.729 us 29.894 us]
                        change: [-15.512% -14.014% -12.561%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe

ajson selector          time:   [3.0875 us 3.1000 us 3.1139 us]
                        change: [-11.651% -10.079% -8.4826%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high mild

gjson selector          time:   [2.0724 us 2.0882 us 2.1050 us]
                        change: [-5.7071% -3.8407% -1.9520%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high severe

ajson multi query       time:   [2.6245 us 2.6365 us 2.6492 us]
                        change: [-9.4889% -8.3135% -7.1778%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild
  3 (3.00%) high severe

gjson multi query       time:   [858.23 ns 861.61 ns 865.37 ns]
                        change: [-10.479% -8.6397% -6.8407%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

serde derive            time:   [9.1830 us 9.2368 us 9.2964 us]
                        change: [-10.407% -8.5730% -6.7286%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

serde derive multi query
                        time:   [1.9318 us 1.9421 us 1.9542 us]
                        change: [-5.3005% -4.0650% -2.8234%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

nom json bench          time:   [1.2689 us 1.2735 us 1.2783 us]
                        change: [-4.7747% -3.7796% -2.8171%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe

Fork: gjson-bench

It seems that ajson is a little bit slower than gsjon.rs :)

opened by piaoger 1

Fix Some warnings and linter issues by clippy
Summary of changes:

Remove deprecate .... and use ..= instead (https://stackoverflow.com/questions/49834574/why-it-is-not-possible-to-use-the-syntax-in-expressions)

Use matches!

Use is_empty instead comparing to 0

Use epsilon while comparing floating numbers

use if let instead of match for a single arm

Remove unnecessary lifetime.
opened by apatniv 1

Handle unicode character in json object keys

Thanks for writing a very useful library.

I think currently library doesn't handle unicode characters in object keys.


    let data = r#"{"sample_unicode\u0041": "abc\u0041\u0042"}"#;

    let name = ajson::parse(data).unwrap();
    dbg!(name);

    {
        dbg!(ajson::get(data, "sample_unicodeA"));

        dbg!(ajson::get(data, "sample_unicode\\u0041"));
    }

    {
        let v: Value = serde_json::from_str(data).unwrap();
        dbg!(&v["sample_unicodeA"]);
    }

Output:

[src/main.rs:10] name = {"sample_unicode\u0041": "abc\u0041\u0042"}
[src/main.rs:13] ajson::get(data, "sample_unicodeA") = None
[src/main.rs:15] ajson::get(data, "sample_unicode\\u0041") = Some(
    "abcAB",
)
[src/main.rs:20] &v["sample_unicodeA"] = String(
    "abcAB",
)

If you willing to accept PRs, I will try to fix this issue and add necessary test cases.

opened by apatniv 1

Some([]) using get_from_read
Hello! I have a json that looks like this:

{ "ids":[1, 2, 3,4, 5] }

When I run this:

extern crate ajson; use std::fs::File; fn main() { let f = File::open("ids.json").unwrap(); let value = ajson::get_from_read(f, "ids.#.1"); println!("{:?}", value); }

I simply get Some([]) as a result. Have I done anything wrong?

Thank you for this library by the way :)
opened by rezicG 1
bracket in array value

Hi,

There is a bug when a bracket sits in array's value.

#[test] fn test_bracket_in_array() { let json = r##"{ "children": ["Sara","Alex]","Jack"], "##; let r = parse(json).unwrap(); assert_eq!(r.get("children.#").unwrap().to_i64(), 3); assert_eq!(r.get("children").unwrap().to_vec(), vec!["Sara", "Alex]", "Jack"]); assert_eq!(r.get("children.1").unwrap(), "Alex]"); assert_eq!(r.get("child*.2").unwrap(), "Jack"); assert_eq!(r.get("c?ildren.0").unwrap(), "Sara"); }

---- test_bracket_in_array stdout ---- thread 'test_bracket_in_array' panicked at 'assertion failed: (left == right) left: ["Sara", "Alex]"], right: ["Sara", "Alex]", "Jack"]', tests/test_ajson.rs:590:5 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace.

opened by kuerant 1

Multiple conditions in queries

Hello! First of all, thanks for this project.

Could you please add conditions operators AND, OR for using multiple conditions in one query?

For example:

let json_string = r#"
{
    "customFields": [
        {
            "projectCustomField": {
                "id": "1",
                "$type": "EnumProjectCustomField"
            },
            "value": {
                "localizedName": null,
                "description": null,
                "name": "High",
                "id": "3",
                "$type": "EnumBundleElement"
            },
            "name": "Priority",
            "id": "1",
            "$type": "SingleEnumIssueCustomField"
        },
        {
            "projectCustomField": {
                "id": "2",
                "$type": "EnumProjectCustomField"
            },
            "value": {
                "localizedName": null,
                "description": null,
                "name": "Bug",
                "id": "3",
                "$type": "EnumBundleElement"
            },
            "name": "Type",
            "id": "2",
            "$type": "SingleEnumIssueCustomField"
        },
    ]
}"#;

let prioroty= ajson::get(json_string, r#"customFields.#($type == "SingleEnumIssueCustomField" && name == "Priority").value.name"#).map(|s| s.to_string());

Todo

opened by rignatev 2