Hjson for Rust

Related tags

Encoding JSON rust hjson serde
Overview

hjson-rust for serde

Build Status crate

Hjson Intro

{
  # specify rate in requests/second (because comments are helpful!)
  rate: 1000

  // prefer c-style comments?
  /* feeling old fashioned? */

  # did you notice that rate doesn't need quotes?
  hey: look ma, no quotes for strings either!

  # best of all
  notice: []
  anything: ?

  # yes, commas are optional!
}

The Rust implementation of Hjson is based on the Serde JSON Serialization Library. For other platforms see hjson.github.io.

This crate is a Rust library for parsing and generating Human JSON Hjson. It is built upon Serde, a high performance generic serialization framework.

Install

This crate works with Cargo and can be found on crates.io with a Cargo.toml like:

[dependencies]
serde = "*"
serde-hjson = "*"

From the Commandline

Install with cargo install hjson

Hjson, the Human JSON.

Usage:
  hjson [options]
  hjson [options] <input>
  hjson (-h | --help)
  hjson (-V | --version)

Options:
  -h --help     Show this screen.
  -j            Output as formatted JSON.
  -c            Output as JSON.
  -V --version  Show version.

Sample:

  • run hjson test.json > test.hjson to convert to Hjson
  • run hjson -j test.hjson > test.json to convert to JSON

Usage

extern crate serde;
extern crate serde_hjson;

use serde_hjson::{Map,Value};

fn main() {

    // Now let's look at decoding Hjson data

    let sample_text=r#"
    {
        # specify rate in requests/second
        rate: 1000
        array:
        [
            foo
            bar
        ]
    }"#;

    // Decode and unwrap.
    let mut sample: Map<String, Value> = serde_hjson::from_str(&sample_text).unwrap();

    // scope to control lifetime of borrow
    {
        // Extract the rate
        let rate = sample.get("rate").unwrap().as_f64().unwrap();
        println!("rate: {}", rate);

        // Extract the array
        let array : &mut Vec<Value> = sample.get_mut("array").unwrap().as_array_mut().unwrap();
        println!("first: {}", array.get(0).unwrap());

        // Add a value
        array.push(Value::String("tak".to_string()));
    }

    // Encode to Hjson
    let sample2 = serde_hjson::to_string(&sample).unwrap();
    println!("Hjson:\n{}", sample2);
}

API

see Rust doc

History

see history.md

Comments
  • Parse error on deserialization of enum value

    Parse error on deserialization of enum value

    Trying to deserialize an struct that contains an enum on Rust nightly. However using hjson i get a parse error while the example seems to work when using json instead.

    thread 'hjson_enum_serialize_deserialize_works' panicked at 'called `Result::unwrap()` on an `Err` value: Syntax(Found a punctuator where a key name was expected (check your syntax or use quotes if the key name includes {}[],: or whitespace), 1, 1)', /Users/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-mac/build/src/libcore/result.rs:845

    Example test-case:

    #![feature(proc_macro)]

#[macro_use]
extern crate serde_derive;
extern crate serde_hjson;

#[derive(Serialize, Deserialize, Debug, PartialEq)]
struct Config {
    color: Color,
}

#[derive(Serialize, Deserialize, Debug, PartialEq)]
enum Color {
    Green,
}

#[test]
fn hjson_enum_serialize_deserialize_works() {
    let config = Config { color: Color::Green };
    let serialized = serde_hjson::to_string(&config).unwrap();
    println!("serialized:\n{}", serialized);
    let deserialized: Config = serde_hjson::from_str(&serialized).unwrap();
    println!("deserialized:\n{:?}", deserialized);
    assert_eq!(config, deserialized);
}

    Best regards, Sander Hahn

    opened by sanderhahn 8
  • Update to Serde 1.0

    Update to Serde 1.0

    Serde 0.9.0 release notes: https://github.com/serde-rs/serde/releases/tag/v0.9.0 Serde 1.0.0 release notes: https://github.com/serde-rs/serde/releases/tag/v1.0.0

    help wanted 
    opened by dtolnay 3
  • tests fail to compile due to quasi

    tests fail to compile due to quasi

    When running "cargo test" in hjson_tests/, it fails to compile with:

    error[E0432]: unresolved import `syntax::parse::parse_tts_from_source_str`
  --> /home/jvalleroy/.cargo/registry/src/github.com-1ecc6299db9ec823/quasi-0.20.1/src/lib.rs:36:37
   |
36 | use syntax::parse::{self, classify, parse_tts_from_source_str, token};
   |                                     ^^^^^^^^^^^^^^^^^^^^^^^^^ no `parse_tts_from_source_str` in `parse`. Did you mean to use `parse_item_from_source_str`?

    and many other errors.

    quasi crate is no longer maintained. Should it be replaced with something else?

    opened by jvalleroy 1
  • Make extern crate linked_hash_map conditional

    Make extern crate linked_hash_map conditional

    The dependency on linked_hash_map is optional, but there was an extern crate linked_hash_map that was not under a conditional flag and this gives compiler errors, when "preserve_order" is not set since the dependency is missing.

    This adds the conditional avoiding trying to import the crate when "preserve_order" is not set.

    opened by najamelan 0
  • Numbers are always be parsed to Value::F64

    Numbers are always be parsed to Value::F64 

    {
  # specify rate in requests/second (because comments are helpful!)
  rate: 1000
}

    This is the example taken from README. And we have to use the value of rate by as_f64 like this:

    let rate = sample.get("rate").unwrap().as_f64().unwrap();

    1000 should be Value::U64 or at least Value::I64.

    Maybe we can represent number values as Value::Number like serde-json does?

    opened by zonyitoo 0
  • hjson-rust fails with Rust 1.48

    hjson-rust fails with Rust 1.48

    We've been using hjson in Nushell, and it's been great. Definitely an improvement to have a more-flexible json format.

    With the latest Rust 1.48, though, we ran into an issue where one of the dependencies is leaving memory uninitialized.

    Here's the full backtrace:

    ---- commands::math::compound_where_paren stdout ----
=== stderr
thread 'main' panicked at 'attempted to leave type `linked_hash_map::Node<std::string::String, value::Value>` uninitialized, which is invalid', /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/mem/mod.rs:658:9
stack backtrace:
   0: rust_begin_unwind
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:483
   1: core::panicking::panic_fmt
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/panicking.rs:85
   2: core::panicking::panic
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/panicking.rs:50
   3: core::mem::uninitialized
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/mem/mod.rs:658
   4: linked_hash_map::LinkedHashMap<K,V,S>::insert
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/linked-hash-map-0.3.0/src/lib.rs:203
   5: <linked_hash_map::serde::LinkedHashMapVisitor<K,V> as serde::de::Visitor>::visit_map
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/linked-hash-map-0.3.0/src/serde.rs:72
   6: <<serde_hjson::value::Value as serde::de::Deserialize>::deserialize::ValueVisitor as serde::de::Visitor>::visit_map
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/value.rs:441
   7: serde_hjson::de::Deserializer<Iter>::parse_value
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:151
   8: <serde_hjson::de::Deserializer<Iter> as serde::de::Deserializer>::deserialize
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:443
   9: <serde_hjson::value::Value as serde::de::Deserialize>::deserialize
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/value.rs:446
  10: <serde_hjson::de::SeqVisitor<Iter> as serde::de::SeqVisitor>::visit
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:537
  11: <serde::de::impls::VecVisitor<T> as serde::de::Visitor>::visit_seq
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-0.8.23/src/de/impls.rs:433
  12: <<serde_hjson::value::Value as serde::de::Deserialize>::deserialize::ValueVisitor as serde::de::Visitor>::visit_seq
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/value.rs:432
  13: serde_hjson::de::Deserializer<Iter>::parse_value
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:147
  14: <serde_hjson::de::Deserializer<Iter> as serde::de::Deserializer>::deserialize
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:443
  15: <serde_hjson::value::Value as serde::de::Deserialize>::deserialize
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/value.rs:446
  16: serde_hjson::de::from_iter
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:813
  17: serde_hjson::de::from_slice
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:846
  18: serde_hjson::de::from_str
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:853
  19: nu_cli::commands::from_json::from_json_string_to_value
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/from_json.rs:71
  20: nu_cli::commands::from_json::from_json::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/from_json.rs:114
  21: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  22: <nu_cli::commands::from_json::FromJSON as nu_cli::commands::command::WholeStreamCommand>::run::__run::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/from_json.rs:36
  23: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  24: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:119
  25: nu_cli::commands::command::Command::run::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/command.rs:321
  26: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  27: nu_cli::evaluation_context::EvaluationContext::run_command::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/evaluation_context.rs:159
  28: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  29: nu_cli::commands::classified::internal::run_internal_command::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/classified/internal.rs:29
  30: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  31: nu_cli::commands::classified::block::run_pipeline::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/classified/block.rs:82
  32: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  33: nu_cli::commands::classified::block::run_block::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/classified/block.rs:55
  34: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  35: nu_cli::cli::process_line::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/cli.rs:1036
  36: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  37: nu_cli::cli::cli::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/cli.rs:479
  38: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  39: futures_executor::local_pool::block_on::{{closure}}
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.5/src/local_pool.rs:317
  40: futures_executor::local_pool::run_executor::{{closure}}
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.5/src/local_pool.rs:87
  41: std::thread::local::LocalKey<T>::try_with
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:272
  42: std::thread::local::LocalKey<T>::with
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:248
  43: futures_executor::local_pool::run_executor
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.5/src/local_pool.rs:83
  44: futures_executor::local_pool::block_on
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.5/src/local_pool.rs:317
  45: nu::main
             at /home/jonathan/Source/nushell/src/main.rs:171
  46: core::ops::function::FnOnce::call_once
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

thread 'commands::math::compound_where_paren' panicked at 'assertion failed: `(left == right)`
  left: `""`,
 right: `"[{\"a\":2,\"b\":1},{\"a\":2,\"b\":2}]"`', crates/nu-cli/tests/commands/math/mod.rs:257:5
stack backtrace:
   0: rust_begin_unwind
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:483
   1: std::panicking::begin_panic_fmt
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:437
   2: main::commands::math::compound_where_paren
             at ./tests/commands/math/mod.rs:257
   3: main::commands::math::compound_where_paren::{{closure}}
             at ./tests/commands/math/mod.rs:249
   4: core::ops::function::FnOnce::call_once
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
   5: core::ops::function::FnOnce::call_once
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/ops/function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

    Assuming I'm reading that correctly, the issue is in linked_hash_map. I looked on their repo and it looks like it's no longer maintained.

    Any ideas if there's a fix for this? We'd definitely love to keep using this package.

    opened by jntrnr 4
  • serde_hjson::from_slice panics on ParseIntError

    serde_hjson::from_slice panics on ParseIntError

    more fuzzing gave me this error:

    thread '' panicked at 'called Result::unwrap() on an Err value: ParseIntError { kind: Overflow }', /home/capitol/project/hjson-rust/hjson/src/util.rs:208:67

    full stacktrace:

        #0 0x561f6f275731 in __sanitizer_print_stack_trace /rustc/llvm/src/llvm-project/compiler-rt/lib/asan/asan_stack.cpp:86:3
    #1 0x561f6f936660 in fuzzer::PrintStackTrace() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x996660)
    #2 0x561f6f95299a in fuzzer::Fuzzer::CrashCallback() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9b299a)
    #3 0x7f153a3933bf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x153bf)
    #4 0x7f153a1b718a in __libc_signal_restore_set /build/glibc-YYA7BZ/glibc-2.31/signal/../sysdeps/unix/sysv/linux/internal-signals.h:86:3
    #5 0x7f153a1b718a in raise /build/glibc-YYA7BZ/glibc-2.31/signal/../sysdeps/unix/sysv/linux/raise.c:48:3
    #6 0x7f153a196858 in abort /build/glibc-YYA7BZ/glibc-2.31/stdlib/abort.c:79:7
    #7 0x561f6f9aeb36 in std::sys::unix::abort_internal::h5c8b2a90c624abaf /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sys/unix/mod.rs:167:14
    #8 0x561f6f997bc5 in std::process::abort::hb13208ae9f5b7133 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/process.rs:1623:5
    #9 0x561f6f9201b6 in libfuzzer_sys::initialize::_$u7b$$u7b$closure$u7d$$u7d$::h2ef829035805c4e9 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9801b6)
    #10 0x561f6f99eed7 in std::panicking::rust_panic_with_hook::h2f4c96dfd8ba524a /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:581:17
    #11 0x561f6f99ea88 in std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h7740abbe2875cb4d /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:484:9
    #12 0x561f6f999ebb in std::sys_common::backtrace::__rust_end_short_backtrace::hcad001df0a36db28 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sys_common/backtrace.rs:153:18
    #13 0x561f6f99ea48 in rust_begin_unwind /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:483:5
    #14 0x561f6fa04460 in core::panicking::panic_fmt::hb15d6f55e8472f62 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/core/src/panicking.rs:85:14
    #15 0x561f6fa040d2 in core::result::unwrap_failed::h110828a80aba3eec /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/core/src/option.rs:1221:5
    #16 0x561f6f2d923d in serde_hjson::util::ParseNumber$LT$Iter$GT$::parse::hba5da05d298b23ff (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x33923d)
    #17 0x561f6f2a58ad in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_tfnns::h7d7fb93a4d3df50b (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x3058ad)
    #18 0x561f6f2abb92 in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_value::h09e41fbc88e5efe4 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x30bb92)
    #19 0x561f6f301ff1 in serde::de::MapVisitor::visit::hf1a5b50f97f17367 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x361ff1)
    #20 0x561f6f310966 in _$LT$linked_hash_map..serde..LinkedHashMapVisitor$LT$K$C$V$GT$$u20$as$u20$serde..de..Visitor$GT$::visit_map::h730c4bb087010ef0 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x370966)
    #21 0x561f6f2aece8 in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_value::h9b2c32860cf298a3 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x30ece8)
    #22 0x561f6f2cd8ba in serde_hjson::de::from_iter::hc227fa3539b40986 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x32d8ba)
    #23 0x561f6f317118 in rust_fuzzer_test_input (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x377118)
    #24 0x561f6f9201e0 in __rust_try (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9801e0)
    #25 0x561f6f91fe3f in LLVMFuzzerTestOneInput (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x97fe3f)
    #26 0x561f6f952edc in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9b2edc)
    #27 0x561f6f95aec0 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9baec0)
    #28 0x561f6f95b87c in fuzzer::Fuzzer::MutateAndTestOne() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9bb87c)
    #29 0x561f6f95dc7f in fuzzer::Fuzzer::Loop(std::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9bdc7f)
    #30 0x561f6f92e239 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x98e239)
    #31 0x561f6f1f22e6 in main (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x2522e6)
    #32 0x7f153a1980b2 in __libc_start_main /build/glibc-YYA7BZ/glibc-2.31/csu/../csu/libc-start.c:308:16
    #33 0x561f6f1f248d in _start (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x25248d)

    Can be reproduced with the following unit test:

    #[cfg(test)]
mod test {
    use crate::{Map,Value};
    use crate::error::Result;

    #[test]
    pub fn parse_int_error() {
        let data: Vec<u8> = vec![47, 97, 47, 65, 58, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 0, 0, 0, 54, 35, 54, 54, 54, 54, 54, 54, 54, 44, 35, 58, 45, 85, 85, 85, 35, 116, 45, 35, 35, 58, 47];

        let mut sample: Result<Map<String, Value>> = crate::from_slice(&data);
    }
}
    opened by alexanderkjall 0
  • serde_hjson::from_slice panics at 'removal index (is 0) should be < len (is 0)'

    serde_hjson::from_slice panics at 'removal index (is 0) should be < len (is 0)'

    more testing of the from_slice function with valid utf8 gave me this error:

    thread '' panicked at 'removal index (is 0) should be < len (is 0)', library/alloc/src/vec.rs:1057:13

    complete stacktrace is:

    ==1202250== ERROR: libFuzzer: deadly signal
    #0 0x55c30665a731 in __sanitizer_print_stack_trace /rustc/llvm/src/llvm-project/compiler-rt/lib/asan/asan_stack.cpp:86:3
    #1 0x55c306d1b660 in fuzzer::PrintStackTrace() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x996660)
    #2 0x55c306d3799a in fuzzer::Fuzzer::CrashCallback() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9b299a)
    #3 0x7f714cb733bf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x153bf)
    #4 0x7f714c99718a in __libc_signal_restore_set /build/glibc-YYA7BZ/glibc-2.31/signal/../sysdeps/unix/sysv/linux/internal-signals.h:86:3
    #5 0x7f714c99718a in raise /build/glibc-YYA7BZ/glibc-2.31/signal/../sysdeps/unix/sysv/linux/raise.c:48:3
    #6 0x7f714c976858 in abort /build/glibc-YYA7BZ/glibc-2.31/stdlib/abort.c:79:7
    #7 0x55c306d93b36 in std::sys::unix::abort_internal::h5c8b2a90c624abaf /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sys/unix/mod.rs:167:14
    #8 0x55c306d7cbc5 in std::process::abort::hb13208ae9f5b7133 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/process.rs:1623:5
    #9 0x55c306d051b6 in libfuzzer_sys::initialize::_$u7b$$u7b$closure$u7d$$u7d$::h2ef829035805c4e9 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9801b6)
    #10 0x55c306d83ed7 in std::panicking::rust_panic_with_hook::h2f4c96dfd8ba524a /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:581:17
    #11 0x55c306d83a88 in std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h7740abbe2875cb4d /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:484:9
    #12 0x55c306d7eebb in std::sys_common::backtrace::__rust_end_short_backtrace::hcad001df0a36db28 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sys_common/backtrace.rs:153:18
    #13 0x55c306d83a48 in rust_begin_unwind /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:483:5
    #14 0x55c306de9460 in core::panicking::panic_fmt::hb15d6f55e8472f62 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/core/src/panicking.rs:85:14
    #15 0x55c306de12a5 in alloc::vec::Vec$LT$T$GT$::remove::assert_failed::h2c0d56e327999482 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/alloc/src/vec.rs:1057:13
    #16 0x55c3066c55fd in serde_hjson::util::StringReader$LT$Iter$GT$::parse_whitespace::hf7084d7b3ccb8e69 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x3405fd)
    #17 0x55c306693981 in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_value::h9b2c32860cf298a3 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x30e981)
    #18 0x55c3066b28ba in serde_hjson::de::from_iter::hc227fa3539b40986 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x32d8ba)
    #19 0x55c3066fc118 in rust_fuzzer_test_input (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x377118)
    #20 0x55c306d051e0 in __rust_try (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9801e0)
    #21 0x55c306d04e3f in LLVMFuzzerTestOneInput (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x97fe3f)
    #22 0x55c306d37edc in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9b2edc)
    #23 0x55c306d3fec0 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9baec0)
    #24 0x55c306d4087c in fuzzer::Fuzzer::MutateAndTestOne() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9bb87c)
    #25 0x55c306d42c7f in fuzzer::Fuzzer::Loop(std::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9bdc7f)
    #26 0x55c306d13239 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x98e239)
    #27 0x55c3065d72e6 in main (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x2522e6)
    #28 0x7f714c9780b2 in __libc_start_main /build/glibc-YYA7BZ/glibc-2.31/csu/../csu/libc-start.c:308:16
    #29 0x55c3065d748d in _start (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x25248d)

    Can be reproduced with this unit test

    #[cfg(test)]
mod test {
    use crate::{Map,Value};
    use crate::error::Result;

    #[test]
    pub fn removal_index() {
        let data: Vec<u8> = vec![47, 42, 44, 45];

        let mut sample: Result<Map<String, Value>> = crate::from_slice(&data);
    }
}
    opened by alexanderkjall 0
  • serde_hjson::from_slice panics on subtract with overflow

    serde_hjson::from_slice panics on subtract with overflow

    testing the from_slice function with valid utf8 gave me this error:

    thread '' panicked at 'attempt to subtract with overflow', /home/capitol/project/hjson-rust/hjson/src/de.rs:294:22

    complete stacktrace:

        #0 0x55a627176731 in __sanitizer_print_stack_trace /rustc/llvm/src/llvm-project/compiler-rt/lib/asan/asan_stack.cpp:86:3
    #1 0x55a627837660 in fuzzer::PrintStackTrace() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x996660)
    #2 0x55a62785399a in fuzzer::Fuzzer::CrashCallback() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9b299a)
    #3 0x7fb82afe03bf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x153bf)
    #4 0x7fb82ae0418a in __libc_signal_restore_set /build/glibc-YYA7BZ/glibc-2.31/signal/../sysdeps/unix/sysv/linux/internal-signals.h:86:3
    #5 0x7fb82ae0418a in raise /build/glibc-YYA7BZ/glibc-2.31/signal/../sysdeps/unix/sysv/linux/raise.c:48:3
    #6 0x7fb82ade3858 in abort /build/glibc-YYA7BZ/glibc-2.31/stdlib/abort.c:79:7
    #7 0x55a6278afb36 in std::sys::unix::abort_internal::h5c8b2a90c624abaf /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sys/unix/mod.rs:167:14
    #8 0x55a627898bc5 in std::process::abort::hb13208ae9f5b7133 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/process.rs:1623:5
    #9 0x55a6278211b6 in libfuzzer_sys::initialize::_$u7b$$u7b$closure$u7d$$u7d$::h2ef829035805c4e9 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9801b6)
    #10 0x55a62789fed7 in std::panicking::rust_panic_with_hook::h2f4c96dfd8ba524a /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:581:17
    #11 0x55a62789fa88 in std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h7740abbe2875cb4d /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:484:9
    #12 0x55a62789aebb in std::sys_common::backtrace::__rust_end_short_backtrace::hcad001df0a36db28 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sys_common/backtrace.rs:153:18
    #13 0x55a62789fa48 in rust_begin_unwind /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:483:5
    #14 0x55a627905460 in core::panicking::panic_fmt::hb15d6f55e8472f62 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/core/src/panicking.rs:85:14
    #15 0x55a6279053ac in core::panicking::panic::h5d1c61fed2502a5f /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/core/src/panicking.rs:50:5
    #16 0x55a6271c0f46 in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_ml_string::h6d7ce8a9cfd6f9b5 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x31ff46)
    #17 0x55a6271a0d95 in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_tfnns::h150292d6fccc2cd0 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x2ffd95)
    #18 0x55a6271b2891 in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_value::hefb12d656723d8ef (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x311891)
    #19 0x55a6271cef2d in serde_hjson::de::from_iter::hc227fa3539b40986 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x32df2d)
    #20 0x55a627218118 in rust_fuzzer_test_input (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x377118)
    #21 0x55a6278211e0 in __rust_try (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9801e0)
    #22 0x55a627820e3f in LLVMFuzzerTestOneInput (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x97fe3f)
    #23 0x55a627853edc in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9b2edc)
    #24 0x55a62785bec0 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9baec0)
    #25 0x55a62785c87c in fuzzer::Fuzzer::MutateAndTestOne() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9bb87c)
    #26 0x55a62785ec7f in fuzzer::Fuzzer::Loop(std::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9bdc7f)
    #27 0x55a62782f239 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x98e239)
    #28 0x55a6270f32e6 in main (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x2522e6)
    #29 0x7fb82ade50b2 in __libc_start_main /build/glibc-YYA7BZ/glibc-2.31/csu/../csu/libc-start.c:308:16
    #30 0x55a6270f348d in _start (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x25248d)

    Can be reproduced with the following unit test:

    #[cfg(test)]
mod test {
    use crate::{Map,Value};
    use crate::error::Result;

    #[test]
    pub fn subtract_overflow() {
        let data: Vec<u8> = vec![39, 39, 39];

        let mut sample: Result<Map<String, Value>> = crate::from_slice(&data);
    }
}
    opened by alexanderkjall 0
  • serde_hjson::from_slice panics on decoding invalid utf8

    serde_hjson::from_slice panics on decoding invalid utf8

    Hi

    I found that the from_slice function panics if the input isn't a valid utf8 string instead of returning an error value.

    Can be reproduced with this unit test:

    
#[cfg(test)]
mod test {
    use crate::{Map,Value};
    use crate::error::Result;

    #[test]
    pub fn invalid_utf8() {
        let data: Vec<u8> = vec![155];

        let mut sample: Result<Map<String, Value>> = crate::from_slice(&data);
    }
}
    opened by alexanderkjall 0
Owner
Hjson
A user interface for JSON.
Hjson
GitHub https://hjson.github.io/
Strongly typed JSON library for Rust

Serde JSON   Serde is a framework for serializing and deserializing Rust data structures efficiently and generically. [dependencies] serde_json = "1.0

null 3.6k Jan 5, 2023
Rust port of simdjson

SIMD Json for Rust   Rust port of extremely fast simdjson JSON parser with serde compatibility. readme (for real!) simdjson version Currently tracking

null 737 Dec 30, 2022
JSON implementation in Rust

json-rust Parse and serialize JSON with ease. Changelog - Complete Documentation - Cargo - Repository Why? JSON is a very loose format where anything

Maciej Hirsz 500 Dec 21, 2022
Rust port of gjson，get JSON value by dotpath syntax

A-JSON Read JSON values quickly - Rust JSON Parser change name to AJSON, see issue Inspiration comes from gjson in golang Installation Add it to your

Chen Jiaju 90 Dec 6, 2022
Get JSON values quickly - JSON parser for Rust

get json values quickly GJSON is a Rust crate that provides a fast and simple way to get values from a json document. It has features such as one line

Josh Baker 160 Dec 29, 2022
This library is a pull parser for CommonMark, written in Rust

This library is a pull parser for CommonMark, written in Rust. It comes with a simple command-line tool, useful for rendering to HTML, and is also designed to be easy to use from as a library.

Raph Levien 1.5k Jan 1, 2023
A rust script to convert a better bibtex json file from Zotero into nice organised notes in Obsidian

Zotero to Obsidian script This is a script that takes a better bibtex JSON file exported by Zotero and generates an organised collection of reference

Sashin Exists 3 Oct 9, 2022
Fontdue - The fastest font renderer in the world, written in pure rust.

Fontdue is a simple, no_std (does not use the standard library for portability), pure Rust, TrueType (.ttf/.ttc) & OpenType (.otf) font rasterizer and layout tool. It strives to make interacting with fonts as fast as possible, and currently has the lowest end to end latency for a font rasterizer.

Joe C 1k Jan 2, 2023
CLI tool to convert HOCON into valid JSON or YAML written in Rust.

{hocon:vert} CLI Tool to convert HOCON into valid JSON or YAML. Under normal circumstances this is mostly not needed because hocon configs are parsed

Mathias Oertel 23 Jan 6, 2023
Typify - Compile JSON Schema documents into Rust types.

Typify Compile JSON Schema documents into Rust types. This can be used ... via the macro import_types!("types.json") to generate Rust types directly i

Oxide Computer Company 73 Dec 27, 2022
A easy and declarative way to test JSON input in Rust.

assert_json A easy and declarative way to test JSON input in Rust. assert_json is a Rust macro heavily inspired by serde json macro. Instead of creati

Charles Vandevoorde 8 Dec 5, 2022
A small rust database that uses json in memory.

Tiny Query Database (TQDB) TQDB is a small library for creating a query-able database that is encoded with json. The library is well tested (~96.30% c

Kace Cottam 2 Jan 4, 2022
A JSON Query Language CLI tool built with Rust 🦀

JQL A JSON Query Language CLI tool built with Rust ?? ?? Core philosophy ?? Stay lightweight ?? Keep its features as simple as possible ?? Avoid redun

Davy Duperron 872 Jan 1, 2023
An implementation of the JSONPath A spec in Rust, with several extensions added on

Rust JSONPath Plus An implementation of the JSONPath A spec in Rust, with several extensions added on. This library also supports retrieving AST analy

Rune Tynan 4 Jul 13, 2022
An HCL serializer/deserializer for rust

hcl-rs This crate provides functionality to deserialize, serialize and manipulate HCL data. The main types are Deserializer for deserializing data, Se

null 56 Dec 31, 2022
Rust libraries and tools to help with interoperability and testing of serialization formats based on Serde.

The repository zefchain/serde-reflection is based on Facebook's repository novifinancial/serde-reflection. We are now maintaining the project here and

Zefchain Labs 46 Dec 22, 2022
Blazing fast Rust JSONPath query engine.

rsonpath – SIMD-powered JSONPath ?? Experimental JSONPath engine for querying massive streamed datasets. Features The rsonpath crate provides a JSONPa

V0ldek 21 Apr 11, 2023
A Rust program that analyzes your TikTok data.

The TikTok JSON analyzer This is a program to analyze your TikTok data and calculate these statistics : Number of logins (in the last 6 months) and lo

Elazrod56 3 May 2, 2023
Hjson for Rust

hjson-rust for serde { # specify rate in requests/second (because comments are helpful!) rate: 1000 // prefer c-style comments? /* feeling ol

Hjson 83 Oct 5, 2022
Slitter is a C- and Rust-callable slab allocator implemented primarily in Rust, with some C for performance or to avoid unstable Rust features.

Slitter is a less footgunny slab allocator Slitter is a classically structured thread-caching slab allocator that's meant to help write reliable long-

Backtrace Labs 133 Dec 5, 2022