Serialize JSON into a canonical format.

Overview

json-canon

Serialize JSON into a canonical format.

Safe for generating a consistent cryptographic hash or signature across platforms.

Follows RFC8785: JSON Canonicalization Scheme (JCS)

JSON cannon

Features

The JSON Canonicalization Scheme concept in a nutshell:

  • Serialization of primitive JSON data types using methods compatible with ECMAScript's JSON.stringify()
  • Lexicographic sorting of JSON Object properties in a recursive process
  • JSON Array data is also subject to canonicalization, but element order remains untouched

Serializers

JavaScript: json-canon

npm version download ci status

const serialize = require('json-canon')

const json = {
  from_account: "543 232 625-3",
  to_account: "321 567 636-4",
  amount: 500,
  currency: "USD"
}

console.log(serialize(json))
// {"amount":500,"currency":"USD","from_account":"543 232 625-3","to_account":"321 567 636-4"}

Rust: json-canon

crates.io version download docs.rs docs ci status

use json_canon::to_string;
use serde_json::json;

let data = json!({
    "from_account": "543 232 625-3",
    "to_account": "321 567 636-4",
    "amount": 500,
    "currency": "USD"
});

println!("{}", to_string(&data)?);
// {"amount":500,"currency":"USD","from_account":"543 232 625-3","to_account":"321 567 636-4"}

Fuzzers

References

Comments
  • [rust] use mutable reference to entries before sort

    [rust] use mutable reference to entries before sort

    bench:

                            time:   [22.139 µs 22.176 µs 22.217 µs]
                            thrpt:  [45.010 Kelem/s 45.093 Kelem/s 45.170 Kelem/s]
                     change:
                            time:   [-23.567% -23.340% -23.129%] (p = 0.00 < 0.05)
                            thrpt:  [+30.087% +30.447% +30.833%
    
    opened by ahdinosaur 0
  • Rust numbers fix

    Rust numbers fix

    • handle number keys
      • closes https://github.com/ahdinosaur/json-canon/issues/5
    • handle numbers greater than JavaScript's Number.MAX_SAFE_INTEGER
      • closes https://github.com/ahdinosaur/json-canon/issues/6
    • add more unit tests
    • add more code docs
    opened by ahdinosaur 0
  • Rust bug: large integers (beyond JavaScript's Number.MAX_SAFE_INTEGER) should error

    Rust bug: large integers (beyond JavaScript's Number.MAX_SAFE_INTEGER) should error

    any integer above JavaScript's Number.MAX_SAFE_INTEGER should error.

    at the moment, we serialize them as is, which could lead to the following problem:

    JSON.stringify(JSON.parse("[1152921504606846976,2305843009213693952,4611686018427387904]"))
    // '[1152921504606847000,2305843009213694000,4611686018427388000]'
    

    another option is we cast integers as f64, but this is lossy and i'd rather throw an error than silently convert data.

    references:

    • https://github.com/l1h3r/serde_jcs/issues/3
    • https://github.com/cyberphone/json-canonicalization/issues/20
    opened by ahdinosaur 0
  • Rust bug: number keys are not properly handled

    Rust bug: number keys are not properly handled

    failing test:

    macro_rules! treemap {
        () => {
            BTreeMap::new()
        };
        ($($k:expr => $v:expr),+) => {
            {
                let mut m = BTreeMap::new();
                $(
                    m.insert($k, $v);
                )+
                m
            }
        };
    }
    
    #[test]
    fn test_object_with_wacky_keys() {
        #[derive(PartialEq, Eq, PartialOrd, Ord, serde_derive::Serialize)]
        #[serde(untagged)]
        enum Key<'a> {
            Str(&'a str),
            Num(u32),
        }
        let expected = r#"{"\n":"Newline","1":"One","2":"Two","3":"Three","4":"Four"}"#;
        let input = treemap![
            Key::Num(2) => "Two",
            Key::Str("4") => "Four",
            Key::Str("1") => "One",
            Key::Num(3) => "Three",
            Key::Str("\u{000a}") => "Newline"
        ];
        let actual = to_string(&input).unwrap();
        assert_eq!(actual, expected);
    }
    
    failures:
    
    ---- test_object_with_wacky_keys stdout ----
    thread 'test_object_with_wacky_keys' panicked at 'assertion failed: `(left == right)`
      left: `"{\"2\":\"Two\",\"3\":\"Three\",\"\\n\":\"Newline\",\"1\":\"One\",\"4\":\"Four\"}"`,
     right: `"{\"\\n\":\"Newline\",\"1\":\"One\",\"2\":\"Two\",\"3\":\"Three\",\"4\":\"Four\"}"`', 
    

    i'll admit i didn't realize numbers could be keys, but that's because i keep thinking that the Rust is deserialized to serde_json::Value (where keys can only be strings) before being serialized, which is just not the case.

    opened by ahdinosaur 0
  • Fuzz tests (and Rust fixes)

    Fuzz tests (and Rust fixes)

    • add JavaScript json-canon-fuzz fuzzer to output test cases
    • fix bug in how Rust json-canon used original bytes for sorting keys
    • fuzz all the things, everything seems to work
    opened by ahdinosaur 0
  • [rust] object pooling

    [rust] object pooling

    inspired by https://www.reddit.com/r/rust/comments/ozpspr/improved_my_base64_encoders_performance_3x_by/,

    i tried to re-use the many ObjectEntry objects that get created (and their byte Vec<u8>'s) using a pool.

    benchmarks look very promising!

                            time:   [13.003 µs 13.027 µs 13.061 µs]
                            thrpt:  [76.564 Kelem/s 76.761 Kelem/s 76.904 Kelem/s]
                     change:
                            time:   [-39.290% -39.190% -39.087%] (p = 0.00 < 0.05)
                            thrpt:  [+64.169% +64.447% +64.716%]
    
    opened by ahdinosaur 0
Owner
Mikey
solarpunk lifestyle infrastructure
Mikey
Get JSON values quickly - JSON parser for Rust

get json values quickly GJSON is a Rust crate that provides a fast and simple way to get values from a json document. It has features such as one line

Josh Baker 160 Dec 29, 2022
Converts cargo check (and clippy) JSON output to the GitHub Action error format

cargo-action-fmt Takes JSON-formatted cargo check (and cargo clippy) output and formats it for GitHub actions. Examples This tool can be used with a v

Oliver Gould 8 Oct 12, 2022
A rust script to convert a better bibtex json file from Zotero into nice organised notes in Obsidian

Zotero to Obsidian script This is a script that takes a better bibtex JSON file exported by Zotero and generates an organised collection of reference

Sashin Exists 3 Oct 9, 2022
Decode Metaplex mint account metadata into a JSON file.

Simple Metaplex Decoder (WIP) Install From Source Install Rust. curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh Clone the source: git c

Samuel Vanderwaal 8 Aug 25, 2022
CLI tool to convert HOCON into valid JSON or YAML written in Rust.

{hocon:vert} CLI Tool to convert HOCON into valid JSON or YAML. Under normal circumstances this is mostly not needed because hocon configs are parsed

Mathias Oertel 23 Jan 6, 2023
Typify - Compile JSON Schema documents into Rust types.

Typify Compile JSON Schema documents into Rust types. This can be used ... via the macro import_types!("types.json") to generate Rust types directly i

Oxide Computer Company 73 Dec 27, 2022
JSON parser which picks up values directly without performing tokenization in Rust

Pikkr JSON parser which picks up values directly without performing tokenization in Rust Abstract Pikkr is a JSON parser which picks up values directl

Pikkr 615 Dec 29, 2022
Strongly typed JSON library for Rust

Serde JSON   Serde is a framework for serializing and deserializing Rust data structures efficiently and generically. [dependencies] serde_json = "1.0

null 3.6k Jan 5, 2023
JSON implementation in Rust

json-rust Parse and serialize JSON with ease. Changelog - Complete Documentation - Cargo - Repository Why? JSON is a very loose format where anything

Maciej Hirsz 500 Dec 21, 2022
Rust port of gjson,get JSON value by dotpath syntax

A-JSON Read JSON values quickly - Rust JSON Parser change name to AJSON, see issue Inspiration comes from gjson in golang Installation Add it to your

Chen Jiaju 90 Dec 6, 2022
rurl is like curl but with a json configuration file per request

rurl rurl is a curl-like cli tool made in rust, the difference is that it takes its params from a json file so you can have all different requests sav

Bruno Ribeiro da Silva 6 Sep 10, 2022
A tool for outputs semantic difference of json

jsondiff A tool for outputs semantic difference of json. "semantic" means: sort object key before comparison sort array before comparison (optional, b

niboshi 3 Sep 22, 2021
Easily create dynamic css using json notation

jss! This crate provides an easy way to write dynamic css using json notation. This gives you more convenient than you think. Considering using a dyna

Jovansonlee Cesar 7 May 14, 2022
A fast and simple command-line tool for common operations over JSON-lines files

rjp: Rapid JSON-lines processor A fast and simple command-line tool for common operations over JSON-lines files, such as: converting to and from text

Ales Tamchyna 3 Jul 8, 2022
A node package based on jsonschema-rs for performing JSON schema validation

A node package based on jsonschema-rs for performing JSON schema validation.

dxd 49 Dec 18, 2022
Tools for working with Twitter JSON data

Twitter stream user info extractor This project lets you parse JSON data from the Twitter API or other sources to extract some basic user information,

Travis Brown 4 Apr 21, 2022
A easy and declarative way to test JSON input in Rust.

assert_json A easy and declarative way to test JSON input in Rust. assert_json is a Rust macro heavily inspired by serde json macro. Instead of creati

Charles Vandevoorde 8 Dec 5, 2022
A fast way to minify JSON

COMPACTO (work in progress) A fast way to minify JSON. Usage/Examples # Compress # Input example (~0.11 KB) # { # "id": "123", # "name": "Edua

Eduardo Stuart 4 Feb 27, 2022
JSON Schema validation library

A JSON Schema validator implementation. It compiles schema into a validation tree to have validation as fast as possible.

Dmitry Dygalo 308 Dec 30, 2022