Fixes #236. Fixes #95.

Builds on #324, #328 and #333.

Limitations:

• No ability to get spans of dotted tables as well as arrays defined with [[key]] (inline tables as well as [] arrays do work). While we could probably special case tables/arrays that are contiguous, in the general case the toml data format allows settings where tables/arrays are spread over parts of the file. In these cases, we set both start and end of the span to 0, as this is easily detectable and no interesting value would ever have such a span. User code needs to be able to handle these cases.

cc: #225, #222

~~The tuple variant hasn't been implemented yet, I'm having trouble figuring it out.~~ See #225 for ongoing discussion -- what should / should not be supported.

Short, the following TOML:

plain = "Plain"
plain_table = { Plain = {} }
tuple = { Tuple = { 0 = 123, 1 = true } }
struct = { Struct = { value = 123 } }
newtype = { NewType = "value" }
my_enum = [
    { Plain = {} },
    { Tuple = { 0 = 123, 1 = true } },
    { NewType = "value" },
    { Struct = { value = 123 } }
]

with the following types:

#[derive(Debug, Deserialize)]
struct Config {
    plain: MyEnum,
    plain_table: MyEnum,
    tuple: MyEnum,
    #[serde(rename = "struct")]
    structv: MyEnum,
    newtype: MyEnum,
    my_enum: Vec<MyEnum>,
}

#[derive(Debug, Deserialize)]
enum MyEnum {
    Plain,
    Tuple(i64, bool),
    NewType(String),
    Struct { value: i64 },
}

deserializes to:

Config {
    plain: Plain,
    plain_table: Plain,
    tuple: Tuple(
        123,
        true
    ),
    structv: Struct {
        value: 123
    },
    newtype: NewType(
        "value"
    ),
    my_enum: [
        Plain,
        Tuple(
            123,
            true
        ),
        NewType(
            "value"
        ),
        Struct {
            value: 123
        }
    ]
}

The use case I am trying to meet is the ability to configure toml to use tripple quotes (''') when useful (such as when there are any newlines.

Here is some example code:

extern crate rustc_serialize;
extern crate toml;

use toml::{encode_str};

#[derive(RustcEncodable)]
struct MyStruct { foo: isize, bar: String }

const LONG_TEXT: &'static str = r#"
this is some text
here is some more
yay
"#;

fn main() {
    let my_struct = MyStruct { foo: 4, bar: LONG_TEXT.to_string() };
    println!("{}", encode_str(&my_struct));
}

This currently prints:

bar = "\nthis is some text\nhere is some more\nyay\n"
foo = 4

I would like for it to be able to print:

bar = '''
this is some text
here is some more
yay
'''
foo = 4

I personally no longer have the time or energy to maintain this crate. I believe this is still pretty widely used as the default TOML implementation for Rust, however, so it'd be good to keep this maintained. I'm opening this to see if there is another who is willing to take over ownership and maintenance of this crate. In the meantime I will not be maintaining this crate other than if a security issue arises.

This is to demonstrate the tripple quote problem I theorized: that if there are ''' in a pretty string there will be issues. However, the issue I hit was not as expected.

The written test prints the following:

EXPECTED:                                              
[example]                                            
text = '''                                          
this is the first line                                                                                   
This has a ''\' in it for no reason                                                                      
this is the third line                               
'''                                                 


RESULT:                                                                                                  
[example]                                           
text = '''                                                                                               
this is the first line                              
This has a ''\\' in it for no reason                
this is the third line                              
''' 

It seems that the round trip is doubling the number of \ it finds?

This is a very rough implementation of round-tripping TOML parser. It correctly round trips all the TOML test suite (except inline arrays). It's very unsuitable for merging at the moment (broken formatting, random layout of modules, unimplemented inline arrays, no public API etc.). Since it's a very invasive change, that make internals more complicated (for no real gain if you just want to simply serialize and deserialize), my question is, should I work on getting it into toml-rs or move this work to a fork (with just API for load-edit-save of TOML documents, leaving attribute-based serializing/deserializing here)?

Hey,

I have a use-case where I perform higher-level diagnostics on top of existing structures, and in order to provide nice errors to the user I need to have access to the span information where a given structure was deserialized from.

Following is a somewhat naive proposal, since I'm not too familiar with the low-level details of how to implement a deserializer for serde.

One option could be to include spans in the tokens returned by the tokenizer, then introduce a generic wrapper (e.g. Spanned<T>) that includes the spans when de-serializing so that they can be used in later errors.

struct Span {
  /// Byte offset where the span starts.
  pub start: usize,
  /// Byte offset where the span ends.
  pub end: usize,
}

struct Spanned<T> {
  pub span: Span,
  pub value: T,
}

#[derive(Deserialize)]
pub struct Foo {
  bar: Spanned<String>,
}

For this to work with toml::Value, the spans would have to be included here, which might exclude it for backwards compatibility. In that case a new set of structures (e.g. toml::SpannedValue) could be introduced.

I created this website using toml-rs and wasm to convert json to toml: https://pseitz.github.io/toml-to-json-online-converter/

The code is pretty straightforward, but sometimes I get the ValueAfterTable error. I have no idea how to fix this, since the input is generic. (Reorder the serde_json::Value?)

#[wasm_bindgen]
pub fn convert_json_to_toml(input:&str) -> Result<String, JsValue>{
 
    let mut toml = String::new();

    let mut deserializer = Deserializer::from_str(input);
    let mut serializer = Serializer::pretty(&mut toml);
    serde_transcode::transcode(&mut deserializer, &mut serializer).map_err(|err| JsValue::from_str(&err.to_string()))?;
  
    Ok(toml)
}

JSON Input: { "kana": {"ent_seq": "1000000"}, "kanji": [] }

See #360.

The second commit exposes de::ErrorKind publicly, if this is not desired I can remove it. It should be fine to expose since it is already marked as non_exhaustive, using #[doc(hidden)].

On April 1 a new version of TOML was published: https://github.com/toml-lang/toml/blob/master/CHANGELOG.md#100-rc1--2020-04-01

The changelog mentions the following changes:

• [x] Clarify in ABNF how quotes in multi-line basic and multi-line literal strings are allowed to be used. (PR in #393)
• [x] Leading zeroes in exponent parts of floats are permitted.
• [x] Clarify that control characters are not permitted in comments.
• [ ] Clarify behavior of tables defined implicitly by dotted keys.
  • [x] spec text:

    Dotted keys define everything to the left of each dot as a table. Since tables cannot be defined more than once, redefining such tables using a [table] header is not allowed. Likewise, using dotted keys to redefine tables already defined in [table] form is not allowed.

  • [ ] spec text (example file):

    The [table] form can, however, be used to define sub-tables within tables defined via dotted keys.

• [x] Clarify that inline tables are immutable.
• [x] Clarify that trailing commas are not allowed in inline tables.
• [x] Clarify in ABNF that UTF-16 surrogate code points (U+D800 - U+DFFF) are not allowed in strings or comments. (ensured by Rust)
• [x] Allow raw tab characters in basic strings and multi-line basic strings.
• [x] Allow heterogenous values in arrays. (was changed in #358)

TOML is getting official compliance tests. The repository contains no actual tests yet but a script to run the compliance tests eventually. I think the best way to reach v1.0 and to avoid bugs is to use the test suite to check both deserialization and serialization.

While this crate already contains a test runner for an almost identical format it may be good to interface with the default Python test runner. If many implementations use the default runner it becomes easy to monitor the correctness of many different implementations.

• [ ] Write script implementing the decoder interface (a serializer for the JSON format already exists) https://github.com/alexcrichton/toml-rs/blob/7a43ac9e9c28afec8414f6d924024f5d33fe8fa2/test-suite/tests/valid.rs#L9-L54
• [ ] Write script implementing the encoder interface
• [ ] Optional: Remove rust test runner (upside: less code, downside: Python is needed to test this crate)

Additionally this is a good opportunity to split the Datetime as suggested in https://github.com/alexcrichton/toml-rs/issues/159#issuecomment-463732912 by @quadrupleslap. The TOML spec and tests make it very clear that these are indeed distinct types. I think it will be easier to use those values if one can be sure that a table contains the correct kind of date or time and does not have to check the value.

The four new types each has a corresponding chrono type:

• DateTime ⟷ chrono::DateTime
• LocalDateTime ⟷ chrono::NaiveDateTime
• LocalDate ⟷ chrono::NaiveDate
• LocalTime ⟷ chrono::NaiveTime

The TOML types can be converted with try_from to chrono types. An error is returned if the date does not exist (e. g. 2019-02-29). In the other direction from is used as no error can occur.

• [ ] Split Datetime
• [ ] Implement conversions to chrono types
• [ ] Implement conversions from chrono types

I'm attempting to read a large-ish TOML document like so:

println!("Reading toml size {}", s.len());
let t = toml::from_str(s).unwrap();

where the str size is around 4MB (this is an auto-generated TOML, obviously). The file is essentially a lot of very small tables in the following format:

[table-id-1]
name = "Foo"
url = "http://example.com"
tags = ["tag-name"]

The loading time is unreasonable, spinning the CPU up to 100% and taking way too long, over a minute before I kill the process.

Am I wrong to expect this library to be able to handle files that big?