Hello,

I'm working on an xmpp library and I need to be able to reopen the parser to consume new bytes.

It's a first step to be able to reset some condition, I use it like that:

loop {
    // My parser is an instance of EventReader<Buffer>,
    // Buffer is a custom struct that hold my buffer and expose some useful methods.
    // It implement Read 
    if self.parser.source().available_data() > 0 {
        self.parser.reopen_parser();
    }
    match self.parser.next() { ... }
}

What do you think?

Signed-off-by: Freyskeyd [email protected]

i took the usage example from the project main page and apply it to my 3 MB xml file with this result:

python: 00.968 sec (CPython/xml.etree, i have Java parser that are ~10x faster that this one) rust: 04.310 sec (without prints traversing only!)

• Win7 64 x
• cargo build --release (rust 1.1)

xml-rs fails to parse XML that contains /> as part of it's body.

Per https://www.w3.org/TR/REC-xml/#syntax

The ampersand character (&) and the left angle bracket (<) must not appear in their literal form, except when used as markup delimiters, or within a comment, a processing instruction, or a CDATA section. If they are needed elsewhere, they must be escaped using either numeric character references or the strings " & " and " < " respectively. The right angle bracket (>) may be represented using the string " > ", and must, for compatibility, be escaped using either " > " or a character reference when it appears in the string " ]]> " in content, when that string is not marking the end of a CDATA section.

It seems like this should be allowed, as only the & and < are reserved and must be escaped. > is just a may be escaped. As an example in xml-rs, <b>></b> is valid XML, just <b>/></b> is invalid XML.

Minimally reproducible example:

cargo.toml

[package]
name = "xml-minimal-error"
version = "0.1.0"
authors = ["Jeff LaJoie <[email protected]>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
xml-rs="0.8"

main.rs

use xml::reader::ParserConfig;

fn main() {
    let xml_str: &[u8] = b"<b>/></b>";
    let parser_config = ParserConfig::default();
    let parser = parser_config.create_reader(xml_str);

    for event in parser {
        match event {
            Err(e) => {
                println!("Error = {:?}", e);
            }
            _ => {},
        }
    }
}

Output

➜  xml-minimal-error git:(master) ✗ cargo run
   Compiling xml-minimal-error v0.1.0 (/Users/jlajoie/workspace/xml-minimal-error)
    Finished dev [unoptimized + debuginfo] target(s) in 0.23s
     Running `target/debug/xml-minimal-error`
Error = Error { pos: 1:4, kind: Syntax("Unexpected token: />") }

Additionally, this example does work in other language XML parsers.

NodeJS Example

package.json

{
  "name": "xml-minimal",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "fast-xml-parser": "^3.19.0"
  }
}

index.js

const parser = require('fast-xml-parser');

let jsonObj = parser.parse("<b>/></b>");

console.dir(jsonObj);

output

➜  node-xml-minimal node index.js
{ b: '/>' }

While parsing a xml document it would be nice to know what the next event in the reader will be so I can dispatch it to the correct parser.

Another way of doing this would be getting the current event (i.e. not having to call next to get an event).

It would be handy if EventWriter implemented a flush() method, that would just flush the underlying sink.

My use-case is a long-running measurement application that occasionally saves a complex XML element into a file backed by a BufWriter. I'd like to flush the writer each time I save a new element so I have the relevant info stored in case of failure.

Looks like xml-rs stopped compiling with the nightly compiler from 2017-05-21. Nightly build from 2017-05-20 still worked.

error: no rules expected the token `flags`
   --> /home/pg/.cargo/registry/src/github.com-1ecc6299db9ec823/xml-rs-0.1.26/src/writer/emitter.rs:115:5
    |
115 |     flags IndentFlags: u8 {
    |     ^^^^^

error: Could not compile `xml-rs`.

When a document is received in chunks (on XMPP for example), it makes sense to initialize the parser on the first chunk, and then feed it data as it comes.

xml::ParserConfig would get a new streaming boolean that would make it never emit xml::XmlEvent::EndDocument before the root tag is closed, a feed method to xml::EventReader, taking a string and emitting newer xml::XmlEvents as they are parsed, and finally add a method to abort the stream.

With these tests enabled people can hack on the parser more confidently. A drawback of using include_bytes! is that cargo doesn't notice changes to any included files (this might just be a cargo bug).

Getting this error as of rust nightly 12-23:

src/namespace.rs:2:5: 2:23 error: unresolved import `core::slice::Items`. There is no `Items` in `core::slice`
src/namespace.rs:2 use core::slice::Items;
                       ^~~~~~~~~~~~~~~~~~

Changed as of this commit: https://github.com/rust-lang/rust/commit/f8cfd2480b69a1cc266fc91d0b60c825a9dc18a7#diff-91f9d2237c7851d61911b0ca64792a88

This would be very helpful to construct error messages when figuring out where in a multi-megabyte XML monstrosity a parser goes off the tracks. It looks like it should be a relatively straightforward change too.

I was playing around with xml-rs in my own project (https://github.com/jdalberg/cwmp) and decided to use QuickCheck for validation. So a bunch of randomness into fields, generate some xml, parse it and compare with input with different permutation of content, and came across what I consider to be a bug in the parser. It would not recoqnice: "?>" as characters.

Looking into xml-rs i could see that in line 127 of reader/lexer.rs the token for "?>" was missing from the function, looking at issue #32 i introduced two new testcases and the two tokens for "]]>" and "?>" in the list of token that are possibly characters.

EventReader never return Result::Err after document end. It returns Ok(EndDocument) over and over instead. It does not depend on the flag ignore_end_of_stream.

So, next code stuck in an endless loop:

use std::io::{BufReader, Cursor};
use xml::{EventReader, ParserConfig};


fn main() {

    let content = "<a></a>";

    let reader = BufReader::new(Cursor::new(content));
    let mut parser = EventReader::new_with_config(reader, ParserConfig::new().ignore_end_of_stream(true /* or false  */) );

    loop {
        if let Err(_) = parser.next() {
            break;
        }
    }
}

The tricky case of <!-- <!--> should be parsed as a single comment, ignoring <! in the comment. xml-rs parses this as two unclosed comments.

Opening this to ask whether that's been considered (a search didn't turn up anything and discussions are not enabled): ParserConfig allows some fairly extensive customisation to the emittable XmlEvent variants, down to 6 (and up to 9, from a default of 8 if I'm reading everything right).

However currently the user still has to "deal" with the un-emittable variants.

Granted most applications will have a single loop processing the input events, but still, would a more type-heavy interface be an option?

Sadly Rust still has no polymorphic or type-based variants, so the syntactic overhead would be fairly large, but it would also be fairly simple code, just annoying to write.