First of all: really nice work! I am looking for a Rust crate to parse a PDF and extract the content from it. For example, to extract each text line and metadata from the first page, to get which font style and/or font family belongs to the given line, etc.

I am asking if this crate supports a good depth of extraction (for example, if the crate supports already font family and font size extraction for a single line, or word, things like that...). Can you give me some information about that please?

Thanks a lot

I've done a bit of comparision benchmarking for extracting URLs from PDF files.

Test file: PDF 1.7 specification https://www.adobe.com/content/dam/acom/en/devnet/pdf/pdfs/PDF32000_2008.pdf

(This file is encrypted with an empty password, that is another feature I would like to bring to lopdf. My code using lopdf cannot yet unscramble strings.)

Using PyPDF2, the process takes 2.7s while just loading the file takes 43s using lopdf. PyPDF2 has some stability / looping issues that are a no-go for me. I would like to improve lopdf performance by a lot.

On load, most of the time is spent in the pom parser in parser.rs.

I see three approches that would be workable:

1. Make the parser a LOT faster somehow. This keeps the current API and objects can be accessed immutably in parallel code.
2. Do on demand parsing of objects. Somewhat complex, can introduce mutability issues, might break the API.
3. Parallelize the parsing using rayon. Would bring a performance gain, but would still consume the same amount of CPU time. Are you open to adding rayon as a dependency ?

From a design perspective, what would be the approaches that make the most sense to you ? I have some time to spend on improving lopdf over the coming weeks.

Hi. I am trying to execute that example todged from [1]

use lopdf::Document;
use lopdf::xobject;

fn main() {
    let mut document = Document::load("example.pdf")?;
    document.version = "1.4".to_string();
    
    // If the stream is loaded correctly
    if let Ok(stream) = xobject::image("image.png") {
        // we need a "page_id", the position coordinates and the size of the image
        document.insert_image(page_id, stream, (50.0, 100.0), (800.0, 500.0));
    }
    
    document.save("image.pdf")?;
}

but I get this error (that image does not exist in xobject):

if let Ok(stream) = lopdf::xobject::image("image.png") {
   |                                       ^^^^^ not found in `lopdf::xobject`

Inspecting the code I see that if it exists [2].

Someone could give me a help with this

Jose

[1] https://www.dariocancelliere.it/blog/2020/09/29/pdf-manipulation-with-rust-and-considerations [2] https://github.com/J-F-Liu/lopdf/blob/fec4729625312da00243b58db2169168fe4ed65d/src/xobject.rs#L29

When I try to read this document: http://mirror.hmc.edu/ctan/macros/latex/contrib/iwhdp/Back_2015.pdf I get:

Not a valid PDF file (prev xref_and_trailer). Mismatch { message: "expect repeat at least 1 times, found 0 times", position: 3117 }

So I was updating and trying out 0.28.0 of lopdf the newest update and noticed when i merge a ton of pdf's together that I use to merge with 0.27 that it now is not a workable PDF. IDK exactly what caused the issue but i can try and look at the changes to see what it could be. But this is a major problem if it can no longer merge adobe PDF files together.

I also saw a noticeable size difference between the PDFs as well will diff them to see what might of changed. old 27,448kb new 27,433kb

While working on tests for rsvg-convert (a tool that's built with librsvg), I found several issues with the Object::as_datetime() implementation. Maybe I'm just using this incorrectly ot it's some other misunderstanding, But I would like to raise the issues here.

Timezone handling

The implementation uses chrono::Local.datetime_from_str(). This method will return an error (ParseError::Impossible)) if the timezone offset in the string doesn't match the local timezone offset. So while this may work for PDF files that are created in the local timezone, it is likely going to fail quite often.

The chrono crate offers another method which is DateTime::parse_from_str(): https://docs.rs/chrono/0.3.0/chrono/datetime/struct.DateTime.html#method.parse_from_str This method seems more appropriate as it can handle different timezone offsets.

In my opinion it would also make sense to consider changing the return value of Object::as_datetime() to return a DateTime in the UTC offset instead of using Local.

Parsing of incomplete dates

I've run into problems because the PDF I tested did not specify the minutes of the timezone offset. So the CreationDate string looked like this:

D:20200211085039+00'

Instead of the proper

D:20200211085039+00'00'

While this was due to a bug in the library that created the PDF, it still seems valid according to the PDF spec. According to the spec all fields after the year are optional. However the code in Object::as_datetime() will raise ParseError(TooShort) unless the complete datetime string is given. So I think the parser should be changed to deal gracefully with the optional fields missing.

Hi, I made a completely new project and copied the example code.

I had to switch to rust nightly because #![feature(field_init_shorthand)] is not allowed on stable. I compiled and ran the get started guide. What I got was a corrupted PDF (I'll attach it if I can).

I do not know why the PDF is corrupted, sadly. It would be nice if someone could look into it.

Uploading example.pdf.txt…

In this PR I updated all libraries, applied various code improvements, fixed tests in case of non enabled chrono feature and done some enrichments on the embed_image feature, now supports buffers.

I also added an useful function for the Document struct called get_object_page(), this function will return the Page ObjectId if found, useful for inserting certain element of another dependent ObjectId and another useful remove_object() function that is able to correctly remove an object from the page and the reference.

I also formatted the code with cargo fmt in order to be compliant to all Rust 2018 rules.

@J-F-Liu please wait before merging this, I'll to check if the embed_image is working properly because I think that is a little buggy. Maybe is not well tested as is not a default feature :-)

I noticed this while writing the nom parser.

This is to make sure this code works in reader.rs:

if let Some(start) = stream.start_position {
    let end = start + length as usize;
    stream.set_content(self.buffer[start..end].to_vec());
}

Another panic for http://ctan.math.washington.edu/tex-archive/macros/latex/contrib/bg/description.pdf

thread 'main' panicked at 'called Result::unwrap() on an Err value: FromUtf8Error { bytes: [139], error: Utf8Error { valid_up_to: 0, error_len: Some(1) } }', libcore/result.rs:945:5

The chrono crate is currently only used for providing a few impls for parsing PDF date objects. However, chrono is a pretty heavy dependency if it is only used for printing the date. The time crate (which chrono itself depends on anyways) is more than enough to do that job (and if it's absolutely necessary to parse date formats, this can also be done with external libraries, there is no need for lopdf to depend on chrono). chrono, however, brings in num-traits, num-*, etc. - very generics-heavy crates that aren't strictly needed for lopdf to work.

Compile time of lopdf:

• chrono on: 19.87s
• chrono off: 15.57s

By default this feature is on, to not break any code, being backwards-compatible. However, since in theory someone could have default-features = false in his Cargo.toml, this is still a "breaking change" in terms of semver.

The only drawback is that with time parsing, the weekday and year day isn't preserved / calculated (strptime doesn't auto-calculate those fields). But for crates that don't need this behaviour, this can be a significant improvement in terms of compile time.

This PR brings the dependency count from 30 to 19 dependencies (11 dependencies removed). If this is merged, I'd be happy if a new version would be released.

I have recreated the the existing example.pdf file with Word, i.e. I created a simple Word file with the text "Hello World!" then exported it as a PDF file. When trying to replace the text in this PDF (as per the provided example), nothing happens. It looks like the "Hello World!" text is not even there.

What would be a proper way to replace the text in a Word-generated PDF?

fn shrink_file(path: &str) {
  let doc = Document::load(path);
  match doc {
    Ok(mut pdf) => {
      pdf.compress();
      pdf.save(format!("{path}__compressed")).unwrap();
    },
    Err(error) => {
      panic!("error {}", error.to_string())
    }
  };
  
}

Output

thread 'main' panicked at 'error Invalid cross-reference table (could not parse xref)', src/main.rs:36:7
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 ELIFECYCLE  Command failed.
 ELIFECYCLE  Command failed with exit code 101.

# parser.rs:304

pub fn content(input: &[u8]) -> Option<Content<Vec<Operation>>> {
    (content_space() * operation().repeat(0..).map(|operations| Content { operations }))
        .parse(input)
        .ok()
}

Returns only Some(Content { operations: [Operation { operator: "q", operands: [] }] }) for the attached sample PDF. It seems like it's not able to parse all the content correctly?

Payslip.pdf

We should fix the documentiation lopdf/pdfutil/README.md to and and some exemples
USAGE: pdfutil [OPTIONS] [SUBCOMMAND]

FLAGS: -h, --help Prints help information -V, --version Prints version information

OPTIONS: -i, --input
-o, --output

SUBCOMMANDS: compress Compress PDF document decompress Decompress PDF document delete_objects Delete objects delete_pages Delete pages delete_zero_length_streams Delete zero length stream objects extract_pages Extract pages extract_stream Extract stream content extract_text Extract text help Prints this message or the help of the given subcommand(s) print_streams Print streams process Process PDF document with specified operations prune_objects Prune unused objects renumber_objects Renumber objects replace_text Replace text