Seeking an optimized DCT implementation with a compatible license.

Options:

• RustDCT Needs benchmarking as current API supports 1D slices only so it needs a transpose step that the in-tree impl doesn't. (ejmahler/rust_dct#2)

Dubious:

• FFTW; licensed under GPL, not compatible with MIT Can provide interface to link in like old UserDCT implementation but composable with the new 3.0-alpha API.

cc @ejmahler

I ran into a situation where I tried to push in a hash of a zero-length value, and instead of erroring, it paniced! This PR fixes the problem, but my Rust is still not very good so I haven't figured out how to write the test correctly. If anyone has any thoughts about how to fix the commented out code, that'd be much appreciated.

This deactivates all unnecessary features of the image dependency. The image dependency usually pulls in a lot of indirect dependencies that you possibly don't want.

In particular this causes issues in my case, because it pulls in rayon, which doesn't work in WebAssembly.

This allows img_hash to be used by projects that depend on image 0.21. Sadly 0.20 is breaking so this change isn't reverse compatible and would require a semver bump.

This patch is relevant for me because I depend on another crate that requires image 0.21 and as far as I can tell this is the only way to use both at once.

Thanks!

works:

let img = image::load_from_memory(&mut buf).unwrap();
let hash = ImageHash::hash(&img, 8, HashType::DCT);

doesn't work:

let img = image::load_from_memory(&mut buf).unwrap();
let hash = ImageHash::hash(&img, 8, HashType::Block);

thread 'main' panicked at 'index out of bounds: the len is 64 but the index is 64', /checkout/src/libcore/slice/mod.rs:2085:14

versions: rustc: 1.28-nightly image = "0.13" # had to peg this to an older version to get it to work

[dependencies.img_hash] version = "^2.0.1" features = ["rust-image"]

stack trace:

thread 'main' panicked at 'index out of bounds: the len is 64 but the index is 64', /checkout/src/libcore/slice/mod.rs:2085:14
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::print
             at libstd/sys_common/backtrace.rs:71
             at libstd/sys_common/backtrace.rs:59
   2: std::panicking::default_hook::{{closure}}
             at libstd/panicking.rs:211
   3: std::panicking::default_hook
             at libstd/panicking.rs:227
   4: std::panicking::rust_panic_with_hook
             at libstd/panicking.rs:511
   5: std::panicking::continue_panic_fmt
             at libstd/panicking.rs:426
   6: rust_begin_unwind
             at libstd/panicking.rs:337
   7: core::panicking::panic_fmt
             at libcore/panicking.rs:92
   8: core::panicking::panic_bounds_check
             at libcore/panicking.rs:60
   9: <usize as core::slice::SliceIndex<[T]>>::index_mut
             at /checkout/src/libcore/slice/mod.rs:2085
  10: core::slice::<impl core::ops::index::IndexMut<I> for [T]>::index_mut
             at /checkout/src/libcore/slice/mod.rs:1958
  11: <alloc::vec::Vec<T> as core::ops::index::IndexMut<I>>::index_mut
             at /checkout/src/liballoc/vec.rs:1717
  12: img_hash::block::blockhash_slow::{{closure}}
             at /home/~/img_hash-2.0.1/src/block.rs:99
  13: img_hash::rust_image::<impl img_hash::HashImage for image::dynimage::DynamicImage>::foreach_pixel
             at /home/~/img_hash-2.0.1/src/rust_image.rs:89
  14: img_hash::block::blockhash_slow
             at /home/~/img_hash-2.0.1/src/block.rs:72
  15: img_hash::block::blockhash
             at /home/~/img_hash-2.0.1/src/block.rs:29
  16: img_hash::HashType::hash
             at /home/~/img_hash-2.0.1/src/lib.rs:279
  17: img_hash::ImageHash::hash
             at /home/~/img_hash-2.0.1/src/lib.rs:85
  18: app::event
             at src/main.rs:63
  19: nannou::run_loop::process_and_emit_glutin_event
             at /home/~/nannou-0.6.0/src/lib.rs:368
  20: nannou::run_loop
             at /home/~/nannou-0.6.0/src/lib.rs:412
  21: <nannou::Builder<M, E>>::run
             at /home/~/nannou-0.6.0/src/lib.rs:147
  22: app::main
             at src/main.rs:14
  23: std::rt::lang_start::{{closure}}
             at /checkout/src/libstd/rt.rs:74
  24: std::panicking::try::do_call
             at libstd/rt.rs:59
             at libstd/panicking.rs:310
  25: __rust_maybe_catch_panic
             at libpanic_unwind/lib.rs:105
  26: std::rt::lang_start_internal
             at libstd/panicking.rs:289
             at libstd/panic.rs:392
             at libstd/rt.rs:58
  27: std::rt::lang_start
             at /checkout/src/libstd/rt.rs:74
  28: main
  29: __libc_start_main
  30: _start

I'm trying to run the example code and was unable to circumvent this error: "the trait img_hash::HashImage is not implemented for image::DynamicImage". Could you look into this??

I've been testing this library with this simple script, but I can't figure out why the produced hash has an invalid size (here 56 instead of 64 bits). The size is also incorrect for DoubleGradient with 135 instead of 128 bits. When using the Mean, Block or DCT hashtype, the resulting hash has the correct length (64 bits). (I'm a novice in Rust, so I might have done something wrong.)

extern crate image;
extern crate img_hash;

use std::path::Path;
use img_hash::{ImageHash, HashType};

fn main() {
    let img = image::open(&Path::new("00002701.jpg")).unwrap();
    let hash = ImageHash::hash(&img, 8, HashType::Gradient);
    println!("{:?}", hash.bitv);
    println!("{}", hash.size());
}

(The image is just a random picture, I am only looking at the hash size here.)

This issue was automatically generated. Feel free to close without ceremony if you do not agree with re-licensing or if it is not possible for other reasons. Respond to @cmr with any questions or concerns, or pop over to #rust-offtopic on IRC to discuss.

You're receiving this because someone (perhaps the project maintainer) published a crates.io package with the license as "MIT" xor "Apache-2.0" and the repository field pointing here.

TL;DR the Rust ecosystem is largely Apache-2.0. Being available under that license is good for interoperation. The MIT license as an add-on can be nice for GPLv2 projects to use your code.

Why?

The MIT license requires reproducing countless copies of the same copyright header with different names in the copyright field, for every MIT library in use. The Apache license does not have this drawback. However, this is not the primary motivation for me creating these issues. The Apache license also has protections from patent trolls and an explicit contribution licensing clause. However, the Apache license is incompatible with GPLv2. This is why Rust is dual-licensed as MIT/Apache (the "primary" license being Apache, MIT only for GPLv2 compat), and doing so would be wise for this project. This also makes this crate suitable for inclusion and unrestricted sharing in the Rust standard distribution and other projects using dual MIT/Apache, such as my personal ulterior motive, the Robigalia project.

Some ask, "Does this really apply to binary redistributions? Does MIT really require reproducing the whole thing?" I'm not a lawyer, and I can't give legal advice, but some Google Android apps include open source attributions using this interpretation. Others also agree with it. But, again, the copyright notice redistribution is not the primary motivation for the dual-licensing. It's stronger protections to licensees and better interoperation with the wider Rust ecosystem.

How?

To do this, get explicit approval from each contributor of copyrightable work (as not all contributions qualify for copyright, due to not being a "creative work", e.g. a typo fix) and then add the following to your README:

## License

Licensed under either of

 * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
 * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)

at your option.

### Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any
additional terms or conditions.

and in your license headers, if you have them, use the following boilerplate (based on that used in Rust):

// Copyright 2016 img_hash developers
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.

It's commonly asked whether license headers are required. I'm not comfortable making an official recommendation either way, but the Apache license recommends it in their appendix on how to use the license.

Be sure to add the relevant LICENSE-{MIT,APACHE} files. You can copy these from the Rust repo for a plain-text version.

And don't forget to update the license metadata in your Cargo.toml to:

license = "MIT/Apache-2.0"

I'll be going through projects which agree to be relicensed and have approval by the necessary contributors and doing this changes, so feel free to leave the heavy lifting to me!

Contributor checkoff

To agree to relicensing, comment with :

I license past and future contributions under the dual MIT/Apache-2.0 license, allowing licensees to chose either at their option.

Or, if you're a contributor, you can check the box in this repo next to your name. My scripts will pick this exact phrase up and check your checkbox, but I'll come through and manually review this issue later as well.

• [x] @cybergeek94
• [x] @forgemo
• [x] @talevy
• [x] @palfrey

• Introduced method 'from_base64(...)' for deserialization
• Removed the field 'size' from ImageHash
• Renamed hash_size() to size()
• Replaced all occurences of uint with usize (uint has become deprecated)

The crate has been completely revamped with a new, much more Rust-y API.

New Features

• Serde support (finally)
• Support for inline storage for hashes with generic arrays
• DCT and Difference-of-Gaussians preprocessing for all hash types
• Hash sizes are rectangular now to allow capturing more detail in one dimension or the other

Additionally, DCT is implemented by the RustDCT crate now which enables the following:

• Precomputing of DCT coefficients is now automatic and per-instance of HasherConfig instead of user-configured per-thread
• DCT is implemented in terms of FFTs now and receives roughly an order of magnitude speedup (need to add benches back to show this)

Breaking Changes

• The old API is completely gone
• Requirement for the image crate now starts at 0.21 (might be able to loosen this)
• Minimum supported Rust version is increased to at least 1.26 (need to establish this exactly)

Getting error:

warning: using multiple versions of crate `image`
src/lib.rs:11:1: 11:27 note: used here
src/lib.rs:11 extern crate gfx_graphics;
              ^~~~~~~~~~~~~~~~~~~~~~~~~~
note: crate name: image
src/lib.rs:37:1: 37:41 note: used here
src/lib.rs:37 extern crate "img_hash" as img_hash_lib;
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: crate name: image

Since the latest version of image-rs library require Rust 1.56.0, I think that img_hash also should use Rust 2021 edition and bump minimal requirements.

For now I can't use image-rs 0.24.0, because img_hash crate cannot interoperate with this version and I have such errors(this works with 0.23)

error[E0277]: the trait bound `image::DynamicImage: Image` is not satisfied
   --> czkawka_core/src/similar_images.rs:606:50
    |
606 |                     let hash = hasher.hash_image(&image);
    |                                       ---------- ^^^^^^ the trait `Image` is not implemented for `image::DynamicImage`
    |                                       |
    |                                       required by a bound introduced by this call

Since it is fairly costly to calculate a hash, it makes sense to serialize an deserialize the ImageHash struct. To do this, the struct should derive Serialize and Deserialize from the serde crate.

When going for a 128 byte image hash, I noticed that the size wasn't actually implemented, even though the doc comment said as such. So, instead of just adding more lengths I tried to just switch it to min_const_generics instead since its been on stable for a while now.

If you prefer not to make the MSRV so high I can drop this commit and just add the extra array implementations.

I noticed some of these libraries are pretty old, and so I set out to update them (and, you know, migrate to the 2018 edition). In so doing, I also realized that the rustdct library kind of changed its API on you, apparently just to make life a little more fun. As such, my most important changes (read: the ones I understand the least) are in dct_2d(), on line 44 of dct.rs. The rest of this is pretty much 100% just me screwing with formatting and implementing advice from cargo clippy.

Anyway, here it is if you want it.

A note: at least after I did this, I did not notice any major difference in performance between dct and non-dct hashes (at least in my tiny use case). Of course, I didn't check to see what difference there was beforehand, and I don't know if the code even works correctly (although it seems to be producing valid hashes for me). There's no telling what, if anything, that indicates. That said, I did completely remove use of transpose from that code path. /shrug