I am consistently getting VMAF results that are 20 to 30 less than if I ran VMAF or libvmaf directly on the same files. This is true for all of the commands, i.e., trying to use crf-search or auto-encode won't work correctly because it can't find the right vmaf (unless i set minimum <70).

Example results: ab-av1:

Z:\Files\Rips\Muxing>ab-av1 vmaf --reference "source_snip.mkv" --distorted "RF22_snip.mkv" --vmaf model="path=vmaf_v0.6.1.json"
(vmaf 29 fps, eta 0s)67.469444

libvmaf:

Z:\Files\Rips\Muxing>ffmpeg -i "RF22_snip.mkv" -i "source_snip.mkv" -lavfi libvmaf="n_threads=6" -f null -
[Parsed_libvmaf_0 @ 0000020d93a4bf80] VMAF score: 93.391178

Not sure if this might be a versioning thing or if I can resolve?

Running on windows vm at the moment with:

• ab-av1 0.4.3
• ffmpeg 2022-10-30-git-ed5a438f05-full_build
• vmaf 2.3.1
• svt-av1 v1.3.0
• opus 0.2-3-gf5f571b (libopus 1.3)

When calculating the predicted encode size percent we compare the sample file size sum to their encoded counterparts. This works fairly well since the samples & encoded samples contain only a video stream.

However, there may also be some container overhead for each sample that introduces inaccuracy. So we could instead measure the lossless sample video stream size sum and compare that to the encoded video streams specifically avoiding any container overhead on each side. Also see discussion in #79.

Measuring video streams a little harder than measuring file sizes and may be a bit less reliable too as we have to trust what ffmpeg tells us. It also seems to be in "kB" precision only, I'm not sure if we can configure that or not.

My current feeling is that this won't significantly improve the sample-encode video stream size predictions over the noise of using sample sizes to deduce the total predicted stream size. So to begin with we should gather some test cases where calculating the samples vs encoded samples using video stream instead of full sample size would be more accurate over many different sample counts.

I'm quite confused about the encoded size estimations I'm getting for my videos with crf-search. I understand they are just predictions based on a limited number of samples, but they don't get near the real encoded size of my files, sometimes not by much, but many times by a huge margin, so I've been trying to understand how it works and where the inaccuracy comes from. At first, I thought it could be caused by differences in encoding parameters or something like this, but now I've been noticing it depends more on how the source file is; how many tracks it has and what percentage of the whole file size corresponds to the video track or to other tracks.

Correct me if I'm wrong, but I'm assuming ab-av1 is comparing the encoding efficiency/percentage between the lossless samples and the encoded samples, and then applying that resulting percentage to the input source file size to get the final predicted encoding size.

For example, when running crf-search, this is what ab-av1 is predicting for one of my full bluray files:

crf 21 VMAF 95.24 predicted full encode size 5.26 GiB (18%) taking 2 hours

The source file is 29,2GB, so the 18% of that results in 5,256MB, as predicted.

The problem I'm finding is that about 10GB out of those 29,2GB are multiple tracks of audio, and the video track is only 19,1GB, so the prediction would be inaccurate, as the audio tracks are not going to be encoded the same way as the video track, which is what actually ab-av1 is calculating. Besides many of those tracks would probably be discarded, and others could be re-encoded, sometimes the same language comes in 2 different audio codecs and quality in 2 tracks, and so on; there is no way to know all of that for every source file and every use case.

My point is that the total size of the final file completely depends on what the user decides to do with the other tracks, but ab-av1 cannot predict that, so I think it should focus on providing the information of what it can actually predict, which is what is actually being calculated with the video encoders and presets; the estimated size of the encoded video track only. Otherwise it gets confusing, it is not clear what that estimation represents, not the whole file size nor only the video track size, at least as it is right now if I'm not mistaken.

So, continuing with the example, I believe the more realistic accurate prediction of the encode size would be to calculate the 18% of the video track only (19,1GB), which results in 3,44GB. Then if I decided to keep all audio tracks untouched, the final file would weight around 13,5GB, far from the currently predicted 5,26GB, or if I decided to select only one language, depending on the quality of that track, it would result in a file around 4GB, also not close to the current prediction. Users can take that into account and consider ab-av1's prediction as a baseline only for the video track, which is quite helpful already.

Would it be possible for ab-av1 to get the size of only the video track from the whole file and then make the encoding prediction based on that? The result would be more accurate, and it would make much more sense in my opinion.

Thank you.

Love the tool but it does not preserve aspect ratio metadata so I currently need to modify the files afterwards.

Input video: 960x480,SAR 119:134, DAR 119:67 Output video: 960x480, SAR 1:1 DAR 2:1

This PR addresses #56.

Instead of erroring on input media with an unknown duration, we set it to zero and then treat all zero-length media as images. There are currently some unwraps that I'm not too happy about, but it "works on my machine™".

I would like to share an idea for the crf-search part that I have encountered in prolonged use:

Sometimes the specified VMAF value is not found by the search, which then ends with "Error: Failed to find a suitable crf" but the user may be fine with a nearby value. For example.

• I specify the VMAF value 98
• but the best result found is 97 -I am fine with that but that 97 found has an excessive estimated file size and the search does not continue to find a different crf with VMAF 97 because it was started for value 98

Basically a VMAF range would have to be accepted to perform the search (in this case it would be 98-97).

For quite some time I have been doing this simply with a bash script and crf-search but there is a lot of overhead as it has to repeat the search several times for different VMAF values even for crfs that have already been analyzed previously.

Hi @alexheretic, many thanks for this contribution!

Unfortunately I haven't been able to debug a blocker and a major annoyance with the utility.(on Ubuntu 21.10):

1. Regardless of the command used, the process fails with "Error: ffmpeg vmaf exit code Some(1)". I tried using ffmpeg-build-script to install ffmpeg and its dependencies, libvmaf should be 2.2.1 (I see you mentioned 1.5.3 as the min requirement?) and I've also compiled the latest vmaf cli and include it in the path ($ ./vmaf --version -> 9e1fb0bb). I don't see anything obvious in VmafOut.
2. After this failure, I can no longer see my terminal's input buffer, and I have to open a new terminal/tab to return to normal (reset doesn't work). This affects all my terminals and terminals emulators (guake, alacritty, terminator, mate's).

Any thoughts on how we can debug this?

The title of this issue is pretty vague, but I've had numerous different problems with using the default SvtAv1EncApp mode.

I first found that there was a problem with an image I was trying to encode when I could view it in Windows Photo Viewer but Chromium could not load it for some reason. Other encoders such as libaom-av1 and libsvtav1 did not have this issue and the image was able to be loaded and displayed properly in a browser.

I did test another image to see if I could give a reproducible sample but something else went wrong instead. This time the image was able to be loaded in my browser but it flickered as if it was showing the image and then a white frame over and over again. The other encoders correctly did not show this flickering.

Here are the source and output images for both issue types. All were encoded at CRF 10. No other options were changed (other than the encoder). I would have embedded them into the issue, but GitHub has yet to add support for AVIF uploads.

images.zip

For clarity: Source image = .png SvtAv1EncApp = .av1.avif libsvtav1 = .svtav1.avif librav1e = .rav1e.avif

Hello, I've installed ab-av1 using cargo install --git https://github.com/alexheretic/ab-av1 to make sure I have the latest version. When I try running the command ~/.cargo/bin/ab-av1 crf-search -i test.mp4 --preset 6 the task panics after 10-40 seconds on any video file. Here is the error with the stack trace:

thread 'main' panicked at 'can not convert float seconds to Duration: value is either too big or NaN', /builddir/build/BUILD/rustc-1.65.0-src/library/core/src/time.rs:736:23
stack backtrace:
   0:     0x55c792d5c889 - <unknown>
   1:     0x55c792d08dae - <unknown>
   2:     0x55c792d335c4 - <unknown>
   3:     0x55c792d5d813 - <unknown>
   4:     0x55c792d5e74d - <unknown>
   5:     0x55c792d5e312 - <unknown>
   6:     0x55c792d5e27c - <unknown>
   7:     0x55c792d5e251 - <unknown>
   8:     0x55c792c66e42 - <unknown>
   9:     0x55c792c80936 - <unknown>
  10:     0x55c792c8f2c1 - <unknown>
  11:     0x55c792cab021 - <unknown>
  12:     0x55c792c873ef - <unknown>
  13:     0x55c792cb6786 - <unknown>
  14:     0x55c792ca6560 - <unknown>
  15:     0x55c792cb5aa7 - <unknown>
  16:     0x7f98b7cc7510 - __libc_start_call_main
  17:     0x7f98b7cc75c9 - __libc_start_main_alias_2
  18:     0x55c792c6a195 - <unknown>
  19:                0x0 - <unknown>

I'm running Fedora 37, if that's important.

Maintaining our own list of known filetypes seems pointless when there are other projects like Infer that do it for us.

Marked as draft until this is answered: If Infer fails to load the file then ProbeErrors are created despite not being thrown by FFProbe. This is because the check happens in the probe function (for convenience). Is this okay or should a more specific error type be created for this? In any case, the whole thing would have failed anyway if the file is unable to be read.

I have some m2ts files I'd like to re-encode into mkv containers. Running this tool seems to be outputting an m2ts file as well, does using the -o option to specify an output file name somehow trigger a container change to the extension specified?

Hey,

I was wondering how the auto encode works, does it specifically try to hit a vmaf of 95 by default? Is there any option to hit the highest achievable, i.e. if a higher vmaf than 95 can he hit, then favour it?

Also, would it be possible to implement a batch/multi file processing on a folder basis, using walkdir with metadata recognition of video files? I'll look into the code to see if it doesn't add to much complexity.

I thought these two features would be a great addition, at least for my use cases.

What are your thoughts?

Would it be possible to one day have ssim2 score as an optional metric for greater psychovisual accuracy over VMAF? Just one example is that VMAF performs poorly in areas of low luma, claiming everything is fine when the video is far from it. Rust implementation: https://github.com/rust-av/ssimulacra2

I have tried running the command "ab-av1 crf-search -e qsv_av1_10bit" (plus -i info) and despite trying multiple ffmpeg builds, including compiling my own including intel-qsv, I get the error 'Ffmpeg encode_sample exit code Some(1)'.

Any fix for this? Trying to determine settings for Av1 QSV 10bit encodes. Thanks!

Specs: 5950x 64GB Ram Intel Arc A770

Fixes #62

This PR saves users from wasting time re-encoding the same image when an image with the best CRF has already been encoded.

I'm not entirely sure how the sample encode cache works, but I assume it doesn't store the samples themselves, so if the best CRF is read from the cache instead of being searched for then we still have to encode it again.

I'm leaving this as a draft for now because I'm not confident that output_path_with_best_crf (at auto_encode.rs:106) is done correctly. It works, but if the sample naming convention ever changes then this will break.

Though it's less usual to run concurrently against the same file I think this should just work anyway. You may, for example, want to try multiple different encoder args for an encoder than uses a ~1 thread on a multi-thread machine.

Currently the temporary dir is shared. Since you can specify --temp-dir conflicts can already by worked around. But it would probably be better if conflicts didn't happen out of the box.

A simple solution is to use a random named nested temp-dir inside --temp-dir for each run.