[BUG] Export video to images causes blockiness

[Rephil2]

Describe the problem

This stackoverflow entry sums everything up neatly with images as reference.
https://video.stackexchange.com/questions/8621/ffmpeg-output-is-blocky
The TLDR is; videos exported to images are in a low quality. I get the exact blockiness as seen in the stackoverflow post.
This issue is quite deceptive as the first few frames are not blocky so it can be easy to overlook for a user (the reason i flagged this as a bug).

My hotfix is to pass -q:v 1 to the FFMPEG cmd as seen bellow (which is already quite convenient).
kwargs = {"out_opts": '-q:v", "1'}
fouv.sample_videos(dataset, force_sample=True, reencode = True , **kwargs)

This comes down to that the ETA library does not include this FFMPEG setting by default.

I cant tell if this is intentional by fiftyone to reduce space but if it isnt then I would propose:
To include this by default in fiftyone/utils/video.py in the function _transform_video()

Code to reproduce issue

I used the code below, but i tried several other ways to export images from a video and got the same results.
fouv.sample_videos(dataset, force_sample=True, reencode = True)

System information

• OS Platform and Distribution : Linux Ubuntu 22.04
• Python version (python --version): Python 3.10.12
• FiftyOne version (fiftyone --version): FiftyOne v0.23.7
• FiftyOne installed from (pip or source): 0.23.7

Willingness to contribute

The FiftyOne Community encourages bug fix contributions. Would you or another
member of your organization be willing to contribute a fix for this bug to the
FiftyOne codebase?

• Yes. I can contribute a fix for this bug independently
• Yes. I would be willing to contribute a fix for this bug with guidance
  from the FiftyOne community
• No. I cannot contribute a bug fix at this time

[swheaton]

Interesting, thanks for the report.

Needs @brimoor 's input, I don't think this code has been touched in a long time.

But with some cursory research, here's some observations.

1. -qscale does not appear to be used in all encoders. Namely, libx264 which outputs H.264 video, uses a Constant Rate Factor argument.
   https://trac.ffmpeg.org/wiki/Encode/H.264

Tip: If you're looking for an output that is roughly "visually lossless" but not technically lossless, use a -crf value of around 17 or 18 (you'll have to experiment to see which value is acceptable for you). It will likely be indistinguishable from the source and not result in a huge, possibly incompatible file like true lossless mode.

Looks like we use a default of 23 and a present of medium?

Downside to crf

The downside is that you can't tell it to get a specific filesize or not go over a specific size or bitrate, which means that this method is not recommended for encoding videos for streaming.

2. qscale of 1 is not recommended by ffmpeg. 2 is the lowest recommended I believe.

There is rarely a need to use -qscale:v 1. Note that if you choose it, libxvid will take much more space than the same video compressed with the native mpeg4 encoder.

3. ffmpeg provides so many tuning options for a reason - you can play with encoding speed, file size, and quality. Seems like the project management triangle where you can only choose 2. I feel like while this may work for your use case, we would need to understand many other use cases to determine the right default. And perhaps offer simpler presets for playing with these things, so that users do not need to hunt down ffmpeg documentation and S/O posts like you did.

[Rephil2]

So I tested values for -q:v 20,15,12,10,8 which all gave blocky results. Below 5 gave good results and it started to clear up starting from 8.

-crf 17,18 and 23 did not improve anything in particular that I noticed.

-preset medium with options -crf 17,18 and 23 didnt work either nor did -preset high or -preset low with -crf 17

For the record this is the default cmd when running you get fouv.sample_videos(dataset, force_sample=True):
['ffmpeg', '-loglevel', 'error', '-vsync', '0', '-i', 'video.mp4', '-vsync', '1', 'image/%06d.jpg'] there is no preset there if that is suppose to be there.

As I was testing I realized i cant (or I dont know how to) actually use kwargs['out_opts'] with this function because they are removed in def _transform_video() since reencoding will always be required by sample_videos(). When in and out options are always reset. This happens in https://github.com/voxel51/fiftyone/blob/develop/fiftyone/utils/video.py: line 903

In regards to your 3rd bullet. I do see what you mean and I get it if this just too specific a case but I thought it was worth bringing up at least.