3

This is a more up-to-date version of this question here, which luckily it avoids being a duplicate of because that question is about VirtualDub rather than FFmpeg.

This is not a subjective question as it can be answered definitely by A/V experts who are familiar with the industry literature that measures such metrics, as proven in the the above question, and there's every reason to suspect that things have changed in the last decade with the amount of development that's been done on both FFmpeg and the algorithms since.

I'm in the process of using FFmpeg to automate my workflow for YouTube uploads, which usually consist of old, rare, low-resolution videos. These need to be upscaled in order to concatenate my own HD outro to them, and often they also have weird aspect ratios.

A suggestion on how to approach upscaling them that I got from Reddit was to do something like this:

ffmpeg ... -vf "scale=iw*6:ih*6:flags=neighbor,scale=1966:-2" ...

This uses the nearest neighbour algorithm to upscale a video as close as possible to the desired resolution x amount of times (in this case going from the original video's 320x176 to 1920x1056) and then uses the standard scale filter to go up the few remaining pixels. This is supposedly better at avoiding artifacts than making one massive jump with the scale filter on its own, and can also be used to downscale by replacing the multiplications with divisions.

Is this the best general FFmpeg approach for upscaling or downscaling videos that I can use in my scripts, or does a better one exist? Or has the research pretty much stayed the same as in slhck's answer from 2012, where lanczos is best for downscaling and bicubic for upscaling?

Hashim Aziz
  • 11,898
  • 35
  • 98
  • 166
  • 1
    What type of content the does video contain? Animation, pixel art, home videos? Which scaling methods looks best to you? The two scales, a single scale, different algorithms? Why not try all [algorithms](https://ffmpeg.org/ffmpeg-scaler.html#scaler_005foptions)? For pixel art there are also the [super2xsai](https://ffmpeg.org/ffmpeg-filters.html#super2xsai) and [hqx](https://ffmpeg.org/ffmpeg-filters.html#hqx) filters to consider. – llogan Apr 11 '21 at 23:01
  • 1
    @llogan The content is all video, filmed on everything from mobile phones to studio cameras, but it's almost always old and low-res. I haven't gone through all the algorithms and judged them based on my perception because that would take a lot of time and would only be relevant to the videos I was able to test on. From my understanding this is what metrics like SSIM, PSNR and VMAF were designed to do away with, so I was hoping that the current literature/benchmarks would shed light on which algorithm is likely to yield the best results in the general case, especially for upscaling. – Hashim Aziz Apr 11 '21 at 23:58
  • 2
    These days the best upscalers are all AI-based. Most of the sites you find via google are trying to sell you their own, in the guise of a review, but this one seems a bit more even-handed - https://www.guru99.com/video-quality-enhancer.html – Tetsujin Apr 12 '21 at 07:26
  • If you want metrics to tell you what to choose, then just make a script that iterates through all of the variations and choose the highest score. That's pretty much the answer to the question. Be aware of the pitfalls and limitations of quality metrics: they are not perfect and the highest score may not look the best. And what looks best to you might not to me. Then there is the whole AI scaling rabbit-hole to consider as mentioned by Tetsujin. If I couldn't avoid the evils of upscaling I would not overthink or overanalyze it and simply choose what looks best to me. – llogan Apr 12 '21 at 17:37

0 Answers0