Video Compression metrix vs Trireme EMPEG

Many if not all of the videos that you watch on Netflix and YouTube have been configured using video quality metrics. Over the last 4 years or so, Netflix’s encoding ladders have been driven by the company’s Video Multimethod Assessment Fusion (VMAF) metric and, before that, Peak Signal to Noise Ratio (PSNR). As well, YouTube uses a neural network based upon constant rate factor encoding, which itself is driven by an internal video quality metric.

Simply stated, video quality metrics attempt to predict how a subjective viewer would rate a particular video, and metrics are comparatively rated based upon the accuracy of these predictions. There are many purists who insist that subjective comparisons are the only valid technique for gauging quality, and indeed, properly administered subjective tests are the gold standard.

However, when you consider that 500 hours of video are uploaded to YouTube each minute, you can appreciate that the service has a strong need to encode its streams as efficiently as possible and a total inability to deploy humans to make it happen. Even Netflix, with a comparatively paltry 2,800 hours of new content in 2019, can’t use human eyes to create the customized encoding ladders for each video. For both companies, and many others, objective quality metrics are essential.

ERROR-BASED METRICS

The first class of metrics are error-based. They compare the compressed image to the original and create a score that mathematically represents the differences between the two images, also called noise or error. The PSNR is a good example. Metrics based upon this approach are simple and easy to compute, but scores often don’t correlate well with subjective ratings because human eyes perceive errors differently.

PERCEPTUAL-BASED MODELS

At a high level, perceptual-based models like the SSIM attempt to incorporate how humans perceive errors, or “human visual system models,” to more accurately predict how humans will actually rate videos. For example, while PSNR estimates absolute errors, Structural Similarity Index (SSIM) is a perception-based model that considers image degradation as perceived change in structural information, while also incorporating important perceptual phenomena, including both luminance masking and contrast masking terms.” In other words, perceptual-based metrics measure the errors and attempt to mathematically model how humans perceive them.

Netflix’s VMAF is another metric that was built to be trained (AI), using what’s called a support vector machine. Since the primary use for VMAF is to help Netflix produce encoding ladders for its per-title encoding, the Netflix training dataset includes clips ranging in resolution from 384x288 to 1080p at data rates ranging from 375Kbps to 20Mbps. Again, by correlating the mathematical result with subjective MOS scores, VMAF became much better at making the 540p vs. 720p decision mentioned above.

As the name suggests, VMAF is a fusion of three metrics, two that measure image quality and one that measures temporal quality, making it a true “video” metric.

Trireme has recently conducted extensive comparative studies to evaluate and analyze the performance of our compression against industry accepted benchmarks. The following studies include multiple videos at various bitrates measured by SSIM, VMAF & PSNR all compared to FFMPEG.

The evaluations were performed on 2 complex video sequences in H.264/AVC compression with 10s and 2s IDR period. The compression with 10s IDR period has been performed from uncompressed video as an input.

The compression with 2s IDR period has been performed from 30 Mbit/s ffmpeg x264 compressed video in the default placebo mode with 2s IDR period as an input, which (logically and obviously) benefits subsequent ffmpeg x264 compression in the default placebo mode, especially near 30 Mbit/s bitrate.

The objective quality difference between EMPEG compression in the default mode and FFMPEG compression in the default placebo mode is bigger than between two video compression standards, such as H.265/HEVC and H.264/AVC according to the attached comparison example in the jpg file. We have provided below a summary of the results and attached the corresponding full studies along with links to the videos.