MPEG-2 SSIM as a gauge for MPEG-4 encoding

Different videos are well different. Even encoded to MPEG-2 with minimal quality loss they have different SSIM and PSNR. Further encoded to MPEG-4, those that have higher SSIM and PSNR while encoded to MPEG-2, would "encode better". This dependency is not linear, i.e. videos with higher SSIM and PSNR for MPEG-2 would not neccessarily encode better to MPEG-4 then those with slightly lower SSIM and PSNR for MPEG-2 - it could be the other way around. However, videos with "way lower" SSIM and PSNR for MPEG-2 would encode to MPEG-4 a lot worse then those with "way higher".

For example, as shown in the table bellow, for MPEG-2 "Infiniti" video has higher SSIM and PNSR then "Budweiser" video, but "Budweiser" encodes better to MPEG-4 and could be squeezed all the way bellow 250kbps at 15 fps at SD resolution (480p) while "Infiniti" needs 300kbps for the same profile. (Visually at 300kbps Infiniti video looks about the same as Budweiser at 250kbps - very few noticable artifacts, while at 250kbps Infiniti is about the same as Budweiser at 200kbps - some noticable artifacrs).

Encode	SSIM	PSNR	Encode	SSIM	PSNR
Infiniti			Budweiser
MPEG-2	86.98	41.91	MPEG-2	74.54	37.74
			153kbps@15fps	71.14	33.82
			203kbps@15fps	74.41	34.06
257kbps@15fps	73.97	40.01	254kbps@15fps	76.33	34.18
309kbps@15fps	75.70	40.47	306kbps@15fps	77.68	40.74
413kbps@15fps	78.35	41.17	511kbps@15fps	81.01	34.46

On the other hand "Dance" video has "way lower" SSIM and PSNR when encoded to MPEG-2 then either Budweiser or Infiniti and when encoded to MPEG-4 cannot be squeezed bellow 500 kbps at 15fps at 480p at all! The following table list SSIM and PSNR for video encoded using different methods at 500 kbps at 15 fps - all has unaceptable video quality.

Encode	SSIM	PSNR
MPEG-2	56.48	34.66
x264	35.19	27.37
divx6	35.32	27.37
divx	5.66	22.59
xvid	5.43	22.59

Similar pattern could be observed with CNN stock footage - videos with higher SSIM/PSNR for MPEG-2 encodes encode fine upto 300 kbps at 15fps, perhaps with the exception of 035-airplane video that regardless of SSIM similar to "Dance" video could be pushed all the way to 300 kbps, alas with noticable artifacts.

Video	SSIM	PSNR	Encode	SSIM	PSNR	Notes
021-war	79.74	39.26	300kbps@15fps	76.07	41.29	Fine
080-reuters	85.28	37.74	500kbps@25fps	62.41	36.35	Fine
035-aero	53.12	32.00	300kbps@15fps	39.13	33.89	Blockiness

And finally, even home videos exhibit similar pattern (MPEG-4 encode at 500fps@15fps)

Video	SSIM	PSNR	Encode	SSIM	PSNR	Notes
Bike	45.27	32.53
Dance	56.48	34.66	500kbps@15fps	35.19	27.37	Worst, 15fps are not enough for motion
Gazon	61.54	35.24	500kbps@15fps	54.45	39.23	Some artifacts, 15 fps are not enough
Udarenia	61.98	32.00	300kbps@15fps	56.52	31.19	Almost fine. 400-500kbps almost do it
Podcast	61.98	34.41

So what is the point of this excersize especially at half the framerate?

• "super clean" videos could be encoded 300fps@15fps at 480p or between 500kbps - 700 kbps at normal framerates.
• SSIM < 50-60 or videos with a lot of motion or black background (like home videos) cannot be encoded at 480p resolution bellow 1Mbps and need to be scaled to 480x360 to be sqeezed into 500 kbps
• scaling framerate to 15-20 fps works for "talking head" video, but is not acceptable for videos with motion, thus once again, the only option is scaling to 480x360.