I have been converting my H265 (HVEC) collection to AV1. I am still trying to find the exact parameters for FFMPEG to produce the optimal output for the devices I use plex on. Its been a little bit of whack a mole so far, but it seems ideal for file size when I get it right.
I've been transcoding my media collection leaving my PC on overnight over months, it's great. My biggest issue is client support for native playback of AV1, naturally.
For what it's worth, AB-AV1 [1] is a pretty awesome tool written in rust which compares random samples from a file at different parameters based on their VMAF score [2] (algorithm from Netflix for human-perceived visual likeness), choosing optimal parameters to save as much space as possible with the loss you're willing to stomach, on a file-by-file basis.
Small plug: I made a nice little python GUI wrapper for ab-av1 [3].
While I agree with you, I find that sometimes the “experience” can improve.
The most common “artifact” of AV1 is to make things slightly more blurry for example. A common H.265 artifact is “blockiness”. I have re-encoded H.265 to AV1 and not only gotten smaller files that playback better on low-end hardware but also display less blockiness while still looking high-resolution and great colour overall.
I always encode 10 bit colour and fast-decode for re-encoding to AV1, even if coming from an 8 bit original.
But then you look at flashback scenes and wonder where the noise has gone.
A lot of movies have purposeful noise, blurriness, snow, and fake artifacts to represent flashback scenes. One level of compression often keeps them okay-ish (like you can tell side by side that it's different, but only when you know what to look for). But these are the scenes that get especially ruined by two layers of compression.
50GB gives assurances that the BluRays are high quality (but not always. I've seen some horrible BluRay encodings...)
As long as you are going from high quality sources, you should be fine. The issue is each transcoding step is a glorified loop-(find something we think humans can't see and delete it)
In other words: the AV1 encoder in your example works by finding 47GBs of data TO DELETE. It's simply gone, vanished. That's how lossy compression works, delete the right things and save space.
In my experience, this often deletes purposeful noise out of animation (there are often static noise / VHS like effects in animation and film to represent flashbacks, these lossy decoders think it's actually noise and just deleted it all changing the feel of some scenes).
--------
More importantly: what is your plan with the 50GB BluRays? When AV2 (or any other future codec) comes out, you'll want to work off the 50GB originals and not off the 3GB AV1 compressed copies.
IMO, just work with the 50GB originals. Back them up, play them as is.
I guess AV1 compression is useful if you have a limited bandwidth (do you stream them out of your basement, across the internet and to your phone or something? I guess AV1 is good for that) But for most people just working with the 50GB originals is the best plan
> In other words: the AV1 encoder in your example works by finding 47GBs of data TO DELETE.
With that reasoning, lossless compression of .wav to .flac destroys >50% of data.
In actuality, you can reconstruct much of the source even with lossy compression. Hell, 320kbps mp3 (and equivalent aac, opus, etc) are indistinguishable from lossless and thus aurally transparant to humans, meaning as far as concerns us, there is no data loss.
Maybe one day we'll get to the point where video compression is powerful enough that we get transparent lossy compression at the bit rates streaming services are offering us.
> In my experience, this often deletes purposeful noise out of animation
AV1 specifically analyzes the original noise, denoises the source then adds back the noise as a synthetic mask / overlay of sorts. Noise is death for compression so this allows large gains in compression ratio.
> AV1 specifically analyzes the original noise, denoises the source then adds back the noise as a synthetic mask / overlay of sorts. Noise is death for compression so this allows large gains in compression ratio.
If said noise still exists after H265.
And there's no guarantee that these noise detection algorithms are compatible with H264, H265, AV1, or future codecs H266 or AV2.
AV1 is not about throwing away more data that the human can’t see. It’s about having better tools.
1. the prediction tools of AV1 are better than those of h265. Better angular prediction, better neighboring pixels filtering, an entirely new chroma from luma prediction tool, an intra-block copying tool, more inter prediction tools, non-square coding units.
2. If the prediction is better, the residuals will be smaller.
3. Those residuals are converted to frequency domain with better tools for AV1 as well (more options than just DCT), so that you have a better grouping of coefficients close to the DC component. (Less zeros interleaving non-zero values.)
4. Those coefficients compress better, with a better entropy coding algorithm too.
You can have exactly the same video quality for h265 and AV1 yet still have a lower bitrate for the latter and with no additional decision made to “find out what humans can’t see.” The only place in the process where you decide to throw away stuff that humans can’t see is in the quantization of the frequency transformed residuals (between step 3 and 4) and the denoising before optional film grain synthesis.
To be clear: you can of course only go down or stay equal in quality when you transcode, due to rounding errors, incompatible prediction modes etc. That’s not under discussion. I’m only arguing about the claim that AV1 is better in general because you throw away more data. That’s just not true.
Yes, in general you find the best high quality source you can get your hands on and then compress that. For us lay people, that would currently be any 4k videos with a high bitrate. In such cases, it doesn't matter much that it is already compressed with AVC or HEVC. Sure, when you compress that again at a lower bitrate, there will some loss of data or quality. But honestly, it doesn't make a discernable difference (after all, you decide what is the video quality acceptable to you by choosing how much more to compress). Ideally, if DVD and Blu-Rays lasted long, we would all just be saving our videos on it. (Assuming there will be any Blu-Ray readers, 10+ years down the lane).
Yes. Deleting data does wonders for the filesize. The question I'm bringing up is one of quality.
If you must delete, delete starting from the 50GB+ original BluRays if at all possible, or some other very high quality source. That way the compression algorithm has the best chance of saving the important scene data.
And keep an eye on the known hard to encode scenes. A lot of the typical shots of a movie are handled well on one set of settings, but suddenly screw up on other scenes (or other animation styles. Anime vs Cartoons vs 3D vs Live Action can have subtle differences leading to quality issues).
It's not easy, and AV1 is our best bet at doing this well so far. But when the future algorithms come out, you need to start over from the best sources of you want AV2 to have a chance.
You should *Never* double compress. (Blu-ray -> H265 -> AV1). This is horrible for the quality. You'll get better results from BluRay -> AV1 by a large margin.
I'd be surprised if there's any noticeable difference between 265 and AV1 when coming from the 265 encoding. 265 has already thrown a lot of stuff away, so there's not much for AV1 to work with.
Maybe if you're going to a lower resolution it would be fine (ie: going from 4k 265 to 720p AV1).
Aside from video, audio compatibility is tricky as well. You can do AAC stereo and most things support that but AAC 5.1 seems to be supported by only some devices so all my video files end up getting stereo AAC, 5.1 AAC, and 5.1 DTS to avoid live transcoding.
IIUC, it's more about the client hardware that determines ability to play without transcoding. You'd have to check the mix of devices you have connecting to it and make a judgement call.
I started to convert a lot of my content in AV1 until I realized that my Nvidia Shield devices won't play AV1. My $30 firestick will play them but I do really prefer the Shield. I guess I'll wait it out and hope for a new Shield (it's been 2019 since Nvidia released one) but i'm not going to hold my breath.
