Deep-dive: Intel Arc’s AV1 video encoding is the future of GPU streaming

Imagine a world the place video streaming may have considerably increased high quality with out utilizing up extra bandwidth—or have the identical high quality whereas having half the influence in your information caps. A world the place Twitch streams aren’t a blocky mess and YouTube movies can truly resemble the visible expertise of enjoying the sport your self.

That world is the longer term promised by AV1: A brand new, open supply video codec that goals to dethrone H.264 as the first video customary after its practically 20 year-long reign. Typically very troublesome to encode, real-time AV1 encoding is now accessible to customers through Intel’s debut Arc Alchemist graphics cards (although the launch has been restricted to China thus far, with a U.S. launch deliberate for later this summer season).

I put Intel’s GPU encoder to the check utilizing a customized Gunnir Arc A380 desktop card to see if it delivers the guarantees we’ve seen from AV1 up to now, and the way it competes in opposition to the present H.264 encoders sometimes used for reside streaming. I additionally need to clarify why all of this issues a lot. There’s rather a lot to this, so let’s dig in.

[ Further reading: The best graphics cards for PC gaming ]

Before H.264 turned the predominant video codec all over the place, on-line video was a large number. I’ve many fond reminiscences of my Windows 98 and Windows XP computer systems being plagued with all method of video participant apps from QuickTime to RealPlayer to the DivX/Xvid gamers of doubtful legality, all to play AMVs or sport trailers downloaded from eMule or LimeWire. Then as soon as YouTube began gaining reputation, all of us needed to cope with .FLV Flash Video information. It was arduous to maintain up. H.264 taking up and be accepted by practically each software, web site, and system felt like magic. But because the years (and years) have handed and video requirements goal at increased resolutions and better framerates, demand elevated for video codecs that operated with increased effectivity.

While H.264 was made effectively royalty-free, H.265 nonetheless has a whole lot of patents and licensing prices tied up in it – which is why you don’t see many shopper purposes supporting it, and nearly no reside streaming platforms settle for it.

YouTube and Netflix converted to nearly solely utilizing VP9 (Google’s own open source video codec) however once more adoption within the shopper software house has been nearly nonexistent, and it appears video streaming giants nonetheless need even increased effectivity.

Adam Taylor/IDG

That’s the place the Alliance for Open Media is available in. AOM is a collaborative effort to develop open-source, royalty-free, and versatile options for media streaming. Backed by basically every big corporation involved in web media together with Google, Adobe, Nvidia, Netflix, Microsoft, Intel, Meta, Samsung, Amazon, Mozilla, and even Apple, AOM’s focus is on creating (and defending through a secure patent overview course of and establishing a authorized protection fund to maintain the tech open) AV1, with AV1 being an ecosystem of open supply video and picture codecs. Tools for metadata and even picture codecs have been developed, however what I’m specializing in right here is the AV1 Bitstream video codec.

It could seem unusual for therefore many massive firms (and competing ones at that) to be working collectively on a singular challenge, however in the end that is an endeavor that advantages all of them. Lower bandwidth prices, higher-quality product, and simpler interoperability for no matter the way forward for streaming media may maintain appears to beat out the advantages of prior philosophies round everybody growing their very own walled-off options.

My solely private concern right here is on the subject of many of those firms’ histories of pushing anti-consumer DRM in terms of video each on-line and offline, and the way these previous actions may affect AV1 implementations.

AV1 within the wild

Adam Taylor/IDG

All that is nice, however how do you truly get AV1 movies? Keeping in thoughts that new codec adoption is often very gradual and AV1 is transferring in a short time, all issues thought of, you’ll be able to truly watch a good bit of AV1 on-line proper now.

Starting with YouTube, you need to go to your YouTube Playback Settings (whereas logged in) and select “Always Prefer AV1” to extend your possibilities of truly being delivered AV1 transcodes of movies. From there, any video at 5K decision or increased ought to have already got AV1 transcodes able to play. YouTube additionally created this “AV1 Beta Launch Playlist” again in 2018 with some pattern movies that had been initially given AV1 copies for some assured testing. Anecdotally, I’ve been seeing increasingly more high-traffic movies that I watch frequently enjoying again in AV1.

Adam Taylor/IDG

The video received’t present any completely different for you within the video participant, however in the event you right-click the video whereas it’s enjoying and click on “Stats for nerds” you need to see “av01” subsequent to “Codecs” if all is working correctly.

Adam Taylor/IDG

Actually with the ability to decode AV1 generally is a blended bag. Most fashionable quad-core PCs from 2017 or later shouldn’t have any problem decoding 1080p AV1 footage on CPU. But when you transcend 1080p, you’ll need hardware-accelerated decoding. Nvidia’s GeForce RTX 30-series graphics playing cards assist AV1 decode (such as the RTX 3050) as do AMD’s Radeon RX 6000-series GPUs and even the iGPUs on Intel 11th Gen and newer CPUs. If you have got this {hardware}, be sure that your drivers are up-to-date and obtain the free AV1 Video Extension from the Microsoft Store, then refresh your browser. Depending in your graphics {hardware}, you’ll even be capable of see the “Video Decode” part of your system doing the work in Windows Task Manager. (Note: All of that is on Windows 10 and 11 solely, Windows 7 shouldn’t be supported.)

Netflix is already streaming AV1 for some movies to appropriate gadgets as a number of TVs, sport consoles (together with the older PlayStation 4 Pro), and a few cell gadgets already assist it. Netflix’s streams are proving to be an exquisite showcase for AV1’s “Film Grain Synthesis” feature — the flexibility for the encoder to research a video file’s movie grain, take away it to cleanly compress the footage, after which present directions to the decoder to re-create it faithfully, with out losing pointless bits on the grain.

Twitch’s former Principal Research Engineer Yueshi Shen additionally shared an AV1 demo with me again in 2020 showcasing what AV1 may deliver to Twitch streaming. Available here, you’ll be able to view principally blocking-free gameplay in 1440p 120FPS delivered with solely 8mbps of bandwidth, 1080p 60FPS utilizing solely 4.5mbps, and 720p 60FPS utilizing solely 2.2mbps. While not a real-world reside streamed check, it’s nonetheless severely spectacular given the terribly poor high quality regular H.264 encoders would produce at these bitrates in regular Twitch streams.

Adam Taylor/IDG

Shen originally projected Twitch to have full AV1 adoption by 2025, with hopes of massive identify content material streaming it as early as this 12 months or 2023. Hopefully with consumer-accessible encoders now accessible, they’ll begin turning on these new options very quickly.

Did Intel ship?

AV1 encoders that run on the CPU have been accessible for fairly some time now, however they’ve been very troublesome to run, taking many hours to course of by means of a pattern even on excessive core depend machines. Performance has been enhancing steadily over the previous couple years and two encoder choices (SVT-AV1 and AOM AV1) are actually accessible to be used as of OBS Studio model 27.2. As I covered when the OBS update released, these are nonetheless powerful to run in real-time, however they’re usable and step one in the direction of shopper AV1 video.

Most of the streaming and content material creator scene have been ready on hardware-accelerated encoders to indicate up on next-generation graphics playing cards, as a substitute. As talked about, Intel, AMD, and Nvidia all added {hardware} AV1 decoders to their earlier era {hardware}, and it’s an assumed-certainty that at the least Nvidia may have an AV1 {hardware} encoder on RTX 4000 {hardware}, if not AMD with RX 7000 as properly. However — and regardless of many delays of their very own — Intel is first to market with GPU AV1 encoders on the brand new Arc Alchemist line of graphics playing cards.

Adam Taylor/IDG

The very first thing I needed to analyze with the brand new A380 GPU was how properly the AV1 encoding truly held up versus currently-available choices. While AV1 as an total codec is extremely promising, the outcomes you get from any codec relies on the encoder implementation and the sacrifices essential to run in real-time. GPU encoders run on fixed-function {hardware} throughout the GPU, permitting for video encoding and decoding with minimal influence to regular 3D workloads, similar to sport efficiency, however don’t at all times produce “ideal” situations.

Adam Taylor/IDG

I’ve previously covered how completely different {hardware} H.264 encoders have improved by means of iterative GPU generations the place it may be seen that early implementations of encoding with Nvidia’s NVENC, for instance, didn’t produce wherever close to as excessive of a high quality as their newer playing cards can. I additionally just lately examined how encoder quality can be somewhat improved through software updates (similar to AMD’s current updates to their AMF encoder). All of because of this whereas there’s loads of room to be hyped for the primary accessible AV1 encoder, there’s additionally room to be dissatisfied by the very first iteration of it, with many extra to return.

The most dependable means I’ve discovered to quantitatively measure video high quality has been Netflix’s VMAF – Video Multi-Method Assessment Fusion. This is an algorithm that assesses video high quality in a means that very carefully matches what precise human viewers really feel about video high quality at a given distance and measurement, somewhat than counting on pure noise measurements similar to with PSNR. Netflix has been constructing this know-how (and blogging about it) for fairly a number of years now, and it’s gotten to a degree the place it very reliably measures variations that assist illustrated what I’d sometimes strive exhibiting you with repetitive side-by-side screenshots.

I’m always testing a large library of lossless gameplay and reside motion video samples, however for ease of presentation, I need to give attention to a number of video games within the FPS style. The fast-paced digicam actions, highly-detailed graphics with a number of particle results, and lots of HUD parts mix to supply a type of “worst case scenario” for video encoding, as nearly each Twitch streamer has struggled with.

My focus is on 1080p 60FPS video, with three major bitrates: 3500kbps, 6000kbps, and 8000kbps. 3500kbps (or 3.5mbps) is the bottom that may sometimes be suggested to ever use at 1080p (at the least, previous to AV1), 6000kbps is the smooth “cap” for Twitch streams, with 8000kbps being an unofficial bandwidth cap that many are in a position to ship to Twitch with out problem. Twitch is the main target right here because of the website offering “Source” high quality streams that don’t undergo the secondary stage of compression that YouTube streams do, which removes a number of the advantages of higher-quality encoders at decrease bitrates.

Adam Taylor/IDG

Here, I examine the same old {hardware} H.264 encoders from Intel, Nvidia, and AMD in opposition to the CPU X264 encoder operating within the “VerySlow” CPU utilization preset — one thing that may’t be run in real-time, however is usually thought of the benchmark for high quality to goal for — in addition to Intel’s AV1 GPU encoder and two SVT-AV1 encode presets that one may “reasonably” (on a high-end CPU like Threadripper) encode in real-time.

These outcomes are, properly, fascinating.

Adam Taylor/IDG

At 6 and 8mbps, Intel Arc’s AV1 encoder goes forwards and backwards between scoring increased than X264 VerySlow and scoring barely decrease (VMAF scores function on a 0-100 scale with 100 being an ideal match for the lossless/uncompressed supply materials), and nonetheless scoring impressively increased than even one of the best GPU H.264 encoders. On its personal, that is already spectacular sufficient. If you’re a sport streamer and you employ a twin PC streaming setup or have the PCIe lanes and slots so as to add a second GPU to your machine, as soon as AV1 is enabled on Twitch you’ll be capable of stream considerably higher-quality streams than most of what’s on your complete web site, on the similar bitrate.

But if we glance over on the decrease 3.5mbps bitrate, Intel’s AV1 encoder soars above any of the H.264 encoders, together with X264 VerySlow. And in a number of the sport assessments, AV1 encoded on the Arc A380 at 3.5mbps scores increased than many of the H.264 choices do at 6mbps (practically double the bandwidth).

Adam Taylor/IDG

Theoretically, if Twitch enabled AV1 streaming proper now, a streamer utilizing an Arc A380 to encode their broadcast would allow everybody concerned on this course of to chop their bandwidth in half — the streamer themselves, the viewer, and Twitch/Amazon — with out taking successful in high quality. It additionally means you’ll be able to instantly get a bounce in high quality with out altering any community necessities on the streamer’s finish.

Moving on from information to precise real-world visuals, the outcomes are nonetheless simply as spectacular.

At 6 and 8mbps, Intel’s AV1 encoder presents “more of the same” at a look in comparison with H.264 encoders. There’s a tiny quantity of added sharpness, however not sufficient to face out, however a really noticeable lack of blocking or artifacting in areas with massive mild modifications or shadows.

Adam Taylor/IDG

Again, you received’t be blown away evaluating outcomes at these increased bitrates, but it surely is an enchancment. AV1 appears to do a very nice job at extra easily mixing collectively areas the place element must be sacrificed, somewhat than creating pixelated-looking blocking that you simply’re used to seeing. Sometimes at a sure distance you may even really feel a bit of the H.264 footage appears to be like “sharper” than the AV1 footage because of the additional “crunch” giving the phantasm of added element, however zooming in, that element isn’t truly there.

Adam Taylor/IDG

At a mere 3.5mbps, Intel’s AV1 encoder does begin to exhibit a small quantity of macro-blocking in gradients, however avoids it throughout detailed sections of the display in comparison with H.264, and actually delivers a view you wouldn’t suppose could be doable at such a low bitrate.

Looming competitors

My VMAF graphs additionally included a pair extra information factors that scored considerably increased than Intel’s AV1 encoder. These are two encodes utilizing the SVT-AV1 CPU encoder. This encoder makes use of “Presets” (much like X264) numbered in such a means that the smaller numbers are tougher to encode with increased high quality (akin to going slower with X264), and better numbers are simpler with worse high quality. In my testing, even on a 32-core Threadripper CPU, 8 and 9 had been the one practical presets to encode with in real-time. So protecting with my theme of slightly-unobtainable benchmark high quality, I included SVT presets 7 and 8 on my graphs.

Adam Taylor/IDG

And as you’ll be able to see, whereas there’s marginal variations between these two particular presets, they each out-score Intel’s AV1 encoder by an extended shot. Much how X264 VerySlow far outperforms any of the GPU H.264 encoders, that is to be anticipated.

I believed it was necessary to incorporate these for 2 causes. First, in the event you’re simply encoding movies for add to YouTube, archives, and so forth, you should utilize these slower encoder profiles with AV1 and exponentially improve your bitrate effectivity (saving on house and add occasions, or rising high quality for the file measurement you’ll have already dedicated to). Secondly, I’m hoping it’s a preview of what we have now to look ahead to from GPU AV1 encoders over the subsequent couple years.

While Intel’s QuickSync Video H.264 encoder (on each the Arc GPUs and 12th Gen iGPUs) is presently main Nvidia and AMD in high quality, earlier generations lagged behind Nvidia (and even AMD if we return far sufficient) which suggests if Nvidia launches RTX 4000 with an AV1 encoder, it may carry out at the least considerably higher than Intel’s providing. Plus, as talked about, that is simply the primary iteration. As {hardware} improves, so will the encoder. I actually need Nvidia and AMD to compete on the AV1 encoder entrance, delivering customers even increased high quality – however I’ve to say I’m fairly stoked with the place our beginning line is right here in Intel’s Arc graphics playing cards.

The way forward for video streaming could be very brilliant, and considerably much less blocky. I’m importing all of my YouTube videos in AV1 transferring ahead and look ahead to streaming within the new format as quickly as platforms enable.

[Disclosure: My Gunnir Photon A380 graphics card unit was sampled to me by Intel for inclusion in my usual encoder quality analysis content on my YouTube channel. I was not paid by Intel for any coverage, am under no obligation to say anything specific, and Intel has not seen anything I post about the GPU prior to publishing. I’ve also been sent GPU samples by Nvidia and AMD for this same purpose.]