Enabling Immersive Experiences with Real-Time 4K HEVC Streaming

By Chris Knowlton, Vice President and Streaming Industry Evangelist, Wowza Media Systems

 

Over the last couple of years, we’ve seen growing interest in immersive virtual reality and 360-degree video experiences, enabled by increasingly high-resolution delivery. In the ever-evolving live-streaming landscape, these innovations are gaining mainstream attention—powering virtual attendance at events such as live sports and concerts—and giving birth to a number of companies whose entire business model is predicated on mainstream acceptance of these technologies.

360-Degree and VR Experiences

 

Anyone who has used Google* Street View or Bing* Streetside in a browser has already experienced interactive 360-degree photography, in which you move through a scene by tapping and panning on your device or laptop. You can have similar experiences today with on-demand and live video streams, too, such as those on the YouTube #360Video channel. Some mobile apps make it more virtual by changing the scene based on the position and motion of your device.

Virtual reality (VR) goggles take 360-degree visual experiences even further by adjusting what you see based on which direction your face is pointing. To these you can add additional sensory cues—platforms such as Virtuix Omni* that allow you to seemingly walk in any direction, plus spatial audio that provides accurate directional sound no matter which way you're facing.

While VR might remind you a bit of the 3D TV craze of a few years ago, the potential for real-world value-added applications seems high. According to VentureBeat, as of October 2015 there were 234 companies in the VR space, employing 40,000 people and generating $13B in value. These businesses are creating the content, technologies, and gear that power these virtual experiences. They include many companies you’ve never heard of, plus the bigger companies you know by name, such as Google, Facebook, Sony, and, naturally, Intel. Digi-Capital estimates market revenue for VR will hit $30B by 2020.

 

High-Resolution Delivery

 

High-definition video is a key part of making immersive experiences so engaging. 4K video (also referred to as Ultra HD or UHD) has been a popular video topic for a while now, with new cameras, screens, supporting technologies, and experiences being announced over the last few years. Despite 4K being one of the hottest consumer electronics trends, there isn’t much content bolstering it today. Immersive experiences call for 4K, however, and will help drive new 4K challenges, innovations, and adoption.

One of the often-overlooked facts about 4K video on televisions is that you need a large screen, or to be closer than normal to the screen, in order to perceive the difference in detail between 4K and 1080p. Conversely, computer monitors, mobile devices, and VR goggles in particular put the screen pixels much closer to our eyes, making lower-quality video more obvious. Palmer Luckey, creator of the Oculus Rift goggles, thinks we’ll need at least 8K resolution to really reach the level of detail required for VR. For now, as more devices come with 4K screens, viewers will increasingly be able to perceive whether they are watching high-resolution video that really makes them feel part of the scene they’re watching.

Once viewers have 4K devices, there are several impediments to reaching them with immersive 4K content, especially for live events. The first is having a fully enabled 4K video workflow to handle four times as many pixels as 1080p video, plus additional data to meet Ultra HD Premium certification, which greatly increases contrast, brightness, color, and audio standards. As a result, production gear for 4K is often more expensive by several multiples, and not all the brands of hardware and software that content producers are used to working with have fully 4K-enabled products.

Another big hurdle in the workflow has been the delivery from streaming servers to viewers. Streaming 4K content requires a lot of bandwidth, with sites such as YouTube generating 4K streams using the H.264 video codec with bit rates of about 20 Mbps. A recent “State of the Internet” report from Akamai indicates the average global connection speed to the Internet is just 5.1 Mbps, suggesting access to 4K streaming may be limited to a small slice of high-bandwidth consumers. If you are transforming content into multiple quality levels and streaming formats, so you can reach almost any viewer regardless of their device or bandwidth, you’ll need to further multiply your computing needs.

 

Bringing It All Together

 

To make the challenges to immersive video experiences less daunting, Intel and Wowza Media Systems are working together to reduce the impediments to live 4K video processing and streaming. Collectively, we’re working on reducing the server footprint that you need to process video for delivery, and on reducing the bandwidth required to stream it—all while maintaining or improving the quality of the video experience for viewers.

Let’s look first at the problem of scaling servers to process video. Wowza* Streaming Engine software helps deliver the right video to any screen in four fundamental ways. It will accept live video and audio streams in many formats, and convert (transcode) them on the fly to other formats more commonly used for streaming. It can simultaneously create multiple renditions of a stream at differing resolutions (trans-sizing) and corresponding bit rates (transrating), ensuring streams are available for every viewer regardless of their device or bandwidth constraints. It can package (transmux) the audio and video into multiple streaming protocols to match those used by each viewer’s player technology.

(Note that the three of these four terms are often combined under the single term “transcoding.” To learn more, see my “What Is Transcoding and Why Is It Critical for Streaming?” blog post)

To do this processing at any production scale using just software, you need large servers with increased cores, RAM, storage, cooling, and power requirements (scaling up), or you need to spread out the load over multiple servers (scaling out). Many of our customers have their servers in fixed-footprint data centers, where space is at a premium. They may want to differentiate their video offerings by adding support for higher-resolution video, but either scaling up or scaling out could be unacceptable.

This is where Intel® HD graphics technology comes into play, with integrated graphics processors built directly into the CPU. Intel® Quick Sync Video is a key component, providing hardware-accelerated video decoding and encoding for Wowza Streaming Engine software to leverage. Using the Intel® Media SDK, Wowza Streaming Engine can offload the heavy processing to an Intel® Xeon® Processor E3 with integrated graphics and free up the CPU to handle a much higher volume of other tasks. Instead of needing to scale up or scale out, our customers can do more with less.

Now let’s look at the other challenge—delivering 4K video. Since we can’t make the “pipes” to each viewer bigger, our only option is to make the video need less bandwidth. We do this with a video codec known as H.265 or HEVC (High Efficiency Video Coding).

The good news is that HEVC can reduce video size by up to 50% while maintaining the same quality. All along the streaming workflow, you reduce your bandwidth and storage needs. More viewers will be able to experience 4K streams, and even those who can’t access 4K will get a higher-quality stream than before. Everyone’s happy, from the content owners to the viewers.

The challenge is that it takes a lot more compute power than H.264, the current mainstream video codec—up to 10 times more. Fortunately, the new Intel Xeon processor E3-1500 v5 family natively supports hardware-accelerated HEVC transcoding. Now, with a single processor, you can transcode two 4K HEVC streams simultaneously, or transcode a full set of resolutions and bit rates from 240p to 4K, allowing you to reach almost any screen with an immersive live streaming experience best suited to each viewer.

Intel showed this powerful new capability in action at Computex 2016 in Taipei. During a Diane Bryant keynote demo, attendees saw a 360-degree live-streamed virtual-reality jazz concert delivered from the legendary Blue Note Jazz Club in New York using hardware-assisted 4K video delivered by Intel Xeon processor E3 family-based servers with Wowza Streaming Engine software.

Want to learn more about how this hardware and software combination enables immersive concert experiences for viewers anywhere at any time? Watch the video: