The neural network just needs a few minutes of video, or even just an audio clip.

Remedy, the developer behind the likes of Alan Wake and Quantum Break, has teamed up with GPU-maker Nvidia to streamline one of the more costly parts of modern games development: motion capture and animation. As showcased at Siggraph, by using a deep learning neural network—run on Nvidia's costly eight-GPU DGX-1 server, naturally—Remedy was able to feed in videos of actors performing lines, from which the network generated surprisingly sophisticated 3D facial animation. This, according Remedy and Nvidia, removes the hours of "labour-intensive data conversion and touch-ups" that are typically associated with traditional motion capture animation.

Aside from cost, facial animation, even when motion captured, rarely reaches the same level of fidelity as other animation. That odd, lifeless look seen in even the biggest of blockbuster games often came down to the limits of facial animation. Nvidia and Remedy believe its neural network solution is capable of producing results as good, if not better than what's produced by traditional techniques. It's even possible to skip the video altogether and feed the neural network a mere audio clip, from which it's able to produce an animation based on prior results.

The neural network is first fed a "high-end production facial capture pipeline based on multi-view stereo tracking and artist-enhanced animations," which essentially means feeding it information on prior animations Remedy has created. The network is said to require only five to 10 minutes of footage before it's able to produce animations based on simple monocular video capture of actors. Compared to results from state-of-the-art monocular and real-time facial capture techniques, the fully automated neural network produces eerily good results, with far less input required from animators.

With the network trained, it's possible to feed in an audio performance, which is then mapped to a 3D model to generate an animation. The process involves "a compact, latent code" that formulates the variations in facial expressions that can't be deduced from the audio recording. That code could potentially be user controlled, which would allow for the resulting animation to be tweaked.

https://youtu.be/VtttfrmfMZw

"Based on the Nvidia Research work we've seen in AI-driven facial animation, we're convinced AI will revolutionise content creation," said Antti Herva, lead character technical artist at Remedy. "Complex facial animation for digital doubles like that in Quantum Break can take several man-years to create. After working with Nvidia to build video- and audio-driven deep neural networks for facial animation, we can reduce that time by 80 percent in large scale projects and free our artists to focus on other tasks."

Nvidia had more AI-based shenanigans in store at Siggraph, training a neural network to recognise the jagged edges of 3D objects in games and replace them with smooth antialiased pixels. The technique supposedly produces sharper images than existing anti-aliasing techniques. Nvidia also used a neural network to speed up ray tracing rendering, a notoriously computationally heavy task, by training the network to distinguish the most useful light paths from paths that don't contribute to the resulting image.

While these techniques are impressive, it may be some time before they make their way into professional workflows.

Speaking of professional workflows, Nvidia also slyly dropped a driver update for its Titan Xp graphics card, which will apparently boost the performance of applications like Maya by up to three times. Such a large performance increase has left many users questioning whether Nvidia had been intentionally holding back Titan Xp performance in order to push users towards its more expensive Quadro graphics cards. The release of the similarly priced AMD Vega Frontier Edition card, which launched in June, may have had something to do with Nvidia's driver update: AMD pitched Vega FE as a combination production card and gaming card, beating the Titan out in several production tasks.

Meanwhile, AMD used Siggraph to launch its new RX Vega 64 and RX Vega 56 gaming graphics cards. RX Vega 64 is said to "trade blows" with the GTX 1080, while RX Vega 56 is said to perform along the same lines as a GTX 1070. Both cards are due for release on August 14 for $500 and $400 (UK pricing TBC).