News Categories

NVIDIA uses AI for denoising, facial animation, trains them in Project Holodeck (Updated)

By Ian Chee - on 1 Aug 2017, 5:30am

NVIDIA uses AI for denoising, facial animation, trains them in Project Holodeck (Updated)

Updated August 1, 2017 at 3:30p.m.: Added more images to further illustrate the AI denoising.

Original article below:

NVIDIA has been making a big push in the world of artificial intelligence (AI), and this year looks to be no different, as the graphics company shows us more of what AI can do to help people in the creative industry. As the company looks to bring AI to the field of computer graphics, we can’t help but think that it’s very fitting to have new announcements made at SIGGRAPH 2017.

Some of the ways that the company is doing so is the employment of AI in denoising and facial animation. Starting with the former, NVIDIA seeks to use AI to shorten the amount of time it usually takes to complete the denoising process.

On the left is the noisy image, and on the right is the result of the AI denoising. <br> Image source: NVIDIA.

Image source: NVIDIA.

In training a neural network for the job, it is fed tens of thousands of image pairs, where one image is clear while the other is missing details, being rendered with only one light ray simulated per pixel compared to the usual hundreds or thousands. The deep neural network then learns from the comparisons of the different types of noise to correct, and as a result can denoise an image in one tenths of a second.

Image source: NVIDIA.

Another application of AI would be in facial animation. The idea here is to have the AI create 3D facial animations based on someone’s speech. Using a number of actors to speak in different emotional tones and languages, the training aims to have the AI understand the changes a face goes through when speaking, and having the AI generate its own speaking model that accurately reflects the movements of a person’s face when they are speaking. This will allow for games and CG movies to localize content for different languages without having to repeat the facial mocap recording process.

With that, NVIDIA also has the OptiX 5.0 SDK that is AI-optimized to just what we described above, as well as the DGX Station that makes use of it.

But what is the NVIDIA DGX Station? Imagine having the power of a supercomputer with 400 CPUs sitting on your desk, while using significantly less power and producing less noise. It comes with four Tesla V100 GPUs and 64GB of total GPU memory. That moves the AI training hardware from an entire room to the top of your desk.

NVIDIA has said that the DGX Station will be on display at SIGGRAPH 2017, where the company will also showcase Project Holodeck. We first caught a glimpse of it during COMPUTEX 2017, but SIGGRAPH is where folks will get the chance to get a hands-on experience of the VR demo featuring photorealistic models and haptic feedback.

The on-stage demo of Project Holodeck during COMPUTEX 2017.

As it turns out, Project Holodeck is also the tool that NVIDIA used in tandem with Isaac, the robot simulator. The idea behind Isaac is to train AI in a virtual environment instead of the real world, so that the AI can explore in the safety of the virtual world, where accidents and mishaps that are inevitable in the learning process does not damage the robot or the environment. NVIDIA will also be demostrating one of the results of Isaac at SIGGRAPH 2017, in the form of a robot with an AI that has been trained in the virtual world of Project Holodeck.

Image source: NVIDIA.

One final announcement that NVIDIA made was regarding its mobile Quadro GPUs. It’s been nearly two years since NVIDIA has ported desktop-level gaming capabilities to notebooks, and earlier this year during COMPUTEX 2017, the company ensured that future gaming notebooks will remain thin, while retaining that kind of graphical processing power. But all that is for the GeForce GTX GPUs for gamers, and not the Quadro series for creative professionals. For the working person who wants a thin and light machine that can be carried anywhere with ease but also needs the graphics processing power of a Quadro card, the solution seems to be an external GPU solution, similar to what ended up as the Alienware Graphics Amplifier.

Image source: NVIDIA.

Available later in September, NVIDIA has partnered with BIZON, Magma (now part of One Stop Systems) and Sonnet to provide external GPU (eGPU) solutions featuring high-end Quadro cards like the GP100 and P6000, alongside an eGPU box to be connected to the mobile workstation. You can also opt to just get the eGPU box if you already have an existing Quadro card that is compatible. Prices of these solutions start at about US$300 (~RM1,284). They do require a Thunderbolt 3 port with full bandwidth, and these boxes can only be used with mobile PCs, and not MacBooks.

By the time you read this, NVIDIA's press conference at SIGGRAPH would have just ended. We'll update this article if we come across additional announcements.

Follow us here for more on NVIDIA.