Title: Deep Illumination: Approximating Dynamic Global Illumination with Generative Adversarial Network,
Abstract: We present Deep Illumination, a novel machine learning technique for
approximating global illumination (GI) in real-time applications using a
Conditional Generative Adversarial Network. Our primary focus is on generating
indirect illumination and soft shadows with offline rendering quality at
interactive rates. Inspired from recent advancement in image-to-image
translation problems using deep generative convolutional networks, we introduce
a variant of this network that learns a mapping from Gbuffers (depth map,
normal map, and diffuse map) and direct illumination to any global illumination
solution. Our primary contribution is showing that a generative model can be
used to learn a density estimation from screen space buffers to an advanced
illumination model for a 3D environment. Once trained, our network can
approximate global illumination for scene configurations it has never
encountered before within the environment it was trained on. We evaluate Deep
Illumination through a comparison with both a state of the art real-time GI
technique (VXGI) and an offline rendering GI technique (path tracing). We show
that our method produces effective GI approximations and is also
computationally cheaper than existing GI techniques. Our technique has the
potential to replace existing precomputed and screen-space techniques for
producing global illumination effects in dynamic scenes with physically-based
rendering quality.