Neural scene baking for permutation invariant transparency rendering with real-time global illuminationOA

Neural rendering provides a fundamentally new way to render photorealistic images.Similar to traditional light-baking methods,neural rendering utilizes neural networks to bake representations of scenes,materials,and lights into latent vectors learned from path-tracing ground truths.However,existing neural rendering algorithms typically use G-buffers to provide position,normal,and texture information about scenes.These are prone to occlusion by transparent surfaces,leading to distortion and loss of detail in rendered images.To address this limitation,we propose a novel neural rendering pipeline that accurately renders the scene behind transparent surfaces with global illumination and variable scenes.Our method separates the G-buffers for opaque and transparent objects,retaining G-buffer information behind transparent objects.Additionally,to render transparent objects with permutation invariance,we have designed a new permutation-invariant neural blending function.We have integrated our algorithm into an efficient custom renderer,achieving real-time performance.Our results show that our method is capable of rendering photorealistic images for variable scenes and viewpoints,accurately capturing complex transparent structures along with global illumination.Our renderer can achieve real-time performance(256×256 at 63 frames/s and 512×512 at 32 frames/s)for scenes with multiple variable transparent objects.