Reconstructing a 3D scene from images is challenging due to the different ways light interacts with surfaces depending on the viewer's position and the surface's material. In classical computer graphics, materials can be classified as diffuse or specular, interacting with light differently. The standard 3D Gaussian Splatting model struggles to represent view-dependent content, since it cannot differentiate an object within the scene from the light interacting with its specular surfaces, which produce highlights or reflections. In this paper, we propose to extend the 3D Gaussian Splatting model by introducing an additional symmetric matrix to enhance the opacity representation of each 3D Gaussian. This improvement allows certain Gaussians to be suppressed based on the viewer's perspective, resulting in a more accurate representation of view-dependent reflections and specular highlights without compromising the scene's integrity. By allowing the opacity to be view dependent, our enhanced model achieves state-of-the-art performance on Mip-Nerf, Tanks&Temples, Deep Blending, and Nerf-Synthetic datasets without a significant loss in rendering speed, achieving >60FPS, and only incurring a minimal increase in memory used.
从图像重建三维场景具有挑战性，因为光与表面的相互作用方式取决于观察者的位置和表面材质。在经典计算机图形学中，材质通常分为漫反射和镜面反射，它们与光的交互方式不同。标准的 3D 高斯点渲染模型难以表示与视角相关的内容，因为它无法区分场景中的物体与光在其镜面反射表面上的交互，从而导致高光或反射效果的缺失。 在本文中，我们提出了一种改进的 3D 高斯点渲染模型，通过引入一个额外的对称矩阵来增强每个 3D 高斯的不透明度表示。此改进使得某些高斯可以根据观察者的视角被抑制，从而更准确地表示视角相关的反射和镜面高光，同时保持场景的完整性。通过使不透明度随视角变化，我们的增强模型在 Mip-Nerf、Tanks&Temples、Deep Blending 和 Nerf-Synthetic 数据集上达到了最先进的性能，同时渲染速度未显著下降（>60FPS），且仅带来了极小的内存开销增加。