期刊
ACM TRANSACTIONS ON GRAPHICS
卷 41, 期 6, 页码 -出版社
ASSOC COMPUTING MACHINERY
DOI: 10.1145/3550454.3555497
关键词
point-based rendering; neural rendering; differentiable rasterization; reflections; catacaustics
资金
- ERC Advanced grant FUNGRAPH [788065]
- GENCI-IDRIS [2022-AD011013409]
This paper addresses the challenges of view-dependent effects, such as reflections, for image-based and neural rendering algorithms. The authors propose a new point-based representation to compute Neural Point Catacaustics, enabling the synthesis of novel views with curved reflectors. Their approach leverages a neural warp field to model catacaustic trajectories of reflections, allowing efficient rendering of complex specular effects.
View-dependent effects such as reflections pose a substantial challenge for image-based and neural rendering algorithms. Above all, curved reflectors are particularly hard, as they lead to highly non-linear reflection flows as the camera moves. We introduce a new point-based representation to compute Neural Point Catacaustics allowing novel-view synthesis of scenes with curved reflectors, from a set of casually-captured input photos. At the core of our method is a neural warp field that models catacaustic trajectories of reflections, so complex specular effects can be rendered using efficient point splatting in conjunction with a neural renderer. One of our key contributions is the explicit representation of reflections with a reflection point cloud which is displaced by the neural warp field, and a primary point cloud which is optimized to represent the rest of the scene. After a short manual annotation step, our approach allows interactive high-quality renderings of novel views with accurate reflection flow. Additionally, the explicit representation of reflection flow supports several forms of scene manipulation in captured scenes, such as reflection editing, cloning of specular objects, reflection tracking across views, and comfortable stereo viewing. We provide the source code and other supplemental material on https://repo-sam.inria.fr/fungraph/neural_catacaustics/
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据