Journal
PROCEEDINGS 2017 INTERNATIONAL CONFERENCE ON 3D VISION (3DV)
Volume -, Issue -, Pages 621-629Publisher
IEEE
DOI: 10.1109/3DV.2017.00076
Keywords
-
Categories
Funding
- EPSRC Programme Grant S3A: Future Spatial Audio for an Immersive Listener Experience at Home [EP/L000539/1]
- BBC as part of the BBC Audio Research Partnership
- EPSRC [EP/P022529/1] Funding Source: UKRI
Ask authors/readers for more resources
In this paper we propose a cuboid-based air-tight indoor room geometry estimation method using combination of audio-visual sensors. Existing vision-based 3D reconstruction methods are not applicable for scenes with transparent or reflective objects such as windows and mirrors. In this work we fuse multi-modal sensory information to overcome the limitations of purely visual reconstruction for reconstruction of complex scenes including transparent and mirror surfaces. A full scene is captured by 360 degrees cameras and acoustic room impulse responses (RIRs) recorded by a loudspeaker and compact microphone array. Depth information of the scene is recovered by stereo matching from the captured images and estimation of major acoustic reflector locations from the sound. The coordinate systems for audiovisual sensors are aligned into a unified reference frame and plane elements are reconstructed from audio-visual data. Finally cuboid proxies are fitted to the planes to generate a complete room model. Experimental results show that the proposed system generates complete representations of the room structures regardless of transparent windows, featureless walls and shiny surfaces.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available