Journal
ACM TRANSACTIONS ON GRAPHICS
Volume 37, Issue 4, Pages -Publisher
ASSOC COMPUTING MACHINERY
DOI: 10.1145/3197517.3201391
Keywords
360 degrees videos; ambisonic audio
Categories
Funding
- National Science Foundation [CAREER-1453101]
- Adobe Research Fellowship
- SoftBank Group
- Direct For Computer & Info Scie & Enginr
- Div Of Information & Intelligent Systems [1453101] Funding Source: National Science Foundation
Ask authors/readers for more resources
Although 360 degrees cameras ease the capture of panoramic footage, it remains challenging to add realistic 360 degrees audio that blends into the captured scene and is synchronized with the camera motion. We present a method for adding scene-aware spatial audio to 360 degrees videos in typical indoor scenes, using only a conventional mono-channel microphone and a speaker. We observe that the late reverberation of a room's impulse response is usually diffuse spatially and directionally. Exploiting this fact, we propose a method that synthesizes the directional impulse response between any source and listening locations by combining a synthesized early reverberation part and a measured late reverberation tail. The early reverberation is simulated using a geometric acoustic simulation and then enhanced using a frequency modulation method to capture room resonances. The late reverberation is extracted from a recorded impulse response, with a carefully chosen time duration that separates out the late reverberation from the early reverberation. In our validations, we show that our synthesized spatial audio matches closely with recordings using ambisonic microphones. Lastly, we demonstrate the strength of our method in several applications.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available