L’arrive´e d’un train en gare de Princeton Junction: Multiple Sweet Spot Binaural 3D Audio for Active Stereoscopic 3D Video

Abstract

Several 3D auralization and visualization methods exist for a range of virtual reality applications, but very little research has been documented on the combination of technologies. This work represents the first attempt to combine multiple listener, moving sweet spot binaural 3D audio with active stereoscopic 3D video. Each auralization and visualization method has its own inherent challenges and benefits, but binaural 3D audio and active stereoscopic 3D video were implemented to offer high fidelity, practically recorded and reconstructed auditory sweet spots, as well as high visual resolution. Head tracking and beam steering were used to create multiple, moving sweet spots. Referred to as 3D3D, the combination was applied to cinematic entertainment to illustrate its potential to improve realism and immersion. A film was produced in Princeton University’s 3D3A laboratory that emphasizes the superior impact of 3D3D by employing it in a progression of audio and video reproduction technologies. Future interdisciplinary applications of 3D3D are far-reaching and diverse, and include training simulators, product and architectural design interfaces, learning environments, and communication systems. 3D3D research is motivated by problems inherent in both 3D video and 3D audio. 3D audio can be difficult for listeners to externally spatialize without appropriate visual cues. Furthermore, viewers of 3D video typically suffer from several adverse symptoms, including eye fatigue, headache, and motion sickness. There is substantial psychological research presented, however, to support the argument that the combination of 3D video and 3D audio can be mutually beneficial by fostering multimodal processing. An original design for a psychological study is proposed to implicitly and explicitly test the success of 3D3D in improving realism and immersion.