Multimodal perception study on virtual 3D curved textures with vision and touch for interactive multimedia systems

Understanding the multimodal rendering of 3D shapes is becoming an important research topic as multimedia and virtual reality technologies are rapidly advancing. This study is aimed to investigate human perceptibility on the curvature and texture changes of 3D virtual surfaces across modalities, vision and touch. Our interest is to obtain perception data that can be used for 3D watermarking or data compression under a virtual reality environment providing multimodal interactions. For this study, we designed two psychophysical experiments to estimate curvature discrimination and texture detection thresholds on curvature surfaces over three conditions: vision only, touch only, and both vision and touch. The results show that touch is dominant at both discriminating curvature surfaces and detecting surface texture changes on a curved surface. In addition, the sensitivity of the both senses to detect texture changes linearly increases as a curvature value increases. Finally, the vision and touch senses compensate each other when both modalities are available at the same time. The thresholds from the present study can potentially be used as the upper limit for selecting watermark strengths or compression in order to ensure imperceptibility in a 3D visuohaptic multimedia systems.