top of page

G-SIAR: Gesture-Speech Interface for AR

For a rapid adoption of natural interaction in Augmented Reality (AR) , the techniques proposed must offer a good precision and it must be easy to learn and perform. Recent research explored free-hand gesture interaction in AR but only a few formal evaluations  were conducted.

We introduce and evaluate two natural interaction techniques: a natural hand interaction, Grasp-Shell, which provides direct physical manipulation of virtual content; and the multi-modal interaction, Gesture-Speech, which combines speech and gestures for an indirect natural interaction. These techniques support object selection, 6 dof movement, uniform scaling, as well as physics-based interaction such as pushing and flinging.

We evaluated and compared these two techniques on fundamental manipulation tasks. The results show that they are a good complement. We proposed that AR interface offers a combination of both interaction techniques to provide better control and interactivity in a single interface.

T. Piumsomboon, D. Altimira, H. Kim, A. Clark, G. Lee, and M. Billinghurst. Grasp-Shell vs Gesture-Speech: A comparison of direct and indirect natural interaction techniques in Augmented Reality. In 13th IEEE International Symposium on Mixed and Augmented Reality (ISMAR'14), pp. 73-82, 2014.

T. Piumsomboon, A. Clark, and M. Billinghurst. G-SIAR: Gesture-Speech Interface for Augmented Reality. In 13th IEEE International Symposium on Mixed and Augmented Reality (ISMAR'14), pp. 365-366, 2014.

Research Article 5
bottom of page