An innovative multimodal virtual platform for communication with devices in a natural way

Chhayarani R. Kinkar; Richa Golash; Akhilesh R. Upadhyay

doi:10.1117/12.907305

8 February 2012 An innovative multimodal virtual platform for communication with devices in a natural way

Chhayarani R. Kinkar, Richa Golash, Akhilesh R. Upadhyay

Proceedings Volume 8289, The Engineering Reality of Virtual Reality 2012; 82890O (2012) https://doi.org/10.1117/12.907305
Event: IS&T/SPIE Electronic Imaging, 2012, Burlingame, California, United States

Abstract

As technology grows people are diverted and are more interested in communicating with machine or computer naturally. This will make machine more compact and portable by avoiding remote, keyboard etc. also it will help them to live in an environment free from electromagnetic waves. This thought has made 'recognition of natural modality in human computer interaction' a most appealing and promising research field. Simultaneously it has been observed that using single mode of interaction limit the complete utilization of commands as well as data flow. In this paper a multimodal platform, where out of many natural modalities like eye gaze, speech, voice, face etc. human gestures are combined with human voice is proposed which will minimize the mean square error. This will loosen the strict environment needed for accurate and robust interaction while using single mode. Gesture complement Speech, gestures are ideal for direct object manipulation and natural language is used for descriptive tasks. Human computer interaction basically requires two broad sections recognition and interpretation. Recognition and interpretation of natural modality in complex binary instruction is a tough task as it integrate real world to virtual environment. The main idea of the paper is to develop a efficient model for data fusion coming from heterogeneous sensors, camera and microphone. Through this paper we have analyzed that the efficiency is increased if heterogeneous data (image & voice) is combined at feature level using artificial intelligence. The long term goal of this paper is to design a robust system for physically not able or having less technical knowledge.

Citation Download Citation

Chhayarani R. Kinkar, Richa Golash, and Akhilesh R. Upadhyay "An innovative multimodal virtual platform for communication with devices in a natural way", Proc. SPIE 8289, The Engineering Reality of Virtual Reality 2012, 82890O (8 February 2012); https://doi.org/10.1117/12.907305

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Data fusion

Sensors

Cameras

Image fusion

Human-computer interaction

System integration

Artificial intelligence

Show All Keywords

Keywords/Phrases

Search In:

Publication Years