Multimodally controlled intelligent telerobot for people with disabilities

Zunaid Kazi; Shoupu Chen; Matthew Beitler; Daniel Chester; Richard Foulds

doi:10.1117/12.263001

17 December 1996 Multimodally controlled intelligent telerobot for people with disabilities

Zunaid Kazi, Shoupu Chen, Matthew Beitler, Daniel Chester, Richard Foulds

Proceedings Volume 2901, Telemanipulator and Telepresence Technologies III; (1996) https://doi.org/10.1117/12.263001
Event: Photonics East '96, 1996, Boston, MA, United States

Abstract

This paper reports on the current status of the multimodal user supervised interface and intelligent control (MUSIIC) project, which is working towards the development of an intelligent assistive telemanipulative system for people with motor disabilities. Our MUSIIC strategy overcomes the limitations of previous approaches by integrating a multimodal RUI (robot user interface) and a semi-autonomous reactive planner that will allow users with severe motor disabilities to manipulate objects in an unstructured domain. The multimodal user interface is a speech and deictic (pointing) gesture based control that guides the operation of a semi-autonomous planner controlling the assistive telerobot. MUSIIC uses a vision system to determine the three-dimensional shape, pose and color of objects and surfaces which are in the environment, and as well as an object-oriented knowledge base and planning system which superimposes information about common objects in the three-dimensional world. This approach allows the users to identify objects and tasks via a multimodal user interface which interprets their deictic gestures and a restricted natural language like speech input. The multimodal interface eliminates the need for general purpose object recognition by binding the users speech and gesture input to a locus in the domain of interest. The underlying knowledge-driven planner, combines information obtained from the user, the stereo vision mechanism as well as the knowledge bases to adapt previously learned plans to perform new tasks and also to manipulate newly introduced objects into the workspace. Therefore, what we have is a flexible and intelligent telemanipulative system that functions as an assistive robot for people with motor disabilities.

Citation Download Citation

Zunaid Kazi, Shoupu Chen, Matthew Beitler, Daniel Chester, and Richard Foulds "Multimodally controlled intelligent telerobot for people with disabilities", Proc. SPIE 2901, Telemanipulator and Telepresence Technologies III, (17 December 1996); https://doi.org/10.1117/12.263001

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available