Sign to Speech

Abstract: This paper addresses the development of a hand gesture recognition system for Indian Sign Language. In this a Sensor Glove is developed which is used by speech impaired people to communicate with others. The Glove converts the hand gestures into corresponding text as well as speech. The Hand glove is composed of 5 flex sensors, a MPU 6050 which combination of both accelerometer and gyroscope, a Microcontroller which is Arduino Pro Micro and Bluetooth transceiver HC-05. The readings from the sensors send by the microcontroller to the Mobile Application using Bluetooth connectivity via Bluetooth transceiver. Mobile applications directly send these readings to the server using UDP (User Datagram Packet) Connection. On the Server side hand gesture is recognized using Artificial Neural Networks (ANNs). A Machine Learning algorithm using Multilevel Perceptron is implemented on Server, to which a input of 78 features extracted from readings are given a and output is obtained as recognition of words or letters corresponding to the gesture. This output is send back to the App where it is displayed as text as well as can be heard as a speech. Thus in this way system helps to give the voice to the speech impaired people.

Keywords: Sign Language, Flex Sensors, 3-axis Accelerometer, Microcontroller, Bluetooth Transceiver, Multilayer Perceptron.