International Journal of Advanced Research in Computer and Communication Engineering

Abstract: Communication barriers between hearing-impaired and hearing individuals remain a significant challenge in everyday interactions. While spoken and written languages are widely supported by modern technologies, sign language communication still lacks accessible and real-time translation solutions. This project presents a Multilingual Speech-to-Sign Language Translator with Avatar, designed to bridge this communication gap using artificial intelligence and human–computer interaction techniques.
The proposed system accepts user input in the form of speech or text, converts it into a target language using multilingual translation models, and represents the translated content through a 3D animated sign language avatar. In addition, the system integrates real-time hand gesture recognition using computer vision techniques to identify basic sign gestures and map them to corresponding textual meanings. This bidirectional interaction enables both hearing and hearing-impaired users to communicate more naturally.
The system architecture combines speech recognition, language translation, text-to-speech synthesis, gesture detection, and avatar animation into a unified web-based platform. By processing inputs locally and rendering sign outputs visually, the system ensures low latency and improved user experience. Experimental evaluation demonstrates accurate speech recognition, smooth avatar animation, and effective translation across multiple languages.
The proposed solution offers an affordable and scalable assistive communication tool that can be deployed in educational institutions, public service centers, and social interaction platforms. By enhancing accessibility and inclusivity, this work contributes toward improving digital communication for the hearing-impaired community while supporting multilingual interaction in real time.

Keywords: Speech-to-Sign Translation, Sign Language Avatar, Gesture Recognition, Multilingual Translation, Assistive Technology, Human-Computer Interaction

Downloads: | DOI: 10.17148/IJARCCE.2026.151143