← Back to VOLUME 15, ISSUE 11, NOVEMBER 2026
This work is licensed under a Creative Commons Attribution 4.0 International License.
SignAI: Intelligent Bidirectional Indian Sign Language Translation Using Skeleton Keypoints
Anand S. Balapure, Vedant Y. Badgujar, Netra A. Jayaswal, Tejaswini B. Sharma, Sheetal P. Patil
π 13 viewsπ₯ 6 downloads
Abstract: Communication between Deaf and hearing individuals remains difficult because interpreters and accessible translation tools are scarce. This paper presents SignAI, a bidirectional Indian Sign Language (ISL) translation prototype that combines Sign-to-Text (S2T) and Text-to-Sign (T2S) translation in a single web application. The S2T module classifies isolated signs from MediaPipe Holistic keypoints using a Conv1DβBiGRUβattention network, supported by motion-based segmentation and confidence gating; recognised glosses are turned into English sentences with a rule-based generator or few-shot prompting of FLAN-T5-base. The T2S module maps text to validated glosses and renders the matching sign clips as a skeleton video, with optional PoseTransformer and fingerspelling support. On a custom 60-class ISL adjective dataset of 1,022 clips, the classifier reaches 74.29% top-1 and 88.57% top-5 accuracy on the test set (n = 140; 95% CI: 66.47β80.81%). At a confidence threshold of 0.65, 89.16% of accepted predictions are correct at 59.29% coverage. A data and code audit and its corrective measures are also reported.
Keywords: Sign language recognition, Indian Sign Language, MediaPipe, skeleton keypoints, bidirectional GRU, sign language production, confidence gating.
Keywords: Sign language recognition, Indian Sign Language, MediaPipe, skeleton keypoints, bidirectional GRU, sign language production, confidence gating.
How to Cite:
[1] Anand S. Balapure, Vedant Y. Badgujar, Netra A. Jayaswal, Tejaswini B. Sharma, Sheetal P. Patil, βSignAI: Intelligent Bidirectional Indian Sign Language Translation Using Skeleton Keypoints,β International Journal of Advanced Research in Computer and Communication Engineering (IJARCCE), DOI: 10.17148/IJARCCE.2026.151116
