Real-Time Sign Language Translator Using Video Prediction

Abstract: Sign language is a visual language in which meaning is conveyed through hand shape, movement, body posture, facial expression, and temporal relationships between gestures. Automatic translation from video to text can reduce communication barriers between sign-language users and people who do not know the language. This paper presents a real-time sign language translator using video prediction, in which a webcam or uploaded video is processed as a sequence rather than as independent images. The proposed framework combines video acquisition, frame preprocessing, hand and pose landmark extraction, spatial feature learning, temporal sequence modelling, sign prediction, text generation, and optional speech synthesis. OpenCV can be used for frame handling, while MediaPipe provides efficient landmark extraction. CNN(Convolutional Neural Network), GRU(Gated Recurrent Unit), LSTM( Long Short-Term Memory), or Transformer models can then learn spatial and temporal patterns in the video. Recent research from 2020 to 2026 shows a transition from static and isolated-sign recognition toward continuous sign- language recognition and end-to-end sign-to-text translation. However, practical deployment remains challenging because of signer variation, occlusion, variable signing speed, limited continuous datasets, non-manual cues, temporal alignment, and computational latency. The proposed research direction therefore emphasizes efficient video prediction, robust temporal modelling, and real-time deployment, with particular relevance to Indian Sign Language.

Keywords: Sign Language Recognition, Video Prediction, Computer Vision, Deep Learning, MediaPipe, Temporal Modeling, Transformer, Indian Sign Language, Real-Time Translation, Speech Synthesis.