A Machine Learning Approach to Lip Reading Automation System

Abstract: Artificial Intelligence is extensively used to detect the movement of lips. It is observed that there is a high Correlation between the visual motion of mouth and corresponding audio data. This fact has been utilized for lip reading and for improving speech recognition. A Convoluted Neural Network would detect the movement of lips and determine the words spoken. The words that are spoken in the video would be detected by the Trained CNN and displayed in the text format. The CNN relies on information provided by the context, knowledge of the language, and any residual hearing. The aim is to verify whether the use of artificial intelligence methods, namely Deep Neural Network, is a suitable candidate for solving this problem. Practically, the focus is on presenting the results in terms of the accuracy of the trained neural network on test data.

Keywords: Artificial Intelligence, Lip Reading, Deep Neural Network, CNN, Machine learning.