Unmasking Deepfakes: Audio-Visual Deepfake Detection Using AI

Abstract: The increasing realism of synthetic and manipulated media has made the authentication of digital video and audio a significant multimedia-forensics problem. Visual-only detectors may miss audio manipulation, while audio-only detectors cannot inspect facial artifacts or temporal inconsistencies. This paper presents a source- grounded description of “Unmasking Deepfakes: Audio-Visual Detection Using AI,” a multimodal detection system documented in the project’s Phase-2 presentation. The visual pathway samples video frames, detects facial regions using YOLOv8, extracts spatial representations using XceptionNet, and models frame-sequence dynamics using a bidirectional long short-term memory (BiLSTM) network. In parallel, the audio pathway extracts the soundtrack, converts it into a Mel-spectrogram representation, and applies a convolutional neural network (CNN) for audio classification. The documented design combines modality-specific information through an attention layer and late-fusion mechanism to produce a Real/Fake prediction and confidence score. A Gradio interface supports video upload and result presentation, while Grad-CAM is identified for visual explanation. The project presentation records the principal processing modules as implemented but states that module/system testing and final performance evaluation remain in progress. Accordingly, this paper contributes a formalized system description, mathematical formulation, inference procedure, and reproducible evaluation plan; it does not claim unverified performance. Dataset identity, split protocol, training configuration, fusion implementation, and measured results must be supplied from experiment logs before quantitative conclusions or deployment claims can be made.

Keywords: Deepfake detection; audiovisual forensics; multimodal learning; YOLOv8; XceptionNet; BiLSTM; Mel- spectrogram; late fusion.