A Multimodal AI Framework for Real-Time Audience Engagement Detection in Virtual Communication

Abstract: The way we globally collaborate has been significantly impacted by virtual communication tools. However, these tools are not effective in conveying nonverbal cues of engagement, which are important in the process of effective human interaction. Real-time audience attention evaluation is a problem that faces the presenter in virtual meetings and classrooms. This study discusses the AI-based methodology that can be used in detecting audience engagement via multimodal emotion recognition.The interactive Speaker Dashboard also displays individual participant engagement scores, which are determined by a Node.js backend server running a Video Analyzer, an Audio Analyzer, and an Engagement Engine. This proposed framework is applicable in various scenarios, such asremote training sessions, corporate meetings, virtual classroom scenarios, and customer support interactions. This framework also indicates better detection accuracy.This sensitivity to real participant behaviour can be seen in the experimental results, which demonstrate that the level of engagement aligns with the attentiveness pattern of live sessions.

Keywords: Engagement detection; Multimodal emotion recognition; Facial expression recognition; Speech emotion analysis; Affective computing; WebRTC; Virtual collaboration; Deep learning; Attention tracking.