AI-Avatar Chatbot

Abstract: To overcome the limitations of traditional text-based conversational interfaces, the authors of this paper introduce a real-time, multimodal system called KARS AI that facilitates more immersive humancomputer interaction, enabling AI avatar chatbots to provide their services. The system features a React.js/Vite frontend, a FastAPI backend, Supabase database, and a cloud database, ensuring seamless text or voice communication with a dynamic 3D avatar representation, and combined with local LLM inference engine - Ollama. The main features are the real-time lip-syncing, facial expressions, and gesture animation, which have been generated with Three.js and React Three Fiber, in addition to low latency communication protocols through WebSocket. The avatar responds to AI states (Listening, Talking, Explaining) to make the experience emotionally engaging, beyond the typical chatbot experience. Advanced features include conversation storing by session, chat history management, avatar customization and therapy mode. Evaluation shows that the system is effective in providing natural, immersive dialogue experiences and is the basis for next- generation conversational AI applications in education, therapy, customer support and virtual companionship. This work is an embodied CTA that aims to combine these technologies in a practical application, resulting in a new research field in the study of embodied conversational agents.

Keywords: Conversational AI, Large Language Models (LLM), 3D Avatar Systems, Multimodal Interaction, Real-time

Chatbots.