Agentic AI for Data Analysis: A RAG-Enhanced Local LLM Framework with Adaptive Visualization

Abstract: This paper presents an Agentic AI Data Analyst model that uses the RAG technique in combination with local large language models deployed using Ollama for privacy protection and low costs associated with intelligent data analysis. This system is no longer dependent on cloud APIs but is capable of providing high quality analysis due to improved incorporation of domain knowledge into the process. The model is coordinated via multi-step reasoning techniques facilitated by LangChain and supported by FAISS vector storage systems for efficient domain knowledge searching. A FastAPI application server runs in the background, providing connectivity to a React front end interface that allows dual modes of data visualization.
Regarding the performance analysis, five open-source language models were tested – LLaMA 3.2 (3B), LLaMA 3.1 (8B), Mistral 7B, Gemma 3 4B, and Gemma 3 12B. In addition, the assessment was conducted in terms of five core criteria: quality of responses, accuracy of the charts generated, depth of insights, rate of hallucinations, and time required for inference. Each of these models was evaluated with and without RAG implementation to determine the exact effects of retrieval augmentation. According to the results of the experiments, RAG offers a performance increase from 25% to 39% in addition to reducing the number of hallucinations by 62-75%. Most notably, RLaMA with 3B parameters and RAG always outperforms Gemma with 12B parameters but without RAG. These results confirm the hypothesis that retrieval of structured domain knowledge is more effective than scaling the number of model parameters. This research shows that lightweight agentic systems locally installed on a computer could serve as a viable alternative to commercial AI solutions.