A Voice-First Personal Finance Assistant Using Tool Calling and a Locally Served Fine-Tuned Small Language Model

Abstract: Large language model assistants can answer questions about personal finances, but can also make mistakes, upload sensitive data or run actions without human approval. FIN-AI is a voice-first personal finance assistant for Indian users with local data storage, rupee amounts, loan instalments, and lakh/crore units. A model is only used to select a tool, calculations are carried out by Python, and answers are phrased with templates; model-proposed modifications are confirmed via Proceed cards. Queries follow a three-tier pipeline: offline intent classifier, a locally hosted Qwen3-1.7B with QLoRA, and cloud LLMs as fallback. Training lessons are created from the real tool registry; the first model was fine-tuned on a 4 GB laptop GPU and later ones on a free cloud T4. In 67 hand-written questions, tool-selection accuracy increased from 71.6% (untuned) to 91.0% after the first fine-tuning (exact McNemar p = 0.0002); the final version scored 89.6% (Wilson 95% interval 80.0–94.8%). The fast path handled 98 of 105 read-only questions offline. Development evaluations are presented, and the data sent to external AI, speech, and market-data services is detailed.

Keywords: Voice assistant, Personal finance, Tool calling, Small language model, QLoRA, Query routing.