Intelligent Anomaly Detection for Digital Transactions Using Machine Learning

Abstract: The exponential proliferation of digital payment mechanisms, such as the Unified Payments Interface (UPI), mobile financial wallets, and high-velocity digital payment gateways, has revolutionized consumer transactions while simultaneously creating fertile attack surfaces for automated cyber fraud. Traditional static rule-based security systems suffer from rigidity against zero-day evasion techniques, high false positive rates, and significant operational overhead. In this research, we propose FinGuard AI, an intelligent anomaly detection framework that couples domain-driven mathematical balance invariance with a Two-Phase Hybrid Machine Learning Engine comprising supervised classification, unsupervised spatial anomaly detection, and explainable decision support. Evaluated on a 20,000- transaction synthetic financial benchmark modeled after PaySim and Mendeley financial schemas (19,200 legitimate and 800 confirmed fraudulent transactions), the framework introduces four specialized balance discrepancy features based on double-entry accounting conservation principles. We perform a comparative evaluation across five machine learning archetypes: Logistic Regression (linear baseline), Decision Tree, Random Forest (100 estimators), a 3-layer Deep Neural Network (MLP 128-64-32), and Isolation Forest (unsupervised). The hybrid engine synthesizes posterior class probabilities with normalized Isolation Forest anomaly scores, weighted at 70 percent and 30 percent respectively, subject to domain safety invariant overrides. On the stratified test split of 4,000 transactions, Random Forest and Decision Tree achieved 100.0 percent accuracy (precision, recall, F1 score, and ROC-AUC of 1.000), Logistic Regression achieved 99.92 percent accuracy, Deep Neural Network achieved 99.85 percent accuracy (0.963 recall), and unsupervised Isolation Forest captured 109 out of 160 fraudulent transactions (97.28 percent accuracy, 0.937 ROC-AUC) without ground-truth label exposure. We explicitly analyze why synthetic data generation and balance discrepancy invariants yield near-perfect test separability, document current deployment limitations, and outline critical directions for real-world validation.

Keywords: Digital Transaction Fraud, Anomaly Detection, Machine Learning, Random Forest, Isolation Forest, Hybrid Detection, Financial Security, Explainable AI.