Aegis: An AI-Based Network Intrusion Detection and Prevention System with Stateful Scan Detection and ISO/IEC 27001 Incident Reporting

Abstract: Most published machine learning intrusion detectors report accuracy above 99% on CIC-IDS2017, yet much of that figure comes from duplicate flows that end up in both the training and test sets. We built an AI-based network intrusion detection and prevention system (NIDS/NIPS) for Windows hosts that pairs flow classification with a stateful scan detector, decoy honeypot ports and a monitor for failed Windows logons. Alerts are mapped to MITRE ATT&CK techniques and ISO/IEC 27001:2022 controls. Before training, we removed host identifiers and 12,537 exact duplicate flows (22.16% of the data), and fitted the scaler on the training split alone. On the cleaned data XGBoost scored 99.62% accuracy, 99.39% macro F1 and a 0.55% false positive rate, with a median latency of 0.38 ms per flow; Random Forest scored 99.25%, 97.54%, 1.22% and 35.55 ms. When we held out the whole PortScan family, both supervised models called over 89% of those flows benign, and Isolation Forest caught only 0.31%. That result is why the system handles scans with a stateful detector instead of the classifier. Detections feed a four-tier escalation policy that writes Windows Firewall rules, but only after allowlist and subnet checks, with automatic expiry, and in dry-run mode unless an administrator switches enforcement on. A pure-Python flow extractor reproduces 56 CICFlowMeter features exactly and approximates 12 more, so live capture no longer needs Java.

Keywords: Network Intrusion Detection, Intrusion Prevention, XGBoost, Isolation Forest, SHAP, Data Leakage, CIC- IDS2017, Port Scan Detection, MITRE ATT&CK, ISO/IEC 27001