SentinelGuard: Explainable Detection of Spyware and Remote Access Trojans on Windows

Abstract: Remote access trojans (RATs) and other spyware can secretly use a computer's camera, contact command- and-control (C2) servers and upload data, while signature-based protection may miss new variants. This paper presents SentinelGuard, a software-only monitor for Windows personal computers that learns the normal behaviour of the PC on which it is installed and combines an unsupervised anomaly detector with exact rules. A per-feature normal-range model and an Isolation Forest operate on seven behavioural features, and SHAP values explain every AI alert in plain language; rules cover camera or microphone use by unexpected applications, per-application C2-style connection bursts, unknown local-network devices and strong unknown WiFi networks. To address RAT delivery, a Downloads guard inspects the Windows Mark-of-the-Web of new files and an optional, read-only email monitor checks attachments and links. A self- protecting learner removes rare upload bursts before training and retrains on a rolling 14-day window. On 215 held-out normal readings the false-alarm rate was 1.4%, and five synthetic attack scenarios were all flagged. In live, safe simulations the AI model detected cryptominer-like CPU load (HIGH) and data upload (CRITICAL) with explanations naming the responsible feature, the C2 rule detected every simulated connection burst, a second PC detected all three demonstration steps including camera use, and test RAT-delivery downloads and emails were flagged within 7 s and 13 s. Self-cleaning lowered a learned upload limit inflated by a burst from about 4,201 KB/s to about 298 KB/s. Camera attribution failed on an updated Windows 11 build, and all attacks were simulated.

Keywords: Anomaly detection, explainable artificial intelligence, Isolation Forest, remote access trojan, SHAP, spyware detection, Windows security.