AI Agents in Cybersecurity: A Systematic Review of Architectures, Applications, Threats, Challenges and Future Directions

Abstract: The integration of artificial intelligence (AI) agents into cybersecurity operations has progressed rapidly from rule-based automation towards tool-augmented and increasingly autonomous systems, yet the literature remains fragmented across architectures, autonomy models, application domains and security risks. This article presents a systematic review of AI-agent architectures, applications, threats, challenges and future directions in cybersecurity. Following PRISMA 2020 guidelines, we searched Crossref and arXiv (January 2021 – September 2026) using agent- and security-related term combinations, screened records against predefined inclusion criteria, and appraised study quality with an eight-item checklist. From 4,309 Crossref and 2,000 arXiv records identified, 55 studies published 2021–2026 were included. We contribute a five-dimension taxonomy covering architecture, intelligence mechanism, tool interaction, coordination and a five-level autonomy scale (advisory to adaptive-autonomous); a synthesis of ten application domains, in which penetration testing, autonomous cyber defence and security-operations-centre automation dominate; and a six-category threat classification spanning input-, model-, tool-, memory-, agent- interaction- and infrastructure-level risks, including prompt injection, tool misuse, memory poisoning and excessive agency. The analysis reveals that single tool-augmented LLM agents prevail, that reported autonomy rarely exceeds supervised semi-autonomy, and that evaluation practices remain fragmented and benchmark-immature. We identify five evidence-based gaps—absence of standardised agentic-security benchmarks, limited real-world deployment evidence, inconsistent autonomy definitions, weak safety evaluation, and insufficient research on securing agents themselves—and derive future research directions, including trustworthy autonomy governance, secure agent–tool protocols and human–agent teaming. The review clarifies the dual role of AI agents as both defensive instruments and an expanding attack surface.

Keywords: AI agents; Agentic AI; Cybersecurity; Large Language Models; Autonomous Cyber Defence; Multi-Agent Systems; Threat Detection; Security Automation