Detection of Mobile Malware (Android) using Machine Learning and Hybrid Analysis

Abstract: The exponential proliferation of mobile devices has catalyzed a parallel surge in Android malware, presenting critical security challenges in information technology. With Android commanding the lion’s share of the global mobile operating system market, it has become the primary target for malicious actors employing sophisticated evasion techniques such as dynamic code loading, reflection, and automated repackaging (obfuscation). Detecting zero-day malware—attacks that exploit previously unknown vulnerabilities—has thus become a paramount objective for security researchers. Traditional detection paradigms, which predominantly leverage signature-based static analysis, are increasingly rendered ineffective by polymorphic malware. Conversely, dynamic analysis, while robust, incurs prohibitive computational overhead, rendering it unsuitable for real-time, on-device application.

In this paper, we propose a novel, intelligent, two-stage hybrid framework that synergizes the efficiency of Deep Learning with the forensic depth of Hybrid Analysis. The proposed model operates on a ”Filter and Focus” principle. The first stage acts as a high-speed filter, employing a 1D Convolutional Neural Network (CNN) to analyze vectorized API call graphs extracted via FlowDroid. To address the ”black box” nature of neural networks, we integrate Gradient-weighted Class Activation Mapping (Grad-CAM) to provide visual explainability of malicious triggers. Furthermore, a Jaccard Similarity module compares these features against known threat signatures. Only applications classified as ’Benign’ or ’Uncertain’ by this stage are forwarded to the second stage, which employs a rigorous hybrid engine combining Mobile Security Framework (MobSF) for static deepinspection and Quark-Engine for dynamic behavioral graphing. Experimental results on a diverse dataset of 13,298 applications (including obfuscated samples from PRAGuard) demonstrate that our hybrid model achieves an accuracy of 97.79%, significantly outperforming standalone Deep Belief Networks (DBN) and Gated Recurrent Units (GRU). The system drastically reduces false positives while maintaining a low average latency, making it a viable solution for scalable, real-world Android security.

Keywords: Android Malware Detection, Network Security, Machine Learning, Static Analysis, Dynamic Analysis, CNN, Grad-CAM, Hybrid Analysis, Deep Belief Network (DBN), Gated Recurrent Unit (GRU), Cyber Threat Intelligence, Opcode Analysis, Adversarial Learning.