Class-imbalance-aware Multiclass Attack Classification Using Random Forest on the Unsw-nb15 Dataset

Abstract: Binary intrusion detection separates normal traffic from malicious traffic, but an operational security system also benefits from identifying the specific attack category associated with an alert. This paper investigates multiclass attack-category classification on the UNSW-NB15 dataset using Random Forest under the dataset's uneven class distribution. A reproducible stratified sample of 20,000 training records and 10,000 testing records was extracted from the supplied training and testing partitions. The target was changed from the binary label used in the earlier studies to the ten-class attack_cat variable, consisting of Normal and nine attack categories. Identifier and binary-label fields were excluded from model inputs to avoid leakage. Numerical attributes were median-imputed and standardized, while categorical attributes were imputed and one-hot encoded. Two Random Forest configurations were evaluated: a standard model and a class-balanced model using balanced_subsample weighting. The standard model achieved 76.13% accuracy and 44.41% macro-F1, whereas the balanced model achieved 75.97% accuracy, 45.04% macro-F1, 77.93% weighted-F1, 46.85% balanced accuracy, and an MCC of 0.686. Class weighting slightly improved macro- level performance and weighted F1 while maintaining comparable overall accuracy. The results also reveal severe difficulty in detecting the rare Analysis, Backdoor, and Worms classes, demonstrating that overall accuracy alone is insufficient for multiclass intrusion-detection evaluation.

Keywords: UNSW-NB15, multiclass classification, intrusion detection, Random Forest, class imbalance, attack categories, network security