A random forest trained only on normal network traffic will flag anything it has not seen before. That is both the strength and the fundamental weakness of anomaly-based intrusion detection. The model does not know what an attack looks like. It knows what ordinary looks like, and it rejects everything else. For a red teamer, this means the target is not a decision boundary between “malicious” and “benign.” The target is the definition of normal itself.

Earlier in this series, we covered ensemble methods and the mechanics of how random forests aggregate decision trees to reduce variance and resist overfitting. We also covered anomaly detection algorithms and the ways their assumptions create exploitable attack surfaces. This entry brings those two threads together in a practical setting, training a random forest classifier on the NSL-KDD dataset to detect anomalous network traffic and examining every step of the data loading process for what it reveals about the model’s eventual blind spots.

Why random forests work for network anomaly detection

A random forest is an ensemble of decision trees, each trained on a bootstrapped sample of the data with a random subset of features considered at every split. In classification, each tree votes for a class and the majority wins. In regression, the outputs are averaged. The randomness injected at both the data and feature level forces diversity among the trees, which reduces the correlation between their errors and produces a model that generalises better than any individual tree.

For anomaly detection specifically, the approach is straightforward. Train the forest exclusively on data representing normal network conditions. When new traffic arrives, run it through the trained ensemble. If the forest produces low-confidence predictions or classifies the input as something it was not trained to recognise, that input gets flagged as anomalous.

This works well in practice because network traffic data is high-dimensional, with dozens of features describing connection duration, byte counts, protocol types, error rates, and service flags. A single decision tree would overfit to noise in that feature space. The ensemble averaging smooths out individual tree errors, and the random feature selection at each split means different trees learn different aspects of what “normal” looks like. The result is a model that captures the full shape of normal traffic rather than memorising specific patterns.

The adversarial implication is worth noting early. Because each tree sees a random subset of features, an attacker who manipulates only one or two features may fool individual trees but is unlikely to flip the majority vote. Giovanni Apruzzese and colleagues at the University of Modena demonstrated in 2020 that random forest-based network detectors show moderate inherent resistance to adversarial perturbations compared to single-model architectures. However, that resistance breaks down under stronger, multi-feature attacks that perturb enough dimensions simultaneously to shift the ensemble consensus.

The NSL-KDD dataset

The original KDD Cup 1999 dataset was the standard benchmark for intrusion detection research for over a decade, but it had problems that distorted results. Redundant records in the training set meant classifiers could achieve high accuracy by memorising duplicates rather than learning genuine patterns. The class distribution was heavily skewed, with some attack types massively over-represented and others nearly absent. Models trained on KDD Cup 99 looked good on paper and performed poorly in practice.

The NSL-KDD dataset, developed by researchers at the Canadian Institute for Cybersecurity, fixes both issues. It removes redundant records from the training and test sets, which eliminates the memorisation shortcut. It rebalances the class distribution so that classifiers cannot coast on majority-class bias. The dataset contains labelled instances of both normal traffic and several categories of malicious activity (DoS, probe, R2L, and U2R attacks), which supports both binary classification (normal versus attack) and multi-class detection targeting specific attack types.

We are using a modified version of NSL-KDD for this exercise. It remains the most widely used benchmark for IDS research because of its compact size, accessibility, and the fact that it exposes class imbalance issues rather than hiding them.

From a red teaming perspective, the dataset itself is informative. The 41 features in NSL-KDD represent the specific dimensions along which the model will learn to distinguish normal from anomalous. That feature list is a map of what the IDS can see, and by extension, what it cannot. If an attacker knows the model was trained on NSL-KDD-style features, they know exactly which aspects of their traffic need to blend in and which dimensions are not being monitored at all.

Downloading the dataset

Before any analysis, we need to retrieve and extract the dataset. The code below downloads the zipped file from the provided URL and extracts it locally.

import requests, zipfile, io

# URL for the NSL-KDD dataset
url = "https://academy.hackthebox.com/storage/modules/292/KDD_dataset.zip"

# Download the zip file and extract its contents
response = requests.get(url)
z = zipfile.ZipFile(io.BytesIO(response.content))
z.extractall('.')  # Extracts to the current directory

This is a standard retrieval pattern, but it is worth flagging a real-world concern. Downloading datasets from remote URLs and extracting them without integrity verification is a supply chain risk. In a production pipeline, you would hash the downloaded file against a known-good checksum before extraction. Adversarial dataset poisoning, where an attacker compromises the data source and injects modified records, is a documented attack vector against ML training pipelines. Sonatype’s 2024 research found malicious packages specifically targeting data science and preprocessing libraries. The pipeline starts here, and so does the attack surface.

Loading and inspecting the data

With the dataset extracted, we need the right libraries and a structured approach to loading it.

Importing libraries

import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import (accuracy_score, precision_score, recall_score,
                             f1_score, confusion_matrix, classification_report)
import seaborn as sns
import matplotlib.pyplot as plt

numpy and pandas handle data loading, cleaning, and manipulation. RandomForestClassifier is the algorithm we will train for anomaly detection. The sklearn.metrics imports provide the evaluation toolkit: accuracy alone is misleading for IDS work because class imbalance means a model that labels everything as “normal” can still score above 90%. Precision, recall, F1, and the confusion matrix tell you what actually matters, specifically how many attacks the model caught and how many it missed. seaborn and matplotlib handle visualisation.

Defining column names

The NSL-KDD dataset ships as a flat text file without headers, so we need to map each column to its correct feature name manually. These 42 column names correspond to the dataset’s feature specification.

# Set the file path to the dataset
file_path = r'KDD+.txt'

# Define the column names corresponding to the NSL-KDD dataset
columns = [
    'duration', 'protocol_type', 'service', 'flag', 'src_bytes', 'dst_bytes',
    'land', 'wrong_fragment', 'urgent', 'hot', 'num_failed_logins', 'logged_in',
    'num_compromised', 'root_shell', 'su_attempted', 'num_root', 'num_file_creations',
    'num_shells', 'num_access_files', 'num_outbound_cmds', 'is_host_login', 'is_guest_login',
    'count', 'srv_count', 'serror_rate', 'srv_serror_rate', 'rerror_rate', 'srv_rerror_rate',
    'same_srv_rate', 'diff_srv_rate', 'srv_diff_host_rate', 'dst_host_count', 'dst_host_srv_count',
    'dst_host_same_srv_rate', 'dst_host_diff_srv_rate', 'dst_host_same_src_port_rate',
    'dst_host_srv_diff_host_rate', 'dst_host_serror_rate', 'dst_host_srv_serror_rate',
    'dst_host_rerror_rate', 'dst_host_srv_rerror_rate', 'attack', 'level'
]

Look at what the model will learn from. The first few features are straightforward connection metadata: duration, protocol_type, service, flag, and byte counts in both directions. Then come behavioural indicators like num_failed_logins, root_shell, and su_attempted, which flag specific suspicious actions within a connection. The count and srv_count features capture traffic patterns over a time window, and the dst_host_* features aggregate behaviour at the destination host level.

The attack column contains the label (the specific attack type or “normal”), and level indicates the difficulty score. Three of these features are categorical (protocol_type, service, flag) and will need encoding before the random forest can use them. The rest are numerical.

For the adversarial perspective, the feature list tells you exactly how granular the model’s view of network traffic is. It tracks error rates, service distribution, and host-level aggregation, but it does not capture payload content, timing jitter at the packet level, or application-layer semantics. An attacker who can make their malicious traffic produce the same statistical profile as normal traffic across these 41 dimensions will be invisible to the model, regardless of what the actual payload contains.

Reading the dataset into a DataFrame

# Read the combined NSL-KDD dataset into a DataFrame
df = pd.read_csv(file_path, names=columns)

This loads the entire dataset into a structured pandas DataFrame with the correct column headers. Before moving to preprocessing and feature engineering, it is worth inspecting what we have.

print(df.head())

Checking the shape, data types, and missing values at this stage catches problems early. A silent column misalignment, where the data has shifted by one column relative to the headers, will corrupt every downstream step without throwing an error. The model will train successfully on garbage and produce confident, wrong predictions. This is exactly the kind of quiet failure that adversarial data poisoning exploits.

What comes next

The dataset is loaded, structured, and ready for inspection. The next steps are preprocessing (encoding categorical features, scaling numerical ones, handling any missing values) and then splitting the data for training and evaluation. Each of those steps encodes assumptions about what normal traffic looks like, and each assumption is a potential blind spot.

The random forest will learn from the 41 features we have defined. It will not learn from anything outside that feature space. When the model goes into production, its view of the network is exactly as wide as these columns and no wider. That constraint is both what makes the model tractable and what makes it beatable.