labs/Lab-6/Lab-6-Solution/05_Supervised_Intrusion_Tree_Solution.ipynb

Module 6 — Lab 5: A Supervised Intrusion Decision Tree

Topics: known-attack classification, fixed test set, tree rules, group recall

Solution version: Run the cells in order, compare the outputs with your work, and read the suggested answers.

Learning objectives

After completing this lab, you will be able to:

• train a binary classifier from labeled normal and attack records;
• apply categorical encoding inside a pipeline;
• evaluate the fixed test set without tuning on it;
• inspect Decision Tree rules and feature importance;
• compare aggregate attack recall with group-level recall.

In [ ]:

from pathlib import Path
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

TRAIN_PATH = Path("data/nsl_kdd_train.csv")
TEST_PATH = Path("data/nsl_kdd_test.csv")

missing_files = [str(path) for path in (TRAIN_PATH, TEST_PATH) if not path.exists()]
if missing_files:
    raise FileNotFoundError(
        "Missing prepared NSL-KDD files: " + ", ".join(missing_files) +
        ". Follow data/README.md and run: python data/prepare_dataset.py"
    )

TARGET_COLUMNS = ["attack_label", "attack_group", "binary_label", "difficulty"]
CATEGORICAL_FEATURES = ["protocol_type", "service", "flag"]


from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder
from sklearn.tree import DecisionTreeClassifier, export_text


from sklearn.metrics import (
    accuracy_score, average_precision_score, confusion_matrix,
    f1_score, precision_score, recall_score,
)


def security_metrics(y_true, y_pred):
    tn, fp, fn, tp = confusion_matrix(y_true, y_pred, labels=[0, 1]).ravel()
    return {
        "accuracy": accuracy_score(y_true, y_pred),
        "attack_precision": precision_score(y_true, y_pred, zero_division=0),
        "attack_recall": recall_score(y_true, y_pred, zero_division=0),
        "attack_f1": f1_score(y_true, y_pred, zero_division=0),
        "false_positives": int(fp),
        "false_negatives": int(fn),
        "false_positive_rate": fp / (fp + tn) if fp + tn else 0.0,
        "alerts_per_10000": 10000 * (fp + tp) / len(y_true),
    }

In [ ]:

train_df = pd.read_csv(TRAIN_PATH, low_memory=False)
test_df = pd.read_csv(TEST_PATH, low_memory=False)
feature_cols = [column for column in train_df.columns if column not in TARGET_COLUMNS]
numeric_features = [column for column in feature_cols if column not in CATEGORICAL_FEATURES]
X_train, y_train = train_df[feature_cols], train_df["binary_label"].astype(int)
X_test, y_test = test_df[feature_cols], test_df["binary_label"].astype(int)

preprocessor = ColumnTransformer([
    ("categorical", OneHotEncoder(handle_unknown="ignore"), CATEGORICAL_FEATURES),
    ("numeric", SimpleImputer(strategy="median"), numeric_features),
])
tree_pipeline = Pipeline([
    ("preprocessor", preprocessor),
    ("tree", DecisionTreeClassifier(
        max_depth=8, min_samples_leaf=5, class_weight="balanced", random_state=42
    )),
])
tree_pipeline.fit(X_train, y_train)

1. Evaluate binary and attack-group performance

In [ ]:

tree_predictions = tree_pipeline.predict(X_test)
tree_probabilities = tree_pipeline.predict_proba(X_test)[:, 1]
tree_metrics = security_metrics(y_test, tree_predictions)
tree_metrics["pr_auc"] = average_precision_score(y_test, tree_probabilities)
display(pd.Series(tree_metrics, name="Decision Tree"))

group_rows = []
for group in ["DoS", "Probe", "R2L", "U2R"]:
    mask = test_df["attack_group"].eq(group).to_numpy()
    group_rows.append({
        "attack_group": group,
        "support": int(mask.sum()),
        "recall": float(tree_predictions[mask].mean()) if mask.any() else np.nan,
    })
group_recall = pd.DataFrame(group_rows).set_index("attack_group")
display(group_recall.round(4))

2. Interpret the bounded tree

In [ ]:

fitted_preprocessor = tree_pipeline.named_steps["preprocessor"]
fitted_tree = tree_pipeline.named_steps["tree"]
transformed_names = fitted_preprocessor.get_feature_names_out()
importance = pd.Series(
    fitted_tree.feature_importances_, index=transformed_names, name="importance"
).sort_values(ascending=False)
top_importance = importance.head(12)
display(top_importance.to_frame())

rule_text = export_text(fitted_tree, feature_names=list(transformed_names), max_depth=3)
print(rule_text)

ax = top_importance.sort_values().plot(kind="barh", figsize=(9, 6), color="#4C78A8")
ax.set_title("Top Decision Tree feature importances")
ax.set_xlabel("Mean decrease in impurity")
plt.tight_layout()
plt.show()

Interpretation warning

High performance on NSL-KDD does not establish production readiness. Tree rules can reflect benchmark artifacts, and attack-group recall can expose failures hidden by the binary score.

Self-check

In [ ]:

assert tree_pipeline.named_steps["tree"].get_depth() <= 8
assert len(tree_predictions) == len(test_df)
assert set(tree_predictions).issubset({0, 1})
assert set(group_recall.index) == {"DoS", "Probe", "R2L", "U2R"}
assert isinstance(rule_text, str) and "class:" in rule_text
assert np.isclose(importance.sum(), 1.0)
assert len(top_importance) <= 12
print("✓ Lab 5 completed")

Suggested answers

1. What does the tree learn that novelty models do not? It directly uses labeled attack patterns and can learn boundaries around known attacks.
2. Why inspect group recall? A high binary score may be dominated by common DoS attacks while rare R2L/U2R attacks remain undetected.
3. Are tree rules signatures? No. They describe one model fitted to one historical benchmark and require external validation.