labs/Lab-6/Lab-6-Practical/05_Supervised_Intrusion_Tree_Lab.ipynb

Module 6 — Lab 5: A Supervised Intrusion Decision Tree

Topics: known-attack classification, fixed test set, tree rules, group recall

Lab version: Complete every TODO, run the cells in order, and use the self-check before opening the Solution.

Learning objectives

After completing this lab, you will be able to:

• train a binary classifier from labeled normal and attack records;
• apply categorical encoding inside a pipeline;
• evaluate the fixed test set without tuning on it;
• inspect Decision Tree rules and feature importance;
• compare aggregate attack recall with group-level recall.

In [ ]:

from pathlib import Path
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

TRAIN_PATH = Path("data/nsl_kdd_train.csv")
TEST_PATH = Path("data/nsl_kdd_test.csv")

missing_files = [str(path) for path in (TRAIN_PATH, TEST_PATH) if not path.exists()]
if missing_files:
    raise FileNotFoundError(
        "Missing prepared NSL-KDD files: " + ", ".join(missing_files) +
        ". Follow data/README.md and run: python data/prepare_dataset.py"
    )

TARGET_COLUMNS = ["attack_label", "attack_group", "binary_label", "difficulty"]
CATEGORICAL_FEATURES = ["protocol_type", "service", "flag"]


from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder
from sklearn.tree import DecisionTreeClassifier, export_text


from sklearn.metrics import (
    accuracy_score, average_precision_score, confusion_matrix,
    f1_score, precision_score, recall_score,
)


def security_metrics(y_true, y_pred):
    tn, fp, fn, tp = confusion_matrix(y_true, y_pred, labels=[0, 1]).ravel()
    return {
        "accuracy": accuracy_score(y_true, y_pred),
        "attack_precision": precision_score(y_true, y_pred, zero_division=0),
        "attack_recall": recall_score(y_true, y_pred, zero_division=0),
        "attack_f1": f1_score(y_true, y_pred, zero_division=0),
        "false_positives": int(fp),
        "false_negatives": int(fn),
        "false_positive_rate": fp / (fp + tn) if fp + tn else 0.0,
        "alerts_per_10000": 10000 * (fp + tp) / len(y_true),
    }

In [ ]:

# TODO 1: load the fixed split and create X_train/y_train/X_test/y_test.
train_df = None
test_df = None
feature_cols = None
numeric_features = None
X_train, y_train, X_test, y_test = None, None, None, None

# TODO 2: build a ColumnTransformer and a DecisionTree pipeline using
# max_depth=8, min_samples_leaf=5, class_weight='balanced', random_state=42.
preprocessor = None
tree_pipeline = None

# TODO 3: fit the pipeline on the fixed training split.

1. Evaluate binary and attack-group performance

In [ ]:

# TODO 4: predict labels/probabilities and calculate security metrics plus PR-AUC.
tree_predictions = None
tree_probabilities = None
tree_metrics = None
display(pd.Series(tree_metrics, name="Decision Tree"))

# TODO 5: calculate recall for DoS, Probe, R2L, and U2R.
group_recall = None
display(group_recall)

2. Interpret the bounded tree

In [ ]:

# TODO 6: obtain transformed feature names, calculate top-12 impurity
# importances, export rules through depth 3, and plot the importance table.
fitted_preprocessor = None
fitted_tree = None
transformed_names = None
importance = None
top_importance = None
rule_text = None
print(rule_text)

Interpretation warning

High performance on NSL-KDD does not establish production readiness. Tree rules can reflect benchmark artifacts, and attack-group recall can expose failures hidden by the binary score.

Self-check

In [ ]:

assert tree_pipeline.named_steps["tree"].get_depth() <= 8
assert len(tree_predictions) == len(test_df)
assert set(tree_predictions).issubset({0, 1})
assert set(group_recall.index) == {"DoS", "Probe", "R2L", "U2R"}
assert isinstance(rule_text, str) and "class:" in rule_text
assert np.isclose(importance.sum(), 1.0)
assert len(top_importance) <= 12
print("✓ Lab 5 completed")

Reflection questions

1. How does supervised learning differ from normal-only novelty detection?
2. Which attack group is hardest for the tree, and why might binary metrics hide it?
3. Why should exported tree rules not be treated as universal IDS signatures?