labs/Lab-5/Lab-5-Practical/03_Decision_Tree_Baseline_Lab.ipynb

Module 5 — Lab 3: Building a Decision Tree Baseline

Topics: stratified split, dummy baseline, bounded tree, leakage-safe pipeline

Lab version: Complete every TODO, run the cells in order, and use the self-check before opening the Solution.

Learning objectives

After completing this lab, you will be able to:

• create a reproducible stratified train-test split;
• fit missing-value imputation on training data only;
• compare a bounded Decision Tree with a dummy baseline;
• evaluate malware precision, recall, F1, false positives, and false negatives;
• explain why trees do not require feature scaling.

In [ ]:

from pathlib import Path
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

DATA_PATH = Path("data/malware_features.csv")

if not DATA_PATH.exists():
    raise FileNotFoundError(
        "Missing data/malware_features.csv. Follow data/README.md, place the three "
        "official UCI CSV files under data/raw/, and run: python data/prepare_dataset.py"
    )


from sklearn.dummy import DummyClassifier
from sklearn.impute import SimpleImputer
from sklearn.metrics import confusion_matrix, precision_score, recall_score, f1_score, accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.tree import DecisionTreeClassifier


def security_metrics(y_true, y_pred):
    tn, fp, fn, tp = confusion_matrix(y_true, y_pred, labels=[0, 1]).ravel()
    return {
        "accuracy": accuracy_score(y_true, y_pred),
        "malware_precision": precision_score(y_true, y_pred, zero_division=0),
        "malware_recall": recall_score(y_true, y_pred, zero_division=0),
        "malware_f1": f1_score(y_true, y_pred, zero_division=0),
        "false_positives": int(fp),
        "false_negatives": int(fn),
        "false_positive_rate": fp / (fp + tn) if fp + tn else 0.0,
    }

In [ ]:

# TODO 1: load the data, exclude source and label from X, and create integer y.
df = None
feature_cols = None
X = None
y = None

# TODO 2: make an 80/20 stratified split with random_state=42.
X_train, X_test, y_train, y_test = None, None, None, None
print("Train shape:", None, "Test shape:", None)

1. Establish a non-learning baseline

Because malware is the majority class, a most-frequent classifier can achieve high accuracy while providing little operational value.

In [ ]:

# TODO 3: fit a most-frequent DummyClassifier and calculate security metrics.
dummy = None
dummy_predictions = None
dummy_metrics = None
pd.Series(dummy_metrics, name="dummy")

2. Train a bounded Decision Tree

The imputer is inside the pipeline, so its medians are learned from training rows only. A shallow tree is intentionally chosen for readability and to reduce overfitting.

In [ ]:

# TODO 4: create a pipeline with median imputation and a Decision Tree.
# Use max_depth=4, min_samples_leaf=10, and random_state=42.
tree_pipeline = None

# TODO 5: fit the pipeline, predict the test set, and calculate security metrics.
tree_predictions = None
tree_metrics = None

comparison = None
display(comparison)

Self-check

In [ ]:

assert X_train.shape[0] + X_test.shape[0] == len(df)
assert abs(y_train.mean() - y_test.mean()) < 0.01
assert set(tree_predictions).issubset({0, 1})
assert tree_pipeline.named_steps["tree"].get_depth() <= 4
assert set(comparison.index) == {"Dummy", "Decision Tree"}
assert comparison.loc["Decision Tree", "malware_f1"] >= comparison.loc["Dummy", "malware_f1"]
print("✓ Lab 3 completed")

Reflection questions

1. Why can the dummy model have deceptively high accuracy?
2. Why does a Decision Tree usually not require feature scaling?
3. What is the security and interpretability benefit of limiting tree depth?