labs/Lab-6/Lab-6-Solution/05_Supervised_Intrusion_Tree_Solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cell-001",
   "metadata": {},
   "source": [
    "# Module 6 — Lab 5: A Supervised Intrusion Decision Tree\n",
    "\n",
    "\n",
    "**Topics:** known-attack classification, fixed test set, tree rules, group recall\n",
    "\n",
    "> **Solution version:** Run the cells in order, compare the outputs with your work, and read the suggested answers."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-002",
   "metadata": {},
   "source": [
    "## Learning objectives\n",
    "\n",
    "After completing this lab, you will be able to:\n",
    "\n",
    "- train a binary classifier from labeled normal and attack records;\n",
    "- apply categorical encoding inside a pipeline;\n",
    "- evaluate the fixed test set without tuning on it;\n",
    "- inspect Decision Tree rules and feature importance;\n",
    "- compare aggregate attack recall with group-level recall."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-003",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "TRAIN_PATH = Path(\"data/nsl_kdd_train.csv\")\n",
    "TEST_PATH = Path(\"data/nsl_kdd_test.csv\")\n",
    "\n",
    "missing_files = [str(path) for path in (TRAIN_PATH, TEST_PATH) if not path.exists()]\n",
    "if missing_files:\n",
    "    raise FileNotFoundError(\n",
    "        \"Missing prepared NSL-KDD files: \" + \", \".join(missing_files) +\n",
    "        \". Follow data/README.md and run: python data/prepare_dataset.py\"\n",
    "    )\n",
    "\n",
    "TARGET_COLUMNS = [\"attack_label\", \"attack_group\", \"binary_label\", \"difficulty\"]\n",
    "CATEGORICAL_FEATURES = [\"protocol_type\", \"service\", \"flag\"]\n",
    "\n",
    "\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.tree import DecisionTreeClassifier, export_text\n",
    "\n",
    "\n",
    "from sklearn.metrics import (\n",
    "    accuracy_score, average_precision_score, confusion_matrix,\n",
    "    f1_score, precision_score, recall_score,\n",
    ")\n",
    "\n",
    "\n",
    "def security_metrics(y_true, y_pred):\n",
    "    tn, fp, fn, tp = confusion_matrix(y_true, y_pred, labels=[0, 1]).ravel()\n",
    "    return {\n",
    "        \"accuracy\": accuracy_score(y_true, y_pred),\n",
    "        \"attack_precision\": precision_score(y_true, y_pred, zero_division=0),\n",
    "        \"attack_recall\": recall_score(y_true, y_pred, zero_division=0),\n",
    "        \"attack_f1\": f1_score(y_true, y_pred, zero_division=0),\n",
    "        \"false_positives\": int(fp),\n",
    "        \"false_negatives\": int(fn),\n",
    "        \"false_positive_rate\": fp / (fp + tn) if fp + tn else 0.0,\n",
    "        \"alerts_per_10000\": 10000 * (fp + tp) / len(y_true),\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-004",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_df = pd.read_csv(TRAIN_PATH, low_memory=False)\n",
    "test_df = pd.read_csv(TEST_PATH, low_memory=False)\n",
    "feature_cols = [column for column in train_df.columns if column not in TARGET_COLUMNS]\n",
    "numeric_features = [column for column in feature_cols if column not in CATEGORICAL_FEATURES]\n",
    "X_train, y_train = train_df[feature_cols], train_df[\"binary_label\"].astype(int)\n",
    "X_test, y_test = test_df[feature_cols], test_df[\"binary_label\"].astype(int)\n",
    "\n",
    "preprocessor = ColumnTransformer([\n",
    "    (\"categorical\", OneHotEncoder(handle_unknown=\"ignore\"), CATEGORICAL_FEATURES),\n",
    "    (\"numeric\", SimpleImputer(strategy=\"median\"), numeric_features),\n",
    "])\n",
    "tree_pipeline = Pipeline([\n",
    "    (\"preprocessor\", preprocessor),\n",
    "    (\"tree\", DecisionTreeClassifier(\n",
    "        max_depth=8, min_samples_leaf=5, class_weight=\"balanced\", random_state=42\n",
    "    )),\n",
    "])\n",
    "tree_pipeline.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-005",
   "metadata": {},
   "source": [
    "## 1. Evaluate binary and attack-group performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-006",
   "metadata": {},
   "outputs": [],
   "source": [
    "tree_predictions = tree_pipeline.predict(X_test)\n",
    "tree_probabilities = tree_pipeline.predict_proba(X_test)[:, 1]\n",
    "tree_metrics = security_metrics(y_test, tree_predictions)\n",
    "tree_metrics[\"pr_auc\"] = average_precision_score(y_test, tree_probabilities)\n",
    "display(pd.Series(tree_metrics, name=\"Decision Tree\"))\n",
    "\n",
    "group_rows = []\n",
    "for group in [\"DoS\", \"Probe\", \"R2L\", \"U2R\"]:\n",
    "    mask = test_df[\"attack_group\"].eq(group).to_numpy()\n",
    "    group_rows.append({\n",
    "        \"attack_group\": group,\n",
    "        \"support\": int(mask.sum()),\n",
    "        \"recall\": float(tree_predictions[mask].mean()) if mask.any() else np.nan,\n",
    "    })\n",
    "group_recall = pd.DataFrame(group_rows).set_index(\"attack_group\")\n",
    "display(group_recall.round(4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-007",
   "metadata": {},
   "source": [
    "## 2. Interpret the bounded tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-008",
   "metadata": {},
   "outputs": [],
   "source": [
    "fitted_preprocessor = tree_pipeline.named_steps[\"preprocessor\"]\n",
    "fitted_tree = tree_pipeline.named_steps[\"tree\"]\n",
    "transformed_names = fitted_preprocessor.get_feature_names_out()\n",
    "importance = pd.Series(\n",
    "    fitted_tree.feature_importances_, index=transformed_names, name=\"importance\"\n",
    ").sort_values(ascending=False)\n",
    "top_importance = importance.head(12)\n",
    "display(top_importance.to_frame())\n",
    "\n",
    "rule_text = export_text(fitted_tree, feature_names=list(transformed_names), max_depth=3)\n",
    "print(rule_text)\n",
    "\n",
    "ax = top_importance.sort_values().plot(kind=\"barh\", figsize=(9, 6), color=\"#4C78A8\")\n",
    "ax.set_title(\"Top Decision Tree feature importances\")\n",
    "ax.set_xlabel(\"Mean decrease in impurity\")\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-009",
   "metadata": {},
   "source": [
    "## Interpretation warning\n",
    "\n",
    "High performance on NSL-KDD does not establish production readiness. Tree rules can reflect benchmark artifacts, and attack-group recall can expose failures hidden by the binary score."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-010",
   "metadata": {},
   "source": [
    "## Self-check"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-011",
   "metadata": {},
   "outputs": [],
   "source": [
    "assert tree_pipeline.named_steps[\"tree\"].get_depth() <= 8\n",
    "assert len(tree_predictions) == len(test_df)\n",
    "assert set(tree_predictions).issubset({0, 1})\n",
    "assert set(group_recall.index) == {\"DoS\", \"Probe\", \"R2L\", \"U2R\"}\n",
    "assert isinstance(rule_text, str) and \"class:\" in rule_text\n",
    "assert np.isclose(importance.sum(), 1.0)\n",
    "assert len(top_importance) <= 12\n",
    "print(\"✓ Lab 5 completed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-012",
   "metadata": {},
   "source": [
    "## Suggested answers\n",
    "\n",
    "1. **What does the tree learn that novelty models do not?** It directly uses labeled attack patterns and can learn boundaries around known attacks.\n",
    "2. **Why inspect group recall?** A high binary score may be dominated by common DoS attacks while rare R2L/U2R attacks remain undetected.\n",
    "3. **Are tree rules signatures?** No. They describe one model fitted to one historical benchmark and require external validation."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}