labs/Lab-6/Lab-6-Practical/05_Supervised_Intrusion_Tree_Lab.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cell-001",
   "metadata": {},
   "source": [
    "# Module 6 — Lab 5: A Supervised Intrusion Decision Tree\n",
    "\n",
    "  \n",
    "**Topics:** known-attack classification, fixed test set, tree rules, group recall\n",
    "\n",
    "> **Lab version:** Complete every `TODO`, run the cells in order, and use the self-check before opening the Solution."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-002",
   "metadata": {},
   "source": [
    "## Learning objectives\n",
    "\n",
    "After completing this lab, you will be able to:\n",
    "\n",
    "- train a binary classifier from labeled normal and attack records;\n",
    "- apply categorical encoding inside a pipeline;\n",
    "- evaluate the fixed test set without tuning on it;\n",
    "- inspect Decision Tree rules and feature importance;\n",
    "- compare aggregate attack recall with group-level recall."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-003",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "TRAIN_PATH = Path(\"data/nsl_kdd_train.csv\")\n",
    "TEST_PATH = Path(\"data/nsl_kdd_test.csv\")\n",
    "\n",
    "missing_files = [str(path) for path in (TRAIN_PATH, TEST_PATH) if not path.exists()]\n",
    "if missing_files:\n",
    "    raise FileNotFoundError(\n",
    "        \"Missing prepared NSL-KDD files: \" + \", \".join(missing_files) +\n",
    "        \". Follow data/README.md and run: python data/prepare_dataset.py\"\n",
    "    )\n",
    "\n",
    "TARGET_COLUMNS = [\"attack_label\", \"attack_group\", \"binary_label\", \"difficulty\"]\n",
    "CATEGORICAL_FEATURES = [\"protocol_type\", \"service\", \"flag\"]\n",
    "\n",
    "\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.tree import DecisionTreeClassifier, export_text\n",
    "\n",
    "\n",
    "from sklearn.metrics import (\n",
    "    accuracy_score, average_precision_score, confusion_matrix,\n",
    "    f1_score, precision_score, recall_score,\n",
    ")\n",
    "\n",
    "\n",
    "def security_metrics(y_true, y_pred):\n",
    "    tn, fp, fn, tp = confusion_matrix(y_true, y_pred, labels=[0, 1]).ravel()\n",
    "    return {\n",
    "        \"accuracy\": accuracy_score(y_true, y_pred),\n",
    "        \"attack_precision\": precision_score(y_true, y_pred, zero_division=0),\n",
    "        \"attack_recall\": recall_score(y_true, y_pred, zero_division=0),\n",
    "        \"attack_f1\": f1_score(y_true, y_pred, zero_division=0),\n",
    "        \"false_positives\": int(fp),\n",
    "        \"false_negatives\": int(fn),\n",
    "        \"false_positive_rate\": fp / (fp + tn) if fp + tn else 0.0,\n",
    "        \"alerts_per_10000\": 10000 * (fp + tp) / len(y_true),\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-004",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO 1: load the fixed split and create X_train/y_train/X_test/y_test.\n",
    "train_df = None\n",
    "test_df = None\n",
    "feature_cols = None\n",
    "numeric_features = None\n",
    "X_train, y_train, X_test, y_test = None, None, None, None\n",
    "\n",
    "# TODO 2: build a ColumnTransformer and a DecisionTree pipeline using\n",
    "# max_depth=8, min_samples_leaf=5, class_weight='balanced', random_state=42.\n",
    "preprocessor = None\n",
    "tree_pipeline = None\n",
    "\n",
    "# TODO 3: fit the pipeline on the fixed training split."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-005",
   "metadata": {},
   "source": [
    "## 1. Evaluate binary and attack-group performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-006",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO 4: predict labels/probabilities and calculate security metrics plus PR-AUC.\n",
    "tree_predictions = None\n",
    "tree_probabilities = None\n",
    "tree_metrics = None\n",
    "display(pd.Series(tree_metrics, name=\"Decision Tree\"))\n",
    "\n",
    "# TODO 5: calculate recall for DoS, Probe, R2L, and U2R.\n",
    "group_recall = None\n",
    "display(group_recall)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-007",
   "metadata": {},
   "source": [
    "## 2. Interpret the bounded tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-008",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO 6: obtain transformed feature names, calculate top-12 impurity\n",
    "# importances, export rules through depth 3, and plot the importance table.\n",
    "fitted_preprocessor = None\n",
    "fitted_tree = None\n",
    "transformed_names = None\n",
    "importance = None\n",
    "top_importance = None\n",
    "rule_text = None\n",
    "print(rule_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-009",
   "metadata": {},
   "source": [
    "## Interpretation warning\n",
    "\n",
    "High performance on NSL-KDD does not establish production readiness. Tree rules can reflect benchmark artifacts, and attack-group recall can expose failures hidden by the binary score."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-010",
   "metadata": {},
   "source": [
    "## Self-check"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-011",
   "metadata": {},
   "outputs": [],
   "source": [
    "assert tree_pipeline.named_steps[\"tree\"].get_depth() <= 8\n",
    "assert len(tree_predictions) == len(test_df)\n",
    "assert set(tree_predictions).issubset({0, 1})\n",
    "assert set(group_recall.index) == {\"DoS\", \"Probe\", \"R2L\", \"U2R\"}\n",
    "assert isinstance(rule_text, str) and \"class:\" in rule_text\n",
    "assert np.isclose(importance.sum(), 1.0)\n",
    "assert len(top_importance) <= 12\n",
    "print(\"✓ Lab 5 completed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-012",
   "metadata": {},
   "source": [
    "## Reflection questions\n",
    "\n",
    "1. How does supervised learning differ from normal-only novelty detection?\n",
    "2. Which attack group is hardest for the tree, and why might binary metrics hide it?\n",
    "3. Why should exported tree rules not be treated as universal IDS signatures?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}