{ "cells": [ { "cell_type": "markdown", "id": "cell-001", "metadata": {}, "source": [ "# Module 6 — Lab 5: A Supervised Intrusion Decision Tree\n", "\n", "\n", "**Topics:** known-attack classification, fixed test set, tree rules, group recall\n", "\n", "> **Solution version:** Run the cells in order, compare the outputs with your work, and read the suggested answers." ] }, { "cell_type": "markdown", "id": "cell-002", "metadata": {}, "source": [ "## Learning objectives\n", "\n", "After completing this lab, you will be able to:\n", "\n", "- train a binary classifier from labeled normal and attack records;\n", "- apply categorical encoding inside a pipeline;\n", "- evaluate the fixed test set without tuning on it;\n", "- inspect Decision Tree rules and feature importance;\n", "- compare aggregate attack recall with group-level recall." ] }, { "cell_type": "code", "execution_count": null, "id": "cell-003", "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "\n", "TRAIN_PATH = Path(\"data/nsl_kdd_train.csv\")\n", "TEST_PATH = Path(\"data/nsl_kdd_test.csv\")\n", "\n", "missing_files = [str(path) for path in (TRAIN_PATH, TEST_PATH) if not path.exists()]\n", "if missing_files:\n", " raise FileNotFoundError(\n", " \"Missing prepared NSL-KDD files: \" + \", \".join(missing_files) +\n", " \". Follow data/README.md and run: python data/prepare_dataset.py\"\n", " )\n", "\n", "TARGET_COLUMNS = [\"attack_label\", \"attack_group\", \"binary_label\", \"difficulty\"]\n", "CATEGORICAL_FEATURES = [\"protocol_type\", \"service\", \"flag\"]\n", "\n", "\n", "from sklearn.compose import ColumnTransformer\n", "from sklearn.impute import SimpleImputer\n", "from sklearn.pipeline import Pipeline\n", "from sklearn.preprocessing import OneHotEncoder\n", "from sklearn.tree import DecisionTreeClassifier, export_text\n", "\n", "\n", "from sklearn.metrics import (\n", " accuracy_score, average_precision_score, confusion_matrix,\n", " f1_score, precision_score, recall_score,\n", ")\n", "\n", "\n", "def security_metrics(y_true, y_pred):\n", " tn, fp, fn, tp = confusion_matrix(y_true, y_pred, labels=[0, 1]).ravel()\n", " return {\n", " \"accuracy\": accuracy_score(y_true, y_pred),\n", " \"attack_precision\": precision_score(y_true, y_pred, zero_division=0),\n", " \"attack_recall\": recall_score(y_true, y_pred, zero_division=0),\n", " \"attack_f1\": f1_score(y_true, y_pred, zero_division=0),\n", " \"false_positives\": int(fp),\n", " \"false_negatives\": int(fn),\n", " \"false_positive_rate\": fp / (fp + tn) if fp + tn else 0.0,\n", " \"alerts_per_10000\": 10000 * (fp + tp) / len(y_true),\n", " }" ] }, { "cell_type": "code", "execution_count": null, "id": "cell-004", "metadata": {}, "outputs": [], "source": [ "train_df = pd.read_csv(TRAIN_PATH, low_memory=False)\n", "test_df = pd.read_csv(TEST_PATH, low_memory=False)\n", "feature_cols = [column for column in train_df.columns if column not in TARGET_COLUMNS]\n", "numeric_features = [column for column in feature_cols if column not in CATEGORICAL_FEATURES]\n", "X_train, y_train = train_df[feature_cols], train_df[\"binary_label\"].astype(int)\n", "X_test, y_test = test_df[feature_cols], test_df[\"binary_label\"].astype(int)\n", "\n", "preprocessor = ColumnTransformer([\n", " (\"categorical\", OneHotEncoder(handle_unknown=\"ignore\"), CATEGORICAL_FEATURES),\n", " (\"numeric\", SimpleImputer(strategy=\"median\"), numeric_features),\n", "])\n", "tree_pipeline = Pipeline([\n", " (\"preprocessor\", preprocessor),\n", " (\"tree\", DecisionTreeClassifier(\n", " max_depth=8, min_samples_leaf=5, class_weight=\"balanced\", random_state=42\n", " )),\n", "])\n", "tree_pipeline.fit(X_train, y_train)" ] }, { "cell_type": "markdown", "id": "cell-005", "metadata": {}, "source": [ "## 1. Evaluate binary and attack-group performance" ] }, { "cell_type": "code", "execution_count": null, "id": "cell-006", "metadata": {}, "outputs": [], "source": [ "tree_predictions = tree_pipeline.predict(X_test)\n", "tree_probabilities = tree_pipeline.predict_proba(X_test)[:, 1]\n", "tree_metrics = security_metrics(y_test, tree_predictions)\n", "tree_metrics[\"pr_auc\"] = average_precision_score(y_test, tree_probabilities)\n", "display(pd.Series(tree_metrics, name=\"Decision Tree\"))\n", "\n", "group_rows = []\n", "for group in [\"DoS\", \"Probe\", \"R2L\", \"U2R\"]:\n", " mask = test_df[\"attack_group\"].eq(group).to_numpy()\n", " group_rows.append({\n", " \"attack_group\": group,\n", " \"support\": int(mask.sum()),\n", " \"recall\": float(tree_predictions[mask].mean()) if mask.any() else np.nan,\n", " })\n", "group_recall = pd.DataFrame(group_rows).set_index(\"attack_group\")\n", "display(group_recall.round(4))" ] }, { "cell_type": "markdown", "id": "cell-007", "metadata": {}, "source": [ "## 2. Interpret the bounded tree" ] }, { "cell_type": "code", "execution_count": null, "id": "cell-008", "metadata": {}, "outputs": [], "source": [ "fitted_preprocessor = tree_pipeline.named_steps[\"preprocessor\"]\n", "fitted_tree = tree_pipeline.named_steps[\"tree\"]\n", "transformed_names = fitted_preprocessor.get_feature_names_out()\n", "importance = pd.Series(\n", " fitted_tree.feature_importances_, index=transformed_names, name=\"importance\"\n", ").sort_values(ascending=False)\n", "top_importance = importance.head(12)\n", "display(top_importance.to_frame())\n", "\n", "rule_text = export_text(fitted_tree, feature_names=list(transformed_names), max_depth=3)\n", "print(rule_text)\n", "\n", "ax = top_importance.sort_values().plot(kind=\"barh\", figsize=(9, 6), color=\"#4C78A8\")\n", "ax.set_title(\"Top Decision Tree feature importances\")\n", "ax.set_xlabel(\"Mean decrease in impurity\")\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "cell-009", "metadata": {}, "source": [ "## Interpretation warning\n", "\n", "High performance on NSL-KDD does not establish production readiness. Tree rules can reflect benchmark artifacts, and attack-group recall can expose failures hidden by the binary score." ] }, { "cell_type": "markdown", "id": "cell-010", "metadata": {}, "source": [ "## Self-check" ] }, { "cell_type": "code", "execution_count": null, "id": "cell-011", "metadata": {}, "outputs": [], "source": [ "assert tree_pipeline.named_steps[\"tree\"].get_depth() <= 8\n", "assert len(tree_predictions) == len(test_df)\n", "assert set(tree_predictions).issubset({0, 1})\n", "assert set(group_recall.index) == {\"DoS\", \"Probe\", \"R2L\", \"U2R\"}\n", "assert isinstance(rule_text, str) and \"class:\" in rule_text\n", "assert np.isclose(importance.sum(), 1.0)\n", "assert len(top_importance) <= 12\n", "print(\"✓ Lab 5 completed\")" ] }, { "cell_type": "markdown", "id": "cell-012", "metadata": {}, "source": [ "## Suggested answers\n", "\n", "1. **What does the tree learn that novelty models do not?** It directly uses labeled attack patterns and can learn boundaries around known attacks.\n", "2. **Why inspect group recall?** A high binary score may be dominated by common DoS attacks while rare R2L/U2R attacks remain undetected.\n", "3. **Are tree rules signatures?** No. They describe one model fitted to one historical benchmark and require external validation." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.6" } }, "nbformat": 4, "nbformat_minor": 5 }