labs/Lab-5/Lab-5-Practical/03_Decision_Tree_Baseline_Lab.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cell-001",
   "metadata": {},
   "source": [
    "# Module 5 — Lab 3: Building a Decision Tree Baseline\n",
    "\n",
    "\n",
    "**Topics:** stratified split, dummy baseline, bounded tree, leakage-safe pipeline\n",
    "\n",
    "> **Lab version:** Complete every `TODO`, run the cells in order, and use the self-check before opening the Solution."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-002",
   "metadata": {},
   "source": [
    "## Learning objectives\n",
    "\n",
    "After completing this lab, you will be able to:\n",
    "\n",
    "- create a reproducible stratified train-test split;\n",
    "- fit missing-value imputation on training data only;\n",
    "- compare a bounded Decision Tree with a dummy baseline;\n",
    "- evaluate malware precision, recall, F1, false positives, and false negatives;\n",
    "- explain why trees do not require feature scaling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-003",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "DATA_PATH = Path(\"data/malware_features.csv\")\n",
    "\n",
    "if not DATA_PATH.exists():\n",
    "    raise FileNotFoundError(\n",
    "        \"Missing data/malware_features.csv. Follow data/README.md, place the three \"\n",
    "        \"official UCI CSV files under data/raw/, and run: python data/prepare_dataset.py\"\n",
    "    )\n",
    "\n",
    "\n",
    "from sklearn.dummy import DummyClassifier\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.metrics import confusion_matrix, precision_score, recall_score, f1_score, accuracy_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "\n",
    "def security_metrics(y_true, y_pred):\n",
    "    tn, fp, fn, tp = confusion_matrix(y_true, y_pred, labels=[0, 1]).ravel()\n",
    "    return {\n",
    "        \"accuracy\": accuracy_score(y_true, y_pred),\n",
    "        \"malware_precision\": precision_score(y_true, y_pred, zero_division=0),\n",
    "        \"malware_recall\": recall_score(y_true, y_pred, zero_division=0),\n",
    "        \"malware_f1\": f1_score(y_true, y_pred, zero_division=0),\n",
    "        \"false_positives\": int(fp),\n",
    "        \"false_negatives\": int(fn),\n",
    "        \"false_positive_rate\": fp / (fp + tn) if fp + tn else 0.0,\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-004",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO 1: load the data, exclude source and label from X, and create integer y.\n",
    "df = None\n",
    "feature_cols = None\n",
    "X = None\n",
    "y = None\n",
    "\n",
    "# TODO 2: make an 80/20 stratified split with random_state=42.\n",
    "X_train, X_test, y_train, y_test = None, None, None, None\n",
    "print(\"Train shape:\", None, \"Test shape:\", None)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-005",
   "metadata": {},
   "source": [
    "## 1. Establish a non-learning baseline\n",
    "\n",
    "Because malware is the majority class, a most-frequent classifier can achieve high accuracy while providing little operational value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-006",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO 3: fit a most-frequent DummyClassifier and calculate security metrics.\n",
    "dummy = None\n",
    "dummy_predictions = None\n",
    "dummy_metrics = None\n",
    "pd.Series(dummy_metrics, name=\"dummy\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-007",
   "metadata": {},
   "source": [
    "## 2. Train a bounded Decision Tree\n",
    "\n",
    "The imputer is inside the pipeline, so its medians are learned from training rows only. A shallow tree is intentionally chosen for readability and to reduce overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-008",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO 4: create a pipeline with median imputation and a Decision Tree.\n",
    "# Use max_depth=4, min_samples_leaf=10, and random_state=42.\n",
    "tree_pipeline = None\n",
    "\n",
    "# TODO 5: fit the pipeline, predict the test set, and calculate security metrics.\n",
    "tree_predictions = None\n",
    "tree_metrics = None\n",
    "\n",
    "comparison = None\n",
    "display(comparison)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-009",
   "metadata": {},
   "source": [
    "## Self-check"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-010",
   "metadata": {},
   "outputs": [],
   "source": [
    "assert X_train.shape[0] + X_test.shape[0] == len(df)\n",
    "assert abs(y_train.mean() - y_test.mean()) < 0.01\n",
    "assert set(tree_predictions).issubset({0, 1})\n",
    "assert tree_pipeline.named_steps[\"tree\"].get_depth() <= 4\n",
    "assert set(comparison.index) == {\"Dummy\", \"Decision Tree\"}\n",
    "assert comparison.loc[\"Decision Tree\", \"malware_f1\"] >= comparison.loc[\"Dummy\", \"malware_f1\"]\n",
    "print(\"✓ Lab 3 completed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-011",
   "metadata": {},
   "source": [
    "## Reflection questions\n",
    "\n",
    "1. Why can the dummy model have deceptively high accuracy?\n",
    "2. Why does a Decision Tree usually not require feature scaling?\n",
    "3. What is the security and interpretability benefit of limiting tree depth?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}