CSAI2026 labs: Lab-2..Lab-6 (Practical + Solution, data), Module-2..Module-6 notebooks and datasets
This commit is contained in:
mikrotik
@@ -0,0 +1,289 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "291ef749",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Module 3 — Lab 6: Capstone — Binary Intrusion Detection with CIC-IDS2017\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"**Topics:** end-to-end ML workflow, CIC-IDS2017, binary classification, evaluation, reporting\n",
|
||||
"\n",
|
||||
"> **Instructions:** Run the cells in order. Complete every `TODO`, then run the **Self-check** cell. Do not open the Solution notebook before attempting the tasks yourself.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "00c8ff6f",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Mission\n",
|
||||
"\n",
|
||||
"Build one complete and leakage-safe intrusion-detection workflow:\n",
|
||||
"\n",
|
||||
"1. load a network-flow table;\n",
|
||||
"2. clean invalid values and duplicate rows;\n",
|
||||
"3. map all attacks to one `ATTACK` class;\n",
|
||||
"4. create one stratified train-test split;\n",
|
||||
"5. train one Random Forest pipeline;\n",
|
||||
"6. evaluate attack precision, recall, F1, and error counts;\n",
|
||||
"7. export a short analyst report.\n",
|
||||
"\n",
|
||||
"The notebook uses `data/cicids2017_binary_sample.csv` when that file is available. Until the instructor prepares the official subset, it uses the included synthetic dataset and prints a clear warning.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "9c2228a9",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from pathlib import Path\n",
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"\n",
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"from sklearn.pipeline import Pipeline\n",
|
||||
"from sklearn.impute import SimpleImputer\n",
|
||||
"from sklearn.ensemble import RandomForestClassifier\n",
|
||||
"from sklearn.metrics import (\n",
|
||||
" ConfusionMatrixDisplay, accuracy_score, confusion_matrix,\n",
|
||||
" precision_score, recall_score, f1_score\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"REAL_DATA_PATH = Path(\"data/cicids2017_binary_sample.csv\")\n",
|
||||
"DEMO_DATA_PATH = Path(\"data/synthetic_security_flows.csv\")\n",
|
||||
"OUTPUT_DIR = Path(\"output\")\n",
|
||||
"OUTPUT_DIR.mkdir(exist_ok=True)\n",
|
||||
"RANDOM_STATE = 42\n",
|
||||
"\n",
|
||||
"DATA_PATH = REAL_DATA_PATH if REAL_DATA_PATH.exists() else DEMO_DATA_PATH\n",
|
||||
"USING_REAL_CICIDS = REAL_DATA_PATH.exists()\n",
|
||||
"\n",
|
||||
"print(\"Dataset:\", DATA_PATH)\n",
|
||||
"if not USING_REAL_CICIDS:\n",
|
||||
" print(\"WARNING: Using synthetic classroom data. Add the prepared CIC-IDS2017 file for the real capstone.\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "c61dcfb0",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1. Load, clean, and define the binary target\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "a2204759",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# TODO 1: load DATA_PATH into raw_df\n",
|
||||
"raw_df = None\n",
|
||||
"\n",
|
||||
"# TODO 2: strip column names, remove duplicates, and replace +/- infinity with NaN\n",
|
||||
"clean_df = None\n",
|
||||
"\n",
|
||||
"# TODO 3: create numeric X and binary y (BENIGN=0, every attack=1)\n",
|
||||
"X = None\n",
|
||||
"y = None\n",
|
||||
"\n",
|
||||
"print(f\"Raw rows: {None}\")\n",
|
||||
"print(f\"Clean rows: {None}\")\n",
|
||||
"print(f\"Numeric features: {None}\")\n",
|
||||
"print(\"Binary class distribution:\")\n",
|
||||
"print(None)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "90e14590",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2. Create one fixed train-test split\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "33430abe",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# TODO 4: create an 80/20 split with stratify=y and random_state=42\n",
|
||||
"X_train, X_test, y_train, y_test = None, None, None, None\n",
|
||||
"\n",
|
||||
"print(\"Training samples:\", None)\n",
|
||||
"print(\"Test samples:\", None)\n",
|
||||
"print(\"Training attack rate:\", None)\n",
|
||||
"print(\"Test attack rate:\", None)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b4f6af84",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 3. Build and train the baseline pipeline\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "aab65468",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# TODO 5: create an imputer + Random Forest pipeline\n",
|
||||
"baseline_pipeline = None\n",
|
||||
"\n",
|
||||
"# TODO 6: fit the pipeline and predict X_test\n",
|
||||
"y_pred = None\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "2c22c49d",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 4. Evaluate the attack class\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "b6c6facd",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# TODO 7: calculate cm and unpack tn, fp, fn, tp\n",
|
||||
"cm = None\n",
|
||||
"tn, fp, fn, tp = None, None, None, None\n",
|
||||
"\n",
|
||||
"# TODO 8: calculate accuracy, attack precision, recall, and F1\n",
|
||||
"baseline_metrics = None\n",
|
||||
"\n",
|
||||
"print(pd.Series(baseline_metrics).round(3))\n",
|
||||
"print(f\"False positives: {fp} | False negatives: {fn}\")\n",
|
||||
"\n",
|
||||
"# TODO 9: plot and save the confusion matrix to output/capstone_confusion_matrix.png\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "dc66b35c",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 5. Inspect feature importance\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "5d1894b7",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# TODO 10: create a descending feature_importance Series\n",
|
||||
"feature_importance = None\n",
|
||||
"\n",
|
||||
"print(feature_importance.head(10))\n",
|
||||
"\n",
|
||||
"# TODO 11: plot the top ten features and save the chart\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9f05106e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 6. Export metrics and write an analyst summary\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "8e66b54f",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# TODO 12: export baseline_metrics to output/capstone_metrics.csv\n",
|
||||
"\n",
|
||||
"# TODO 13: create a summary with dataset type, row count, feature count,\n",
|
||||
"# attack rate, precision, recall, F1, false positives, false negatives,\n",
|
||||
"# and a recommendation. Save it to output/capstone_summary.txt.\n",
|
||||
"summary = None\n",
|
||||
"\n",
|
||||
"print(summary)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9b0c0e2f",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Self-check\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "b28ca2a2",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"assert isinstance(clean_df, pd.DataFrame)\n",
|
||||
"assert set(y.unique()) == {0, 1}\n",
|
||||
"assert X.shape[1] >= 5\n",
|
||||
"assert abs(y_train.mean() - y_test.mean()) < 0.02\n",
|
||||
"assert cm.shape == (2, 2)\n",
|
||||
"assert len(feature_importance) == X.shape[1]\n",
|
||||
"assert (OUTPUT_DIR / \"capstone_confusion_matrix.png\").exists()\n",
|
||||
"assert (OUTPUT_DIR / \"capstone_feature_importance.png\").exists()\n",
|
||||
"assert (OUTPUT_DIR / \"capstone_metrics.csv\").exists()\n",
|
||||
"assert (OUTPUT_DIR / \"capstone_summary.txt\").exists()\n",
|
||||
"print(\"✓ Capstone lab completed\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "c7bf86d2",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Capstone questions\n",
|
||||
"\n",
|
||||
"1. How many attacks did the model miss, and why does that number matter operationally?\n",
|
||||
"2. Which features received the highest importance? Do they make sense for network flows?\n",
|
||||
"3. Why does random row splitting provide an optimistic estimate for some security datasets?\n",
|
||||
"4. What additional validation would you require before deploying this detector?\n",
|
||||
"5. If the notebook used synthetic data, which conclusions must be rechecked on the official CIC-IDS2017 subset?\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Reference in New Issue
Block a user