CSAI2026 labs: Lab-2..Lab-6 (Practical + Solution, data), Module-2..Module-6 notebooks and datasets
This commit is contained in:
mikrotik
@@ -0,0 +1,213 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cell-001",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Module 5 — Lab 3: Building a Decision Tree Baseline\n",
|
||||
"\n",
|
||||
" \n",
|
||||
"**Topics:** stratified split, dummy baseline, bounded tree, leakage-safe pipeline\n",
|
||||
"\n",
|
||||
"> **Solution version:** Run the cells in order, compare the outputs with your work, and read the suggested answers."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cell-002",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Learning objectives\n",
|
||||
"\n",
|
||||
"After completing this lab, you will be able to:\n",
|
||||
"\n",
|
||||
"- create a reproducible stratified train-test split;\n",
|
||||
"- fit missing-value imputation on training data only;\n",
|
||||
"- compare a bounded Decision Tree with a dummy baseline;\n",
|
||||
"- evaluate malware precision, recall, F1, false positives, and false negatives;\n",
|
||||
"- explain why trees do not require feature scaling."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "cell-003",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from pathlib import Path\n",
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"\n",
|
||||
"DATA_PATH = Path(\"data/malware_features.csv\")\n",
|
||||
"\n",
|
||||
"if not DATA_PATH.exists():\n",
|
||||
" raise FileNotFoundError(\n",
|
||||
" \"Missing data/malware_features.csv. Follow data/README.md, place the three \"\n",
|
||||
" \"official UCI CSV files under data/raw/, and run: python data/prepare_dataset.py\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"from sklearn.dummy import DummyClassifier\n",
|
||||
"from sklearn.impute import SimpleImputer\n",
|
||||
"from sklearn.metrics import confusion_matrix, precision_score, recall_score, f1_score, accuracy_score\n",
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"from sklearn.pipeline import Pipeline\n",
|
||||
"from sklearn.tree import DecisionTreeClassifier\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def security_metrics(y_true, y_pred):\n",
|
||||
" tn, fp, fn, tp = confusion_matrix(y_true, y_pred, labels=[0, 1]).ravel()\n",
|
||||
" return {\n",
|
||||
" \"accuracy\": accuracy_score(y_true, y_pred),\n",
|
||||
" \"malware_precision\": precision_score(y_true, y_pred, zero_division=0),\n",
|
||||
" \"malware_recall\": recall_score(y_true, y_pred, zero_division=0),\n",
|
||||
" \"malware_f1\": f1_score(y_true, y_pred, zero_division=0),\n",
|
||||
" \"false_positives\": int(fp),\n",
|
||||
" \"false_negatives\": int(fn),\n",
|
||||
" \"false_positive_rate\": fp / (fp + tn) if fp + tn else 0.0,\n",
|
||||
" }"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "cell-004",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df = pd.read_csv(DATA_PATH, low_memory=False)\n",
|
||||
"feature_cols = [column for column in df.columns if column not in {\"source\", \"label\"}]\n",
|
||||
"X = df[feature_cols].apply(pd.to_numeric, errors=\"coerce\")\n",
|
||||
"y = df[\"label\"].astype(int)\n",
|
||||
"\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(\n",
|
||||
" X, y, test_size=0.20, stratify=y, random_state=42\n",
|
||||
")\n",
|
||||
"print(\"Train shape:\", X_train.shape, \"Test shape:\", X_test.shape)\n",
|
||||
"print(\"Malware rate — train/test:\", y_train.mean().round(4), y_test.mean().round(4))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cell-005",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1. Establish a non-learning baseline\n",
|
||||
"\n",
|
||||
"Because malware is the majority class, a most-frequent classifier can achieve high accuracy while providing little operational value."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "cell-006",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"dummy = DummyClassifier(strategy=\"most_frequent\")\n",
|
||||
"dummy.fit(X_train, y_train)\n",
|
||||
"dummy_predictions = dummy.predict(X_test)\n",
|
||||
"dummy_metrics = security_metrics(y_test, dummy_predictions)\n",
|
||||
"pd.Series(dummy_metrics, name=\"dummy\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cell-007",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2. Train a bounded Decision Tree\n",
|
||||
"\n",
|
||||
"The imputer is inside the pipeline, so its medians are learned from training rows only. A shallow tree is intentionally chosen for readability and to reduce overfitting."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "cell-008",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"tree_pipeline = Pipeline([\n",
|
||||
" (\"imputer\", SimpleImputer(strategy=\"median\", add_indicator=True)),\n",
|
||||
" (\"tree\", DecisionTreeClassifier(\n",
|
||||
" max_depth=4,\n",
|
||||
" min_samples_leaf=10,\n",
|
||||
" random_state=42,\n",
|
||||
" )),\n",
|
||||
"])\n",
|
||||
"tree_pipeline.fit(X_train, y_train)\n",
|
||||
"tree_predictions = tree_pipeline.predict(X_test)\n",
|
||||
"tree_metrics = security_metrics(y_test, tree_predictions)\n",
|
||||
"\n",
|
||||
"comparison = pd.DataFrame([dummy_metrics, tree_metrics], index=[\"Dummy\", \"Decision Tree\"])\n",
|
||||
"display(comparison.round(4))\n",
|
||||
"display(pd.DataFrame(\n",
|
||||
" confusion_matrix(y_test, tree_predictions, labels=[0, 1]),\n",
|
||||
" index=[\"Actual benign\", \"Actual malware\"],\n",
|
||||
" columns=[\"Predicted benign\", \"Predicted malware\"],\n",
|
||||
"))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cell-009",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Self-check"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "cell-010",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"assert X_train.shape[0] + X_test.shape[0] == len(df)\n",
|
||||
"assert abs(y_train.mean() - y_test.mean()) < 0.01\n",
|
||||
"assert set(tree_predictions).issubset({0, 1})\n",
|
||||
"assert tree_pipeline.named_steps[\"tree\"].get_depth() <= 4\n",
|
||||
"assert set(comparison.index) == {\"Dummy\", \"Decision Tree\"}\n",
|
||||
"assert comparison.loc[\"Decision Tree\", \"malware_f1\"] >= comparison.loc[\"Dummy\", \"malware_f1\"]\n",
|
||||
"print(\"✓ Lab 3 completed\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cell-011",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Suggested answers\n",
|
||||
"\n",
|
||||
"1. **Why can dummy accuracy look high?** Malware is the majority class, so always predicting malware is often correct but creates many false positives.\n",
|
||||
"2. **Why no scaler?** Decision Trees compare one feature with a threshold; multiplying a feature by a constant changes the threshold but not the ordering of samples.\n",
|
||||
"3. **Why bound the depth?** A shallow tree is easier to audit and is less likely to memorize small training regions."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Reference in New Issue
Block a user