CSAI2026 labs: Lab-2..Lab-6 (Practical + Solution, data), Module-2..Module-6 notebooks and datasets
This commit is contained in:
mikrotik
@@ -0,0 +1,252 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1f325e4e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Module 3 — Lab 3: A Random Forest Security Pipeline\n",
|
||||
"\n",
|
||||
"**Topics:** Scikit-learn Pipeline, imputation, Random Forest, fit, predict\n",
|
||||
"\n",
|
||||
"> **Solution version:** This notebook contains the complete solution. Run the cells in order and compare the results with your Lab notebook.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e9ba9f17",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Learning objectives\n",
|
||||
"\n",
|
||||
"After completing this lab, you will be able to:\n",
|
||||
"\n",
|
||||
"- create one reusable preprocessing-and-model pipeline;\n",
|
||||
"- train a Random Forest binary classifier;\n",
|
||||
"- generate predictions for unseen data;\n",
|
||||
"- explain why tree models do not require feature standardization.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "5dc04708",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from pathlib import Path\n",
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"DATA_PATH = Path(\"data/cicids2017_binary_sample.csv\")\n",
|
||||
"RANDOM_STATE = 42\n",
|
||||
"\n",
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"from sklearn.pipeline import Pipeline\n",
|
||||
"from sklearn.impute import SimpleImputer\n",
|
||||
"from sklearn.ensemble import RandomForestClassifier\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3746fff2",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1. Prepare the binary classification data\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "96d91cc2",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df = pd.read_csv(DATA_PATH)\n",
|
||||
"df.columns = df.columns.str.strip()\n",
|
||||
"df = df.drop_duplicates().replace([np.inf, -np.inf], np.nan)\n",
|
||||
"\n",
|
||||
"X = df.drop(columns=\"Label\").select_dtypes(include=\"number\")\n",
|
||||
"y = df[\"Label\"].astype(str).str.strip().str.upper().ne(\"BENIGN\").astype(int)\n",
|
||||
"\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(\n",
|
||||
" X,\n",
|
||||
" y,\n",
|
||||
" test_size=0.2,\n",
|
||||
" random_state=RANDOM_STATE,\n",
|
||||
" stratify=y,\n",
|
||||
")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "69eb8c19",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2. Build one pipeline\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "0dbae88b",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"pipeline = Pipeline([\n",
|
||||
" (\"imputer\", SimpleImputer(strategy=\"median\")),\n",
|
||||
" (\"model\", RandomForestClassifier(\n",
|
||||
" n_estimators=100,\n",
|
||||
" max_depth=12,\n",
|
||||
" random_state=RANDOM_STATE,\n",
|
||||
" n_jobs=-1,\n",
|
||||
" )),\n",
|
||||
"])\n",
|
||||
"\n",
|
||||
"print(pipeline)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ea59d402",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Why there is no `StandardScaler`\n",
|
||||
"\n",
|
||||
"Random Forest makes decisions by comparing feature values with thresholds. Scaling a column does not change the ordering of its values, so standardization is not required for this model family. The imputer remains inside the pipeline because the model needs finite values.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "c8f9a97a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 3. Train and predict\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "c6194b17",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"pipeline.fit(X_train, y_train)\n",
|
||||
"y_pred = pipeline.predict(X_test)\n",
|
||||
"\n",
|
||||
"print(\"Predictions:\", y_pred[:20])\n",
|
||||
"print(\"Number of test samples:\", len(y_pred))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b5ad6c87",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 4. Predict one flow\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7eca199f",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"one_flow = X_test.iloc[[0]]\n",
|
||||
"one_prediction = int(pipeline.predict(one_flow)[0])\n",
|
||||
"\n",
|
||||
"print(\"Prediction:\", one_prediction)\n",
|
||||
"print(\"Meaning:\", \"ATTACK\" if one_prediction == 1 else \"BENIGN\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "90d72108",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Self-check\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "98f48da9",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"assert list(pipeline.named_steps) == [\"imputer\", \"model\"]\n",
|
||||
"assert pipeline.named_steps[\"model\"].n_estimators == 100\n",
|
||||
"assert len(y_pred) == len(y_test)\n",
|
||||
"assert set(np.unique(y_pred)).issubset({0, 1})\n",
|
||||
"assert one_prediction in {0, 1}\n",
|
||||
"print(\"✓ Lab 3 completed\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "6100563d",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Reflection questions\n",
|
||||
"\n",
|
||||
"1. What problem does the imputer solve inside the pipeline?\n",
|
||||
"2. Why is a pipeline safer than manually transforming train and test data in separate cells?\n",
|
||||
"3. What does `random_state=42` contribute to the experiment?\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "aeab0fb6-54bf-4e88-87bc-400d658d67b7",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "75b25828-0978-465b-a789-3181b0c25f38",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "8b198739-c8b0-4355-a44e-08f9c205768b",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "10091acc-3fbd-4e91-9bb4-f994516ce995",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Reference in New Issue
Block a user