labs/Lab-3/Lab-3-Practical/06_Capstone_CICIDS2017_Binary_IDS_Lab.ipynb

Module 3 — Lab 6: Capstone — Binary Intrusion Detection with CIC-IDS2017

Topics: end-to-end ML workflow, CIC-IDS2017, binary classification, evaluation, reporting

Instructions: Run the cells in order. Complete every TODO, then run the Self-check cell. Do not open the Solution notebook before attempting the tasks yourself.

Mission

Build one complete and leakage-safe intrusion-detection workflow:

1. load a network-flow table;
2. clean invalid values and duplicate rows;
3. map all attacks to one ATTACK class;
4. create one stratified train-test split;
5. train one Random Forest pipeline;
6. evaluate attack precision, recall, F1, and error counts;
7. export a short analyst report.

The notebook uses data/cicids2017_binary_sample.csv when that file is available. Until the instructor prepares the official subset, it uses the included synthetic dataset and prints a clear warning.

In [ ]:

from pathlib import Path
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import (
    ConfusionMatrixDisplay, accuracy_score, confusion_matrix,
    precision_score, recall_score, f1_score
)

REAL_DATA_PATH = Path("data/cicids2017_binary_sample.csv")
DEMO_DATA_PATH = Path("data/synthetic_security_flows.csv")
OUTPUT_DIR = Path("output")
OUTPUT_DIR.mkdir(exist_ok=True)
RANDOM_STATE = 42

DATA_PATH = REAL_DATA_PATH if REAL_DATA_PATH.exists() else DEMO_DATA_PATH
USING_REAL_CICIDS = REAL_DATA_PATH.exists()

print("Dataset:", DATA_PATH)
if not USING_REAL_CICIDS:
    print("WARNING: Using synthetic classroom data. Add the prepared CIC-IDS2017 file for the real capstone.")

1. Load, clean, and define the binary target

In [ ]:

# TODO 1: load DATA_PATH into raw_df
raw_df = None

# TODO 2: strip column names, remove duplicates, and replace +/- infinity with NaN
clean_df = None

# TODO 3: create numeric X and binary y (BENIGN=0, every attack=1)
X = None
y = None

print(f"Raw rows: {None}")
print(f"Clean rows: {None}")
print(f"Numeric features: {None}")
print("Binary class distribution:")
print(None)

2. Create one fixed train-test split

In [ ]:

# TODO 4: create an 80/20 split with stratify=y and random_state=42
X_train, X_test, y_train, y_test = None, None, None, None

print("Training samples:", None)
print("Test samples:", None)
print("Training attack rate:", None)
print("Test attack rate:", None)

3. Build and train the baseline pipeline

In [ ]:

# TODO 5: create an imputer + Random Forest pipeline
baseline_pipeline = None

# TODO 6: fit the pipeline and predict X_test
y_pred = None

4. Evaluate the attack class

In [ ]:

# TODO 7: calculate cm and unpack tn, fp, fn, tp
cm = None
tn, fp, fn, tp = None, None, None, None

# TODO 8: calculate accuracy, attack precision, recall, and F1
baseline_metrics = None

print(pd.Series(baseline_metrics).round(3))
print(f"False positives: {fp} | False negatives: {fn}")

# TODO 9: plot and save the confusion matrix to output/capstone_confusion_matrix.png

5. Inspect feature importance

In [ ]:

# TODO 10: create a descending feature_importance Series
feature_importance = None

print(feature_importance.head(10))

# TODO 11: plot the top ten features and save the chart

6. Export metrics and write an analyst summary

In [ ]:

# TODO 12: export baseline_metrics to output/capstone_metrics.csv

# TODO 13: create a summary with dataset type, row count, feature count,
# attack rate, precision, recall, F1, false positives, false negatives,
# and a recommendation. Save it to output/capstone_summary.txt.
summary = None

print(summary)

Self-check

In [ ]:

assert isinstance(clean_df, pd.DataFrame)
assert set(y.unique()) == {0, 1}
assert X.shape[1] >= 5
assert abs(y_train.mean() - y_test.mean()) < 0.02
assert cm.shape == (2, 2)
assert len(feature_importance) == X.shape[1]
assert (OUTPUT_DIR / "capstone_confusion_matrix.png").exists()
assert (OUTPUT_DIR / "capstone_feature_importance.png").exists()
assert (OUTPUT_DIR / "capstone_metrics.csv").exists()
assert (OUTPUT_DIR / "capstone_summary.txt").exists()
print("✓ Capstone lab completed")

Capstone questions

1. How many attacks did the model miss, and why does that number matter operationally?
2. Which features received the highest importance? Do they make sense for network flows?
3. Why does random row splitting provide an optimistic estimate for some security datasets?
4. What additional validation would you require before deploying this detector?
5. If the notebook used synthetic data, which conclusions must be rechecked on the official CIC-IDS2017 subset?