{ "cells": [ { "cell_type": "markdown", "id": "cell-001", "metadata": {}, "source": [ "# Module 6 — Lab 1: Auditing NSL-KDD Flow Records\n", "\n", "\n", "**Topics:** network flows, fixed split, labels, attack groups, leakage\n", "\n", "> **Solution version:** Run the cells in order, compare the outputs with your work, and read the suggested answers." ] }, { "cell_type": "markdown", "id": "cell-002", "metadata": {}, "source": [ "## Learning objectives\n", "\n", "After completing this lab, you will be able to:\n", "\n", "- distinguish network-flow features from targets and audit fields;\n", "- preserve the predefined NSL-KDD train/test split;\n", "- describe binary labels and the DoS, Probe, R2L, and U2R groups;\n", "- identify attack labels that appear only in the test set;\n", "- document imbalance, duplicates, and benchmark limitations." ] }, { "cell_type": "code", "execution_count": 1, "id": "cell-003", "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "\n", "TRAIN_PATH = Path(\"data/nsl_kdd_train.csv\")\n", "TEST_PATH = Path(\"data/nsl_kdd_test.csv\")\n", "\n", "missing_files = [str(path) for path in (TRAIN_PATH, TEST_PATH) if not path.exists()]\n", "if missing_files:\n", " raise FileNotFoundError(\n", " \"Missing prepared NSL-KDD files: \" + \", \".join(missing_files) +\n", " \". Follow data/README.md and run: python data/prepare_dataset.py\"\n", " )\n", "\n", "TARGET_COLUMNS = [\"attack_label\", \"attack_group\", \"binary_label\", \"difficulty\"]\n", "CATEGORICAL_FEATURES = [\"protocol_type\", \"service\", \"flag\"]" ] }, { "cell_type": "markdown", "id": "cell-004", "metadata": {}, "source": [ "## 1. Load and validate the fixed split" ] }, { "cell_type": "code", "execution_count": 2, "id": "cell-005", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train shape: (125973, 45)\n", "Test shape: (22544, 45)\n", "Model features: 41\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
durationprotocol_typeserviceflagsrc_bytesdst_byteslandwrong_fragmenturgenthot...dst_host_same_src_port_ratedst_host_srv_diff_host_ratedst_host_serror_ratedst_host_srv_serror_ratedst_host_rerror_ratedst_host_srv_rerror_rateattack_labelattack_groupbinary_labeldifficulty
00tcpftp_datasf49100000...0.170.000.000.000.050.00normalNormal020
10udpothersf14600000...0.880.000.000.000.000.00normalNormal015
20tcpprivates0000000...0.000.001.001.000.000.00neptuneDoS119
30tcphttpsf23281530000...0.030.040.030.010.000.01normalNormal021
40tcphttpsf1994200000...0.000.000.000.000.000.00normalNormal021
\n", "

5 rows × 45 columns

\n", "
" ], "text/plain": [ " duration protocol_type service flag src_bytes dst_bytes land \\\n", "0 0 tcp ftp_data sf 491 0 0 \n", "1 0 udp other sf 146 0 0 \n", "2 0 tcp private s0 0 0 0 \n", "3 0 tcp http sf 232 8153 0 \n", "4 0 tcp http sf 199 420 0 \n", "\n", " wrong_fragment urgent hot ... dst_host_same_src_port_rate \\\n", "0 0 0 0 ... 0.17 \n", "1 0 0 0 ... 0.88 \n", "2 0 0 0 ... 0.00 \n", "3 0 0 0 ... 0.03 \n", "4 0 0 0 ... 0.00 \n", "\n", " dst_host_srv_diff_host_rate dst_host_serror_rate \\\n", "0 0.00 0.00 \n", "1 0.00 0.00 \n", "2 0.00 1.00 \n", "3 0.04 0.03 \n", "4 0.00 0.00 \n", "\n", " dst_host_srv_serror_rate dst_host_rerror_rate dst_host_srv_rerror_rate \\\n", "0 0.00 0.05 0.00 \n", "1 0.00 0.00 0.00 \n", "2 1.00 0.00 0.00 \n", "3 0.01 0.00 0.01 \n", "4 0.00 0.00 0.00 \n", "\n", " attack_label attack_group binary_label difficulty \n", "0 normal Normal 0 20 \n", "1 normal Normal 0 15 \n", "2 neptune DoS 1 19 \n", "3 normal Normal 0 21 \n", "4 normal Normal 0 21 \n", "\n", "[5 rows x 45 columns]" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "train_df = pd.read_csv(TRAIN_PATH, low_memory=False)\n", "test_df = pd.read_csv(TEST_PATH, low_memory=False)\n", "feature_cols = [column for column in train_df.columns if column not in TARGET_COLUMNS]\n", "numeric_features = [column for column in feature_cols if column not in CATEGORICAL_FEATURES]\n", "\n", "assert list(train_df.columns) == list(test_df.columns)\n", "print(\"Train shape:\", train_df.shape)\n", "print(\"Test shape:\", test_df.shape)\n", "print(\"Model features:\", len(feature_cols))\n", "display(train_df.head())" ] }, { "cell_type": "markdown", "id": "cell-006", "metadata": {}, "source": [ "## 2. Audit classes, groups, and unseen attack labels" ] }, { "cell_type": "code", "execution_count": 3, "id": "cell-007", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
traintest
binary_label
0673439711
15863012833
\n", "
" ], "text/plain": [ " train test\n", "binary_label \n", "0 67343 9711\n", "1 58630 12833" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
traintest
attack_group
DoS459277460
Normal673439711
Probe116562421
R2L9952885
U2R5267
\n", "
" ], "text/plain": [ " train test\n", "attack_group \n", "DoS 45927 7460\n", "Normal 67343 9711\n", "Probe 11656 2421\n", "R2L 995 2885\n", "U2R 52 67" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Attack labels seen only in test: ['apache2', 'httptunnel', 'mailbomb', 'mscan', 'named', 'processtable', 'ps', 'saint', 'sendmail', 'snmpgetattack', 'snmpguess', 'sqlattack', 'udpstorm', 'worm', 'xlock', 'xsnoop', 'xterm']\n" ] } ], "source": [ "train_binary = train_df[\"binary_label\"].value_counts().sort_index()\n", "test_binary = test_df[\"binary_label\"].value_counts().sort_index()\n", "train_groups = train_df[\"attack_group\"].value_counts()\n", "test_groups = test_df[\"attack_group\"].value_counts()\n", "\n", "train_attack_labels = set(train_df.loc[train_df[\"binary_label\"] == 1, \"attack_label\"])\n", "test_attack_labels = set(test_df.loc[test_df[\"binary_label\"] == 1, \"attack_label\"])\n", "unseen_test_attacks = sorted(test_attack_labels - train_attack_labels)\n", "\n", "display(pd.DataFrame({\"train\": train_binary, \"test\": test_binary}).fillna(0).astype(int))\n", "display(pd.DataFrame({\"train\": train_groups, \"test\": test_groups}).fillna(0).astype(int))\n", "print(\"Attack labels seen only in test:\", unseen_test_attacks)" ] }, { "cell_type": "markdown", "id": "cell-008", "metadata": {}, "source": [ "## 3. Audit quality and leakage risks" ] }, { "cell_type": "code", "execution_count": 4, "id": "cell-009", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'train_exact_duplicates': 0, 'test_exact_duplicates': 0}\n", "Columns excluded from modeling: ['attack_group', 'attack_label', 'binary_label', 'difficulty']\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
train_missingtest_missingtrain_uniquetest_unique
num_outbound_cmds0011
is_host_login0022
is_guest_login0022
land0022
logged_in0022
root_shell0022
protocol_type0033
wrong_fragment0033
num_shells0034
su_attempted0033
\n", "
" ], "text/plain": [ " train_missing test_missing train_unique test_unique\n", "num_outbound_cmds 0 0 1 1\n", "is_host_login 0 0 2 2\n", "is_guest_login 0 0 2 2\n", "land 0 0 2 2\n", "logged_in 0 0 2 2\n", "root_shell 0 0 2 2\n", "protocol_type 0 0 3 3\n", "wrong_fragment 0 0 3 3\n", "num_shells 0 0 3 4\n", "su_attempted 0 0 3 3" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "quality_summary = pd.DataFrame({\n", " \"train_missing\": train_df[feature_cols].isna().sum(),\n", " \"test_missing\": test_df[feature_cols].isna().sum(),\n", " \"train_unique\": train_df[feature_cols].nunique(dropna=True),\n", " \"test_unique\": test_df[feature_cols].nunique(dropna=True),\n", "})\n", "duplicate_counts = {\n", " \"train_exact_duplicates\": int(train_df.duplicated().sum()),\n", " \"test_exact_duplicates\": int(test_df.duplicated().sum()),\n", "}\n", "forbidden_model_columns = sorted(set(TARGET_COLUMNS) & set(train_df.columns))\n", "\n", "print(duplicate_counts)\n", "print(\"Columns excluded from modeling:\", forbidden_model_columns)\n", "display(quality_summary.sort_values(\"train_unique\").head(10))" ] }, { "cell_type": "markdown", "id": "cell-010", "metadata": {}, "source": [ "## Benchmark note\n", "\n", "NSL-KDD is useful for learning a reproducible IDS workflow. Its fixed test set includes attacks that are absent from training, but it remains a legacy benchmark and does not represent modern production traffic." ] }, { "cell_type": "markdown", "id": "cell-011", "metadata": {}, "source": [ "## Self-check" ] }, { "cell_type": "code", "execution_count": 5, "id": "cell-012", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✓ Lab 1 completed\n" ] } ], "source": [ "assert len(feature_cols) == 41\n", "assert set(CATEGORICAL_FEATURES).issubset(feature_cols)\n", "assert set(train_df[\"binary_label\"].unique()) == {0, 1}\n", "assert set(test_df[\"attack_group\"].unique()).issubset({\"Normal\", \"DoS\", \"Probe\", \"R2L\", \"U2R\"})\n", "assert len(unseen_test_attacks) > 0\n", "assert set(feature_cols).isdisjoint(TARGET_COLUMNS)\n", "assert quality_summary.shape[0] == 41\n", "print(\"✓ Lab 1 completed\")" ] }, { "cell_type": "markdown", "id": "cell-013", "metadata": {}, "source": [ "## Suggested answers\n", "\n", "1. **Why keep the fixed split?** It preserves the benchmark design and exposes attack labels that training never saw.\n", "2. **Why exclude `difficulty`?** It was derived from previous classifiers and is not a network measurement available to a real detector.\n", "3. **Why inspect group counts?** Binary metrics can hide poor detection of rare R2L or U2R records." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.6" } }, "nbformat": 4, "nbformat_minor": 5 }