All six notebooks use the fixed NSL-KDD train/test split. NSL-KDD retains the 41 KDD network features while reducing the duplicate-record problem of the original KDD Cup 1999 release.

Download

The official UNB NSL-KDD page documents the dataset, citation, and redistribution conditions, but it no longer provides direct file downloads.

Download these two files from the NSL-KDD GitHub mirror:

KDDTrain+_20Percent.txt
KDDTest+.txt

A preserved copy is also available through the NSL-KDD Zenodo record. If the Zenodo record does not include the 20% training file, download KDDTrain+.txt; the preparation command accepts it when --full-train is supplied.

Place the files as follows:

Practical-6/
└── data/
    ├── prepare_dataset.py
    └── raw/
        ├── KDDTrain+_20Percent.txt
        └── KDDTest+.txt

Prepare the classroom tables

From Practical-6, run:

python data/prepare_dataset.py

To use the full training set instead, place KDDTrain+.txt under data/raw/ and run:

python data/prepare_dataset.py --full-train

The utility:

applies the official 41 feature names;
adds attack_label, attack_group, binary_label, and difficulty;
validates numeric and categorical columns;
rejects unknown attack labels so mapping errors remain visible;
preserves the original fixed train/test split;
writes nsl_kdd_train.csv, nsl_kdd_test.csv, and feature_manifest.csv.

Quick verification

from pathlib import Path
import pandas as pd

train = pd.read_csv(Path("data/nsl_kdd_train.csv"))
test = pd.read_csv(Path("data/nsl_kdd_test.csv"))

print("Train:", train.shape)
print("Test:", test.shape)
print(train["binary_label"].value_counts().sort_index())
print(test["attack_group"].value_counts())

Each prepared table must have 41 model features plus these four audit/target columns:

attack_label, attack_group, binary_label, difficulty

Citation and redistribution

The official UNB page permits redistribution but requires citation of NSL-KDD and the associated paper:

M. Tavallaee, E. Bagheri, W. Lu, and A. Ghorbani, “A Detailed Analysis of the KDD CUP 99 Data Set,” CISDA, 2009.

Retain this citation when redistributing prepared classroom data.

Important limitations

NSL-KDD is derived from traffic collected for the KDD Cup 1999 benchmark.
The fixed test set contains attack types not present in the training set; this supports a useful generalization discussion but does not make the benchmark modern.
Class prevalence does not represent a production SOC.
IP addresses, packet payloads, timestamps, and modern encrypted-flow characteristics are absent.
The dataset has no dedicated botnet label.

Dataset setup — CTU-13 for Lab 7

CTU-13 contains 13 real-network botnet scenarios with flow-level Botnet, Normal, and Background labels. Lab 7 uses the labeled bidirectional .binetflow file from one scenario; PCAP files and executable samples are not required.

Recommended download

Use the official CTU-13 dataset page and download Scenario 8, listed as CTU-Malware-Capture-Botnet-49. Locate this file inside the scenario's detailed-bidirectional-flow-labels directory:

capture20110816-3.binetflow

Scenario 8 is recommended because it matches the botnet detection reference used for this practical. Another labeled CTU-13 scenario can be used, but class counts and model results will differ.

The official page also provides the complete CTU-13-Dataset.tar.bz2 archive. The compressed archive is approximately 1.9 GB, so downloading one scenario is usually more convenient for a classroom exercise.

Placement and preparation

Place the .binetflow file below data/raw/:

Practical-6/
└── data/
    ├── prepare_ctu13.py
    └── raw/
        └── capture20110816-3.binetflow

From Practical-6, run:

python data/prepare_ctu13.py data/raw/capture20110816-3.binetflow

If exactly one .binetflow file exists below data/raw/, the input argument may be omitted:

python data/prepare_ctu13.py

The utility reads the source in chunks and retains at most 20,000 records from each label family. To adjust the classroom sample size or memory usage:

python data/prepare_ctu13.py data/raw/capture20110816-3.binetflow --per-class 30000 --chunk-size 100000

It creates:

data/ctu13_botnet_flows.csv

The loader accepts both common CTU-13 flow schemas:

the compact 15-column export containing Dir;
the extended Argus export containing directional counters such as SrcPkts, DstPkts, and DstBytes but no Dir.

For the extended schema, the classroom direction feature is derived as bidirectional, source_only, destination_only, or unknown from the source and destination packet counters. When available, the original DstBytes value is used instead of reconstructing it from total and source bytes.

Label policy

Botnet receives binary_label = 1.
Normal receives binary_label = 0.
Background receives binary_label = -1 because its status is uncertain.

Lab 7 trains and evaluates on Botnet versus Normal only. Background flows are scored for investigation but excluded from supervised performance metrics.

Safety and leakage controls

Use only .binetflow flow records; do not download or execute malware samples.
Source and destination IP addresses are retained for analyst triage but excluded from model features.
Raw label fields are excluded from model features.
The preparation utility applies bounded sampling so the notebook does not need to load the complete raw scenario.
A same-scenario random split is suitable for instruction but does not demonstrate cross-botnet generalization.

Citation

S. Garcia, M. Grill, J. Stiborek, and A. Zunino, “An empirical comparison of botnet detection methods,” Computers & Security, vol. 45, pp. 100–123, 2014. https://doi.org/10.1016/j.cose.2014.05.011

CTU-13 is distributed under the Creative Commons Attribution license. Preserve the dataset citation and attribution in redistributed course materials.

README.md Unescape Escape

Dataset setup — NSL-KDD