# Dataset setup — UCI Malware Static and Dynamic Features

All six notebooks use pre-extracted tabular features from the official UCI dataset **Malware Static and Dynamic Features VxHeaven and Virus Total**.

## Download

1. Open the [UCI dataset page](https://archive.ics.uci.edu/dataset/541/malware%2Bstatic%2Band%2Bdynamic%2Bfeatures%2Bvxheaven%2Band%2Bvirus%2Btotal).
2. Select **Download** and extract the archive.
3. Create `data/raw/` if it does not already exist.
4. Copy these three files into `data/raw/`:

```text
staDynBenignLab.csv
staDynVxHeaven2698Lab.csv
staDynVt2955Lab.csv
```

The expected layout is:

```text
Practical-5/
└── data/
    ├── prepare_dataset.py
    └── raw/
        ├── staDynBenignLab.csv
        ├── staDynVxHeaven2698Lab.csv
        └── staDynVt2955Lab.csv
```

## Prepare the classroom table

From the `Practical-5` directory, run:

```bash
python data/prepare_dataset.py
```

The utility:

- finds the three required files under `data/raw/`;
- normalizes column names and adds deterministic occurrence suffixes when
  case- or punctuation-distinct UCI fields collapse to the same snake-case name;
- keeps columns shared by all three sources;
- removes obvious identifiers and pre-existing target columns;
- converts usable columns to numeric values;
- adds `source` and `label` (`0 = benign`, `1 = malware`);
- retains missing values so imputation can be fitted on training data inside the notebooks;
- writes `data/malware_features.csv` and `data/feature_manifest.csv`.

## Quick verification

```python
from pathlib import Path
import pandas as pd

path = Path("data/malware_features.csv")
df = pd.read_csv(path)
print(df.shape)
print(df["label"].value_counts().sort_index())
print(df["source"].value_counts())
```

The original release contains approximately 6,248 rows: 595 benign records and 5,653 malware records. The exact number of retained feature columns can vary if the source archive changes column formatting.

## Provenance and license

- Dataset: Malware Static and Dynamic Features VxHeaven and Virus Total
- Source: UCI Machine Learning Repository
- DOI: [10.24432/C58K6H](https://doi.org/10.24432/C58K6H)
- License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

Retain the citation and license when redistributing derived classroom tables.

## Important limitations

- The class distribution is strongly imbalanced.
- Benign and malware samples originate from different collections, so source-specific artifacts may inflate random-split performance.
- The features represent a historical dataset and should not be treated as a current production benchmark.
- No raw malware is required or permitted for this practical.