80 lines
2.7 KiB
Markdown
80 lines
2.7 KiB
Markdown
# Dataset setup — UCI Malware Static and Dynamic Features
|
|
|
|
All six notebooks use pre-extracted tabular features from the official UCI dataset **Malware Static and Dynamic Features VxHeaven and Virus Total**.
|
|
|
|
## Download
|
|
|
|
1. Open the [UCI dataset page](https://archive.ics.uci.edu/dataset/541/malware%2Bstatic%2Band%2Bdynamic%2Bfeatures%2Bvxheaven%2Band%2Bvirus%2Btotal).
|
|
2. Select **Download** and extract the archive.
|
|
3. Create `data/raw/` if it does not already exist.
|
|
4. Copy these three files into `data/raw/`:
|
|
|
|
```text
|
|
staDynBenignLab.csv
|
|
staDynVxHeaven2698Lab.csv
|
|
staDynVt2955Lab.csv
|
|
```
|
|
|
|
The expected layout is:
|
|
|
|
```text
|
|
Practical-5/
|
|
└── data/
|
|
├── prepare_dataset.py
|
|
└── raw/
|
|
├── staDynBenignLab.csv
|
|
├── staDynVxHeaven2698Lab.csv
|
|
└── staDynVt2955Lab.csv
|
|
```
|
|
|
|
## Prepare the classroom table
|
|
|
|
From the `Practical-5` directory, run:
|
|
|
|
```bash
|
|
python data/prepare_dataset.py
|
|
```
|
|
|
|
The utility:
|
|
|
|
- finds the three required files under `data/raw/`;
|
|
- normalizes column names and adds deterministic occurrence suffixes when
|
|
case- or punctuation-distinct UCI fields collapse to the same snake-case name;
|
|
- keeps columns shared by all three sources;
|
|
- removes obvious identifiers and pre-existing target columns;
|
|
- converts usable columns to numeric values;
|
|
- adds `source` and `label` (`0 = benign`, `1 = malware`);
|
|
- retains missing values so imputation can be fitted on training data inside the notebooks;
|
|
- writes `data/malware_features.csv` and `data/feature_manifest.csv`.
|
|
|
|
## Quick verification
|
|
|
|
```python
|
|
from pathlib import Path
|
|
import pandas as pd
|
|
|
|
path = Path("data/malware_features.csv")
|
|
df = pd.read_csv(path)
|
|
print(df.shape)
|
|
print(df["label"].value_counts().sort_index())
|
|
print(df["source"].value_counts())
|
|
```
|
|
|
|
The original release contains approximately 6,248 rows: 595 benign records and 5,653 malware records. The exact number of retained feature columns can vary if the source archive changes column formatting.
|
|
|
|
## Provenance and license
|
|
|
|
- Dataset: Malware Static and Dynamic Features VxHeaven and Virus Total
|
|
- Source: UCI Machine Learning Repository
|
|
- DOI: [10.24432/C58K6H](https://doi.org/10.24432/C58K6H)
|
|
- License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)
|
|
|
|
Retain the citation and license when redistributing derived classroom tables.
|
|
|
|
## Important limitations
|
|
|
|
- The class distribution is strongly imbalanced.
|
|
- Benign and malware samples originate from different collections, so source-specific artifacts may inflate random-split performance.
|
|
- The features represent a historical dataset and should not be treated as a current production benchmark.
|
|
- No raw malware is required or permitted for this practical.
|