236 lines
7.5 KiB
Plaintext
236 lines
7.5 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cell-001",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Module 5 — Lab 1: Auditing a Malware Feature Dataset\n",
|
|
"\n",
|
|
"\n",
|
|
"**Topics:** provenance, schema, imbalance, missing values, leakage\n",
|
|
"\n",
|
|
"> **Solution version:** Run the cells in order, compare the outputs with your work, and read the suggested answers."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cell-002",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Learning objectives\n",
|
|
"\n",
|
|
"After completing this lab, you will be able to:\n",
|
|
"\n",
|
|
"- load a prepared table of static and dynamic malware features safely;\n",
|
|
"- distinguish identifiers, provenance fields, features, and the binary label;\n",
|
|
"- audit class imbalance, source composition, missing values, infinite values, and duplicates;\n",
|
|
"- identify columns that must not enter a predictive model;\n",
|
|
"- document dataset limitations before modeling.\n",
|
|
"\n",
|
|
"> **Safety boundary:** This practical uses numeric feature tables only. Do not download, open, or execute malware samples."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "cell-003",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from pathlib import Path\n",
|
|
"import numpy as np\n",
|
|
"import pandas as pd\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"\n",
|
|
"DATA_PATH = Path(\"data/malware_features.csv\")\n",
|
|
"\n",
|
|
"if not DATA_PATH.exists():\n",
|
|
" raise FileNotFoundError(\n",
|
|
" \"Missing data/malware_features.csv. Follow data/README.md, place the three \"\n",
|
|
" \"official UCI CSV files under data/raw/, and run: python data/prepare_dataset.py\"\n",
|
|
" )"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cell-004",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 1. Load the prepared table\n",
|
|
"\n",
|
|
"The preparation utility adds two non-feature columns: `source` records provenance and `label` uses `0 = benign`, `1 = malware`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "cell-005",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"df = pd.read_csv(DATA_PATH, low_memory=False)\n",
|
|
"required_columns = {\"source\", \"label\"}\n",
|
|
"if not required_columns.issubset(df.columns):\n",
|
|
" raise ValueError(f\"Missing required columns: {required_columns - set(df.columns)}\")\n",
|
|
"\n",
|
|
"feature_cols = [column for column in df.columns if column not in required_columns]\n",
|
|
"print(\"Dataset shape:\", df.shape)\n",
|
|
"print(\"Number of candidate features:\", len(feature_cols))\n",
|
|
"display(df.head())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cell-006",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 2. Audit labels and provenance\n",
|
|
"\n",
|
|
"`source` is useful for detecting collection bias, but it must never be used as a predictor. In this dataset, collection source and class are closely related."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "cell-007",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"class_counts = df[\"label\"].value_counts().sort_index()\n",
|
|
"class_proportions = df[\"label\"].value_counts(normalize=True).sort_index()\n",
|
|
"source_by_label = pd.crosstab(df[\"source\"], df[\"label\"], margins=True)\n",
|
|
"\n",
|
|
"print(\"Class counts (0 = benign, 1 = malware):\")\n",
|
|
"display(pd.DataFrame({\"count\": class_counts, \"proportion\": class_proportions}))\n",
|
|
"print(\"Source by label:\")\n",
|
|
"display(source_by_label)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cell-008",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 3. Audit data quality and leakage risks\n",
|
|
"\n",
|
|
"Missing values will be imputed inside a training pipeline in later labs. Imputing before the split would allow test-set statistics to influence training."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "cell-009",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"numeric_features = df[feature_cols].apply(pd.to_numeric, errors=\"coerce\")\n",
|
|
"infinite_count = int(np.isinf(numeric_features.to_numpy(dtype=float)).sum())\n",
|
|
"missing_summary = numeric_features.isna().mean().sort_values(ascending=False).rename(\"missing_rate\")\n",
|
|
"duplicate_count = int(df.duplicated().sum())\n",
|
|
"constant_features = numeric_features.nunique(dropna=True).loc[lambda values: values <= 1].index.tolist()\n",
|
|
"\n",
|
|
"suspicious_tokens = (\"label\", \"class\", \"target\", \"source\", \"filename\", \"hash\", \"md5\", \"sha\")\n",
|
|
"suspicious_columns = [\n",
|
|
" column for column in feature_cols\n",
|
|
" if any(token in column.lower() for token in suspicious_tokens)\n",
|
|
"]\n",
|
|
"\n",
|
|
"print(\"Exact duplicate rows:\", duplicate_count)\n",
|
|
"print(\"Infinite numeric values:\", infinite_count)\n",
|
|
"print(\"Constant candidate features:\", len(constant_features))\n",
|
|
"print(\"Suspicious feature names:\", suspicious_columns[:20])\n",
|
|
"display(missing_summary.head(10).to_frame())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cell-010",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 4. Write the dataset risk statement\n",
|
|
"\n",
|
|
"Record three facts before modeling:\n",
|
|
"\n",
|
|
"1. The positive class is the majority class.\n",
|
|
"2. Benign and malware rows come from different source collections.\n",
|
|
"3. A random split can measure interpolation within this historical table, but not performance on future malware campaigns."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cell-011",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Self-check"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "cell-012",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"assert {\"source\", \"label\"}.issubset(df.columns)\n",
|
|
"assert len(feature_cols) > 0\n",
|
|
"assert set(df[\"label\"].dropna().unique()) == {0, 1}\n",
|
|
"assert set(feature_cols).isdisjoint({\"source\", \"label\"})\n",
|
|
"assert class_counts.loc[1] > class_counts.loc[0]\n",
|
|
"assert len(missing_summary) == len(feature_cols)\n",
|
|
"assert infinite_count == 0\n",
|
|
"print(\"✓ Lab 1 completed\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cell-013",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Suggested answers\n",
|
|
"\n",
|
|
"1. **Why exclude `source`?** The source is strongly associated with the label. A model could learn the collection process instead of malware behavior.\n",
|
|
"2. **Why delay imputation?** The imputer must learn medians from training data only; otherwise test information leaks into training.\n",
|
|
"3. **Why is a random split limited?** Samples from the same historical collections appear on both sides of the split, so the test set does not represent future drift or a new collection process."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "0c3b5e17-8d80-4c4b-a68f-7d6e085b04f0",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "a20ec3d1-331a-46ab-8e18-f3cd73ff9fc3",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.6"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|