253 lines
8.2 KiB
Plaintext
253 lines
8.2 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "979aed09",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Module 4 — Lab 1: Inspecting the SMS Spam Dataset\n",
|
|
"\n",
|
|
"\n",
|
|
"**Topics:** text data, duplicates, conflicting labels, class distribution\n",
|
|
"\n",
|
|
"> **Solution version:** This notebook contains the complete solution. Run the cells in order and compare the results with your Lab notebook.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c9b13285",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Learning objectives\n",
|
|
"\n",
|
|
"After completing this lab, you will be able to:\n",
|
|
"\n",
|
|
"- load the original UCI tab-separated text file;\n",
|
|
"- audit missing values, exact duplicates, and conflicting labels;\n",
|
|
"- create a binary target without modifying the message text;\n",
|
|
"- describe class imbalance and message-length differences;\n",
|
|
"- explain why aggressive cleaning may remove useful security signals.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "8c2b2a4a",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from pathlib import Path\n",
|
|
"import pandas as pd\n",
|
|
"\n",
|
|
"DATA_PATH = Path(\"data/SMSSpamCollection\")\n",
|
|
"\n",
|
|
"if not DATA_PATH.exists():\n",
|
|
" raise FileNotFoundError(\n",
|
|
" \"Missing data/SMSSpamCollection. Follow data/README.md to download \"\n",
|
|
" \"and extract the official UCI SMS Spam Collection.\"\n",
|
|
" )\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "184f1a06",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 1. Load the original collection\n",
|
|
"\n",
|
|
"The source file has no header. Each row contains a label, one tab character, and the message text.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "f376803d",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"raw_df = pd.read_csv(\n",
|
|
" DATA_PATH,\n",
|
|
" sep=\"\\t\",\n",
|
|
" header=None,\n",
|
|
" names=[\"label\", \"message\"],\n",
|
|
" encoding=\"utf-8\",\n",
|
|
")\n",
|
|
"\n",
|
|
"print(\"Raw shape:\", raw_df.shape)\n",
|
|
"print(raw_df[\"label\"].value_counts(dropna=False))\n",
|
|
"raw_df.head(10)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4ae84500",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 2. Audit the records\n",
|
|
"\n",
|
|
"Exact duplicates can cross a random train-test split and make evaluation too optimistic. A more serious problem occurs when identical text has different labels. Audit both conditions before modeling.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "177c72a1",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"audited_df = raw_df.copy()\n",
|
|
"audited_df[\"label\"] = audited_df[\"label\"].astype(\"string\").str.strip().str.lower()\n",
|
|
"audited_df[\"message\"] = audited_df[\"message\"].astype(\"string\").str.strip()\n",
|
|
"audited_df = audited_df.dropna(subset=[\"label\", \"message\"])\n",
|
|
"audited_df = audited_df[audited_df[\"message\"].ne(\"\")]\n",
|
|
"audited_df = audited_df[audited_df[\"label\"].isin([\"ham\", \"spam\"])]\n",
|
|
"\n",
|
|
"exact_duplicate_count = int(audited_df.duplicated(subset=[\"label\", \"message\"]).sum())\n",
|
|
"label_counts_per_message = audited_df.groupby(\"message\")[\"label\"].nunique()\n",
|
|
"conflicting_messages = label_counts_per_message[label_counts_per_message.gt(1)].index\n",
|
|
"\n",
|
|
"print(\"Exact duplicate rows:\", exact_duplicate_count)\n",
|
|
"print(\"Conflicting message texts:\", len(conflicting_messages))\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b9d41a33",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 3. Build the clean classroom table\n",
|
|
"\n",
|
|
"Remove ambiguous messages before exact duplicates, then map `ham → 0` and `spam → 1`. Keep the original text intact.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "b0b915b1",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"clean_df = audited_df[~audited_df[\"message\"].isin(conflicting_messages)].copy()\n",
|
|
"clean_df = clean_df.drop_duplicates(subset=[\"label\", \"message\"]).reset_index(drop=True)\n",
|
|
"clean_df[\"target\"] = clean_df[\"label\"].map({\"ham\": 0, \"spam\": 1}).astype(int)\n",
|
|
"\n",
|
|
"print(\"Clean shape:\", clean_df.shape)\n",
|
|
"print(clean_df[\"target\"].value_counts().sort_index().rename(index={0: \"ham\", 1: \"spam\"}))\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "74762f79",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 4. Describe the classes and message lengths\n",
|
|
"\n",
|
|
"Length is used only for exploration. The later models will learn from the message text itself.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "1334e9da",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"clean_df[\"character_count\"] = clean_df[\"message\"].str.len()\n",
|
|
"clean_df[\"word_count\"] = clean_df[\"message\"].str.split().str.len()\n",
|
|
"\n",
|
|
"length_summary = clean_df.groupby(\"label\").agg(\n",
|
|
" message_count=(\"message\", \"size\"),\n",
|
|
" median_characters=(\"character_count\", \"median\"),\n",
|
|
" median_words=(\"word_count\", \"median\"),\n",
|
|
")\n",
|
|
"class_distribution = clean_df[\"label\"].value_counts(normalize=True)\n",
|
|
"\n",
|
|
"print(length_summary)\n",
|
|
"print(\"Class proportions:\")\n",
|
|
"print(class_distribution)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "34c637b9",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Security note: do not over-clean\n",
|
|
"\n",
|
|
"URLs, phone numbers, currency symbols, punctuation, unusual capitalization, and deliberate misspellings may contain spam evidence. Begin with the vectorizer defaults. Add normalization only when an experiment demonstrates that it improves validation performance without hiding operationally important signals.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "27f4bb63",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Self-check\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "870a2fbe",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"assert list(raw_df.columns) == [\"label\", \"message\"]\n",
|
|
"assert clean_df[\"message\"].notna().all()\n",
|
|
"assert clean_df[\"message\"].ne(\"\").all()\n",
|
|
"assert set(clean_df[\"label\"].unique()) == {\"ham\", \"spam\"}\n",
|
|
"assert set(clean_df[\"target\"].unique()) == {0, 1}\n",
|
|
"assert not clean_df.duplicated(subset=[\"label\", \"message\"]).any()\n",
|
|
"assert not clean_df[\"message\"].isin(conflicting_messages).any()\n",
|
|
"assert class_distribution.loc[\"ham\"] > class_distribution.loc[\"spam\"]\n",
|
|
"assert {\"character_count\", \"word_count\"}.issubset(clean_df.columns)\n",
|
|
"print(\"✓ Lab 1 completed\")\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "3a766ad5",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Suggested answers\n",
|
|
"\n",
|
|
"1. **Why should exact duplicates be removed before the train-test split?** \n",
|
|
" A duplicate can otherwise appear in both partitions, allowing the model to receive nearly identical training and test evidence and inflating the reported score.\n",
|
|
"2. **Why might removing URLs and currency symbols reduce detection quality?** \n",
|
|
" Those tokens often carry direct spam intent. Removing them can discard discriminative evidence rather than remove harmless noise.\n",
|
|
"3. **Why is accuracy alone risky for this dataset?** \n",
|
|
" The ham class is much larger, so a model can obtain high accuracy while missing many spam messages. Spam precision, recall, F1, and the confusion matrix are also required.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "dcda6cf6-3ba4-4229-8991-8efb2209d294",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.6"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|